It has generally been acknowledged that data is everything. And while it may be everything, it needs time to mature within an organisation, so that it can grow from just an asset into a competitive advantage. Thus, it needs a model to rely on in the process.
A data maturity model is the foundational guideline that establishes the best practices of enterprise data management, with a focus on defining, implementing, improving and evaluating data across an organisation.
Defined by different layers of maturity, the approach towards data starts with building awareness that data has become paramount to running even the most isolated business processes and ends with extracting the most relevant insights from data, with a view to improving business performance.
Data awareness
At the first layer of maturity, the approach to data is rather reactive and isolated inside each specific project. At this point, data may exist in a raw state, unprocessed, uncleaned, unconsolidated, or not be taken care of at all. Some data processes may be well defined inside each project, but they are not transmitted across the organisation, resulting in a non-strategic view of the data. This could result in suboptimal decisions on the implementation of goals to be achieved, workflows to be enforced and tools to be used, which could impact the organisation in the long term.
Understanding the data, gathering the relevant business requirements and criteria, and documenting the entities, processes and their relationships into data models and diagrams will enable the organisation to define the role that data has for it as well as a long-term data plan.
Data as an asset
After raising awareness of the importance of data, data becomes a critical asset, and defining a policy becomes imperative to manage it in a systematic manner. At this point, having consolidated data is the main objective. Involving relevant data roles and building capabilities empowers the organisation to establish practices and processes in a transparent manner. Architecture, analysis, engineering, integration, governance and quality assurance of data constitute the recipe for the data management strategy. It is only through the cross-functional dependencies of these areas that data processes can improve and become optimal.
This entire layer of data maturity encompasses projects around the consolidation and integration of different data sources, leading to the creation of data warehouses, data lakes and big data platforms, depending on the business needs. Data lakes and data warehouses are both widely used for storing big data, but with different objectives.
A data lake is a vast pool of raw data, the purpose of which is not yet defined. A data warehouse is a repository for structured, filtered data that has already been processed for a specific purpose, one of them being reporting. As the variety of data types increases, along with the data volume within an organisation, it becomes imperative to increase the velocity at which the data must be processed. Thus, big data platforms with cloud computing capabilities should be explored at this stage.
All these solutions to store data and treat it properly as an asset constitute the solid basis for the next maturity layers.
Data as a mindset
After data consolidation, another shift happens at the third level of maturity where data management guidelines become the norm. Following them consistently allows for implementing the mindset at an organisational level, with data being treated as a scope in itself to ensure long-term success. At this point, having accurate, relevant and clean data is the key.
Metadata management becomes mandatory, with data dictionaries, data catalogues and business rules enabling the consolidation of key business terms, metrics and data assets. Accompanied by data lineage and keeping traceability for audit trails, the organisation will be ready to implement data regulation practices. The entire data life cycle is thus managed in a transparent manner, with clear documentation of how the data was created, used and archived. On top of that, access rules close the data security gap, which is essential for how the data is valued at the next maturity levels.
By profiling the data, problems can be discovered early, and costs can be avoided in the long run. Issues with the uniqueness of keys, duplicated data, lack of data validation upon input and missing, misleading or improperly formatted data could harm the next stages of generating knowledge out of data.
Data as an insight
As data now has value beyond the asset that it represents, it can become the source of insights that leverage the ultimate advantage in a competitive context. At the fourth layer of maturity, analysing the data through statistical and quantitative techniques uncovers paths unforeseen before.
At this level, it is very important to make the data visible to the relevant audience within the organisation. Focusing on both analytic and operational reports, generating relevant insights is the main objective.
Descriptive analytics is leading the way now, be it custom or self-serviced, in the form of consolidated reports or ad-hoc analyses. Backing up these visualisations with a solid reporting strategy will allow the relevant business stakeholders to make the right decision at the right moment.
Data as an advantage
At the fifth level of maturity, data is seen as essential for identifying opportunities to improve processes in a competitive market.
Structured or unstructured, the data holds knowledge which can be extracted through scientific methods and algorithms. Predictive algorithms, such as regression and classification, can be used to predict future events and trends. Supervised algorithms can use prior knowledge to learn the relationship between the input and the output that can be observed in data. Unsupervised learning, while not relying on labelled output, must infer the natural structure already existing in the data.
Data needs that may arise at this maturity level can be related to predictive analytics, data classification and segmentation and anomaly detection. Further insights can be extracted with natural language processing techniques. Any question that can be answered through machine learning, deep learning and artificial intelligence (AI) techniques sits within this ultimate layer.
Of course, sharing your best practices with the industry is also part of this final level of maturity.
As we can see, a data maturity model holds the journey from first encountering data to developing a long-term relationship that is here to stay.