Finding a Way in the Data Labyrinth – Episode 2 - How to Navigate the Data Evolution?

In this episode of the blog, we will discuss possible steps to find a way in the Data Labyrinth. 

At the start of the data transformation journey, it is necessary to define a solid strategy and a clear vision. From developing a data and AI roadmap incorporating strategic imperatives to identifying use cases and priorities. Here, it is essential to unpack the organization’s data maturity by doing an in-depth assessment. This assessment can then be used to create a data and AI-operating model that delivers what a company needs - both in the long and the short term. This model should lead to a data platform that has the following capacities:

- Possibility to integrate any source type
- Suitable for various types of usage patterns
- Ability to deliver ready-to-use datasets for reporting, APIs, AI models, and data exchange with other organizations
- Flexible scaling up and down of compute and storage capacity
- In-built building blocks tailored to identified use cases
- Good governance

For a successful implementation, the model must be validated and supported by both the business and IT stakeholders.

Next comes the “Start Small, Think Big” or incubate phase:  identifying new ideas, experimenting, and innovating by creating pilots to ensure that the transformation swiftly gain speed and the time to value decreases. The data estate modernization must be visible to the users and with plenty of interaction between IT and the business. To gain the trust of the users and their managers, it is essential to come up with quick results and show that the new strategy has a positive impact on their work. It is ideal to confront all the stakeholders - business and IT with new concepts like AI and machine learning to make them comfortable with new ways of using the company data assets. Also, test the tools in this phase to assess and see which ones best fit the requirements.

After the incubation step, it is time to build and deploy the data and AI platform in the organization. Once the tools are selected, the plans created, and the necessary resources allocated, the migration of the old data systems can finally go ahead. The new data platform will be rolled out in a step-by-step manner, department by department, delivering relevant services and benefits regarding operational costs, project budgets, and productivity.

Data governance is set up or adapted. Change management initiatives are initiated within the organization to inform, teach, and convince the user community about the new ways of working. It is made clear to the organization that data modernization is a bit of work and will involve the whole organization.

For example, setting up an AI environment is much simpler than asking a data scientist to test some algorithms and show users fancy results. To avoid the valley of death - which means that an AI effort never passes the stage of POC, a whole new and specific process around AI needs to be implemented. The elements of this process include, among other things, identifying business opportunity, problem analysis and problem-solving, finding the correct algorithms for the problem at hand, DataOps that identify, collect, validate, prepare, and store datasets for machine learning purposes, developing and testing user-friendly AI applications, integrating the solution in the daily business, improving the models, and monitoring the results.

When the new data foundation is ready, it is time to reap the advantages of the hard work and see how business blooms using a data-driven strategy. But to keep the brand-new platform relevant, there should be monitoring and support, and the platform must adapt to new needs and changing technology. It should also be scalable to receive a large amount of data or to serve a growing user community.    

Reading this would make one believe that setting up and implementing a data-modernization strategy looks like a (long) walk in the park - one has to follow the signs. But of course, things are never that simple!

In the next episode, let us look at the riddles, traps, and dangerous beasts hidden in the data labyrinth.

Coming soon: Episode 3 – Unveiling Traps and Charting Success