Finding a Way in the Data Labyrinth – Episode 3 – Unveiling Traps and Charting Success
It is important for any organization to move away from their conservative mindset when they look at data modernization. However, it is not the only thing that stops them in this futuristic journey. There are multiple challenges apart from the advent of new and changing technology.
In the last two episodes of the data labyrinth blogs, we have discussed the data modernization journey. We have shared insights on why data modernization is essential, along with information on how to navigate the data evolution journey through a roadmap.
In the concluding episode, let us talk about the riddles, traps, and dangerous beasts hidden in our data labyrinth.
The first and most important barrier that affects the successful implementation of a data modernization project is a very ordinary one: money. Finding the resources to buy tools and hire the needed knowledge can be cumbersome. Hence, the business should be the initiator of such a program! A strong business sponsor with money and leverage is a very good means to help find the way in the data labyrinth. In order to keep the inflow of cash coming in regularly, it is essential to involve the clients.
They should not wait for years before the results of their hard work start showing up. Instead, they should be involved right from the beginning. Their feedback must be taken, and they must be made aware of the next steps. One thing that cannot be repeated enough is that businesses should drive the change toward a data-powered organization. If this is not the case, just stop!
The second difficulty on the path toward a data-driven organization is changing the organization’s culture. Most humans are resistant to change. People like to keep doing the same things when everything is going well and perceive adapting to a new way of working as a threat. They are afraid of not coping with the new technology, of losing their job, or of becoming irrelevant. So, include change management in the short- and long-term planning.
It is obvious that the input needed to get a perfect output must be correct, relevant, and qualified. In other words, it is essential to manage data quality, data availability, and metadata. Not only must the data be in pristine health, but the company's organization must also be adapted to get the most out of the data assets: placing the right profiles in the right place, clearing tasks and responsibilities, and fostering a seamless collaboration between IT and business. Without a good data governance process, most of the work on the data platform is in vain, something that is not always recognized at the management level.
To make things even more complicated, there are a lot of different tools available on the market. Choosing the right one to set up an efficient data architecture can be nerve-wracking. What kind of extraction or transformation tool is needed? To cloud or not to cloud? Lakehouse, Data Lake, Data Sea, Data Pond, or Data Swamp? Spark, Flink, or Kafka? R or Python? It is not easy to find the right combination. One must seek the help of a neutral third party (ok, consultants).
To summarize, there is a way through the Data Labyrinth, but unfortunately, it is riddled with traps: lack of a good sponsor, not enough resources, no long-term vision, opposition from the users, lack of governance, the spaghetti of tools... To be relevant, organizations must take the necessary steps to become data-driven, but at the same time, think before beginning the data journey! Take a bit of time to assess the current situation and write out a strategy and roadmap. One must realize that a big project like this is a company-wide efforts and that IT and business must work together to achieve a successful transformation. Also, don’t be afraid to ask for advice and help: different types of specialized profiles are required during the different phases of data modernization, and for cost reasons, it is better just to hire them for the period they are needed. A neutral third party can also provide new ideas and a clear look at the best way to reach the nirvana of the data-powered organization.
Maybe they can act as your beautiful Crete princess.
![[Missing text '/pageicons/altmail' for 'English']](/Static/img/email.png)
With more than 25 years of experience in data projects, Jan, from a close proximity, has seen the evolution of primitive data warehouses to the huge quantities of data processing behemoths that allows the creation of semi-intelligent machines that can disrupt existing business processes.
In a very modest way, he contributed to this great change by working on plenty of data projects in different roles and all over the world. Jan wants to use his experiences to help organizations in their quest to become data-driven and, at last, use the huge potential of their data to their benefit.
As a Dutch speaking Belgian who works in Luxembourg and lives in France, Jan has also mastered the art of combining the impossible, which is a great help in his daily work!