Digital transformation: the growing data problem
As digital transformation has gathered pace across all sectors over the past couple of years, fundamentally changing the way we work with its introduction of new technologies, 2020 will see one of its early benefits become a growing challenge for businesses: data. Collecting data One of the first upsides of digitizing working processes and the way we store information was access to customer data. Initially, it was simply a way of improving customer service: having access to transactional data from a CRM system, for example, giving a real-time snapshot of a client relationship. But in a very short period of time, nearly everything has gone digital, meaning data can today be harvested from almost anywhere, such as social media, the web, voice recordings and IoT devices, and in areas as diverse as demographics, behaviour patterns and interactions. Known as ‘big data’, this unstructured data is very different to traditional transactional data, coming in multiple raw forms, and as such requires specialist data analytics skills to integrate and interpret. The return from this investment in data analytics talent for organisations is of course in-depth insight to give a competitive edge. But big data is coming at an unprecedented rate today, its volume estimated to be doubling every six months, with new sources frequently added into the mix. This results in storage challenges, as well as integration issues. On its own, its usefulness is limited, but combined with the corporate process data, it can be a powerful weapon, meaning CIO’s will need to bring data scientists more into the day-to-day, rather than retain them as separate siloed entities, if they want meaningful results. Storing data Traditionally, data storage is cheap, which has meant it’s been easy for businesses to adapt to harnessing big data from multiple sources. Even though storage requirements are increasing exponentially, the advent of data repositories has given analytics a home, as well as a platform to integrate the unstructured digital content with the regular data gathered and present the fuller customer picture. However, regulatory changes, such as the recent introduction of GDPR, have brought data management and governance to the fore. Vast quantities of stored data present both a cyber and compliance risk if they are stored for too long, which is becoming a problem as the incoming volume of data starts to outpace the manual capabilities of the data analysts. Integrating data Organisations at the forefront of data management are increasingly incorporating AI, in particular machine learning, into their data analytics strategy, to keep up with the massive quantities of incoming data and to provide better business decisions. Machine learning is able to adapt to new forms of data, discovering new patterns or rules in it, often without human intervention. Its sophistication is already at a level where it can be used for data derived from NLP and image recognition, for example, negating some of the problems in the multiple forms and sources of unstructured data. While human data science skills are still crucial, automation is being introduced into the data architecture to speed up the extraction and integrating of the unstructured data, to ensure it is not stored in a data lake for longer than is necessary. Not only is the process more efficient from an internal perspective, the single view of a customer and all related data is achievable faster and mitigates any data governance concerns. Exactly what the data landscape will look like by the end of 2020 is still unknown, but those businesses addressing the changing nature of its collection and analysis now through the implementation of AI, will be leading the way, come what may. If you would like any help or advice on your transformation project or specialist talent, please don't hesitate to contact us today.
