Big Data? Smart Data!
In the age of data, the companies that succeed are those that manage to analyze data intelligently. In her guest article, Nicole Schnetzer shows how reducing and cleaning data, as well as asking precise questions, can contribute to business success.
Business processes for companies have changed fundamentally due to globalization and digitalization. The barriers to entry for new market participants have fallen. Digital platforms, which benefit many millions of people as a network, are pushing analog providers out of the market or disrupting entire industries. Today, companies need to adapt and develop in an agile manner in order to survive the competition. Data is the raw material for driving agility and the organizational learning cycles of companies.
Imagine the earth as a huge field of snow: Children, men and women leave footprints with their winter boots. Animal tracks appear. In a very short space of time, a network of many and widely ramified tracks is created. At some point, the smooth, white surface has become a densely interwoven tangle of tracks. It is no longer possible to tell which track leads where and which child belongs to which parent.
In order to identify traces in this apparent chaos and to be able to assign relationships, the variety of traces must be narrowed down with precise questions and many of the impressions removed. This is the only way to recognize any patterns and find answers to questions. The supercomputer in The Hitchhiker's Guide to the Galaxy took several million years to give the answer "42" to the question "about life, the universe and all the rest". Nobody could do anything with the answer "42". The question was asked in a vague way.
Banished from the data warehouse to the data swamp
Many companies do the exact opposite. They collect data indiscriminately and fill data warehouses and data lakes. Much of the data is generated in real time. It comes from sensors, devices, video/audio, networks, log files, transactional applications, the web and social media. Much of the data ends up in a data swamp. This is data that can no longer be used for meaningful business analysis with a reasonable amount of resources.
The maze of traces has not only become too big, it is also outdated. Data is particularly valuable when it is processed in real time. Outdated data is of little use to companies at a time when highly dynamic developments are being driven forward. Their resources are better used if they concentrate on collecting relevant data.
With a view to the snowfield: companies should only record those tracks that specifically fit the company's needs and are relevant to their industry.
What is big data?
In today's terminology, the vast amounts of data that are collected are big data. It is characterized by a large volume, a great variety (unstructured data), high speed (data processing frequency) and a high degree of veracity (trustworthiness/representational power of the data).
This results in a certain value of the data: Not all data collected is valuable. Good data quality, accuracy and completeness of the data - these are all important factors for subsequent, correct analyses. Big data alone does not provide any insights. Smart data is needed to derive meaningful insights from the data.
Reduction and structuring of data
A first step in making big data usable is to reduce the volume. Only useful information for solving problems is retained. If necessary, the variety of data must also be limited using a screening process. The value and veracity of the data increases with this initial cleansing (see infographic).
The existing unstructured data is now structured and further cleansed. Processes such as text and speech recognition (natural language processing) are used for this. The data is now available to the company as smart data. A downstream analysis platform processes the data into knowledge that is immediately understood. Company decision-makers are able to derive insights from this data in order to tap into potential.
One example:
Customer experience analytics (customer insights): In this case, big data is generated from all transactions and points of contact between customers and the company. Smart data is data that makes it possible to analyze which customers are in which phase of the customer life cycle or how satisfied they are. Findings from this enable the company to tap into potential.
Sifting through challenges, using smart data
In order to benefit from the collected data, companies are required to formulate questions relating to current and future challenges. Along these lines, data should be collected, organized and made available to decision-makers as smart data. The time pressure to offer customers beneficial solutions and to act proactively in the market continues to increase. To achieve this, data insights must be continuously and automatically integrated into day-to-day business and management processes. This is the only way for companies to be able to optimize them or initiate new developments.
One example:
Employee Experience Analytics (Employee Insights): Only data (smart data) that is highly relevant in terms of employee satisfaction, employee engagement and motivation is collected from employee touchpoints. With the corresponding analyses, a company is able to respond proactively to trends.
Benefits of smart data
- Reducing uncertainties in the decision-making process
- Data-based management and decision-making increases acceptance of decisions (full transparency)
- Predictive scenarios help to recognize what is coming next
- Reduced costs due to increased efficiency in processing and analyzing data
- Acceleration of organizational learning cycles due to faster decisions
* Nicole Schnetzer is Head of Marketing at Echonovum.
