Bill Wilson
Chief Sustainability Officer and Head of Data & Intelligence Solutions, NTT DATA UK
More from this contributor
Five years ago, data marketplaces were hard to find. I set out to discover whether there was a way to monetize smart city data, by offering it on an open platform to anyone who would buy. Now there are many more choices – and data marketplaces built into tools that data analysts and data scientists are already using.
The growth of data marketplaces is the observable outworking of a growing trend for B2B data sharing. Snowflake, one of the vendors in this space, has a ‘data cloud’ graphic (below) that shows the rapid growth of peer-to-peer data sharing, even in the last two years.
Figure: Data cloud growth
These trends have been enabled by a combination of factors including:
- Data pricing models that establish a benchmark in what has been a nascent market until recently.
- A realization by organizations that they only have part of the answer – or that third parties can provide an answer faster than looking for data internally.
- Increasing examples of data being used for secondary purposes – the data collected by one firm can be reused by another for a completely different purpose (and the speed of the pandemic left many of us data professionals scrabbling for secondary data to help decision-makers with crucial judgment calls).
- Changing attitudes towards data sharing – again, the pandemic showed what was possible.
- New data sources which cleverly side-step traditional data collection – the market for earth observation data, for example, will double in the next decade as the number of use cases extends, and frequency and richness improve.
- Technology changes that make data sharing discoverable, real-time, rapidly deployable and with the necessary levels of governance.
Data partnerships are driven by the need for new business growth models and resilience in the face of greater uncertainty. It means firms can no longer be digital islands.
While these are some of the enablers of data sharing, data partnerships are potentially wider and deeper, driven by the need for new business growth models and resilience in the face of greater uncertainty. It means firms can no longer be digital islands. Five examples of data partnerships are discussed below.
1. Transparency and resilience in the value chain
Recent years have shown the importance of resilience in the supply chain. Diversification is only a solution if new suppliers do not share exposure to the same firms two tiers back in the supply chain. Understanding exposure to supply shocks has been driven up the CEO agenda in the wake of a series of ‘black swan’ events, and data partnerships are a means to gather information to make that assessment.
Transparency is crucial. The carbon footprint of suppliers – and how customers use and dispose of products – all have an impact, necessitating data partnerships up and down the value chain.
Transparency is crucial for several reasons, including carbon accounting. The carbon footprint of suppliers and how customers use and dispose of products all have an impact, necessitating data partnerships up and down the value chain. How does that become an engine of growth? Part of the answer is about access to the market – we’re working in the agricultural sector helping farmers respond to new demand signals from supermarkets for low carbon produce. Farmers who don’t make changes will find themselves accessing a smaller and smaller market. As carbon accounting tightens up, firms that have done a good job in their supply chains will flourish.
Another part of the answer is efficiency – tracking downstream materials and driving circularity more effectively can cut production costs by $600 billion per year across Europe.
Forces in the supply chain are therefore driving data partnerships like never before.
2. Offering new – integrated – services to customers
One example of data partnerships for better customer service is found in the automotive industry, where we are seeing a desire for stronger collaboration between vehicle manufacturers (who own connected vehicle data) and dealers (who often own the customer relationship). The richness of vehicle data – if properly used – can enable a much better vehicle servicing and repair experience at the dealer.
Taking the connected vehicle opportunity further, my team is working on providing insights based on time and space associations. We can, for example, associate a vehicle with a driver’s mobile handset based on time and place alone – and then use those insights to offer new services, or better tailor the driving experience.
Recent technology breakthroughs allow personal data to be joined across organizations where neither party can see personal data held by the other. What they can share is insight which does not contain personal data.
Normally this would create data privacy concerns, but recent technology breakthroughs allow personal data to be joined across organizations where neither party can see personal data held by the other. What they can share is insight which does not contain personal data. The technique is called a ‘data clean room’ and is an incredibly powerful tool because of the fine-grained control that is allowed over exactly what is shared – and it’s a great way to overcome underlying fears about data partnerships.
3. Heightened need for personalized digital services in the metaverse
Given that the metaverse digital experience is the product, the quality and richness of the digital experience must be superior to what exists today – driven by better data about avatars. Advertising of those services in the metaverse will look different to Web 2.0, but successful brand placement will be even more tailored to the individual. Some of the data sharing in the metaverse will be between informed virtual consumers sharing selected data with digital services providers. However, aggregated behavioral information will have an even greater information exchange value between commercial entities who form data partnerships.
4. Data-hungry AI models
An explosion of AI can only be supported by an explosion in training data. How is ChatGPT able to give such authoritative (if not always accurate) answers? Part of the answer is the 570GB of open-source text it was trained on.
No matter what the AI use case, there’s a good chance that some external data either provides more ‘ground truth’ or more training data. The insurance industry provides many examples of this effect today – as firms try to gain more specific data about individual risks. Sometimes this is external data such as in catastrophic risk modeling; in other cases, the insurance customer exchanges detailed data for a better premium. Among the insurance customers we work with, these data partnerships are now allowing for automated settlement of claims through sophisticated AI models, driving customer satisfaction and slashing operational costs.
5. Data partnerships in the public sector
Changes in the public sector provide an interesting counterpoint. NTT DATA helps to run the Civil Service Data Challenge in the UK where government employees suggest ideas for more effective use of data across central government. Over half the ideas submitted concern data sharing between departments: somewhere else in government, data exists that allows a service to be more efficient or effective. Senior officials have been evaluating competition entries this year which include data partnerships to tackle modern slavery, child poverty and government grant fraud.
Some of the historic barriers to data partnerships in government resemble those in the commercial sector including inertia, lack of standards, data quality and technology barriers.
Diffuse boundaries
As data partnerships continue to blossom, the data boundaries of organizations will blur to an even greater extent than they do today. The technology enablers are now mainstream, enabling data partnerships to be a key engine of growth in uncertain times.
