Will Small Data be the new Big Data?

January 23, 2013


By Tom Robinson, David Lancefield and Carlo Gagliardi

Data is a vital resource in the new digital economy. But at present, the private sector is a largely closed data environment. A consumer’s personal data (their ‘small data’) is either stored offline, separated into digital silos or managed by businesses, unseen by consumers. This will change as open data principles are adopted. We believe new ways of using ‘small data’ could cause a fundamental shift in the economics of doing business.

There are many reasons to support this view - the network externalities of open data in the private sector are better understood than before [1] [2], the stock and flow of digital data is on an explosive growth path [3],  social pressure for control over the collection and use of personal data is growing [4] and government is actively supporting open data initiatives [5] . We are moving towards a more open data future. But what might it look like, and what are the implications?

We think that it’s likely that individuals will control “vaults” of personal data, stored securely, with flexibility over who has access to what, for how long, and with what privileges. These sources of data will contribute to a personal ‘Digital Identity’, incorporating additional information held by businesses as a result of new technologies and innovations - like e-health devices, in-car telematics and smart meters.

We believe that these ‘Digital Identities’ - combined with new consumer tools for interacting with multiple vendors at once - could transform the economics of doing business, for three main reasons.

First, the business environment will become a four party system. An open data future creates a potential market for fourth party organizations working to help and empower consumers. These fourth parties will assist consumers with managing their data and - with permission - act on their behalf to save them money and improve or provide new services.

Second, trust between business and consumers will become a highly valued asset, as consumers will only share their personal data with organisations they trust. The result will be the emergence of  ‘buying brands’ that cultivate relationships with businesses in adjacent sectors, using their trusted positions to provide complete service and product solutions centred around activities like health, home and leisure.

Third, the ‘Intention Economy’ described by Doc Searls [6] - where buyers rather than sellers are in charge - could become a reality. Consumers backed by personal data and the right digital infrastructure will be free to signal to the market their specific personal requirements and intent to buy. Suppliers can then bid directly to individuals to provide the precise product or service they desire. Consumers have wants and needs beyond the simple choices offered by business; an open data future could allow them to articulate these needs. In economic terms, we’ll see a more efficient market in which there are clearer signals, lower transaction costs and better targeting of products and services.

This open data future could be closer than we think. Personal Data Store services like MyDex, Personal.com and the Locker Project already exist. Services like Intently that facilitate intention signalling and reverse auctions are being actively developed and programming languages like KRL for personal data control and interaction are being adopted.

Pressure originating from government initiatives like ‘midata’ and action to promote consumer privacy and autonomy are likely to be key drivers moving us in this direction. High performing businesses should be able to tap in to new opportunities to take their brand, trust and product to new customers. And it’s an opportunity for new entrants to take on the fourth party role at the centre of this new digital order.

[1] BIS, “ Better Choices: Better Deals”, Dec 2012
[2] World Economic Forum, “Unlocking the value of Personal Data”, Oct 2012
[3] “Data production will be 44 times greater in 2020-“ – SCS, Data rEvolution
[4] Emergence and uptake of technologies like DoNotTrack and browser add-ons like Mozilla’s Collusion demonstrate that internet users are concerned by how much data businesses already hold on them, how it is gathered and what is done with it.
[5] Through the data.gov.uk and midata programmes
[6] Doc Searls, Berkman Center for Internet & Society, Harvard University, ‘The Intention Economy’


Tom Robinson  |  + 44(0)20 721 34302

David Lancefield  |  +44 (0)20 7213 2263

Carlo Gagliardi  |  +44 (0)20 7213 5659