Big data, big deal?

28 September 2017

Benefits and challenges associated with Big Data for results measurements

In our last blog, we explored the use of big data for monitoring and evaluation (M&E) purposes in international development. We concluded that big data can provide us with great insights into aid effectiveness; it can show whether development projects have been successful and facilitate a more adaptive process toward designing programmes that deliver better outcomes for disadvantaged people. However, current projects are still in the pilot phase and much still remains to mainstream the use of these techniques. With this blog we would like to build on our first blog by exploring the potential benefits and practical challenges associated with applying big data.

Benefits of big data:

Picture1 Availability of data The data is readily available and therefore no new data collection processes need to be put in place. As a result less costs need to be made for collecting data using household surveys. For example, existing satellite imagery can be used to determine the effects of reforestation projects. Satellites can show whether the size of the land used for forestation is increasing or decreasing and thus whether a reforestation programme is successful.
Picture2 Timeliness of data Big data is also considered to be close to real-time and can be tracked on a continuous basis. For example, GPS information from mobile phones can be used to track school attendance and time spent at school. Projects that are aimed at increasing school attendance and learning could benefit from this information in determining whether interventions lead to the planned results.
Picture3 Volume of data It usually comes in high volumes. Digital tools including data mining and machine learning can be used to analyse these larger volumes of data and identify patterns. For example, these tools can be used to analyse transactional data on mobile phone payments. This analysis can provide information on the income levels of people and the extent to which programmes focused on poverty alleviation are successful or approaches need to change.
Picture4 Unstructured data M&E tends to rely strongly on large amounts of structured data gathered for instance through surveys. Big data analysis tools allow practitioners to tap into large amounts of unstructured data including conversations, photos and videos. This can provide us valuable information such as changing opinions on certain topics or changes in people’s environments.

However, there are challenges of big data:

Picture5 There are ethical considerations to take into account when using big data. Is it ethical to collect information on movements of children to assess whether they are attending school? Respecting individual rights and liberties, such as the right to poverty is key in using big data. This is easier said than done and needs to be assessed on a case-by-case basis.
Picture6 Analysis of big data for M&E purposes comes with methodological challenges. More often than not a clear counterfactual often doesn’t exist. The data frequently is naturally biased because not everyone has access to technology. Information is only collected of those people who have access to mobile phones or the internet, which can result in misrepresentation of the actual situation.
Picture7 Especially in the international development sector, quality of data is a challenge. There is limited influence on how the data is gathered, and the organisations responsible for doing so might change their methodology without you even knowing. Limited data availability can be caused by the fact that data is not shared publicly or data is not being gathered at all, such is the case with people that are unconnected to internet.
Picture8 Finally, specific expertise is needed for analysing large volumes of big data with digital tools. M&E experts typically do not have this experience and therefore we need to work closely with data analysts. This cooperation doesn’t come automatically and both evaluators and data analysts need to invest in building a relationship.

Big data can provide added value, if used correctly. In projects where there is a large amount of unstructured data, or where there are few privacy concerns big data can offer huge insights. This was also one of the conclusions in the annual UKES evaluation: “Data linkage, the availability of big data, and the options remote sensing provides are increasing the number of questions we can answer. Data visualisation opens up doors to better understanding and communication of data”[1]. However, big data probably has less to offer to projects that are targeted towards communities with no access to internet. Whether the opportunities outweigh the challenges will therefore need to be assessed on a case by case basis taking these contextual factors into account.

Picture10

How we apply big data in M&E will also be different in each sector. Satellite imagery, for instance, is more applicable in agricultural projects than in gender-related projects. For gender-related projects social media monitoring can be more useful. For example, it can be used to determine the effects of women’s economic empowerment campaigns. It is possible to analyse the extent to which people within a geographic location are positively or negatively associated with a certain topic on social media. By analysing changes in these associations, it becomes possible to draw conclusions on the success of the campaign.

If we look at scale of benefits and challenges of this specific case, we see the benefits outweighing the challenges. The social media data is readily available and close to real time. Specific tools are available that are able to analyse these large amounts of unstructured data. As a result, the specific expertise needed for analysis is limited and so are the ethical challenges; people choose to share information on social media. Methodological challenges are still present as not all people use social media and therefore results might be skewed. These challenges should therefore be taken into account when interpreting the results.

Ultimately big data has the potential to show whether development projects have been successful and facilitate a more adaptive process toward designing programmes that deliver better outcomes for poor people. As sectors provide an interesting lens to look at big data, we’ll explore this more detail in our next blog.

This blog is the second of a “digital in development” collaboration series between PwC Kenya, PwC Netherlands and PwC UK.

[1] https://www.evaluation.org.uk/index.php/events-courses/ukes-conference/232-ukes-2017-annual-evaluation-conference-programme-2

 

Maaike Platenburg

Maaike Platenburg | Manager
Profile | Email | +44 (0)7841 803162

 

More articles by Maaike Platenburg

Comments

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated and will not appear until the author has approved them.