Photo: kras99/Adobestock
Innovative technologies of the Fourth Industrial Revolution (4IR) are transforming and modernizing the way data is generated, collected, and analyzed across different industries and fields of study.1 Satellite imagery, blockchain, and mobile applications are some of the new methods that are being used to collect more timely data and statistics. These technologies are also increasing the speed, volume, granularity, and sources of generated data, also known as “big data.”2 New, high-frequency data sets are being produced through digital platforms, often by the private sector or by individuals, that are directly related to real-world problems. For example, data provided by online restaurant reservations and hourly credit card spending has allowed policymakers to gauge the course of the Covid-19 pandemic. The Economist argues that the fast production and abundance of new data is transforming the economics and public policy fields into an emerging “third-wave economics.”
Stakeholders use this wealth of collected information for different purposes. Data is most often collected by private actors for commercial gain. In the public policy field, data can enable analysts to better identify certain societal problems, help policymakers make educated decisions, and assist evaluators in monitoring programs’ results and effectiveness. For regular citizens and civil society organizations, the availability of timely data can be one tool to push governments, companies, and other institutions to increase transparency and accountability.
In the field of international development, these innovative data collection technologies are also being applied to monitor and address complex global development challenges such as financial crises, climate change, and pandemics. For example, tracking and taking action on Covid-19 requires timely data on the spread of the virus and new variants, testing volumes, hospitalization and death rates, and vaccination progress. More widely, innovative data collection technologies can be employed to track the United Nations’ Sustainable Development Goals (SDGs).
While there is a global drive to make decisions based on robust data, there are certain risks and barriers surrounding its collection and use. Developing countries’ governments grapple with competing priorities and limited budgets to collect timely data. Initial calculations from the Global Partnership for Sustainable Development Data estimate that collecting the indicators needed to track the SDGs will cost approximately $17.7 billion in the poorest countries and $27.6 billion in middle-income countries. In addition, these innovative data collection methods rely on modern technologies, both of which require digital connectivity and associated skills in “small data” and “big data” analytics—critical building blocks that are often missing in many developing countries.
Complicating matters, donors who fund development projects tend to design and procure data collection and analytic systems in a very bespoke way. Each aid organization has its own data framework and formatting design. As a result, their reporting obligations are not standardized and harmonized, making it hard for recipients to respond to multiple requirements. This approach results in numerous custom-built data collection systems that often miss the mark on functionality and delivery timelines. Such procurement strategies also fail to take advantage of readily available “commercial-off-the-shelf” products that could serve the international development community.
At the same time, applying innovative technologies to data collection raises security and privacy risks for individuals, companies, and governments alike. There is a naive assumption in the development community that the data collected is being utilized to address noble causes such as tackling poverty, strengthening democracies, or responding to health challenges. Despite an underlying assumption of a “shared ethic” in data collection and use, both domestically and internationally, in reality there are very disparate beliefs on how to apply such technologies. In fact, these innovations are subject to misuse and abuse: bad actors exploit data to steal people’s identities, commit financial fraud, perpetrate false narratives in development, spread disinformation and misinformation, and facilitate digital authoritarianism.
Within this evolving context, the U.S. government has a role in shaping the innovative data collection architecture. Working with the wider donor community, federal agencies such as the U.S. Agency for International Development (USAID), the Department of State, and the Department of Defense have a responsibility to ensure ethical and responsible data collection through these new technologies. USAID, through its 2020–2024 Digital Strategy has already begun to outline what role data can play in development and how to maintain and support digital ecosystems globally.
There is enormous potential for use of innovative technologies to improve the quality of data collected and to make international development more effective. Back in 2013, a UN High-Level Panel on the Post-2015 Development Agenda called for a “data revolution” in international development to improve the quality of statistics and information by leveraging new technologies.3 In fact, some experts argue that “we are in the midst of a new model of international development with data at the center of everything.”
Technological developments in data collection and analysis include the use of geographic information systems (GIS) and imagery analysis, crowdsourcing, and mobile technology (see Box 1). These new methods are applied on their own or in combination with traditional data collection systems such as household surveys, censuses, and administrative data. They offer many potential applications in international development, both to address sectoral challenges (e.g., health, financial inclusion, and the environment) and cross-cutting issues such as increased transparency, capacity building, and monitoring and evaluation of projects and programs (see Appendix for sectoral examples).
There are four groups of stakeholders collecting and using development data: (1) governments and international organizations, which use data to support their policymaking decisions and improve service delivery; (2) non-governmental and other civil society organizations, which use data to monitor the effects of government policies; (3) individuals, who use data to monitor and access public and commercial services; and (4) companies, which use data in the production process and to understand customer demand. These stakeholders work across different sectors, tracking challenges and progress in areas such as financial inclusion, the environment, agriculture and land use, humanitarian interventions, and democratic elections.
Policymakers and development actors can better target policy challenges with more accurate and specific data. Collecting data helps analysts and decisionmakers identify sources of problems in many sectors, such as pinpointing communities that are suffering from food insecurity or from flooding or droughts. Data also helps better understand the barriers to access for services such as education, banking, healthcare, and clean water. In addition, by tracking certain indicators and evaluating programs, data informs current initiatives and future programming; it can ensure that development programs are implemented as efficiently and effectively as possible to achieve the desired outcomes.
In the context of the Covid-19 pandemic, mechanisms such as GIS, imagery analysis, and biometric data present unique opportunities for contact tracing, vaccine distribution, and community outreach. GIS and imagery analysis enable researchers to understand the relationship between specific health trends linked to certain locations. While GIS may not be suited to address the non-geographic factors at play (such as vaccine hesitancy and Covid-19 misinformation), some of the most significant barriers to global vaccine access are systemic and geographic in nature. For example, GIS can help pinpoint populations at increased risk from Covid-19, identify infrastructure available for vaccine storage and distribution, and unpack other gaps in healthcare access. GIS unlocks detailed information that can help policymakers identify communities that may be far from or unable to reach health centers—thus informing more effective and targeted development assistance, such as to construct new vaccination sites or use medical drones to bring vaccines to inaccessible communities. In sum, GIS and imagery analysis can be directly applied to support overall vaccine distribution, not only for Covid-19 but also for other neglected diseases such as polio, measles, and tuberculosis.
Although emerging technologies are making it easier to collect real-time data on specific development problems, countries face various barriers to their adoption. The rapid spread of technology and digitalization also creates risks if these tools end up in the wrong hands. On the collection side, many countries do not have adequate infrastructure and human resources to collect even the most basic national statistics—and subnational governments may fare even worse. Moreover, there are concerns surrounding the quality of the data in terms of its accuracy, fragmentation, and trust. On the data-use side, bad actors are collecting and using data for nefarious purposes, such as suppressing free speech, persecuting minorities, stealing individuals’ identities, and committing financial fraud.
Looking ahead, policymakers and donors need to recognize these challenges when applying these technologies and ensure that the data architecture protects users, standardizes metrics, and strengthens government capacity to effectively use and sustain these frameworks.
Innovative data collection methods hinge upon some basic building blocks; they require digital infrastructure, a comprehensive legal framework to address data privacy concerns, and interoperability of systems. However, in many countries where data collection has the greatest potential to improve livelihoods and remedy pressing development challenges, governments lack the resources—such as reliable electricity or internet connectivity—to either access or utilize innovative technology. Particularly tight budgets and shortages of experienced and skilled data analysts may prevent policymakers from fully capitalizing on innovations in data collection. This is further exacerbated by the fact that collecting data can be a lengthy process and therefore requires a sustained financial commitment rather than just a one-time investment. This funding gap allows wealthier nations to reap more of the benefits of the data revolution than poorer countries.
At the same time, employing these technologies also requires human capital. Necessary training and expertise include digital skills to operate the technology as well as analytical capacities to unpack, organize, and understand the data being collected. People in developing countries often lack education and skills related to innovative technology and collection methods. While there have been efforts to address these weaknesses through training initiatives, digital literacy remains a significant barrier in many contexts.
In addition, some of these new methodologies used to collect and analyze data are not standardized across countries, companies, and institutions. In the development field, programs and projects are often over-individualized and produce customized data that cannot be replicated or referenced for future use. While some projects and contexts are unique and require flexible indicators, having common markers across data sets would allow better use within and across development sectors.
The quality of the data collected also deserves some scrutiny. Some of the new methods to collect data do not always produce information that is valid and accurate. During the data validation process, data is reviewed prior to analysis, thereby improving the integrity of data sets and preventing misinterpretation or misapplication. Standardizing data can further ensure the quality and reliability of the information collected.
While innovative technologies provide ample opportunities to improve data collection and policy decisions, they can also be misused. The data generated by such technologies carries certain risks if it ends up in the wrong hands. Malicious actors are employing technology to collect data they can use to infringe on individual liberties and subvert populations. For example, OpenStreetMap includes information on the location of non-governmental organizations (NGOs) in Afghanistan; now that the Taliban controls the country, this may pose a risk to civil society members.
Governments and politicians in both authoritarian and democratic systems have turned to the internet and automated surveillance systems to further consolidate power and control people, a phenomenon known as digital authoritarianism. With innovations in technology comes a rise in censorship and other forms of disruption. In Chad, for example, the internet was shut down for 472 days between 2018 and 2019 for “security reasons,” a move that enabled President Idriss Deby to extend his reign. And in 2020, there were 155 internet shutdowns in 20 countries, mainly in the Middle East and Africa, to prevent further dissemination of dissent.
Governments and politicians have also utilized digital technologies to spread disinformation to undermine trust in institutions so they can retain power. In misinformation and disinformation campaigns—both in traditional media and on the internet and social platforms—data is misrepresented and altered to promote a specific message. Misinformation and disinformation have markedly impacted elections and democratic institutions, public health and safety (such as during the Covid-19 pandemic), governance, and other development issues. A recent report by CollaborateUp illustrates some of these fault lines. In sub-Saharan Africa, disinformation about Covid-19 vaccines has not only resulted in people refusing to receive the vaccines but also refusing to take proven preventive measures against the disease. In Latin America and the Caribbean, Chinese state media has leveraged social networks and advertising to increase the use of Chinese-produced vaccines, mainly Sinovac and Sinopharm, by incorrectly reporting and broadcasting that they are more effective than they actually are. In Africa, Western media has long mischaracterized the region as a continent of despair, with negative stories perpetrating an overinflated narrative of high risk. The unseen benefit in all of this is that while improvements in digital technologies have increased avenues for misinformation and disinformation—such as deepfakes and disinformation articles auto-generated by bots—these same channels have the ability to detect false information, albeit with fewer financial resources and capabilities.
Bad actors can also use the information collected through technical channels to conduct fraudulent activities, especially in the financial system. Data stored in financial institutions bears risks for individuals (e.g., customers’ accounts may be hacked or spoofed) and to the integrity of the country’s banking system (such as through money laundering and terrorism financing). For example, the Cambodian Financial Intelligence Unit was unable to supervise the country’s casino and real-estate sectors; this allowed organized crime groups to launder money, prompting the Group of Seven’s Financial Action Task Force to put Cambodia on its watchlist in 2019.
These cases illustrate that without proper frameworks and controls, bad actors can use technologies and data to exploit and abuse individuals and companies—and even interfere in the sovereignty of a nation. Given the rapid pace of technological innovation and data generation, mitigation efforts need to preempt further misuse and abuse.
As the world strives to meet the SDGs by 2030, many stakeholders are working to improve the quality of data for development. Multilateral efforts led by the United Nations include the UN Statistical Commission, UNdata, the Global Partnership for Sustainable Development Data, the SDGs Data Lab, the Global Working Group on Big Data for Official Statistics, and the World Data Forum.
U.S. government agencies have their own initiatives that are incorporating modern data collection mechanisms into their development programming. For example, USAID’s GeoCenter team applies geospatial technology, data analytics, and visualization techniques to improve the planning, design, and monitoring and evaluation of programs. Another related initiative at USAID is the Digital Ecosystem Framework, a practical guide to understand a country’s digital architecture and identify areas of action or intervention. It hinges upon three pillars (digital infrastructure, digital economy, and digital society, rights, and governance) and four cross-cutting issues (inclusion, cybersecurity, emerging technologies, and geopolitical positioning). In addition, the Data Collaboratives for Local Impact program—a joint initiative by the President’s Emergency Plan for AIDS Relief and the Millennium Challenge Corporation—“seeks to improve the capacity of individuals, communities and organizations to use data to solve problems relating to HIV/AIDS, global health, gender equality, and economic growth.”
International organizations working with governments, U.S. agencies, and other donors can leverage these new technologies for data collection. Some areas for action include investing in the wider digital ecosystem, focusing on tools to prevent bad actors from misusing data, developing common standards in data collection, and combining the use of data sets and technologies to increase development impact.
Promoting inclusive access to these innovative data collection technologies is paramount. Yet the capacity to access and utilize them relies on investment in the wider “digital ecosystem”—that is, the underlying infrastructure and systems as well as an enabling environment that allows a society to use digital technology. Actions to strengthen the digital ecosystem include investing in digital connectivity (e.g., electricity access and broadband and mobile networks), strengthening digital governance, improving digital and data literacy, and supporting long-term and continued data collection efforts.
There is a greater need to increase data literacy within government bureaucracies. Some low- and middle-income countries have burgeoning digital ecosystems where emerging data analysts can directly contribute to in-country efforts to employ data for development. This will require hiring and training the right people—local data analysts who have experience with query and computing languages—to help extract, combine, and analyze multiple data sets.
At the same time, establishing “data insights teams” in organizations such as USAID and within developing-country governments can increase their capacity to address multiple development challenges holistically and effectively. That is, instead of treating each project and development goal separately, development experts and data analysts should be used in a centralized manner to facilitate data analysis and subsequent policy decisions. Development organizations can set up a central data insights team to process, analyze, and maintain data sets spanning multiple projects and initiatives in a single place. The Bill & Melinda Gates Foundation, for example, recently instituted this approach within its Global Health program. Furthermore, to improve outcomes, data and dashboarding should be integrated into the life cycle of development programs, not just at the beginning and end of the program.
Improving data governance and the data privacy and rights of individuals is also vital. Data collection comes with risks linked to people’s privacy and how this data is being used. Efforts to keep sensitive data private are particularly crucial.
The increasingly online nature of global activity produces substantial amounts of digital trails that trace personal, social, economic, and political transactions and decisions. Malign actors and governments can easily use this information to undermine policies, development objectives, and confidence in institutions. Overabundant data may also endanger vulnerable populations, which may not consider issues of privacy and consent.
Policymakers should be cognizant and vigilant of how digital trails can be used and misused and how governments and malign actors can wield data to control, oppress, or abuse citizens. Several actions can be taken to improve data governance, privacy, and rights:
Common standards and terminology are necessary so that data can be used more widely and remain relevant (i.e., to facilitate data interoperability). In that regard, when using these new technologies, there should be a common language of data metrics to prevent constant translation between agencies, industries, and institutional data models.
For example, in the monitoring and evaluation (M&E) field, standardizing indicators on practitioners’ surveys and geolocating services would allow for increased coordination in development spheres. Efforts to reduce competition and increase standardization and collaboration include the International Aid Transparency Initiative, which allows donors to share M&E data, and USAID’s Development Information Solution, which aims to standardize M&E data inputs. While more such initiatives are emerging, they should continue to be promoted and developed across the U.S. government, the private sector, multilateral organizations, and allied governments.
Finally, different technologies and data sets should be used together to maximize development impact. Using these tools in combination can provide better insights into development issues than one technology or data set can on its own. While certain technologies—such as GIS and AI—are impressive on their own, they are more effective when used together. Technologies that combine geographic information with knowledge management systems can help produce forecasts, inform choices, and support strategic planning. For example, GIS data combined with AI contribute to disaster mitigation efforts by predicting areas more susceptible to landslides by overlaying topographical maps with rainfall data. Meta tagging, which assigns key descriptors to help sort and locate data, is also useful.
Private-public partnerships can enable the combination of different technologies and elicit broader collaboration across the collection, analysis, and effective use of data. Working with in-country partners from various levels of government is also critical to coordinating data gathering among local and national development efforts.
Innovative data collection technologies offer countless opportunities to analyze and address existing development challenges and emerging crises, including the Covid-19 pandemic. Yet, as the world becomes increasingly digital, how data is collected and used will be critical. While technology such as AI, ML, and GIS offer promising applications for data collection, it is also important to acknowledge that the new universe of data tools is not perfect and can be subject to misuse and abuse, from identity theft and financial fraud to facilitating digital authoritarianism. Applying innovative technologies to data collection will continue to raise security and privacy risks for individuals, companies, and governments alike. That is why it is critical for the United States to continue investing in a wider digital development ecosystem abroad. Working with the donor community and the private sector, the U.S. government has a role in shaping the global digital architecture to ensure ethical and responsible data collection and use.
Daniel F. Runde is senior vice president, director of the Project on Prosperity and Development, and holds the William A. Schreyer Chair in Global Analysis at the Center for Strategic and International Studies (CSIS) in Washington, D.C. Romina Bandura is a senior fellow with the Project on Prosperity and Development and the Project on U.S. Leadership in Development at CSIS. Elena Méndez Leal is a program coordinator with the Project on Prosperity and Development at CSIS.
The authors would like to thank Noa Levin and Veronica Rice for their invaluable research efforts critical to producing this brief.
This report is made possible with the generous support of Premise.
This report is produced by the Center for Strategic and International Studies (CSIS), a private, tax-exempt institution focusing on international public policy issues. Its research is nonpartisan and nonproprietary. CSIS does not take specific policy positions. Accordingly, all views, positions, and conclusions expressed in this publication should be understood to be solely those of the author(s).
© 2022 by the Center for Strategic and International Studies. All rights reserved.
Please consult the PDF for references.
The Monitoring of the Andean Amazon Project (MAAP), led by the NGO Amazon Conservation Association, uses high-tech tools for monitoring deforestation in real time. Launched in 2015, MAAP draws on satellite technology from five sources—Landsat, Planet Labs, DigitalGlobe, Sentinel Hub, and PeruSAT-1—to deliver information on deforestation and its drivers. MAAP currently covers 83 percent of the Amazon Basin across Peru, Brazil, Colombia, Ecuador, and Bolivia, combining remote-sensing technologies (e.g., drones, radar, and analytical algorithms) to track deforestation, identify illegal acts, and help halt such activity. In 2020, MAAP used data from its real-time Amazon Fires Monitoring app to document over 2,500 major fires across the Amazon Basin (see map below).
Major fires in 2020 (orange dots) within the Amazon Basin (blue line).
Source: “MAAP #129: Amazon Fires 2020 – Recap of Another Intense Fire Year,” MAAP, November 30, 2020, https://maaproject.org/2020/amazon-fires-recap/. Reprinted with permission.
Innovative technologies are helping improve agricultural productivity and land-use policies and programs by maintaining up-to-date, informative data. For example, GIS can generate images of land and crop use. Together, imagery analysis and AI can then fill gaps in project information and generate cropland maps, validate models through predictive maps, and create descriptive data and statistics that can be presented as analysis. One example is the Visualizing Insights on Fertilizer for African Agriculture (VIFAA) program in Nigeria, led by Development Gateway. The VIFAA Nigeria Dashboard monitors fertilizer price, fertilizer use, and product availability and maintains a plant directory to inform farmers and local governments so they can improve and better facilitate land and crop use in Nigeria.
Source: “Nigeria Fertilizer Dashboard,” AfricaFertilizer.org, https://vifaanigeria.org/#/nigeria/home. Reprinted with permission.
New technologies are also used to collect information that can help aid groups provide better humanitarian assistance, such as delivering potable water and meeting related sanitation needs. The international non-profit information management service iMAAP teamed up with Premise, USAID, and the Department of State’s Bureau for Population, Refugees, and Migration to map and track migrant and refugee settlements in Colombia. In addition to mapping, crowdsourcing can identify needs in the community and fill in information gaps. Members of migrant communities can upload information to the platform, which informs USAID and other humanitarian organizations on how to assist these groups. Each submission requires answers to specific questions, an exact location, and photographs of the area.
Source: “Detection and Profiling of New Settlements Through Satellite Imagery and Mobile Apps,” iMMAP, https://immap.org/news/detection-and-profiling-of-new-settlements-through-satellite-imagery/. Reprinted with permission.
Source: Premise International Development Team, “How New Premise Software Updates Are Saving Lives In Colombia,” Premise, https://www.premise.com/how-new-premise-software-updates-are-saving-lives-in-colombia/. Reprinted with permission.
Innovative technologies are also applied to tracking election results through citizen observation. Together, the National Democratic Institute’s (NDI) DemTech Initiative, TimbaObjects, and DemCloud created the open-source data management system Apollo, which collects information from observers (via SMS, smartphone app, or the internet), verifies the submitted information, and aggregates the data to analyze election results. The dashboard has been implemented in countries such as Malawi, Nigeria, Liberia, Uganda, Morocco, Tunisia, and Azerbaijan.
In Côte D’Ivoire, Apollo was used to monitor the 2015 presidential election. Observers used cell phones, SMS messages, and standardized checklists to provide real-time data on key aspects of voting and ballot counting, including the opening and closing times for polling stations, presence of appropriate electoral materials, successful deployment of new biometric authentication technology, and adherence to the electoral code. This observation also enabled an independent estimate of the election results.
On election day, observers were sent to 755 polling stations across the country, covering each of the 14 districts, 31 regions, and 107 departments of Côte d’Ivoire. Together, 5,285 observer reports in the form of 38,000 coded text messages were aggregated in Apollo, generating 396,375 data points for rapid review and processing by trained statisticians. According to observers using the Apollo system, the election was free and fair, with a few minor instances of intimidation or violence at polling stations.
Sample SMS response rate to questions about election-day conditions
Source: “Apollo,” NDI/DemTech, last updated December 9, 2021, https://dem.tools/guides-and-tools/apollo. Reprinted with permission.
1 This report builds on an earlier CSIS study. See Erol Yayboke, Erin Nealer, and Charles Rice, Harnessing the Data Revolution to Achieve the Sustainable Development Goals: Enabling Frogs to Leap (Washington, DC: Center for Strategic and International Studies, September 2017), https://csis-website-prod.s3.amazonaws.com/s3fs-public/publication/170901_Yayboke_HarnessingDatatSDGs_Web.pdf.
2 The World Bank 2021 Development report classifies data collection according to intent (public use or private use) and method (traditional means such as censuses, surveys, and labor force studies versus new means that use, for example, digital platforms, biometrics, and geospatial intelligence). See World Bank Group, World Development Report 2021: Data for Better Lives (Washington, DC: World Bank, 2021), 28, https://www.worldbank.org/en/publication/wdr2021.
3 According to the panel, “Better data and statistics will help governments track progress and make sure their decisions are evidence-based; they can also strengthen accountability. This is not just about governments. International agencies, CSOs and the private sector should be involved. A true data revolution would draw on existing and new sources of data to fully integrate statistics into decision making, promote open access to, and use of, data and ensure increased support for statistical systems.” See UN High Level Panel on the Post-2015 Development Agenda, A New Global Partnership: Eradicate Poverty and Transform Economies through Sustainable Development (New York, NY: United Nations, 2013), 24, https://www.post2020hlp.org/the-report/.
