Can machine learning and computational text analysis help us make sense of vast numbers of documents, such as development project evaluation files dating back to the 1970s? This is a question that Dr. Ruth Carlitz, assistant professor of Political Science at the University of Amsterdam, seeks to answer in her research on how politics conditions the effectiveness of foreign aid initiatives. To gain a better understanding of the methods that could refine her grant proposals and future research projects, Carlitz turned to the ODISSEI Social Data Science (SoDa) Team for assistance. While the team has helped many researchers with practical computational assistance, their expertise can also be utilised to address a wide range of research questions.
Carlitz is a political scientist with expertise in the political economy of international development. She has a special interest in countries that are reliant on foreign aid. At the moment Carlitz is mostly focused on reproductive and maternal health care issues and the impact of politics in not just the recipient countries, but also the donor countries on this issue. Carlitz started to think about this fifteen years ago when she was living and working in Tanzania: “It was very clear that there was a connection between politics and political talking points that were happening in the United States and the aid that people in Tanzania actually got.”
The challenge of evaluating foreign aid initiatives
Even though most foreign aid initiatives write reports to judge the effectiveness of such programmes, it is not easy to compare these evaluations. Carlitz notes the difference between American and European agencies. “The U.S. as a foreign aid donor does not tend to collaborate with international projects as much and that means the results are evaluated differently than European projects. The European approach is to look at data and evaluate them in a numerical way, whereas the United States Agency for International Development (USAID) outsources the evaluation process to the private sector, which makes those data hard to analyse for researchers.” One of the ways in which computational methods could help Carlitz understand data sources such as development project evaluation files is to use machine learning or computational text analysis on evaluation reports from the past fifty years.
Interdisciplinarity in Computational Methods
A colleague at the University of Amsterdam (Dr. Eelke Heemskerk) recommended Carlitz to get in contact with the SoDa team for a consultation. “The first consultation was great. I’ve read papers that use machine learning techniques and understand what they’re doing on a conceptual level, but was unclear on what is actually involved in implementing these techniques. You have to really understand the data and know what they are in order to run successful machine learning applications.” Since Carlitz is working on a grant proposal that could potentially implement certain machine learning techniques or text analysis on large datasets, the SoDa team’s assistance was very helpful.
The first consultation inspired Carlitz to think about other ways she could utilise computational methods in future projects. “I would like to think about how I might apply similar techniques to trace subnational allocations of ‘climate finance’ initiatives, such as aid that countries in the Global South receive to help them adapt to and mitigate the consequences of climate change.” Carlitz would like to utilise computational methods by using her expertise to innovate on the conceptual side of such projects, rather than the technical side. “It is not my ambition to be an expert on machine learning, but I would really like to understand more so that I can better supervise students who are looking to implement computational methods in their projects. To go: ‘oh for this problem, it is probably best to go with this or that method.’” As more interdisciplinary researchers like Carlitz recognize the value of leveraging computational methods for their projects and seek assistance from experts in the field, social sciences will gain access to more nuanced and data-driven insights, paving the way for impactful research and real-world applications.
Relevant links
- Dr. Ruth Carlitz
- About SoDa
- Article: First Aid for Data Questions: how SoDa can help develop web scrapers and AI models
- Follow SoDa on Twitter
Photo by Wesley Tingey on Unsplash