Feed the Monster: Impact and technical evaluation

Motivated to ensure Syrian children have an opportunity to learn to read Arabic and continue their lifelong learning process, the Norwegian government funded the EduApp4Syria competition. The competition sought to develop an open source smartphone application that could build foundational literacy skills in Arabic and improve psychosocial well-being for Syrian refugee children. The competition was coordinated by the Norwegian Agency for Development Cooperation (Norad) in cooperation with the Norwegian University of Science and Technology (NTNU); All Children Reading: A Grand Challenge for Development (ACR GCD)—a partnership including the United States Agency for International Development (USAID), World Vision, and the Australian Government; mobile operator Orange; and the Inter-Agency Network for Education in Emergencies (INEE). Two games were shortlisted as winners through the two-phased competition: Antura and the Letters and Feed the Monster (FTM). 

ACR GCD, Digital Learning for Development (DL4D), and United Nations Children’s Fund (UNICEF) Office of Innovation, supported an evaluation of the two apps that sought to assess the effects on literacy learning and psychosocial outcomes. The evaluation also assessed the technical and gaming aspects of the two apps and compared these against commonly accepted measures of quality and evidence-based practice. 

Integrated Services, Indigenous Solutions (INTEGRATED), in partnership with Consortium for Research and Evaluation of Advanced Technologies in Education (CREATE) of New York University, conducted an impact evaluation using a longitudinal quasi-experimental design to estimate the impacts of the EduApp4Syria games (apps) on children’s literacy and psychosocial outcomes over time. In this design, we compared growth in literacy outcomes for two groups of children (each using one of the apps) to a group of children in matched environments who did not have access to the apps. All children in the study had little or no schooling and lived in the Azraq refugee camp in Jordan. This design relies on comparing two groups that experience the same historical trends and events over time. Simultaneously, INTEGRATED and CREATE conducted a technical evaluation, working closely with ACR and DL4D to refine key app evaluation questions to identify usability improvements for the next release of the games, provide feedback on the open beta versions built in Phase 2 of the competition, and inform improvements to be applied in Phase 3. The qualitative evaluation was conducted in collaboration with CREATE researchers who are among the foremost experts in their specialization: assessing game use and engagement among children playing digital learning games. The contributions of CREATE ensured depth in qualitative gaming use data used while triangulating conclusions.