Processes for Introducing and Incorporating Digital Innovations into LMIC Settings: Examples of (1) Machine Learning for Strategic Program Planning & Evaluation; and, (2) Mobile phone surveys.

Near ubiquitous access to mobile phones globally has catalyzed discourse on the potential of phone surveys for use in the monitoring of population health. In contrast to resource and time intensive face to face surveys, phone surveys offer respondents the option of being interviewed over a personal or shared mobile phone in the privacy of their own home. Their increasing use, particularly in low and middle income countries (LMICs) where disease burden is highest, has allowed for the rapid, routine, and low cost measurement of population based health outcomes as well as key outcomes among health care providers, including knowledge. Despite their promise, these surveys are often not developed using rigorous pre-testing activities essential for ensuring quality data, including cognitive, reliability, and validity testing. In the absence of these activities, emerging data could over- or under-estimate the burden of disease and/or health care practices under assessment. Further, to improve the standardization of phone survey assessments, research must be undertaken to systematically test the effects of alternative survey modalities on factors influencing cost and key survey metrics including contact, response, completion and refusal rates as well as demographic representativeness. In this panel, we present findings from India on the essential steps required to develop valid and reliable phone surveys for use in measuring a range of health outcomes among women and frontline health workers in India. We start by reviewing methods on cognitive testing to enhance structured surveys, reflecting on the use of scales as response options. We next review efforts to assess the reliability of phone surveys for measuring outcomes among pregnant and postpartum women including knowledge and women’s experiences during pregnancy and childbirth. Finally, we present data on factors influencing response, and completion rates.

This session will explore how to effectively incorporate machine learning solutions to guide allocation of funding, staff, and supplies to optimize health outcomes in global health programs. It will share best practices and use cases for how to: - frame machine learning within the context of a holistic analytics strategy; - identify scenarios that present greatest possible return on investment from machine learning; - stand up the appropriate technology environment for machine learning solutions; - select and prepare datasets for use in machine learning solutions; - decide methods for identifying model of best fit; - interpret results and their accuracy; - integrate machine learning solutions into decision-making processes; - and provide access to machine learning solutions and their parameters and results via visualizations and dashboards. To illustrate how these strategies have been implemented in an actual intervention, and how the complexities of intervention size and geographical complexity can come into play, this session will present a use case for how machine learning is currently being used to optimize the diagnosis of HIV positive clients in Lesotho, Nigeria, and Ethiopia to increase the effectiveness and reach of testing campaigns. In this use case, data scientists collaborate with in-country implementers to collect several quarters of site-level testing results, then analyze the data using machine learning to identify a series of practical improvements that could increase overall HIV testing yields. The analysis is done using R programs running on an AWS-enabled data science platform. The results of these analyses are made available via dashboards and used by decision-makers ranging from in-country implementers to global leadership to identify how to most effectively target limited resources during iterative program planning cycles. This solution is being rolled out to several other countries where the HIV burden is high.