AI for Assisting NGOs in improving Maternal and Child Health Outcomes
AI for Assisting NGOs in
Improving Maternal and Child
Health Outcomes
MOTIVATION
India has a maternal mortality ratio of 113 and child mortality ratio of 2830 per 100,000 live births. Lack of access to preventive care information is a major contributing factor for these deaths. ARMMAN is an Indian non-profit that leverages mobile health technology and automated messaging services to deliver critical preventive care information to beneficiaries (new and expecting mothers) during pregnancy and after delivery in an effort to mitigate these staggering numbers.
Unfortunately, a key challenge in such information delivery programs is that a significant fraction of beneficiaries drop out of the program. Yet, non-profits often have limited health-worker resources (time) to place crucial service calls for live interaction with beneficiaries to prevent such engagement drops. The goal of this project is to design ways in which AI can be leveraged to better predict beneficiaries who are at a higher risk of dropping out and then planning the limited resources optimally to maximize beneficiary engagement with the program.
ONGOING &
RECENT WORK
While the health workers at non-profits such as ARMMAN reach out individually to beneficiaries, such programs still suffer from declining engagement. We have deployed SAHELI, a system to efficiently utilize the limited availability of health workers for improving maternal and child health in India. SAHELI uses the Restless Multiarmed Bandit (RMAB) framework to identify beneficiaries for outreach. It is the first deployed application for RMABs in public health, and is already in continuous use by our partner NGO, ARMMAN. We have already reached ∼ 100K beneficiaries with SAHELI, and are on track to serve 1 million beneficiaries by the end of 2023. This scale and impact has been achieved through multiple innovations in the RMAB model and its development, in preparation of real world data, and in deployment practices; and through careful consideration of responsible AI practices. Specifically, in this paper, we describe our approach to learn from past data to improve the performance of SAHELI’s RMAB model, the real-world challenges faced during deployment and adoption of SAHELI, and the end-to-end pipeline.
Motivated by the randomized controlled trials (RCTs) run in partnership with ARMMAN, we consider the task of evaluating policies of algorithmic resource allocation through RCTs. Such policies are tasked with optimizing the utilization of limited intervention resources, with the goal of maximizing the benefits derived. Evaluation of such allocation policies through RCTs proves difficult, notwithstanding the scale of the trial, because the individuals’ outcomes are inextricably interlinked through resource constraints controlling the policy decisions. Our key contribution is to present a new estimator leveraging our proposed novel concept, that involves retrospective reshuffling of participants across experimental arms at the end of an RCT. We identify conditions under which such reassignments are permissible and can be leveraged to construct counterfactual trials, whose outcomes can be accurately ascertained, for free. We prove theoretically that such an estimator is more accurate than common estimators based on sample means — we show that it returns an unbiased estimate and simultaneously reduces variance. We demonstrate the value of our approach through empirical experiments on synthetic, semisynthetic as well as real case study data and show improved estimation accuracy across the board.
We build on previous work proposing ‘Decision Focused Learning (DFL)’ that aims to integrate the optimization problem in the learning pipeline. Previous works have only shown the applicability of DFL in simulation setting. In collaboration with ARMMAN, we conduct a large-scale field study consisting of 9000 beneficiaries for 6 weeks and track key engagement metrics in a mobile health awareness program. To the best of our knowledge this is the first real-world study involving Decision Focused Learning. We demonstrate that beneficiaries in the DFL group experience statistically significant reductions in cumulative engagement drop, while those in the Predict-then-Optimize group do not. This establishes the practicality of use of decision focused learning for real world problems. We also demonstrate that DFL learns a better decision boundary between the RMAB actions, and strategically predicts parameters for arms which contribute most to the final decision outcome.
This paper studies restless multi-armed bandit (RMAB) problems with unknown arm transition dynamics but with known correlated arm features. The goal is to learn a model to predict transition dynamics given features, where the Whittle index policy solves the RMAB problems using predicted transitions. However, prior works often learn the model by maximizing the predictive accuracy instead of final RMAB solution quality, causing a mismatch between training and evaluation objectives. To address this shortcoming, we propose a novel approach for decision-focused learning in RMAB that directly trains the predictive model to maximize the Whittle index solution quality. We present three key contributions: (i) we establish differentiability of the Whittle index policy to support decision-focused learning; (ii) we significantly improve the scalability of decision-focused learning approaches in sequential problems, specifically RMAB problems; (iii) we apply our algorithm to a previously collected dataset of maternal and child health to demonstrate its performance. Indeed, our algorithm is the first for decision-focused learning in RMAB that scales to real-world problem sizes.
Partnering with ARMMAN, develop restless multi-arm bandit (RMAB) techniques for designing engagement interventions focused on the realities of large scale and model uncertainty in real-world planning. We address these dual challenges with GROUPS, a double oracle–based algorithm for robust planning in RMABs with scalable grouped arms. The algorithm identifies several "worst-case" environments within the uncertainty set, then plans against them to formulate a robust policy. This is enabled by a new per-arm index notion of regret, "Whittle index regret" that allows efficient search of the uncertainty space. Using real-world data from ARMMAN, we show that GROUPS produces robust policies for environments with over 300,000 arms that reduce minimax regret by up to 50%, halving the number of preventable missed voice messages to connect more mothers with life-saving maternal health information
This paper describes our work in assisting ARMMAN and similar non-profits that employ automated messaging programs to deliver timely preventive care information to beneficiaries (new and expecting mothers) during pregnancy and after delivery. To assist non-profits in optimizing their limited health-worker resources, we developed an AI system, called Restless Multi-Armed Bandits (RMABs). One key technical contribution in this system is a novel clustering method of offline historical data to infer unknown parameters required as an input to the RMAB. Our second major contribution is evaluation of our RMAB system in collaboration with ARMMAN, via a real-world service quality improvement study. The study compared strategies for optimizing service calls to 23003 participants over a period of 7 weeks to reduce engagement drops. We show that the RMAB group provides statistically significant improvement over other comparison groups, reducing ∼ 30% engagement drops. To the best of our knowledge, this is the first study demonstrating the utility of RMABs in real world public health settings. We are transitioning our RMAB system to ARMMAN for real-world use.
A shorter version of this paper, earning Best Paper Award, also appeared as: Restless Bandits in the Field: Real-World Study for Improving Maternal and Child Health Outcomes (NeurIPS 2021 Workshop on Machine Learning for Public Health)
[paper]
In partnership with ARMMAN, we analyze call records of over 300,000 women registered in the program created by ARMMAN and try to identify women who might not engage with these call programs that are proven to result in positive health outcomes. We build machine learning based models to predict the long term engagement pattern from call logs and beneficiaries’ demographic information, and discuss the applicability of this method in the real world through a pilot validation. Through a randomized controlled trial, we show that using our model’s predictions to make interventions boosts engagement metrics by 61.37%. We then formulate the intervention planning problem as restless multi-armed bandits (RMABs), and present preliminary results using this approach.
This work models the limited-resource planning scenario as a restless multi-armed bandit (RMAB) problem, where each beneficiary is assumed to transition from one state to another depending on the intervention provided to them. In practice, the transition probabilities of beneficiaries are unknown a priori, and hence, we propose a mechanism for the problem of balancing the explore-exploit trade-off. Empirically, we find that our proposed mechanism outperforms the baseline intervention scheme maternal healthcare dataset.
A shorter version of this paper also appeared as: Learning Index Policies for Restless Bandits with Application to Maternal Healthcare (AAMAS 2021, Extended Abstract).
PROJECT
CONTRIBUTORS
Milind Tambe
Aditya Mate
Kai Wang
Jackson A. Killian
Shahin Jabbari
Andrew Perrault
Lily Xu
Shresth Verma
Aparna Taneja
Vineet Nair
Aparna Hegde
Neha Madhiwalla
Paula Rodriguez Diaz
Sanket Shah
PUBLICATIONS
Aditya Mate*, Lovish Madaan*, Aparna Taneja, Neha Madhiwalla, Shresth Verma, Gargi Singh, Aparna Hegde, Pradeep Varakantham, and Milind Tambe. 2/2022. “Field Study in Deploying Restless Multi-Armed Bandits: Assisting Non-Profits in Improving Maternal and Child Health.” In AAAI Conference on Artificial Intelligence. Vancouver, Canada
Aditya Mate*, Lovish Madaan*, Aparna Taneja, Neha Madhiwalla, Shresth Verma, Gargi Singh, Aparna Hegde, Pradeep Varakantham, and Milind Tambe. 12/2021. “Restless Bandits in the Field: Real-World Study for Improving Maternal and Child Health Outcomes.” In MLPH: Machine Learning in Public Health NeurIPS 2021 Workshop
Siddhart Nisthala, Lovish Madaan, Aditya Mate, Harshavardhan Kamarthi, Anirudh Grama, Divy Thakkar, Dhyanesh Narayanan, Suresh Chaudhary, Neha Madhiwala, Ramesh Padhmanabhan, Aparna Hegde, Pradeep Varakantham, Balaram Ravindran, and Milind Tambe. 5/5/2021. “Selective Intervention Planning using Restless Multi-Armed Bandits to Improve Maternal and Child Health Outcomes.” In AAMAS workshop on Autonomous Agents for social good.
Arpita Biswas, Gaurav Aggarwal, Pradeep Varakantham, and Milind Tambe. 8/2021. “Learn to Intervene: An Adaptive Learning Policy for Restless Bandits in Application to Preventive Healthcare.” In International Joint Conference on Artificial Intelligence (IJCAI).
Arpita Biswas, Gaurav Aggarwal, Pradeep Varakantham, and Milind Tambe. 5/7/2021. “Learning Index Policies for Restless Bandits with Application to Maternal Healthcare (Extended abstract).” In International Conference on Autonomous Agents and Multiagent Systems (AAMAS).