AI for Global Health and Public Health

Modeling to Inform Disease Control, Screening, Treatment Policies, and Prevention Interventions

 

  

MOTIVATION

AI tools can be used to inform public health policy.  For example, predictive analytics can be used to identify risk factors for disease; and optimization frameworks (whether single stage or repeated) can be used to identify when to screen or treat disease, or which risk groups to target given limited resources. Optimization frameworks can also be used to channel limited resources towards at-risk individuals to improve their adherence to healthy habits. We describe several projects and potential project areas below.

CURRENT PROJECTS

Improving Maternal and
Child Health Outcomes in
Partnership with ARMMAN

Maternal and Child Health

This project Improving Maternal and Child Health Outcomes in Partnership with ARMMAN aims at using AI for improving Maternal and Child Health Outcomes by predicting which beneficiaries are at a risk of dropping out from automated health information delivery programs. 

Learning Loss Functions for Predict-then-Optimize

Predict-then-Optimize (PtO) is a framework for using machine learning to perform decision-making under uncertainty. As the name suggests, it proceeds in two steps—first, you make predictions about the uncertain quantities of interest and then, second, you make the required decisions assuming that these predictions are accurate. However, these decisions are only optimal if the input predictions are accurate. To evaluate the quality of our decisions for a given prediction, we check how well they would perform on the ground truth values of the quantities of interest (from the dataset) as opposed to the predictions. Let’s use an example to make all these different steps in the PtO pipeline concrete.

HIV Prevention
Among Homeless Youth

The HIV Prevention Among Homeless Youth project focuses on the development of decision support systems for homeless youth drop-in center staff, who need to find the most influential homeless youth to raise awareness about HIV (and other STDs) among their peers, and to drive the homeless youth community towards safer behaviors.

PREVIOUS WORK

Algorithmic Social Interventions for Social Work and Public Health

Combatting COVID-19

Coronavirus

The COVID-19 outbreak has caused an unprecedented global reaction with countries taking drastic steps to combat the pandemic. Mathematical modeling and multi-agent based analysis of the pandemic allows better understanding of the disease spread and may help inform policy at the national and regional level. We use tools and modeling techniques from AI to help understand the situation better and design aids that may help policymakers design better solutions in the fight against this pandemic. Read more about Teamcore's efforts to stop the spread of COVID-19.

  

Using Machine Learning
& Multi-Agent Planning
to Fight Tuberculosis

Tuberculosis Health Post

Fighting and Preventing Tuberculosis in India

Tuberculosis is one of the top 10 killers in the world and is especially prevalent in India. AI can help across the entire pipeline of care, from decision support tools for planning active screening routes, to predictive algorithms for resource constrained health workers to deliver targeted interventions to patients.

  

Using Social Networks
for Prevention Interventions

Click here to read more on this initiative.

Two young men

Substance abuse prevention among homeless youth

Research has consistently documented levels of cocaine, heroin, methamphetamine, alcohol, and marijuana use and abuse among these adolescents that far exceed that of housed adolescents.  This Social Networks and Substance Abuse Prevention for Homeless Youth project aims to use algorithms to determine the best group formations to prevent regular use of hard drugs among homeless youth.

 

Army officer with a young child

Suicide prevention among active duty military and homeless youth

One of the fundamental questions facing social science is how social networks and the cognitions people have about their networks affect their mental states and mental health.  AI techniques present an opportunity to dynamically model social networks and the messages transmitted across those networks to create predictive models of influence unavailable with standard statistical techniques.  View Predictive Modeling for Early Identification of Suicidal Thinking.

  

  

PROJECT
PARTICIPANTS

Sze-chuan Suen
Milind Tambe
Bryan Wilder
Han Ching Ou
Dana Goldman
Eric Rice
Carl Castro
Anthony Fulginiti
Anamika Barman-Adhikari
Aditya Mate

Phebe Vayanos
Aida Rahmattalabi
Jackson A. Killian
Maimuna Majumder
Marie Charpignon
Angel N. Desai
Shahin Jabbari
Andrew Perrault
Lily Xu
Shresth Verma

Aparna Taneja
Vineet Nair
Aparna Hegde
Neha Madhiwalla
Paula Rodriguez Diaz
Sonja Johnson-Yu
Sanket Shah

  

SPONSORS

California HIV/AIDS Research Program
The California HIV/AIDS Research Program (CHRP) logo

Army Research Office

  

RELATED
PUBLICATIONS

Sanket Shah, Arun Suggala, Milind Tambe, and Aparna Taneja. 5/1/2024. “Efficient Public Health Intervention Planning Using Decomposition-Based Decision-Focused Learning.” International Conference on Autonomous Agents and Multiagent Systems (AAMAS). Auckland, New Zealand.Abstract

The declining participation of beneficiaries over time is a key concern in public health programs. A popular strategy for improving retention is to have health workers `intervene' on beneficiaries at risk of dropping out.  However, the availability and time of these health workers are limited resources. As a result, there has been a line of research on optimizing these limited intervention resources using Restless Multi-Armed Bandits (RMABs). The key technical barrier to using this framework in practice lies in estimating the beneficiaries' RMAB parameters from historical data. Recent research on Decision-Focused Learning (DFL) has shown that estimating parameters that maximize beneficiaries' cumulative returns rather than predictive accuracy, is essential to good performance. 

Unfortunately, these gains come at a high computational cost because of the need to solve and evaluate the RMAB in each DFL training step. Consequently, past approaches may not be sustainable for the NGOs that manage such programs in the long run, given that they operate under resource constraints. In this paper, we provide a principled way to exploit the structure of RMABs to speed up DFL by decoupling intervention planning for different beneficiaries. We use real-world data from an Indian NGO, ARMMAN, to show that our approach is up to two orders of magnitude faster than the state-of-the-art approach while also yielding superior model performance. This enables computationally efficient solutions, giving NGOs the ability to deploy such solutions to serve potentially millions of mothers, ultimately advancing progress toward UNSDG 3.1.

Panayiotis Danassis, Shresth Verma, Jackson A. Killian, Aparna Taneja, and Milind Tambe. 8/2023. “Limited Resource Allocation in a Non-Markovian World: The Case of Maternal and Child Healthcare.” International Joint Conference on Artificial Intelligence (IJCAI).Abstract
The success of many healthcare programs depends on participants' adherence. We consider the problem of scheduling interventions in low resource settings (e.g., placing timely support calls from health workers) to increase adherence and/or engagement. Past works have successfully developed several classes of Restless Multi-armed Bandit (RMAB) based solutions for this problem. Nevertheless, all past RMAB approaches assume that the participants' behaviour follows the Markov property. We demonstrate significant deviations from the Markov assumption on real-world data on a maternal health awareness program from our partner NGO, ARMMAN. Moreover, we extend RMABs to continuous state spaces, a previously understudied area. To tackle the generalised non-Markovian RMAB setting we (i) model each participant's trajectory as a time-series, (ii) leverage the power of time-series forecasting models to learn complex patterns and dynamics to predict future states, and (iii) propose the Time-series Arm Ranking Index (TARI) policy, a novel algorithm that selects the RMAB arms that will benefit the most from an intervention, given our future state predictions. We evaluate our approach on both synthetic data, and a secondary analysis on real data from ARMMAN, and demonstrate significant increase in engagement compared to the SOTA, deployed Whittle index solution. This translates to 16.3 hours of additional content listened, 90.8% more engagement drops prevented, and reaching more than twice as many high dropout-risk beneficiaries.
Aditya Mate. 10/16/2022. “Optimization and Planning of Limited Resources for Assisting Non-Profits in Improving Maternal and Child Health.” INFORMS Doing Good with Good OR.Abstract

The maternal mortality rate in India is appalling, largely fueled by lack of access to preventive care information, especially in low resource households. We partner with non-profit, ARMMAN, that aims to use mobile health technologies to improve the maternal and child health outcomes.

 

To assisst ARMMAN and such non-profits, we develop a Restless Multi-Armed Bandit (RMAB) based solution to help improve accessibility of critical health information, via increased engagement of beneficiaries with their program. We address fundamental research challenges that crop up along the way and present technical advances in RMABs and Planning Algorithms for Limited-Resource Allocation. Transcending the boundaries of typical laboratory research, we also deploy our models in the field, and present results from a first-of-its-kind pilot test employing and evaluating RMABs in a real-world public health application.

Jackson A. Killian, Lily Xu, Arpita Biswas, and Milind Tambe. 8/2022. “Restless and Uncertain: Robust Policies for Restless Bandits via Deep Multi-Agent Reinforcement Learning.” In Uncertainty in Artificial Intelligence (UAI).Abstract
We introduce robustness in restless multi-armed bandits (RMABs), a popular model for constrained resource allocation among independent stochastic processes (arms). Nearly all RMAB techniques assume stochastic dynamics are precisely known. However, in many real-world settings, dynamics are estimated with significant uncertainty, e.g., via historical data, which can lead to bad outcomes if ignored. To address this, we develop an algorithm to compute minimax regret--robust policies for RMABs. Our approach uses a double oracle framework (oracles for agent and nature), which is often used for single-process robust planning but requires significant new techniques to accommodate the combinatorial nature of RMABs. Specifically, we design a deep reinforcement learning (RL) algorithm, DDLPO, which tackles the combinatorial challenge by learning an auxiliary "λ-network" in tandem with policy networks per arm, greatly reducing sample complexity, with guarantees on convergence. DDLPO, of general interest, implements our reward-maximizing agent oracle. We then tackle the challenging regret-maximizing nature oracle, a non-stationary RL challenge, by formulating it as a multi-agent RL problem between a policy optimizer and adversarial nature. This formulation is of general interest---we solve it for RMABs by creating a multi-agent extension of DDLPO with a shared critic. We show our approaches work well in three experimental domains.
Vineet Nair, Kritika Prakash, Michael Wilbur, Aparna Taneja, Corrine Namblard, Oyindamola Adeyemo, Abhishek Dubey, Abiodun Adereni, Milind Tambe, and Ayan Mukhopadhyay. 7/2022. “ADVISER: AI-Driven Vaccination Intervention Optimiser for Increasing Vaccine Uptake in Nigeria.” In International Joint Conference on AI (IJCAI) 7/2022. Abstract
More than 5 million children under five years die from largely preventable or treatable medical conditions every year, with an overwhelmingly large proportion of deaths occurring in under-developed countries with low vaccination uptake. One of the United Nations’ sustainable development goals (SDG 3) aims to end preventable deaths of new-borns and children under five years of age. We focus on Nigeria, where the rate of infant mortal-ity is appalling. We collaborate with HelpMum, a large non-profit organization in Nigeria to design and optimize the allocation of heterogeneous health interventions under uncertainty to increase vaccination uptake, the first such collaboration in Nigeria. Our framework, ADVISER: AI-Driven Vaccination Intervention Optimiser, is based on an integer linear program that seeks to maximize the cumulative probability of successful vaccination. Our optimization formulation is intractable in practice. We present a heuristic approach that enables us to solve the problem for real-world use-cases. We also present theoretical bounds for the heuristic method. Finally, we show that the proposed approach out-performs baseline methods in terms of vaccination uptake through experimental evaluation. HelpMum is currently planning a pilot program based on our approach to be deployed in the largest city of Nigeria, which would be the first deployment of an AI-driven vaccination uptake program in the country and hopefully, pave the way for other data-driven programs to improve health outcomes in Nigeria.
Han-Ching Ou. 3/31/2022. “Sequential Network Planning Problems for Public Health Applications.” PhD Thesis, Computer Science, Harvard University.Abstract

In the past decade, breakthroughs of Artificial Intelligence (AI) in its multiple sub-area have made new applications in various domains possible. One typical yet essential example is the public health domain. There are many challenges for humans in our never-ending battle with diseases. Among them, problems involving harnessing data with network structures and future planning, such as disease control or resource allocation, demand effective solutions significantly. However, unfortunately, some of them are too complicated or unscalable for humans to solve optimally. This thesis tackles these challenging sequential network planning problems for the public health domain by advancing the state-of-the-art to a new level of effectiveness.

In particular, My thesis provides three main contributions to overcome the emerging challenges when applying sequential network planning problems in the public health domain, namely (1) a novel sequential network-based screening/contact tracing framework under uncertainty, (2) a novel sequential network-based mobile interventions framework, (3) theoretical analysis, algorithmic solutions and empirical experiments that shows superior performance compared to previous approaches both theoretically and empirically.

More concretely, the first part of this thesis studies the active screening problem as an emerging application for disease prevention. I introduce a new approach to modeling multi-round network-based screening/contact tracing under uncertainty. Based on the well-known network SIS model in computational epidemiology, which is applicable for many diseases, I propose a model of the multi-agent active screening problem (ACTS) and prove its NP-hardness. I further proposed the REMEDY (REcurrent screening Multi-round Efficient DYnamic agent) algorithm for solving this problem. With a time and solution quality trade-off, REMEDY has two variants, Full- and Fast-REMEDY. It is a Frank-Wolfe-style gradient descent algorithm realized by compacting the representation of belief states to represent uncertainty. As shown in the experiment conducted, Full- and Fast-REMEDY are not only being superior in controlling diseases to all the previous approaches; they are also robust to varying levels of missing
information in the social graph and budget change, thus enabling
the use of our agent to improve the current practice of real-world
screening contexts.

The second part of this thesis focuses on the scalability issue for the time horizon for the ACTS problem. Although Full-REMEDY provides excellent solution qualities, it fails to scale to large time horizons while fully considering the future effect of current interventions. Thus, I proposed a novel reinforcement learning (RL) approach based on Deep Q-Networks (DQN). Due to the nature of the ACTS problem, several challenges that the traditional RL can not handle have emerged, including (1) the combinatorial nature of the problem, (2) the need for sequential planning, and (3) the uncertainties in the infectiousness states of the population. I design several innovative adaptations in my RL approach to address the above challenges. I will introduce why and how these adaptations are made in this part.

For the third part, I introduce a novel sequential network-based mobile interventions framework. It is a restless multi-armed bandits (RMABs) with network pulling effects. In the proposed model, arms are partially recharging and connected through a graph. Pulling one arm also improves the state of neighboring arms, significantly extending the previously studied setting of fully recharging bandits with no network effects. Such network effect may arise due to regular population movements (such as commuting between home and work) for mobile intervention applications. In my thesis, I show that network effects in RMABs induce strong reward coupling that is not accounted for by existing solution methods. I also propose a new solution approach for the networked RMABs by exploiting concavity properties that arise under natural assumptions on the structure of intervention effects. In addition, I show the optimality of such a method in idealized settings and demonstrate that it empirically outperforms state-of-the-art baselines.

  •  
  • 1 of 5
  • »