Strategic and counterfactual reasoning in AI-assisted decision making
- From finance and healthcare to criminal justice and transportation, various domains that involve critical decisions, traditionally made by humans, are increasingly incorporating artificial intelligence (AI) systems into their decision making processes. Modern AI systems excel at processing vast amounts of data and solving complex problems at a speed and scale unimaginable for humans. However, complete automation of high-stakes decisions is often undesirable due to legal, ethical, and societal concerns. A promising approach, which has attracted significant attention in the machine learning literature, lies in human-AI collaboration: decision making pipelines that leverage the computational strengths of AI systems to enhance the overall quality of decisions while maintaining a degree of human control. In this context, I focus on AI-assisted decision making scenarios characterized by complexity and uncertainty, specifically requiring strategic reasoning about others’ actions and counterfactual reasoning about alternatives to past decisions.
First, I focus on strategic reasoning and introduce methods based on game-
theoretic modeling to support policy design in strategic environments. These methods enable a decision maker in a resource allocation scenario to design policies, informed by a predictive model, that maximize their utility while accounting for strategic responses from individuals who gain knowledge about the policy and aim to receive a beneficial decision. I provide algorithms for two distinct scenarios with varying levels of information available to individuals: a fully transparent scenario
where the policy is disclosed and a partially transparent scenario where the decision maker provides actionable recommendations to individuals rejected by the policy.
Then, I shift focus to counterfactual reasoning and develop methods based on causal modeling to enhance the counterfactual reasoning capabilities of a human decision maker in a sequential decision making task. These methods aim to improve the decision maker’s learning process from past experiences by identifying critical time steps where different actions could have led to better outcomes. Specifically, I consider settings where a decision maker observes the state of the environment over time and takes a series of interdependent actions that result in an observed outcome. For both discrete and continuous states, I formalize the problem of finding alternative action sequences, close to the observed one, that would have achieved a better counterfactual outcome, and I provide efficient algorithmic solutions.
Finally, I investigate how people perceive responsibility in human-AI teams. In this context, I propose a computational model based on counterfactual simulations to predict how an external observer attributes responsibility to a human and an AI agent collaborating towards a common goal. To evaluate the model’s predictions, I develop a simulation environment that generates stylized instances of sequential human-AI collaboration and conduct a human-subject study in which participants
make responsibility judgments about the two agents.