explainable decision-making

At its core, explaining a decision is a process of reverse inference: it involves backtracking from the decision to the reasons that support it. My PhD work argues that formal logic can serve as an explainability layer atop AI systems (see figure above). The idea is not to replace data-driven or other AI approaches with purely logical ones. Instead, the decisions of an AI system, whether symbolic or sub-symbolic, are captured in a formal logical language, which, thanks to its structured semantics, expressivity, and scrutability, supports exactly this kind of inference and reasoning. The result is explanations that are rigorous, flexible, and human-aware.

Within this layer, we distinguish between two kinds of explanations:

• A monolithic explanation is generated with respect to the AI system’s knowledge base alone. It is a minimal set of the system’s knowledge that entails the decision.
• A human-aware explanation is generated with respect to both the AI system’s knowledge base and the human user’s knowledge base. The user may reason about the problem with a different model than the system, so a decision that follows from the system’s knowledge may not follow from the user’s. A human-aware explanation updates the user’s knowledge base so that the decision follows from it. This notion is rooted in theory of mind, the ability to attribute mental models to others while recognizing that these models may differ from one’s own, and it generalizes the model reconciliation problem from the planning community to any problem that can be expressed in a logic for which satisfiability of subsets can be decided (Vasileiou et al., 2022).

Building on this foundation, we developed:

• efficient algorithms for computing explanations by exploiting hitting sets (Vasileiou et al., 2021);
• extensions to logic programs (Tran Cao et al., 2021) and to probabilistic scenarios (Vasileiou et al., 2025; Tang et al., 2025);
• personalized explanations via knowledge forgetting (Vasileiou & Yeoh, 2023);
• visualizations for conveying explanations to users (Kumar et al., 2022);
• dialectical explanations through structured argumentative dialogues (Vasileiou et al., 2024);
• explanations for agent scheduling problems, including a hybrid logic–LLM system for course scheduling (Vasileiou et al., 2023; Vasileiou & Yeoh, 2025).

This work is collected in my dissertation, Explainable Decision-Making: From Formal Logic to AI Systems with Explainable Behavior, which received the ICAPS Outstanding Dissertation Award and the Turner Best Dissertation Award from Washington University in St. Louis.