6  Conclusion

In this work, we introduced a human-centric model explanation evaluation framework grounded in causal reasoning. Central to this framework is a process that starts with eliciting well-defined target explanatory questions. We then revisited the Shapley explanation literature in light of this framework, highlighting the importance of distinguishing between model and world-level explanations. Distinguishing between the two and recognizing the type of explanatory question that is of interest to the explainee allows for a more principled approach to selecting an appropriate Shapley-based method, or in some cases, to determining that a Shapley explanation is not appropriate.

In surveying the Shapley explanation literature, a number of potential avenues for future research emerged. First, there is an open question as to whether a causal perspective justifies non-Shapley explanation methods in particular circumstances. In particular, if a fully-specified SCM can be elicited, then should a machine learning approach for generating predictions have been taken in the first place? As a second example, if model-level explanations are desired, what are the benefits of using Shapley values instead of an individual conditional expectation plot (for local explanations) or a partial dependence plot (for global explanations)? Zhao and Hastie (2021) have shown that PDP and ICE plots have causal interpretations in situations where the complement set satisfies the backdoor criterion.

The debate over reference distributions within the Shapley explanation literature has parallels in the counterfactual explanation literature, highlighting the need for additional unification efforts within XAI. Specifically, Wachter, Mittelstadt, and Russell (2018) introduced the notion of counterfactual explanations to XAI and advocated for an unconditional distribution. Subsequent work has been divided on whether a marginal or interventional approach should be taken. These parallels suggest that a causal perspective may provide a useful foundation for such unification efforts. In fact, there is some work already moving in this direction (Viswanathan and Zick (2021), Beckers (2022)).

Beckers, Sander. 2022. “Causal Explanations and XAI.” arXiv:2201.13169 [Cs], February. http://arxiv.org/abs/2201.13169.

Viswanathan, Vignesh, and Yair Zick. 2021. “Model Explanations via the Axiomatic Causal Lens.” arXiv:2109.03890 [Cs], September. http://arxiv.org/abs/2109.03890.

Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2018. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” arXiv:1711.00399 [Cs], March. http://arxiv.org/abs/1711.00399.

Zhao, Qingyuan, and Trevor Hastie. 2021. “Causal Interpretations of Black-Box Models.” Journal of Business & Economic Statistics 39 (1): 272–81. https://doi.org/10.1080/07350015.2019.1624293.