Explainability Frameworks for Large-Scale Generative Models

Abstract

Explainability -- the ability to provide human-understandable accounts of model behaviour, generation decisions, and output provenance -- is a critical requirement for responsible deployment of large-scale generative AI systems in regulated domains and high-stakes applications. While explainability for classification and regression models has a mature literature (LIME, SHAP, attention visualisation), the extension of explainability methods to large-scale generative models presents unique challenges: the open-ended output space, the autoregressive generation process, the multi-layer attention architecture, and the trillion-parameter scale all complicate the application of standard explainability techniques. This paper proposes the Generative Explainability Framework (GEF), a comprehensive methodology for explaining large-scale generative model behaviour across four explanation types: token attribution (which input tokens most influenced each generated token), training data attribution (which training examples most influenced the generation), concept-level explanation (which high-level concepts drove generation choices), and counterfactual explanation (what minimal input changes would produce different outputs). GEF integrates four corresponding explanation methods: Integrated Gradients for LLMs (IGLLM), influence function approximation (IFA), concept activation vectors for generation (CAVG), and generation counterfactual search (GCS). GEF is evaluated on explanation faithfulness, completeness, and user comprehension across three LLM applications: medical diagnosis explanation, legal document drafting, and code generation. GEF achieves 84.2% explanation faithfulness (SD = 3.8%) -- the degree to which explanations accurately reflect model behaviour -- and 71.6% user comprehension accuracy (SD = 5.4%) in a user study with 96 domain professionals. The study contributes the GEF specification, a Generative Explanation Quality (GEQ) evaluation framework, and empirical evidence that multi-type explanation packages substantially improve user understanding of generative AI behaviour over single-explanation-type approaches.