User Trust Modeling in Responsible AI Systems

Abstract

User trust in AI systems -- the degree to which users rely on AI outputs, recommendations, or decisions in ways that are appropriate to the system's actual reliability and limitations -- is a central construct in responsible AI design. Miscalibrated trust, whether excessive (overtrust) or insufficient (undertrust), undermines the beneficial outcomes that AI-assisted decision- making seeks to achieve and increases the risk of harmful over-reliance or unjustified rejection of accurate AI guidance. Despite extensive theoretical treatment of AI trust, validated computational models of user trust that can be integrated into responsible AI systems to personalise oversight, explanation, and interaction strategies remain scarce. This paper proposes the Dynamic Trust Calibration Model (DTCM), a Bayesian trust model that continuously estimates individual user trust levels from interaction signals -- decision patterns, override frequency, explanation engagement, and confidence-accuracy alignment -- and uses these estimates to trigger personalised interventions designed to maintain appropriate trust calibration. The DTCM is evaluated through a longitudinal user study involving 168 participants interacting with an AI clinical advisory system over six weeks across three experimental conditions: DTCM-adaptive interventions, static explanation baseline, and no-explanation control. DTCM-adaptive conditions achieve a 36.8% improvement in trust calibration accuracy (SD = 5.4%), a 28.9% reduction in overtrust incidents (SD = 4.7%), and a 31.4% reduction in undertrust incidents (SD = 4.9%) compared to the static baseline. The study contributes the DTCM specification, a Trust Calibration Index (TCI) measurement instrument, and empirical evidence that dynamic trust modelling produces superior responsible AI interaction outcomes compared to static explanation approaches.