Diffusion Models for High-Fidelity and Controllable Content Generation

Abstract

Diffusion models have established themselves as the dominant paradigm for high-fidelity visual content generation, achieving state-of-the-art quality in image and video synthesis. However, achieving high fidelity and precise controllability simultaneously remains a central challenge: the stochastic nature of diffusion sampling that enables high diversity and quality is inherently at tension with the deterministic control required for precise content specification. This paper proposes the Fidelity-Control Diffusion (FCD) framework, a unified approach to simultaneously optimising generation fidelity and controllability in diffusion models through three novel mechanisms: adaptive classifier- free guidance (ACFG) that dynamically adjusts guidance strength based on semantic content difficulty; hierarchical conditioning injection (HCI) that provides conditioning signals at multiple diffusion timestep ranges to enable both coarse-grained structural control and fine-grained detail control; and semantic consistency regularisation (SCR) that enforces semantic alignment between conditioning signals and generated outputs through a consistency loss during fine-tuning. The FCD framework is evaluated on image generation (text-to-image, layout-to- image, semantic segmentation-to-image) and video generation (text-to- video, pose-to-video) benchmarks. FCD achieves a 28.4% improvement in controllability precision (CLIP-score improvement = 0.082) while maintaining 98.1% of baseline generation fidelity (FID increase from 14.2 to 14.5) -- substantially improving the fidelity-controllability trade-off compared to standard CFG approaches. On layout-to-image generation, FCD achieves 91.4% object placement accuracy (SD = 2.1%), a 24.8% improvement over ControlNet. The study contributes the FCD framework, an ACFG scheduling algorithm, and a Fidelity-Controllability Trade-off (FCT) evaluation benchmark.