Unified Architectures for Multimodal Content Generation

Abstract

The proliferation of specialised generative architectures -- distinct models for text, image, audio, video, and code generation -- has produced impressive per-modality performance but fragmented the generative AI landscape into a collection of task-specific systems that require separate deployment, maintenance, and orchestration overhead. Unified architectures -- single model systems capable of generating high-quality content across multiple modalities without per-modality specialisation -- represent the next frontier in generative intelligence but remain significantly behind specialist models in output quality. This paper proposes the Universal Content Generator (UCG), a unified autoregressive architecture for high-quality multimodal content generation based on a single transformer backbone with modality-adaptive tokenisation, shared attention with modality-aware positional encoding, and a mixture-of-decoders output head that routes generation to modality-specific decoder networks while sharing the primary attention stack. UCG supports five generation modes: text, image (512x512), audio (speech and music), code, and video (4-second clips at 16fps). The UCG architecture is trained using a curriculum that progresses from unimodal pre-training through bimodal alignment to full five-modality joint training on a 1.8T token multimodal corpus. UCG evaluation on 15 generation benchmarks across all five modalities demonstrates that UCG closes 84.2% of the quality gap between unified and specialist models on average, with text generation matching GPT-3.5-level performance, image generation achieving FID = 22.1 (competitive with mid-tier specialist models), and code generation achieving HumanEval pass@1 = 48.4%. UCG introduces a 31.7% parameter reduction versus the ensemble of specialist models required for equivalent modality coverage. The study contributes the UCG architecture, a multimodal curriculum training methodology, and a Unified Generation Quality (UGQ) evaluation suite.