Knowledge-Integrated Large Language Models for Reliable Text Generation

Abstract

Large language models generate fluent and contextually coherent text but remain prone to hallucination -- the generation of plausible-sounding but factually incorrect content -- a fundamental reliability limitation that constrains their deployment in knowledge-intensive applications where factual accuracy is critical. The hallucination problem arises from the tension between the generative objective (predicting plausible next tokens) and the factual accuracy objective (grounding claims in verifiable knowledge), which are not aligned during standard language model training. This paper proposes the Knowledge-Integrated LLM (KILLM) architecture, a framework that integrates structured knowledge graph (KG) representations into the LLM generation process through three novel mechanisms: a KG-conditioned attention layer that aligns token generation with entity and relation representations from a structured knowledge base; a dynamic knowledge retrieval controller that adaptively retrieves relevant KG subgraphs for each generation step; and a factual consistency verifier that evaluates generated claims against KG triples before output and triggers correction when factual inconsistency is detected. KILLM is evaluated on four knowledge-intensive generation benchmarks -- FEVER fact verification, Natural Questions open-domain QA, MedQA clinical QA, and SciFact scientific claim verification -- and compared against RAG, chain-of-thought prompting, and direct LLM baseline. KILLM achieves a 34.2% reduction in hallucination rate (SD = 3.8%) relative to direct LLM baseline and a 18.4% reduction relative to RAG, while maintaining 96.4% of baseline fluency as measured by human evaluation. The study contributes the KILLM architecture, a hallucination taxonomy for LLM generation, and an open-source evaluation toolkit for knowledge-grounded text generation benchmarking.