Synthetic Data Generation for Privacy-Preserving Machine Learning

Abstract

Synthetic data generation -- the production of artificial datasets that statistically resemble real sensitive data without disclosing individual records -- has emerged as a critical enabling technology for privacy-preserving machine learning in domains where real data collection is constrained by privacy regulations, ethical requirements, or data scarcity. While differentially private synthetic data generation has been an active research area, the application of large generative AI models -- particularly diffusion models, VAEs, and LLMs -- to synthetic data generation has introduced substantially more realistic and statistically faithful synthetic datasets at the cost of increased privacy risk from memorisation and overfitting. This paper proposes the Generative Privacy-Preserving Synthesis (GPPS) framework, a unified methodology for high-fidelity synthetic data generation with formal privacy guarantees across tabular, time-series, text, and image data modalities. GPPS integrates four generation strategies: differentially private variational autoencoder synthesis (DP-VAES) for tabular data; differentially private diffusion model synthesis (DP-DMS) for image data; privacy-preserving LLM-based text synthesis (PP-LLMS) for text data; and federated synthetic generation (FSG) for distributed private data settings. Each strategy provides formal differential privacy (DP) guarantees with specified epsilon budgets. GPPS is evaluated on six benchmark datasets across the four modalities using a unified evaluation framework covering data utility, privacy risk, and downstream ML performance. GPPS achieves mean data utility score of 0.874 (SD = 0.042) at epsilon = 1.0 (strong privacy) -- substantially outperforming prior DP synthesis baselines (0.712, SD = 0.058) -- while maintaining formal DP guarantees verified by the Renyi Differential Privacy accountant. Downstream ML performance on GPPS synthetic data achieves 91.4% of real-data model performance on average. The study contributes the GPPS specification, a Synthetic Data Quality Index (SDQI), and the first cross-modal DP synthetic data evaluation benchmark.