A methodology for synthetic data generation and validation applied to agricultural engineering

Abstract

The scarcity of high-quality experimental data, coupled with the high costs of laboratory and sensory analyses, are significant barriers to the development of computational modeling and artificial intelligence in the field of Agricultural Sciences. This thesis introduces a reverse engineering framework for generating synthetic data based on Cholesky decomposition, which can reconstruct multivariate distributions while preserving statistical properties and structural dependence between variables. As a case study, the technique was employed in a model for the sensory quality of Arabica coffee using a public dataset from the Coffee Quality Institute (𝑛 = 207 samples). The method was divided into four stages: (i) data extraction and preprocessing (ii) statistical synthesis preserving first and second order moments (iii) multiple validation of statistical fidelity and (iv) evaluation of predictive power via the Train Synthetic, Test Real (TSTR) protocol. The results showed high structural fidelity, obtaining a correlation matrix similarity of 96.20% and a Kolmogorov-Smirnov (KS) test similarity of 85.81%; all variables exceeded the technical acceptance threshold of 70%. In RF regression, only the models trained with synthetic data achieved R² = 0.9498, a Mean Absolute Error (MAE) of only 0.216, and 100% accuracy within a margin of error of ±10%. For the ordinal classification of sensory quality, the approach achieved 89.37% accuracy and a Weighted Quadratic Kappa Coefficient of 0.9558, showing almost perfect agreement with the standards of the Specialty Coffee Association (SCA). As its main scientific contribution, this thesis establishes a reverse engineering framework for reconstructing synthetic databases from aggregated statistics, demonstrating that the simultaneous preservation of statistical fidelity and predictive utility can be achieved without the use of deep generative models. The proposed approach expands the possibilities for secure data sharing, reduces dependence on costly experimentation, and offers a computationally efficient and easily reproducible solution with high potential for technology transfer to applications in Agricultural Engineering, Agronomy, and other areas that depend on reliable experimental databases.