DOI: 10.5281/zenodo.17661435
title: Dataset of diclosulam bioactivity and soil physicochemical attributes in weakly weathered soils

description: Zenodo Description 

1. Overview of the Repository

This repository provides a comprehensive, fully documented set of experimental data, soil physicochemical attributes, processed variables, mathematical expressions, fitted parametric models, and reproducible computational procedures used in the study:

Interpretable artificial intelligence modeling of pre-emergence herbicide bioactivity in weakly weathered soils for optimized dose recommendations – Part I: diclosulam.

The material consolidates the complete workflow used to construct an interpretable hybrid AI framework for modeling diclosulam bioactivity in weakly weathered soils presenting high variability in chemical composition. All scripts have been exported in PDF format, each containing the full commented Python source code and the corresponding methodological explanation.

The repository supports scientific transparency, reproducibility, traceability of computational steps, and future extensions by other research groups.

 

2. Dataset Structure

The repository is organized into the following directories:

Data_Raw/  Original bioassay measurements, soil physicochemical attributes, and auxiliary inputs.

Data_Processed/  Cleaned and standardized data tables, reconstructed variables, selected features, optimized mathematical expressions, and fitted parameters.

Scripts/  Twelve PDF documents (script_01 to script_12) containing the commented Python code for each computational stage.

Figures/  Dose–response curves, 3D surfaces (real and predicted), comparative plots, sensitivity analyses, and statistical evaluation graphics.

README_MASTER.pdf  Global documentation of the modeling pipeline and execution sequence.

All files follow a consistent naming convention, ensuring clarity and reproducibility.

 

3. Detailed Description of the Scripts

Below is a detailed scientific description of the functionality of each script (PDF):

1. script_01_preprocess_raw_soil_data.pdf

Reads raw experimental data, enforces numerical types, removes inconsistencies, and produces a standardized table for subsequent analysis. This step ensures internal data coherence prior to model development.

2. script_02_rename_variables_using_keymap.pdf

Applies a variable renaming keymap to harmonize nomenclature across datasets. This avoids semantic ambiguity and ensures consistent naming throughout the modeling pipeline.

3. script_03_shap_sensitivity_analysis.pdf

Performs explainable sensitivity analysis using SHAP values for an XGBoost model. Identifies the most influential variables driving CDP behavior and exports rankings, feature impact statistics, and graphical summaries.

4. script_04_process_selected_variables_and_generate_scatter_plots.pdf

Reconstructs selected variables using linear segmentation and generates scatter plots of CDP against the most influential soil attributes. Produces min–max ranges and exports a processed dataset for model refinement.

5. script_05_xgboost_threshold_optimization_and_curve_generation.pdf

Uses Optuna to jointly optimize (i) the correlation threshold for dynamic feature selection and (ii) the XGBoost hyperparameters. Generates multivariate CDP curves as a function of DDH and exports prediction tables along with MAE and R² measures.

6. script_06_plot_predicted_and_real_curves.pdf

Produces comparative plots of predicted versus experimental CDP curves. Outputs tables of reconstructed points and graphical evaluations of model fidelity.

7. script_07_hybrid_nm_de_parametric_curve_fitting.pdf

Implements a hybrid optimization scheme that combines Nelder–Mead and Differential Evolution to fit a parametric function capturing CDP as a function of DDH and an additional soil attribute. Includes stabilization constraints and exports optimized parameters and the final expression.

8. script_08_compare_expressions_against_experimental_values.pdf

Evaluates several candidate mathematical expressions (including the optimized hybrid function) against experimental data using MAE, MSE, RMSE, R², and MAPE. Generates comparative plots and pointwise error analyses.

9. script_09_hypothesis_testing_expressions.pdf

Executes statistical hypothesis testing (Shapiro–Wilk, paired t-test, and Wilcoxon test) to determine whether the optimized expression significantly outperforms other candidate models. Produces test results and significance assessments.

10. script_10_hypothesis_testing_simulated_errors.pdf

Validates methodological consistency using simulated error distributions. Tests the behavior of the statistical framework under controlled scenarios to examine its sensitivity and discriminative capability.

11. script_11_soil_specific_dose_response_fits.pdf

Fits individual dose–response curves for each soil using an exponential parametric structure. Exports corresponding PDF figures and fitted parameters, enabling detailed inspection of soil-specific behaviors.

12. script_12_cdp_surface_real_predicted_3d.pdf

Generates and compares three-dimensional surfaces of real (experimental) and predicted CDP values. Uses linear interpolation and 3D rendering to visualize interactions between PHA, DDH, and CDP. Includes real, predicted, and overlaid surfaces.

 

4. Scientific Purpose

The repository documents a fully interpretable artificial intelligence workflow to replace discrete soil-specific dose–response curves with a unified continuous response surface driven by soil physicochemical descriptors. This approach supports:

transparent and explainable AI modeling;generation of optimized herbicide dose recommendations;reduction of unnecessary chemical input;environmental sustainability;reproducibility and methodological auditing.

The dataset and scripts together enable complete reconstruction of the study’s computational backbone and provide a framework for expansion to other herbicides and soil classes.

 

5. Authors and Contributions

1. Wesley Pacheco Calixto (Corresponding Author) — AI modeling, pipeline integration, mathematical formulation, dataset construction.2. Viviane Margarida Gomes Pacheco — Data organization, verification, methodological support.3. Danielle Resende Almeida — Bioassay execution and experimental validation.4. Ademir Xavier Souza — Statistical guidance and methodological evaluation.5. Clóves Gonçalves Rodrigues — Soil analysis and physicochemical characterization.6. Antonio Paulo Coimbra — Scientific review, analytical consulting, theoretical validation.