DOI: 10.5281/zenodo.12802588
title: Near-infrared (NIR) soil spectral library using the NeoSpectra Handheld NIR Analyzer by Si-Ware

description: Up-to-date information on soil properties and the ability to track changes in soil properties over time are critical for improving multiple decisions on soil security at various scales, ranging from global climate change modeling and policy to national level environmental and development planning, to farm and field level resource management. Diffuse reflectance infrared spectroscopy has become an indispensable laboratory tool for the rapid estimation of numerous soil properties to support various soil mapping, soil monitoring, and soil testing applications. Recent advances in hardware technology have enabled the development of handheld sensors with similar performance specifications as laboratory-grade near-infrared (NIR) spectrometers.

Here, we've compiled a hand-held NIR spectral library (1350-2550 nm) using the NeoSpectra Handheld NIR Analyzer developed by Si-Ware. Each scanner is fitted with Fourier-Transform technology based on the semiconductor Micro Electromechanical Systems (MEMS) manufacturing technique, promising accuracy, and consistency between devices.

This library includes 2,106 distinct mineral soil samples scanned across 9 of these portable low-cost NIR spectrometers (indicated by serial no). 2,016 of these soil samples were selected to represent the diversity of mineral soils found in the United States, and 90 samples were selected across Ghana, Kenya, and Nigeria. 519 of the US samples were selected and scanned by Woodwell Climate Research Center. These samples were queried from the USDA NRCS NSSC-KSSL Soil Archives as having a complete set of eight measured properties (TC, OC, TN, CEC, pH, clay, sand, and silt). They were stratified based on the major horizon and taxonomic order, omitting the categories with less than 500 samples. Three percent of each stratum (i.e., a combination of major horizon and taxonomic order) was then randomly selected as the final subset retrieved from KSSL's physical soil archive as 2-mm sieved samples. The remaining 1,604 US samples were queried from the USDA NRCS NSSC-KSSL Soil Archives by the University of Nebraska - Lincoln to meet the following criteria: Lower depth <= 30 cm, pH range 4.0 to 9.5, Organic carbon <10%, Greater than lower detection limits, Actual physical samples available in the archive, Samples collected and analyzed from 2001 onwards, Samples having complete analyses for high-priority properties (Sand, Silt, Clay, CEC, Exchangeable Ca, Exchangeable Mg, Exchangeable K, Exchangeable Na, CaCO3, OC, TN), & MIR scanned.

All samples were scanned dry 2mm sieved. ~20g of sample was added to a plastic weighing boat where the NeoSpectra scanner would be placed down to make direct contact with the soil surface. The scanner was gently moved across the surface of the sample as 6 replicate scans were taken. These replicates were then averaged so that there is one spectra per sample per scanner in the resulting database.

A subset of 1,976 US topsoil samples were used to create Cubist models for 8 soil properties including bulk density (BD, <2mm fraction, 1/3 Bar, units in grams per cubic centimeter), calcium carbonate (CaCO3, <2mm fraction, units in weight percent), clay content (percent), buffered ammonium-acetate exchangeable potassium (Ex. K, units in centimoles of charge per kilogram of soil), pH, sand content (percent), silt content (percent), and soil organic carbon (SOC, estimated after inorganic carbon removal, units in weight percent). Two strategies were evaluated for handling scanner-to-scanner variability: averaging scans per sample (avg) versus retaining replicate scans across all scanners (reps) during model building. Cubist avg models and cubist reps models are provided here for the 8 soil properties outlined in “.qs” file format, and can be opened and worked with in the R programming language. The subset of 1,976 samples has also been provided here for reproducibility (1976_NSlibrary_withmetadata.csv).

The repository contains:



Neospectra_database_column_names.csv: describes the variables (columns) of site and soil data, and the range of NIR and MIR spectra. Both Neospectra_WoodwellKSSL_avg and Neospectra_WoodwellKSSL_reps share the same columns. The CSV is composed of the file name, column name, type, example, and description with measurement unit.

Neospectra_project_summary.txt: the summary of the project with purpose, the origin of soil samples, instrumentation, and brief SOP.

Neospectra_WoodwellKSSL_avg_MIR.csv: the equivalent MIR spectra of neospectra samples' list that was fetched from the KSSL database and formatted to the OSSL specifications.

Neospectra_WoodwellKSSL_avg_soil+site+NIR.csv: soil, site, and Neospectra's NIR. Each row contains the averaged spectra for a given scanner and soil sample (1 spectra per scanner per soil sample). Soil and site info is filled within the same soil sample.

Neospectra_WoodwellKSSL_reps_soil+site+NIR.csv: soil, site, and Neospectra's NIR. Each row contains one replicated spectra of a given scanner (6 repeats per scanner per soil sample). Soil and site info is filled within the same soil sample.

1976_NSlibrary_withmetadata.csv: reproducible matrix for model calibration.

Models:



log..bd_model_nir.neospectra_cubist_AVG_ossl_na_v1.2.qs: Cubist average model for log(1+BD).



log..caco3_model_nir.neospectra_cubist_AVG_ossl_na_v1.2.qs: Cubist average model for log(1+CaCO3).




clay_model_nir.neospectra_cubist_AVG_ossl_na_v1.2.qs: Cubist average model for clay.




log..k.ex_model_nir.neospectra_cubist_AVG_ossl_na_v1.2.qs: Cubist average model for log(1+Ex. K).




ph.h2o_model_nir.neospectra_cubist_AVG_ossl_na_v1.2.qs: Cubist average model for pH.




sand_model_nir.neospectra_cubist_AVG_ossl_na_v1.2.qs: Cubist average model for sand.




silt_model_nir.neospectra_cubist_AVG_ossl_na_v1.2.qs: Cubist average model for silt.




log..soc_model_nir.neospectra_cubist_AVG_ossl_na_v1.2.qs: Cubist average model for log(1+SOC).




log..bd_model_nir.neospectra_cubist_REPS_ossl_na_v1.2.qs:  Cubist replicate model for log(1+BD).




log..caco3_model_nir.neospectra_cubist_REPS_ossl_na_v1.2.qs: Cubist replicate model for log(1+CaCO3).




clay_model_nir.neospectra_cubist_REPS_ossl_na_v1.2.qs: Cubist replicate model for clay.




log..k.ex_model_nir.neospectra_cubist_REPS_ossl_na_v1.2.qs: Cubist replicate model for log(Ex. K).




ph.h2o_model_nir.neospectra_cubist_REPS_ossl_na_v1.2.qs: Cubist replicate model for pH.




sand_model_nir.neospectra_cubist_REPS_ossl_na_v1.2.qs: Cubist replicate model for sand.




silt_model_nir.neospectra_cubist_REPS_ossl_na_v1.2.qs: Cubist replicate model for silt.




log..soc_model_nir.neospectra_cubist_REPS_ossl_na_v1.2.qs: Cubist replicate model for log(1+SOC).
description: We thank the USDA NRCS National Soil Survey Center for providing access to their soil archives and for continuing to promote the use of soil spectroscopy. This project was funded by USDA National Institute of Food and Agriculture Award #2020-67021-32467; USDA National Institute of Food and Agriculture Award #2018-67007-28529; Woodwell Fund for Climate Solutions; and Foodshot Global.