dataesr/scientific-paragraphs-categorization huggingface dataset:
Spikes Spikes are a sudden rise or a fall in pixel values, that may be generated due to instrument malfunction, environment, or imperfection in electronic circuitry. They often mask the detail in an image and lead to inaccurate analysis. One way to deal with spikes is manual supervision, which requires human attention and is time-consuming. Hyperspectral data contain a massive amount of information, so inspecting them manually is a tedious task. Interpolation, smoothing, or removing spikes based on the neighboring pixel  (Behrend, Tarnowski, & Morris, 2002; Zhang & Henson, 2007)  is also good choices. Other options for spike identification include density distribution, median and median modified Wiener filters. Cannistraci, Montevecchi, and Alessio (2009).
Background removal To efficiently process HS images, background removal is used. The non-informative background or outliers in the dataset, caused by the instrument error or radiation, complicates the processing. To clean up and extract the relevant information from the hyperspectral cube, traditional image pre-processing techniques such as histogram detection to detect sharp changes in an image or manual selection of a threshold value can be used  Nawaz and Yan (2020a) ,  Khan, Nawaz, Guoxia, and Yan (2019), and Saeed, Murad, Nawaz, and Irum (2017).
Semi-supervised learning for hyperspectral imaging Due to the high cost and difficulty of obtaining labeled data, semisupervised learning has been widely used in hyperspectral imaging. Semi-supervised learning algorithms in this context use both labeled and unlabeled data to learn more robust and discriminative representations of hyperspectral data. The labeled data is used to train a classifier or perform dimensionality reduction in semi-supervised learning for hyperspectral imaging  (He, Liu, Wang, & Hu, 2017). The unlabeled data is then used to improve the algorithm's performance by incorporating it into the learning process. This is typically accomplished by incorporating a regularization term that encourages the learned representation to be smooth and coherent across the data, with the underlying structure of the data expected to align.
Graph-based semi-supervised learning Graph-based methods have been widely used in hyperspectral imaging for various tasks such as classification, segmentation, and anomaly detection. Semi-supervised learning methods based on graphs have shown promising results for processing hyperspectral data. In graphbased methods, a graph is used to represent the relationships between pixels or spectral bands in the data. These methods can propagate labels or information from labeled data to unlabeled data on the graph, and then optimize the learned representation to be consistent with the propagated information. In  Hong, Gao, et al. (2020) , authors proposed Graph Convolution Networks (GCNs) to classify hyperspectral images. GCNs are a type of neural network that can operate directly on graphs, which makes them well-suited for working with hyperspectral data, as each pixel in a hyperspectral image can be represented as a node in a graph.
The proposed method is divided into two major steps. First, the hyperspectral image is used to create a graph in which each pixel is represented as a node and the edges are defined based on the spatial proximity of the pixels. Second, on the graph, a GCN is trained to classify the image. Overall, the paper presents a novel and effective approach for classifying hyperspectral images using GCNs, which has the potential to improve the accuracy and efficiency of this important application. In  Hong, Yokoya, Chanussot, Xu, and Zhu (2019) , a novel framework for dimensionality reduction of hyperspectral images using semisupervised learning was proposed. The proposed framework entails creating a graph to represent the relationships between the pixels in the hyperspectral image and learning a low-dimensional representation of the data using a multitask regression approach.
Document image analysis Forensic examination of document image analysis is also an exciting venue for imaging spectroscopy. The conventional document imaging system uses pattern recognition techniques for three-channel RGB images. Several important applications of document imaging can benefit from imaging spectroscopy; such as signature segmentation, writer identification, forgery detection, and historical document image analysis. A comprehensive review of hyperspectral document image analysis is given in  Qureshi et al. (2019). Visualizing the spectrum bands at different wavelengths may provide useful insights into challenging document image analysis problems. In addition, combining the discriminative spectral signature with deep neural networks can pave the way for further studies. Deep neural networks require a huge amount of data, however available hyperspectral datasets consist of only a few samples.
Mastectomy Patients with early-stage breast cancer could select between a lumpectomy with radiation or a mastectomy alone. Even though both operations had the same survival rates, re-excision rates in breast lumpectomy were stated to be approximately 40% in  Fisher et al. (2002). Residual tumors, which were not visible to the surgeon during the procedure, were usually identified near the resected material's periphery. As a result, intraoperative tumor assessment is vital for full resection. Kho et al. recently investigated the application of imaging spectroscopy for human breast margin assessment using specimens from 18 patients with a dataset of 22,000 spectra connected with histology using support vector machine  (Kho et al. , 2019). Their study was tested on six lumpectomy specimens. With accuracies of 93%, 84%, 70%, and 99%, invasive carcinoma, ductal carcinoma in situ, connective tissue, and adipose tissue were correctly classified as a tumor or healthy tissue, respectively, demonstrating the potential use of imaging spectroscopy for margin assessment during breast-conserving surgery.
But with the recent upsurge in urbanization rate all around the world, UGS is decreasing. For urban microclimate stability, UGS is an important parameter, and previously many airborne datasets have been used for UGS monitoring but nearly all fail to some extent when differentiating various UGS classes (i. e. , trees from grassland, etc. )  (Chen et al. , 2018). HSI sensors provide the detailed spectral and spatial context of UGS and allow monitoring of UGS types as well. For example, Lv et al. (  2009 ) performed UGS mapping using the Multiple Endmember Spectral Mixture Analysis (MESMA) and Hyperion HSI over the Palo Alto region in the United States. They achieved an overall accuracy of 85% and concluded that MESMA is the best-suited technique for differentiating UGS from heterogeneous urban landscapes.
AeroRIT dataset The AeroRIT hyperspectral dataset  (Rangnekar et al. , 2020 ) is designed to facilitate aerial hyperspectral scene understanding. The dataset includes a radiance-calibrated hyperspectral scene sampled at every 10th band, between 397-1003 nm. The semantic map pixels were labeled with ENVI 4. 8. 2, using individual hyperspectral signatures and the geo-registered RGB images as references. This dataset is a significant step towards developing robust algorithms for hyperspectral airborne sensing that can perform advanced tasks like vehicle tracking and occlusion handling.
ARAD-HS dataset The ARAD-HS dataset  (Arad et al. , 2022) , also known as the ARAD 1 K natural spectral image dataset, is the largest collection of natural hyperspectral images published in June 2022. The dataset contains 950 publicly available train images and 50 test images. The publicly available images have been published as 1-channel hyperspectral images in the 400-700 nm range and 16-channel multi-spectral images in the range of 400-1000 nm.
AVIRIS dataset The Airborne Visible InfraRed Imaging Spectrometer (AVIRIS) dataset is a collection of hyperspectral images in 224 contiguous spectral channels with wavelengths from 400 to 2500 nanometers, developed by National Aeronautics and Space Administration (NASA), USA. The main objective of the AVIRIS project is to identify, measure, and monitor constituents of the Earth's surface and atmosphere based on molecular absorption and particle scattering signatures. Research with AVIRIS data is predominantly focused on understanding processes related to the global environment and climate change. The AVIRIS dataset is available for download from the AVIRIS Data Portal and the AVIRIS-NG Data Portal. The data can be used for a wide range of applications, including environmental studies, and climate change research.
Chikusei dataset The Chikusei Dataset  (Yokoya & Iwasaki, 2016 ) is an airborne hyperspectral dataset, captured by the Headwall Hyperspec-VNIR-C imaging sensor over agricultural and urban areas in Chikusei, Ibaraki, Japan. Ground truth of 19 classes was collected via a field survey and visual inspection using high-resolution color images. The dataset has been used for various research purposes, including image super-resolution and image classification.
ICVL hyperspectral database The ICVL database  (Arad & Ben-Shahar, 2016)  includes a variety of scene images  (Arad & Ben-Shahar, 2016). A Specim PS Kappa DX4 hyperspectral camera and a rotating stage for spatial scanning were used to capture the images. Images were obtained with a spatial resolution of 1392 × 1300 over 519 spectral bands (400-1000 nm at roughly 1. 25 nm increments). The (. raw) files include raw ENVI data from the camera, whereas the (. hdr) files contain the headers needed to decode it. The database contains several datasets, including Bostwana (Hyperspectral Remote Sensing Scenes, 2020), acquired by the NASA EO-1 satellite over the Okavango Delta. Botswana contains 200 images, and it is used for hyperspectral image reconstruction from RGB images.
Spectral laboratory The Spectral Laboratory at the University of Eastern Finland has collected a variety of spectral datasets Spectral Color Research Group, including a dataset for spectral images related to oral and dental health, spectral images of Nordic sawn timbers (Garcia Peraza Herrera,  Horgan, Ourselin, Ebner, & Vercauteren, 2023) , hyperspectral images of textile  (Porebski, Alimoussa, & Vandenbroucke, 2022) , and religious icons  (Mirhashemi, 2019). These datasets can be used for research in medical, material science, and texture analysis.
HyTexiLa The Hyperspectral Texture images acquired in Laboratory (HyTex-iLa) dataset provides high spectral and spatial resolution reflectance images of 112 materials to study spatial and spectral textures. The data is in the range from 405. 37 nm to 995. 83 nm at 3. 19 nm intervals and 186 channels. The dataset consists of high-resolution images of food, stone, vegetation, and wood categories.
SpecTex HyperSpectral Texture Image Database SpecTex is a database of spectral images that includes sixty textile samples with various texture patterns, developed for spectral texture analysis. The image cubes are 640 × 640 × 39 in dimension, and the spectral data is captured in visible wavelengths ranging from 400 nm to 780 nm at 5 nm intervals  (Hirvonen et al. , 2014).
Hyperspectral datasets for forensic analysis iVision HHD  (Islam et al. , 2022)  is a dataset of handwritten document images, developed by the Institute of Space Technology, Islam-abad, Pakistan, and Wageningen University & Research, the Netherlands. The dataset contains handwriting samples from 54 subjects. The individuals belong to different age groups, and professions, and have different writing patterns. Each hyperspectral cube in the collection has a spatial resolution of 512 × 650 pixels and 149 spectral channels in the 478-901 nm spectral region. This dataset can be used for forensic examination of document images, including, writer identification, handwritten optical character recognition, and ink mismatch detection. Other hyperspectral document image datasets include  (Kim et al. , 2010)  for historical document image analysis, and Borba, Jawhari, Honorato, and de Juan (2017) for forensic analysis of documents.
Hyperspectral medical images datasets Several hyperspectral medical image datasets have been developed to facilitate research in this area: • Hyperspectral Brain Dataset: This public database contains invivo hyperspectral human brain images. It was created for research in brain cancer detection, and it contains 36 images in the range of 400-1100 nm, with a spectral resolution of 2-3 nm  (Fabelo et al. , 2019). • Hyperspectral Dermoscopy Dataset:This dataset  (Gu et al. , 2018)  provides hyperspectral images for early screening of skin cancers using image-based approaches. It contains 330 images, in the range of 465-630 nm, containing 16 spectral channels, with a spatial resolution of 512 × 512.
ISET hyperspectral database This is a set of ISET hyperspectral scenes created from data acquired using a hyperspectral imaging system (Hyspex VNIR-1600) that detects image radiance in narrow bands spanning from 400 to 1000 nm  (Xiao et al. , 2009). There are four main types of categories in the ISET hyperspectral database with a different number of scenes. A MATLAB toolbox for processing hyperspectral data hsdar  (Lehnert et al. , 2018)  An R package for processing, feature extraction and classification of hyperspectral data hsdar HSImage  (Brown & Moser, 2018)  A Python and C++ library for interactive ENVI-BIL hyperspectral images HSImage HybridSN  (Roy, Krishna, Dubey, & Chaudhuri, 2019)  A spatio-spectral method for HSI classification using Pytorch HybridSN Hyperspectral  (Vali, Comai, & Matteucci, 2020)  Deep Learning for Land-cover Classification in Hyperspectral Images Hyperspectral HSIToolbox  (Dhaene, Žižakić, Huang, Li, & Pižurica, 2023)  A web-based application for the classification of hyperspectral images HSIToolbox HSI-Tradition-to-Deep  (Ahmad et al.
, 2021)  A Github repo of traditional to Deep learning models for HSI classification HSI-Traditional-to-Deep Scyllarus  (Habili & Oorloff, 2015)  A C++ API for processing of hyperspectral images SCYLLARUS spectrai  (Horgan & Bergholt, 2021)  A PyTorch based framework for processing spectral data spectrai Specim  (Behmann et al. , 2018)  A commercial software for hyperspectral imaging Specim There are hyperspectral datasets for other applications as well, such as face recognition  (Di et al. , 2010) , wood analysis  (Hirvonen et al. , 2014) , and Rosis-3  (Ghamisi et al. , 2017)  for urban data classification. However, to fully exploit the potential of imaging spectroscopy in various applications, large open-source hyperspectral databases are still needed.
AI software tools for HSI datasets In the last two decades, several artificial intelligence (AI) based tools are developed to process hyperspectral images as discussed in Table  4. Scyllarus is a set of software tools for processing hyperspectral data and producing useful results  (Habili & Oorloff, 2015). Scyllarus offers customers usable data for a variety of applications by utilizing sophisticated and advanced image processing algorithms developed at Data61. Whatever sector you operate in, there is a good chance that imaging spectroscopy can help you. Scyllarus added three frequently used AI tools to the hyperspectral image processing procedure.
Scyven (scyllarus visualization environment) Scyllarus is the founder of this hyperspectral visualization tool. Scyven (Scyllarus Visualization Environment) helps us to examine Hyperspectral images and analyze them to find spectral structures. Scyven is a simple graphical user interface with the tremendous capability of the Scyllarus C++ API. Scyven can be used to find facts about sceneries that the naked eye or typical RGB images could not have. It can increase the use of Hyperspectral image data by using powerful processing techniques. Scyven is compatible with both Windows and Linux, and it does not require any particular hardware to handle data. You can collect data in other places and then import it into Scyven. Automatic Illuminant Recovery, Reflectance Recovery, and Material Discovery are among the features.
It uses SAM (Spectral Angle Mapping), SVM (Support Vector Machines), LSU (Linear Spectral Unmixing), and PCA (Principal Component Analysis) techniques for Spectral Library Classification. The Scyllarus MATLAB Toolbox was created to help researchers with hyperspectral and multispectral image processing techniques. Scyllarus is implemented inside this Toolbox separately from the C++ API. It is more comprehensive and includes features that are not accessible in the C++ API, such as, Deterministic annealing, K-means, Image reformatting, and quantization techniques. It represents spectral data with smooth functions based on Gaussian functions, such as NURBS (Non-uniform Rational B-Splines).
Conclusion and future work In this paper, we presented a thorough review of the contemporary landscape of imaging spectroscopy, placing particular emphasis on hyperspectral and multispectral image acquisition, processing, and analysis. Our comprehensive exploration covered various critical aspects of the field, including applications, pre-processing techniques, the availability of databases, and the ever-evolving landscape of software tools and image processing frameworks for analyzing these images in recent years. We addressed the techniques and challenges involved in the acquisition, processing, and analysis of hyperspectral images across a wide range of application areas, including medicine, food and agriculture, remote sensing of environment, minerals extraction, biometrics, and document analysis.
Therefore, one potential area requiring special attention is how to reduce the amount of raw imaging spectroscopic data during acquisition and/or storage while retaining high-quality spectral characteristics. The insufficient availability of publicly accessible labeled imaging spectroscopic data and the difficulty of maintaining such large databases presents another key challenge. Therefore, research in the area of developing large public imaging spectroscopic datasets presents a great opportunity and holds significant value. A related challenge to the limited availability of public databases is the lack of open-source imaging spectroscopy software tools and packages, which hinders the full potential growth of this field. Expanding these resources could lead to more valuable solutions and opportunities for researchers and industry professionals.
Therefore, future research in the area of developing large imaging spectroscopic datasets presents a promising opportunity and holds significant value in terms of contribution to the knowledge base. Developing novel algorithms for near/real-time processing and interpretation of imaging spectroscopy data are other significant challenges, offering tremendous research prospects that can enhance the utilization of this technology for the greater good. For instance, imaging spectroscopy coupled with advanced computer vision algorithms can offer potential solutions for numerous industrial applications, including but not limited to food processing, recycling, mining, and exploration industries. Similarly, when combined with powerful deep learning frameworks capable of processing vast amounts of data for enhanced accuracy, we anticipate that imaging spectroscopy will assume an increasingly dominant and influential role in various airborne and ground-based research and industrial applications.
, 2014) Historical documents Historical document Analysis - 365-1100 nm 70+ 256 × 256 Classification, detection, and segmentation (Kim, Zhuo, Deng, Fu, & Brown, 2010) HyTexiLa (Khan, Texture Analysis and Material 112 405-995 nm 186 1800 × 2000 Classification and identification Mihoubi, Mathon, Identification Thomas, & Hardeberg, 2018) Hyperspectral Human Brain cancer detection 36 400-1000 nm - 2-3 nm Classification and detection Brain Image Database (Fabelo et al. , 2019) Hyperspectral Melanoma detection 330 465-630 nm 16 512 × 272 pixels Classification and detection Dermoscopy Dataset (Gu, Partridge, & Zhou, 2018) ICVL (Arad & Environment 203 1000-1300 nm 31 1392 × 1300 Classification Ben-Shahar, 2016) iVision HHID (Islam Document forensic analysis 270 478-901 nm 149 512 × 650 pixel Classification and recognition et al.
Material To investigate the bistable behavior of the domes both in experiments and simulations, the chosen constituent material is a rubber-like photocurable resin that goes under the commercial name TangoBlack FLX 930 (Stratasys, France). To date, such polymer material has been used to produce different types of highly deformable structures such as porous Voronoi-type materials  [56]  and hierarchical beams  [8]. Tan-goBlack FLX 930 has a hyperelastic behavior and its constitutive law has been modeled using the Neo-Hookean model  [57] , implemented in Abaqus FE commercial software  [58]. This model is described using a strain energy potential function that depends on both the deviatoric and volumetric component of the strain invariants, i. e. : 𝑊 = 𝐶 10 (𝐼 1 -3) + 1 𝐷 1 (𝐽 𝑒𝑙 -1) 2 (2) where 𝐼 1 is the first strain invariant and 𝐽 𝑒𝑙 is the elastic volume strain which accounts for the total and thermal volume strains.
Acknowledgments This work has benefited from the financial support of the  LabEx LaSIPS  ( ANR-10-LABX-0032-LaSIPS ) managed by the  French National Research Agency  under the '' Investissements d'avenir'' program  ( ANR-11-IDEX-0003 ). The authors would also like to acknowledge  Zahra Hooshmand-Ahoor  ( LMS, Ecole Polytechnique ) for her help with samples manufacturing,  Xavier Dassonville  and  Frédéric Lesage  ( La Fabrique, CentraleSupélec ) for their help with the experimental set-up manufacturing and  Yves Bernard  ( GeePs, CentraleSupélec ) for providing the laser measurement system.
Experimental settings The computational experiments are conducted on a CentOS Linux machine with 8 GB of RAM and an Intel Xeon CPU E5-2680 v4 processor at 2. 40 GHz. All the linear and mixed-integer linear programs are addressed with the state-of-the-art solver IBM CPLEX 12. 10, for which a single thread is used. The algorithms are implemented in C++, and CPLEX is called through the Concert Technology. A testbed of 525 benchmark instances 1  from Otto et al. (  2013 ) is considered, each one having 100 tasks. The instances are divided into 21 groups based on the probability distribution used for generating task processing times, the structure of the precedence graph, and the order strength (OS). The latter is a density indicator of precedence graphs computed as 2 • | A | / (n • (n -1)). Each group contains 25 instances, and their characteristics are presented in Table  2.
62  More details on these calculations are provided in the Methods section. We numerically verified the consistency between the two methods at low frequencies in all systems (see SI sec. S4 and S5). In Figure  2 , we compare our simulations with experimental results obtained with different spectroscopic techniques. [56] [57] [58] [59]  The experimental datasets were converted from permittivity to conductivity using the relation  61, 63   σ(ω) = iωε 0 [ε(ω) -1]. The simulation results for the 1 M system reported in Fig. 2 (a) show that σ(ω) reaches a global maximum at ω ≈ 1. 5×10 14 rad/s corresponding to the water librational band. 64  At lower frequencies, we find a shoulder resembling the Debye-like modes of polar fluids and a plateau corresponding to the static conductivity σ(0). The same panel also shows a semi-quantitative agreement between simulations and experiments, across the whole frequency range.