dataesr/scientific-paragraphs-categorization huggingface dataset:
Approximately 6. 7 and 2. 5 million individuals were diagnosed with fungal skin diseases and recurrent VVC, respectively. Compared with the findings of Foxman et al. and Ruhnke et al. [2, 7] , VVC prevalence was much lower in our study. However, there are major differences that could explain this discrepancy. First, the work of Foxman et al. was based on an online survey consisting of questions on a variety of topics, which was given to more than 6,000 women aged 16-65 years  [2]  epidemiology studies published between 1975 and 2013, which reported fungal infection rates in Germany  [7]. By contrast, we used data obtained from gynecological practices. Second, the definition of VVC varied between the three studies, as the authors of the first study looked at a past history of VVC  [2] , the authors of the second study investigated the diagnosis of recurrent VVC (at least four episodes in one year)  [7] , and we focused on any VVC episode diagnosed between 2014 and 2016.
Image Analysis Tracer concentration can be estimated from the intensity of light transmitted through the sample containing dyed fluid. Videos were recorded of each experiment and light intensities were converted into concentration values for comparison with the numerical simulation results. Greyscale intensities were used for simplicity. Normalised intensity was found by dividing each video frame by the background image, which was taken from the first frame of the video before solute was present. This removed the effect of any uneven illumination and eliminated background features continuously present in the images. A Wiener filter was applied to reduce random noise in the images. The MATLAB function wiener2 (MathWorks 2018) was used with a 20 by 20 pixel neighbourhood. A mask was also applied to mask out areas of the sample not containing flow channels.
Acknowledgements We thank  Energi Simulation  for supporting the chairs held by  Sebastian Geiger  and  Eric Mackay. We thank  EPSRC for supporting Julien Maes' position (Grant Number   EP/P031307/1 ). We are grateful to  Mohamed Arab  and  Jim Buckmann  for help with the collection and interpretation of the SEM data. Open Access This article is distributed under the terms of the Creative Commons Attribution 4. 0 International License (http://creativecommons. org/licenses/by/4. 0/),which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Acknowledgements We gratefully acknowledge the support of several colleagues who contributed to this study, including  Ren Sophorn ,  Polin Sroy ,  Meas Rith ,  Myrthe Pareyn ,  Nimol Khim  and  Mathilda Mushiete  for assisting with data collection and/or blood sample processing. This work was funded through the  Department of Economy, Science and Innovation of the Flemish government. We acknowledge the support of the  Belgian Co-operation (Directorate-General for Development Co-operation)  for the Institutional collaboration between CNM Cambodia and the  Institute of Tropical Medicine, Belgium. MBT received doctoral research funding through a  2014 Erasmus Mundus Joint Doctorate Fellowship, Specific Grant Agreement   2014-0681 , and through a  Les Amis des Instituts Pasteur à Bruxelles 2017 Research Fellowship.
We present a dataset of close range hyperspectral images of materials that span the visible and near infrared spectrums: HyTexiLa (Hyperspectral Texture images acquired in Laboratory). The data is intended to provide high spectral and spatial resolution reflectance images of 112 materials to study spatial and spectral textures. In this paper we discuss the calibration of the data and the method for addressing the distortions during image acquisition. We provide a spectral analysis based on non-negative matrix factorization to quantify the spectral complexity of the samples and extend local binary pattern operators to the hyperspectral texture analysis. The results demonstrate that although the spectral complexity of each of the textures is generally low, increasing the number of bands permits better texture classification, with the opponent band local binary pattern feature giving the best performance.
The analysis of hyperspectral images is performed for image data exploration, classification and quantification tasks. There are various multivariate analysis techniques that are applied to hyperspectral images for data decomposition, pre-processing and performing regression or classification analyses. Some of the methods for hyperspectral data analysis include the principal component analysis  [5] , linear discriminant analysis  [6] , partial least squares discriminant analysis  [7] , K-nearest neighbor algorithm  [8] , support vector machine  [9] , least-squares support vector machine  [10] , artificial neural network  [11] , deep recurrent neural network  [12]  and deep convolutional neural networks  [13]. Hyperspectral imaging is widely used in a number of applications, e. g. , remote sensing  [14] , medical diagnosis  [15] , cultural heritage  [16, 17] , face recognition  [18] , food quality control  [19] , color management and printing  [20, 21].
A number of applications have been developed through the multivariate analysis of hyperspectral images, including the investigation of dried blood spots  [22] , the measurement of oil and protein contents in peanut varieties  [23] , the classification of muscle foods  [24] , the detection of defects on peaches  [25]  and numerous other applications. Hyperspectral images contain more spectral information compared to color images. The high dimensionality of hyperspectral image data is an open challenge, and a trade-off between high information content and practical handling is often required. To evaluate this compromise and assess its performance, high quality data with high spatial and high spectral resolutions are required. Creating such a dataset takes time and effort, but it is necessary to the research community.
Considering these needs, we present a hyperspectral image dataset of 112 textured objects falling into five different categories. Images are reflectance data that span the visible and near infrared (NIR) parts of the electromagnetic spectrum. The data is referred to as HyTexiLa (Hyperspectral Texture images acquired in Laboratory). The idea behind creating this dataset is to provide a platform for the benchmark analysis of various applications and processing. The areas where this data could be used are in the fields of image processing, computational imaging and computer vision, such as surface identification, spatio-spectral analysis of textured surfaces, image sensor simulation, color reproduction, image relighting and so on. The availability of a high spectral and spatial resolution dataset will provide easy access for the evaluation of different techniques and will allow the results to be compared.
In this paper, we present that dataset and analyze its spatial and spectral properties. We define the image acquisition protocol, the distortions that occur during the acquisition of the objects, the method for the correction of such distortions and the effect of such corrections. Our focus in this paper is to present a hyperspectral dataset to the community and to provide an analysis on our dataset as a benchmark for further research. We provide the methodology for pre-processing the raw hyperspectral data before using it for the intended applications. This paper is organized as follows: First we analyze the existing hyperspectral datasets and then, in Section 3, we discuss the hyperspectral image acquisition protocol and the processing of spectral data for addressing the distortions that occur during the acquisition.
Comparison of HyTexiLa with Existing Hyperspectral Datasets Several hyperspectral image datasets are publicly available. In this section, we provide an overview of the existing hyperspectral image datasets that consist of close range images, including natural scenes and indoor images of objects. In this section, we provide a brief review of existing hyperspectral image datasets and compare them with the properties of our proposed dataset. We do not include the remote sensing datasets in this section, because remote sensing does not assume the same imaging model. We also exclude reflectance datasets obtained from pointwise spectrometers because they will not contain any spatial relationship between spectra as well as those multispectral image datasets which contain less than 10 channels. Table  1  gives an overview of the existing hyperspectral image datasets.
The datasets in  [26] [27] [28] [29] [30] [31] [32] [33] [34]  consist of outdoor scenes which include vegetation and buildings. The wavelength range in  [26, 30, [34] [35] [36] [37] [38]  is from 400 nm to 700 nm, while the datasets  [27, 30, 31]  have spectral ranges from 400 nm to 720 nm. The spectral sampling in  [26, 27, [29] [30] [31] [32] [33] 35, 36, 39 ] is 10 nm. The dataset in  [28, 32, 40, 41]  has a spectral range of 400 nm to 1000 nm. The datasets in  [35, 36]  consist of images of objects taken indoors. There are some datasets which consist of a specific category of objects, like paintings  [42] , leaves from apple tree  [43] , textiles  [39] , honey  [41]  and wood samples  [44]. Most of the available datasets are sampled or re-sampled at 10 nm intervals, while some of them have different sampling intervals, e.
g. ,  [28]  has 1. 25 nm sampling step, while datasets in  [41, 43]  have spectral sampling intervals of 6 nm and 4. 7 nm, respectively. In this paper, we provide a new dataset of hyperspectral images that has been made publicly available. This dataset consists of samples from five different categories of materials. The spectral range of this dataset is from 405. 37 nm to 995. 83 nm. As identified in Section 2, there are only a few datasets that span both the visible and NIR regions. The information in NIR is valuable for material classification  [45] [46] [47] , and the identification of textile fibers  [48]  and minerals  [49]. Although some of the hyperspectral datasets provide information in the NIR region, they are either for specific samples, as in  [41] [42] [43] , or consist of outdoor scenes with many objects, as in  [28, 32].
Each image in our dataset consists of one specific, rather flat, object. The spectral resolution of data is also an important aspect to be considered. In most of the hyperspectral image datasets mentioned in Table  1 , the spectral sampling interval is 10 nm, which is considered enough to representat the spectra  [50]. A spectral sampling of 5 nm was recommended for Munsell spectra in  [50] , but it is also worth noting that more closely sampled data may be required when spectra are less smooth. One of such cases is the presence of spiky illumination spectral power distributions, where the use of more accurate spectral intervals is required. The availability of high resolution spectral and spatial data will allow the development and testing of advanced resampling methods. There is an existing dataset for textile samples  [39]  but it contains spectral information in the visible region and the sampling has intervals of 5 nm between the bands.
Some datasets consist of food items  [41, 45]  but do not provide high spatial resolution. Similarly, the datasets containing images of urban and natural scenes do not provide high spatial resolution for each material. The CAVE dataset  [35]  contains detailed images of various materials that are captured from close range, but they are only in the visible region. Our dataset was spectrally sampled at about 3. 19 nm and can be used with simulations of most fluorescent tube and light emitting diode-based illumination sources. The advantage of our dataset when compared with already available datasets is the availability of high resolution spectral and spatial data. The spectral range in our dataset covers both the visible and NIR regions. Each image in the dataset contains a unique material, and it can be used as ground-truth spectral data for surface identification applications.
Objects in the Dataset The different objects chosen for the dataset were divided into five classes: textile, wood, vegetation, food and stone. These samples are a set of textured object surfaces that exhibit different properties while remaining fairly flat. The imaged objects were reasonably flat so that the blurring effect due to distance variations from camera was minimal during the scanning process. After the completion of image acquisition, we analyzed each image thoroughly and discarded a few samples which were too blurry. We also excluded materials that exhibited high gloss in the considered viewing conditions to maintain a simple imaging model, as in Equation (  1 ), which allows easy computation of reflectance values: ∀k ∈ [K], ∀(x, y) ∈ [M] × [N], I k (x, y) = R k (x, y) • S(λ k ).
(1) Finally, for this dataset we provide 65 textile samples of various types and colors. They were selected among textile samples provided by the humanitarian association Don De Soie located in Templeuve in France. In the wood category, there are 18 different samples of wood that can be found in the area around Mjøsa lake in Norway. There are 15 images for the vegetation category, which consist of leaves, flowers and one mushroom. The food category consists of 10 samples of tea, coffee and spices, and there are four samples in the stone category. Image labels include the material category, the acquisition sequence number and a specific name if the material has been precisely identified. For the textile, wood, and vegetation samples, the labels also include the color of the material (suffix _colorname if there are at least two samples of the same type but with different colors) and/or the acquisition side (suffix _back if the back of the material has been acquired together with the front).
( 10 ) Reflectance images are cropped to 1024 × 1024 pixels which corresponds to the minimum of the most relevant regions of interest (ROIs) of the 112 selected textures (ROIs that really contain texture elements). Figure  5  shows a sRGB rendering of two radiance images. For each radiance image, I, the subimage, P, selected for illumination estimation is displayed as a red dashed square, and the texture region that we retain as the reflectance image support, R, is displayed as a green solid square. Note that pixel values that have undergone specular reflection of the light in the radiance image can have values that are higher than the SG-3051 tile and produce values over one in the reflectance image. We decided to keep them unchanged in the final dataset so that the original corrected radiance image can be retrieved by multiplication with the original illumination spectra (which is provided in the metadata of the reflectance image file, see Section 3.
Dataset Description The dataset is available on the websites of the team/laboratory of the authors (http://color. univlille. fr/datasets and https://www. ntnu. edu/web/colourlab/software). Each one of the 112 texture images is provided as both (i) a hyperspectral reflectance image in ENVI format and (ii) a sRGB reconstructed image in PPM format. (i) The ENVI format consists of two files: a binary file (extension. raw) of single-precision floating-point variables (32-bit) that stores the raw hyperspectral reflectance image in the BSQ (Band SeQuential) data organization (channel by channel) and a text file (extension. hdr) which contains the metadata of the image. The. hdr text file holds:.
There are different conclusions by various authors for the number of effective dimensions that can represent the spectral information. As an example, for the Munsell reflectance dataset, different authors have provided different dimensions that effectively represent it, ranging from 3 to 18 dimensional representations  [66]. The required dimensionality of data mainly depends on the statistical properties of the spectra being considered and the information that one wants to preserve or emphasize. A review of the work done for finding the dimension of spectral data is provided in  [66].
Spectral Analysis of the Proposed Dataset In this section we investigate the effective spectral dimension of the acquired hyperspectral images. We do not assume any a priori knowledge about the data and provide a benchmark analysis in the spectral domain. Having these goals in mind, we used the principal component analysis (PCA) technique, which is mainly used to reduce the dimensionality of data consisting of a large number of inter-related variables, while retaining the variance in data  [61]. PCA has the ability to de-correlate the spectral data and provide variance information through simple computational methods. This ability makes it one of the most popular techniques for the computation of the effective spectral dimension in hyperspectral imaging  [67]. For the computation of effective spectral dimensions, each image is processed individually.
For an effective representation of the spectral data, we select only the first few eigenvalues, and the projection of spectra on the corresponding eigenvectors is able to provide adequate representation of the acquired data. This means that if we select the principal components containing 95% variance, the rest of principal components can be discarded with a loss of 5% of information. The decision to keep a certain percentage of the variance depends on the application for which a reduced set of representative data is required. For this analysis, we computed the effective dimension for each hyperspectral image which retained 99% and 95% of the total variance  [66]. Figure  11a  shows the number of components required to retain 99% of variance, and Figure  12  gives information about the effective spectral dimensions of the whole dataset and also about each of the five categories of materials.
If the desired value of retained variance is reduced to 95%, then most of the images in our dataset can be defined with one or two principal components, while six images from the "textile" require a third component. Figure  12b  represents the number of images that require a certain number of dimensions to retain 95% of the variance. From these figures, it can be observed that the number of effective dimensions is dependent on the nature of the materials being imaged and may be used as a feature to estimate the category of imaged object. The reason why some of the images require a higher dimension for the description of their spectral properties is due to the presence of anomalies, for example, a tiny part with specular reflection which causes the effective dimension to rise. Figure  11  shows that by reducing the threshold for retained variance, the number of dimensions required to represent the spectral data is reduced.
Interpretation of the Effective Dimension The effective dimension analysis of our dataset on the basis of PCA, as described in Section 4. 1, provided a number indicating the number of dimensions required to retain a certain percentage of variance. However, it did not provide a physical meaning to the retained spectra since the principal components contain both positive and negative values. In order to obtain the information about the retained spectra, we used the non-negative matrix factorization (NMF)  [68] -based decomposition on the hyperspectral images in our dataset. NMF is used in remote sensing hyperspectral images for spectral unmixing of the acquired data  [69, 70]. The advantage of NMF over PCA is that it extracts sparse and meaningful features from the given data, and the non-negativity constraint ensures the output is interpretable in terms of its physical meaning.
NMF is able to compute the spectral signature of end-members in a given hyperspectral image and the presence of each end-member in each pixel in the image in the form of abundance maps. The assumption behind this computation is the linear mixing model which states that the spectral signature of each pixel is a linear combination of the spectral signature of end members at a particular spatial location. In NMF-based decomposition, the number of components to be retained is provided by the user, which is mostly based on the prior knowledge of the end-members. For our dataset, we used PCA to compute the required number of components. For the illustration of NMF based decomposition results, we provide two examples in Figures  13  and 14 , containing samples of a leaf and textile, respectively.
There is a third component in this sample which corresponds to a line in the abundance map (Figure  14h ) and is probably caused by some component in the textile which is not easily visible in a color image. Although we have provided only two examples from the NMF-based decomposition of the spectral data, the results indicate that such an analysis can provide interesting information about the spectral composition of the materials present in an image. For example, the amount of chlorophyll in vegetation, information about pigments and dyes in a textile, and the quality of food materials on the basis of their spectral analysis. We used PCA to get information about the spectral dimension before decomposing it through NMF. From the comparison of spectral behavior and the effective dimension, it can be seen that effective spectral dimension of most of the objects in our dataset is within two to eight.
However, this dimension should not be directly related to the required number of spectral channels needed for acquisition. Although there are some studies which have aimed to find the number of filters that can best be used to acquire the maximum spectral information  [71] , there is no agreement on the optimum number of channels since it is dependent on the nature of surface spectral properties. Nevertheless, knowledge of the effective spectral dimension is useful for dimensionality reduction and obtaining the components of spectra by decomposing it. In the next section, we study the impact of dimensionality reduction on classification performances on the hyperspectral dataset using Local Binary Pattern (LBP)-based descriptors.
Due to its discrimination power and computational efficiency, LBP has become a popular approach for various applications  [72]. LBP-based texture classification was first performed on gray-level images, since the original operator only used the spatial information of texture  [73]. Later, Palm  [74]  showed that classification based on a color analysis outperforms the single use of intensity spatial information. Thus, texture feature extraction is extended to the color domain by taking both spatial and color textural information into account. Here, we propose to extend the color LBP-based descriptors to any K-channel radiance image, I. Then, we propose to study the discriminative power of these descriptors on our dataset with respect to an increasing number of channels.
( 15 ) The final texture feature is the concatenation of the histogram of LBP[I PPI ] and the histogram of LCC[I], and its size is 2 • 2 P. • Luminance-Opponent Band Angles LBP (L-OBALBP): As for L-LCCLBP, this approach first applies the LBP operator to the PPI I PPI , and then Lee et al. [79]  considers the angle between each pair of bands, (k, l) ∈ [K] 2 , k = l, as OBALBP[I] (k,l) (p) = ∑ q∈N P,d s arctan I k q I l q + η , arctan I k p I l p + η • 2 (q) , ( 16 ) where η is a constant of small value. The final texture feature is the concatenation of the histogram of LBP[I PPI ] and the K(K -1) histograms of {OBALBP (k,l) [I]} k =l∈[K]. Its size is (1 + K(K -1)) • 2 P. 5. 2. Assessment on Our Proposed Dataset.