dataesr/scientific-paragraphs-categorization huggingface dataset:
Some datasets are annotated with only bounding boxes (UECFOOD100  [13] , UECFOOD256  [14] ), while others include polygon or mask annotations (MyFood Dataset  [15] , FoodSeg103  [16] , UECFoodPixComplete  [17] ). There are four main methods for collecting images for food image datasets. First, images can be captured in a standardized laboratory environment (eg. UNIMIB2016  [18] ), which ensures high resolution and good quality images. However, this method typically limits the number of collected images. Second, images can be downloaded from the internet, either from social networks  [19]  or search engines  [15]. This approach allows collection of large number of images, but can also result in a large number of non-food images that need to be sorted. Downloaded images may vary in quality, including blurry images, images with text, low-resolution images, or retouched images.
Third, images can be collected directly from users  [20] , which provides a realistic representation of reallife scenarios. However, implementing this method can be challenging, as it requires a large number of users and an extended period to collect a substantial amount of images. Finally, some datasets are build with images from other existing datasets. For instance UECFoodPixComplete  [17]  dataset was build by annotating UECFOOD100  [13]  images. Likewise, Food201-Segmented is made of Food-101  [21]  segmented images (see Table. 1). The Table. 1 lists, at the present stage of our investigation, the only publicly available food image datasets for detection and segmentation tasks. Theses datasets are classified based on their main characteristics, such as their usage, number of classes, total number of images, method of image collection, and the origin of the dishes represented in the images.
Chiang et al. [36]  proposed a model based on Mask Region-based Convolutional Neural Network (Mask-RCNN) with a union post-processing technique. In 2021, Wu et al. [16]  proposed a semantic segmentation method consisting of a recipe learning (ReLeM) and an image segmentation modules. They used a Long short-term memory (LSTM) network as an encoder, a vision transformer architecture as an decoder and they achieved 43,9% mIoU with their dataset FoodSeg103. Okamoto and Yanai  [17]  used DeepLabv3+ model on their UECFoodPixComplete dataset and obtained 55. 5% of mIoU. Liang et al. [20]  introduced a model called ChineseFoodSeg to address challenges specific to Chinese food images, such as blurred outlines, rich colors, and varied appearances. Their model outperformed DeepLabv3+, U-Net, and Mask-RCNN on the ChinesseDia-betesFood187 dataset, achieving an accuracy of 94% and mIoU of 79%.
However, their proposed method is more complex and less time-efficient compared to DeepLabV3+. Sharma et al. [37]  proposed a novel architecture named Gourmet-Net which incorporates both channel and spatial attention informations in an expanded multi-scale feature representation using advanced Waterfall Atrous Spatial Pooling (WASPv2)  [38]  module with channel and spatial attention mechanisms. GourmetNet achieves state-of-the-art performance on the UNIMIB2016 and UECFood-Pix datasets. Achieving on these datasets a mIoU of 71. 79% and 65. 13% respectively. A more recent work,  [39]  proposed a Bayesian version of DeepLabv3+ and GourmetNet  [37]  to perform multi-class segmentation of foods. It is worth noting that the quality of the image dataset plays a significant role in the performance of these models.
CamerFood10 dataset The process of building our dataset began by compiling a list of the 42 most consumed foods in Cameroon  [40]. To gather a large number of images, a Python script was developed to automatically scrape and download images from search engines (such as Google Image and Bing). We also manually downloaded images from some social media platforms (including Instagram, Twitter  and Facebook). This initial collection resulted in approximately 50,000 images. However, collected images contain a lot of noise, duplicated images, blurred images, low resolution images, small size images and irrelevant classes images. The next step was to clean up the data. For this purpose, a number of tasks were carried out. Firstly, images with a height or width less than 400 pixels were removed as they typically lacked sufficient useful information.
Duplicate and visually similar images were also eliminated to avoid redundancy. Additionally, non-food related images were filtered out, focusing solely on foodrelated content. Blurred and low quality images were also removed. After that, we ended up with 3,067 images. The VGG Image Annotator (VIA) tool  [41]  was then used to annotate the images with polygon annotations. CamerFood10 dataset, was obtained after further refinement as removing images of classes with very few occurrences. The final dataset consists of 1,241 images and 1,513 annotated food items. Our dataset is divided into a training set of 1,032 images and a validation set with 209 images. Fig. 3  illustrates the distribution of images in the CamerFood10 dataset, while Fig. 2  show some samples images of CamerFood10 dataset.
Backbone and features extraction Inspired by  [42]  and its works on road boundaries estimation, we employ reduced ResNet50  [9]  network (without the 4th block) for features extraction. We have found that using a backbone model with a small depth yields better results with our dataset. We present in Fig. 5  the backbone architecture with the different extracted features layers. In our proposed mid-DeepLabv3+ model architecture, the shapes of the low, middle, and deep layers are (128 × 128 × 64), (64 × 64 × 128), and (32 × 32 × 256), respectively. In the ResNet50  [9]  model, these layers correspond to conv2 block3 2 relu, conv3 block4 2 relu, and conv4 block6 2 relu. By incorporating the middle layer, we effectively introduce additional levels of detail and context to improve the segmentation performance of our model.
Encoder The encoder path is the same as DeepLabv3+  [8]. The depth-level features layer (conv4 block6 2 relu) extracted from the backbone is passed trough a Atrous Spatial Pyramid Pooling (ASPP)  [43]  module. ASPP captures multi-scale information by incorporating atrous convolutions at different rates. It includes one 1x1 convolution, three convolutions with 3x3 kernel size (with dilation rates  (6, 12, 18)  respectively) and the image-level features with image average pooling. As shown in Fig. 4 , the resulting features from the five branches are concatenated together and passed through a 1x1 convolution. As DeepLabv3+  [8] , we used Atrous Separable Convolution instead of traditional 3x3 convolutions in the Encoder an Decoder paths. This technique factorizes a standard convolution into a depthwise convolution followed by a pointwise convolution (i.
Attention module In our mid-DeepLabv3+ proposed model we further add attention mechanism after each features extraction layer. Attention mechanisms have proven to be effective in computer vision, allowing models to focus on relevant parts of the input by assigning different weights to different regions  [44]. Several works have demonstrated that incorporating attention can enhance the performance of semantic segmentation models, including DeepLabv3+  [45] [46] [47]. In our work, we have chosen to use the SimAM (Simple Attention Module)  [10]  attention model. SimAM whose algorithm (1) is shown below, is a lightweight attention module that does not introduce additional parameters. It directly estimates 3D weights, instead of expanding 1D or 2D weights as in other spatial and channel attention mechanisms.
Despite its simplicity, SimAM performs comparably to popular attention mechanisms, while having zero additional parameters. In this work we tested multiple attention mechanism such as SE (Squeeze-and-Excitation)  [48] , CBAM  [49] , BAM  [50] , ECA  [51] , Triplet Attention  [52]  and Coordinate Attention  [53]. In our model architecture shown in Fig. 4 , SimAM attention module is represent red-colored blocks. This attention module is added after each extracted backbone layers. It enable the model to focus on important features and improve segmentation performance.
Experiments and Results To evaluate our work, we first compared the performance of our proposed model on our dataset with other popular benchmark models. The results obtained on CamerFood10 dataset are shown in Table. 4. Additionally, we tested our model on the publicly available MyFood dataset  [15]  and compared the results (Table. 5) with their bestreported results. The evaluation metrics used include mean Intersection-Over-Union (mIOU) and mean Pixel-Accuracy (mPA). Furthermore, we computed mIoU for each class in the Camer-Food10 dataset to assess its balance, and the results are presented in Table. 6.
GoumetNet [37] is a novel architecture published by Sharma et al. in 2021. It incorporates both channel attention and spatial attention information in an expanded multi-scale feature representation using advanced Waterfall Atrous Spatial Pooling (WASPv2) module  [38]. GourmetNet refines the feature extraction process by merging features from multiple levels of the backbone through the two attention modules. The refined features are processed with the advanced multi-scale waterfall module that combines the benefits of cascade filtering and pyramid representations.
Mean Pixel Accuracy (mPA) It is the ratio of pixels correctly classified (see Eq. 1). This metric is easy to understand but high accuracy does not necessary imply superior segmentation ability. This metric does not fit for imbalanced dataset :    [53]  62. 41 89. 83 +900 TripletAtt  [52]  62. 20 89. 88 +11,800 SE  [48]  64. 33 90. 27 +11,200 BAM  [50]  63. 78 90. 03 +33,200 CBAM  [49]  60. 79 89. 11 +11,500 mPA = 1 N N i=1 T P i + T N i T P i + T N i + F P i + F N i. (1).
Experimental environment We conducted our experiments using a compute node on Grid5000  [58]  equipped with a GeForce GTX 1080 Ti. The Tensorflow, Keras API and Python 3 were utilized for all experiments. During the training phase, we employed an input image size of 512px × 512px, the standard Adam optimizer and Categorical cross-entropy loss. To adjust the learning rate, we adopted the ReduceOnPlateau learning schedule with an initial learning rate of 1e-4. The validation loss was monitored, and we set the patience parameter to 25 and the reduce factor to 0. 8. The models were trained for 250 epochs on the CamerFood10 dataset. All datasets and code of each model used in this paper are publicly available here 3.
mid-DeepLabv3+ performance analysis In this experiment we explore how the different contributions impact the model performances. We first evaluate performance metrics for DeepLabV3+ with the reduced ResNet50 (see Fig. 5 ) backbone which we consider as our baseline model. Then, we test different version of the proposed model by alternately adding to the baseline network the SimAM attention mechanism at different places, the middle layer (Mid-Layer ) and both at the same time. Our proposed model, named mid-DeepLabv3+, is the one obtained by adding both the SimAM module after the backbone extracted layers (See Fig. 3 ). In.
Comparison with other benchmark models This third experiment is conducted to access how our model perform in comparison to other popular benchmark models for image segmentation on the same dataset. We compare our model to DeepLabv3+  [8] , GourmetNet  [37]  and Unet  [56]. As shown in Table. 4, for DeepLabv3+ we tested Aligned Xception  [8]  as the original paper, ResNet101  [9]  and ResNet50  [9]  as presented in  [43]. All the models is build with output stride = 16. Table. 4 reports the results obtained using the CamerFood10 dataset. Mid-DeepLabv3+ model.
Evaluation on another dataset In this experiment, the performance of the proposed model is evaluated when tested on other datasets. For this purpose, we choose MyFood dataset published by Freitas et al. [15] , made up of images of most consumed Brazilian food. MyFood dataset has 9 classes and contains 1,250 images (see Table. 1). With MyFood we trained the same models as with Camefood10 in the previous second experiment. The obtained results are reported in Table. 5. We found that our results were comparable to those obtained by Freitas et al. [15]. We observed that our model mid-DeepLabv3+ outperformed benchmark models on MyFood dataset.
CamerFood10 classes performance analysis Finally we compute mIoU for each class of our CamerFood10 dataset in order to further investigate how the proposed model performs for each class of food. In Table. 6, we provide the mIoU results for each class of CamerFood10. Classes are ranked from the high mIoU to the lowest. Upon analyzing the results, we observed low performance for classes with very few occurrences (as shown in Fig. 3 ), such as "Taro", "Yellow soup", "Koki", and "Beans". We also obtained lower results for classes with high intra-class variation, such as "Tomato soup" and "Bobolo". On the other hand, higher results were achieved for classes with many occurrences and minimal intra-class variation, such as "White rice", "Puff-puff", and "Fried plantains". Figure  6  shows some predicted masks obtained with mid-DeepLabv3+.
Conclusion and Discussion This paper addresses the issue of developing a vision-based dietary assessment system (VBDA) for African foods. Focusing on the image analysis phase, we have made two main contributions. on CamerFood10. Compared to DeepLabv3+ with the same backbone, mid-DeepLabv3+ achieved +10. 74% mIoU performance improvement while having two time less parameters. As part of our future work, on the one hand, we intend to expand the number of images in the data set and, on the other hand, investigate new methods for improving the performance of the model. In addition to accurately estimate calories of food in a given image, we also plan to explore methods for estimating the amount of each food item.
This paper describes OpenSpyrit, an open access and open source ecosystem for reproducible research in hyperspectral single-pixel imaging, composed of SPAS (a Python single-pixel acquisition software), SPYRIT (a Python single-pixel reconstruction toolkit) and SPIHIM (a single-pixel hyperspectral image collection). The proposed OpenSpyrit ecosystem responds to the need for reproducibility in single-pixel imaging, which is currently lacking due to limited access to data and reconstruction algorithms. The SPIHIM collection currently contains 140 hypercubes that are acquired using SPAS and reconstructed using SPYRIT. The SPIHIM hypercubes are obtained by inverse Hadamard transformation of the raw data. They have a size of 64 × 64 × 2048 for a spectral resolution of 2. 3 nm and a spatial resolution that is comprised between 182.
Our datasets -SPIHIM  [26]  -are made publicly available following FAIR (findability, accessibility, interoperability, reusability) principles  [27]. We share the raw data together with the Hadamard inverse transformed hypercubes and the hypercubes reconstructed thanks to deep learning. Our reconstruction method is implemented in the Python package SPYRIT  [28] , which allows the method to be retrained from scratch or to evaluate the corresponding Pytorch models that we make available. Our acquisition software is released via the SPAS package  [29]. We believe the OpenSpyrit ecosystem can considerably ease the training and/or benchmarking of hyperspectral single-pixel image reconstruction algorithms.
While the denoised completion operator G dc is kept fixed during training, we optimize the parameters of the convolutional neural network in a supervised manner argmin ℓ (ℓ) -G ( (ℓ) ) 2 , (7) where { (ℓ) ∈ R } 1≤ℓ ≤ is an image database and { (ℓ) ∈ R } 1≤ℓ ≤ are the associated measurements computed according to Eq. (  1 ) and Eq. (3).
SPAS: Single-Pixel Acquisition Software SPAS  [29]  is an open source python package for single-pixel acquisition, which has been been tested to control a DMD 4100 (0. 7" XGA VIS, ViALUX) and an AvaSpec-ULS2048CL-EVO spectrometer. It therefore requires the ALP4lib  [34]  package for DMD control and the MSL-Equipment  [35]  package for spectrometer control. SPAS provides simple functions for the initialization of the DMD and spectrometer, for the upload of a sequence of patterns into the DMD memory, and for the acquisition that is triggered by an external signal provided by the DMD. Each pattern is displayed on the DMD during a given illumination time and the external trigger is sent to the spectrometer for synchronization. During acquisition, the spectra are stored in the spectrometer's internal memory and are transferred to the computer via a callback function, which allows other tasks such as image reconstruction to be run in parallel.
SPYRIT: Single-Pixel Reconstruction Toolkit SPYRIT is a Python package for single-pixel image reconstruction  [28]  that relies on PyTorch  [37]  and can be easily installed using the pip package management system. It implements the forward model of Eq. (  1 ), the preprocessing step of Eq. (  3 ), and the reconstruction step of Eq. (  5 ) via classes that inherit from nn. module. This allows the full pipeline to be seen from an image acquisition to its reconstruction as a neural network, in addition to full exploitation of PyTorch functionalities (e. g. , data loaders, network architectures, loss functions, optimizers, etc. ). The source code of the SPYRIT package is available on GitHub and benefits from continuous integration with automated release of Python packages, dedicated benchmarks for regression testing, and integrated documentation.
Experimental setup Our setup is composed of an illumination arm, a DMD, and a light collection arm as depicted in Fig. 2. The illumination arm is composed of a white LED lamp (Thorlabs LIUCWHA) and a bi-telecentric lens system (Edmund Optics TECHSPEC® Large Format Telecentric 62902, https://github. com/openspyrit/spyrit-examples magnification 0. 9x) that forms the image of the object in the active plane of a DMD (ViALUX GmbH DLP V-700, 1024 x 768 micromirrors, 13. 7 µm pitch). The DMD is made of a matrix of microscopic mirrors that can be individually tilted to either +24 • (ON state) or -24 •  that focuses light at the entrance of an optical fiber (1500 µm core diameter, NA = 0. 39, FT1500 UMT) connected to a compact spectrometer (Avantes AvaSpec-ULS2048CL-EVO, Λ = 2048 spectral channels, 515-750 nm, entrance slit of 200 µm, 1200 lines/mm grating).
Experimental data for setup characterization For the purpose of reproducibility, we characterize the spatial and spectral resolution of the setup. To do so, we consider five objects from the SPIHIM collection. To determine the spatial resolution, we consider the Star Sector resolution target (Thorlabs, R1L1S2P, see top row of Fig. 4 ) and the USAF resolution target (Edmund, USAF 1951 38256, see bottom row of Fig. 4 ). Both samples are illuminated in transmission using a cold white LED array light source (Thorlabs, LIUCWHA). To determine the spectral resolution, we consider the light spots of a Mercury-Argon calibration lamp (Ocean Optics HG-1 with characteristic peaks at 546, 577, 579, 697, 707, 727, and 738 nm). As a more realistic reference image, we consider the image of a cat taken from the STL-10  [38]  test set.
Training of the DC-Net We train a DC-Net using SPYRIT  [28]  for a number of measurements of 4096, 2048, 1024, and 512. In particular, we consider the = 100, 000 images that correspond to the 'test' subset of the ImageNet dataset  [39]. The original images are randomly cropped to 128×128 and are normalized between -1 and 1. The image domain denoiser D is a U-Net with three downsampling steps separated by a max pooling operation and three upsampling steps separated by transposed convolutions. The number of filters in the contracting path is increased from 16 to 32, then to 64. Each layer is separated by a ReLU and batch normalization layer. Our U-Net has a total of 499,985 learnable parameters. We solve Eq. (  7 ) using the ADAM optimizer  [40] , with an initial learning rate of 10 -3 , which is halved every 10 epochs, for a maximum of 30 epochs.
e. , we chose such that E(  2 2 ) is maximal), as suggested in  [42]. We reconstruct the hypercubes in the least square sense using Eq. (  4 ) (see first column of Fig. 7 ) as well as with the data driven DC-Net given by Eq. (  5 ) (see second, third, and fourth columns of Fig. 7 ). All reconstructed images are obtained by keeping a single spectral channel at = 579 nm. The Hadamard matrices are constructed such that the set of patterns at a given resolution is included in the set of patterns at higher resolutions. For instance, a full basis scan at resolution 64 (i. e. , = 4, 096 measurements taken from patterns of size = 64 × 64) is equivalent to an acquisition at resolution 128 (i. e. , = 16, 384 measurements taken from patterns of size = 128 × 128) undersampled by a factor of four. To illustrate this, we reconstruct the USAF target at resolution 64 (see second row of Fig.
7 ) and at resolution 128 (see third row of Fig. 7 ). More hypercubes reconstructed at resolution 128 can be found in Section 2 of Supplement 1 where we consider fifteen different objects and four different spectral channels. The least-squares reconstruction is independent of the choice for and depends on only, This can be observed by comparing the images of the second row of Fig. 7  to those of the third row of Fig. 7. In particular, the bars of the elements 5 and 6 of the group 4 can be resolved in the deep reconstruction, but not in the least-squares reconstruction (see the red rectangles in Fig. 7 ). This improvement is observed even if no such piecewise-constant binary images are present in the ImageNet database used for training. As expected, lowering the number of measurements leads to a loss of spatial resolution, which can be evaluated from the reconstructions of the resolution targets (see second, third, and fourth rows of Fig.
Discussion Images collected by hardware-driven commercial systems are relatively abundant. Among the main hyperspectral databases, the AVIRIS collection (e. g. , see the NASA website  [43] ) contains thousands of hypercubes over 214 spectral channels taken from airborne platforms. The BGU ICVL collection  [44]  currently contains 201 hypercubes over 519 spectral channels corresponding to indoor and outdoor scenes taken from the ground. Very few databases offer hypercubes with more than a thousand spectral bands. In contrast to existing datasets, our computational device provides access to 2,048 spectral channels. Our imaging system natively acquires a 64 × 64 × 2048 hypercube with a spectral resolution of 2. 3 nm, while the spatial resolution can be adjusted between 182 µm and 15 µm using a DMD-based hardware zoom that can achieve a ×12 magnification with no modification of the optical components.
Irrespective of the imaging configuration, all the patterns of a 64 × 64 Hadamard basis are acquired, resulting in a total of = = 4,096 Hadamard patterns split into 8,192 positive and negative patterns. We also consider accelerated acquisitions for which < 4,096 patterns are acquired. The fully sampled datasets can be downsampled a posteriori to simulate an accelerated acquisition with different acceleration factors. Typically, the 64 × 64 Hadamard patterns are resized to fill the largest square region possible on the DMD, which corresponds to 768 × 768 micromirrors. In this case, each pixel of the Hadamard patterns corresponds to an area of 12 × 12 micromirrors. However, it is also possible to display the patterns on smaller fields of view, acting.