dataesr/scientific-paragraphs-categorization huggingface dataset:
COCO-stuff  [33] , an extended version of COCO with more than 1,800,000 image, includes 181 classes  [34]. PASCAL VOC  [35]  is dedicated to pedestrian detection with 500,000 tagged images. It includes 181 classes of 20 different objects. CIFAR-100  [36]  includes 60,000 images with objects from 100 classes. INRIA has developped its own dataset dedicated to pedestrian detection. Caltech dataset is also dedicated to the detection of people and pedestrians. Overall, these datasets are relatively rich concerning the most commonly detected objects (pedestrians, cars, buses, bicycles, etc. ). The COCO dataset is also the reference in terms of object detection and is ideal for comparing different object detection models. The latter offers different databases adapted to the training phases and to the test of neural networks.
YOLOv3, for example, uses ImageNet to train the first 53 layers of its network and establish the reference databases. This dataset being very dense, it is a good choice to pre-train YOLOv3. This model then uses other databases to perform detection and classification, such as COCO for example. For more specific applications, MCIndoor20000 dataset  [37]  includes more than 20,000 digital images from three different indoor object categories, including doors, stairs, and hospital signs. Figure  1  shows some examples of MCIndoor2000 dataset images including doors and handles, as well as a sample of our custom ESIGELEC dataset. The wheelchair is supposed to navigate indoors as well outdoors. For this reason, the datasets dedicated to the autonomous vehicle are also interesting to take into account.
In KITTI  [38] , the authors present one of the widely used dataset in road environment for autonomous driving research. KITTI is a calibrated, synchronized, and timestamped autonomous driving dataset. The KITTI dataset was collected through instrumented vehicle by different kinds of sensors such as: color and grayscale stereo cameras, a velodyne 3D laser scanner and a high precision GPS/IMU navigation system. The platform contains real-world traffic situations with both static and dynamic objects  [38]. NuTonomy scenes (called NUScenes  [39] ) is a multimodal dataset for automotive driving applications, with information from several kind of sensors such as: 6 cameras, 5 radars and 1 LiDAR. It is fully annotated and comprises 1000 scenes, 3D bounding boxes for 23 classes and 8 attributes. It has 100× as many images than the pioneering KITTI dataset.
Hardware Architecture The robotic electric wheelchair of the IRSEEM laboratory is an Invacare, Bora model, from which all the proprietary electronics have been removed and replaced by our own elements: 1. An on-board computer with 8 GB RAM and 250 GB SSD, running on Linux Ubuntu 16. 04 LTS; 2. A Roboteq engine driver; 3. An Xbox joystick instead of the original one, which can be connected to the wheelchair by an USB or a bluetooth connection to manually control the wheelchair; 4. A Wi-Fi router to get a wireless LAN on the wheelchair; 5. An embedded Human-Machine Interface (HMI) with a touch screen; 6. The sensing part is composed of an Intel RealSense D435 camera and an Realsense T265 camera. The D435 camera provides color and depth images, whereas the T265 camera provides fisheye images and odometry data in the form of position (cartesian coordinates) and orientation (quaternions).
In addition, an on-board computer is used, with Ubuntu 16. 04 as operating system and 8 GB RAM and 250 GB SSD, for all on-board processing. The interaction between the wheelchair and the user is done via a touch screen which is connected to the on-board computer via an HDMI connection, or using the Wi-Fi connection. A Roboteq engine driver is used to control the engines. The hardware description of the wheelchair is shown in Figure  2. In our project, the wheelchair should be autonomous in decisions. Because of the deep learning algorithm involved in the object detection process, we needed a GPUcapable embedded computer. For this reason, we have implemented the entire software developments onto an Nvidia Jetson TX2 board. This board offers several advantages: GPU computing power for our deep learning algorithm, software interface fully compatible with image processing and computer vision libraries.
• Image acquisition: two kind of images are acquired through the D435 camera: RGB (color) and depth images; • Object detection and depth estimation: objects (doors, handles) are detected using the color images provided by the D435 camera. Distance estimation is carried out using the depth images, and provides distance measurements to be associated with the detections; • Tracking: using a classified object and an associated depth, the position of the 3D object is added to the semantic map using the odometry data provided by the T265 camera. The wheelchair software is fully developed under the Robotic Operating System (ROS) robotic middleware  [40]. Because ROS is a multi-platform framework, portability between desktop and embedded platform was completely transparent, and required no code refactor.
Object Detection and Distance Measurement 4. 1. Object Detection To perform object detection, we have used the powerful YOLOv3 neural network. For this project, the keypoints we detect are doors ans door handles. Since these classes are under-represented in the YOLOv3 training dataset (i. e. , ImageNet [28]), we have composed a custom dataset to perform the network training. We have extracted 755 door images from the MCIndoor20000 dataset  [37] , consisting of tagged images containing various wheelchair indoor objects. As we could not find an open dataset with sufficient representation door handles, we have developed our own dataset, consisting of 1885 images, which we combined with door images from the MCIndoor20000 dataset. We supervised the labeling of 2640 images from two combined datasets using a semi-automatic labeling tool we developed.
This tool is a simple drag-to-select python script, that allowed at first a manual annotation of about 1000 object per class. We then used these annotated images to retrain YOLOv3. From this retrained model, we were able to classify a larger dataset. We selected the properly labeled images from this extended dataset, that we again used to further refine the model. We finally proceeded to fine tune YOLOv3 on the required classes. For this transfer learning process, we trained only the neural network classification layers. Figure  4  shows some qualitative results of the detection process after re-training YOLOv3 on the recognition of doors, and door handles.
Object Tracking Using the combination of position and depth associated with each item extracted by the detection algorithm, we rely on the SORT (Simple Online and Realtime Tracking) algorithm  [21]  to track the different objects in the scene while the wheelchair is moving. On a video stream, multiple objects (doors and door handles) are detected and assigned a given distance. SORT inspects the detected objects, and determines if a given object is newly seen, or if the movement of the object is a consequence of the wheelchair's movements. This algorithm is based on a Kalman filter and provides a unique identification number to each new detected object. SORT keeps track of several objects simultaneously and filters the positions of noisy objects associated with moving boundary boxes. Finally, we use odometric data from T265 RealSense camera to estimate the wheelchair displacement.
We combine this data with the position of the object in order to visualize a 3D semantic map of the environment containing the detected and tracked objects, as shown in Figure  5. In order to validate these algorithms, we created out an experimental dataset in the ANL IRSEEM laboratory. A set of four doors with different colours and handle shapes were placed in the coverage area of a motion capture Vicon system. Once equipped with reflective markers, the doors and wheelchair were located by a Vicon system providing their position and orientation with millimetric precision at a frequency of 100 Hz  [42]. This way, we obtain a high rate ground truth distance between the wheelchair and the doors and handles. The different doors have been placed along an arc of a circle (see Figure  6 ), with the aim of evaluating the door detection when multiple elements are present simultaneously in the images.
In this experiment, ADAPT's wheelchair moves through the scene and, by changing orientation, finds itself several times facing the different doors. The distance measurements between the doors, handles, and the wheelchair is then compared to the ground truth provided by the Vicon system. Figure  9  presents these results. For a total of 804 doors detections, the actual distance and the distance estimated by the D435 RealSense camera, as well as the difference between these two values, are observed. Table  2  provides the numerical results of this experiment. We observe a mean error of 18. 1 cm, corresponding to 3. 8% error on the estimation of the objects distance which is a good result regarding the indoor environment constraints of the wheelchair. These values are below the data provided by the manufacturer.
29% and 0. 80% on a set of 866 images. It is likely that the low recall rate of handles is due to the lack of diversity in our dataset. The recent updates of YOLO does provide advanced tools for automatic data augmentation, which were not available at the time of our experimentation. Data augmentation and gathering of a more extensive dataset will be part of our future work, and should improve the current results. For this purpose, we plan on developing our own data collection system for the dataset, materialized by a D435 RealSense camera instrumented on the wheelchair as shown in Figure  1. We will introduce a protocol for images collection in indoor environment that will take into account not only the ANL as a controlled environment, but also some corridors (with doors and handles) inside the IRSEEM building.
In addition, we also plan to increase our dataset diversity using deep learning-based data augmentation built within YOLOv5. Data augmentation performs transformations (i. e. , rotations, adding noise, mixing subsets of images, etc. ) on the base training data to expose the model to a wider range of semantic variation than the base training set  [43]. When tested on our desktop computer which is equipped with a Nvidia GTX 980 GPU, we are able to process the image flow of the camera (30 FPS) in real-time. When tested on the Jetson TX2 GPU, we get a framerate of 5. 8 FPS at the same 640 × 480 resolution, which is still suitable for our use case in which the wheelchair moves at at most 1 m per second. We did consider resizing the images when using the Jetson TX2. We do tend, in this case, to have a faster inference.
Distance Estimation Evaluation For the distance evaluation between the camera (which is mounted on the wheelchair) and the different objects in the indoor environment, we used in previous works different CNN models dedicated to depth estimation like Monodepth  [12] , Monodepth2  [1, 44] , and MadNET  [17, 45]. In this paper, we have carried out distance measurements by directly using the RealSense D435 camera (without any deep learning model) because of embeddedsystem constraints related to the wheelchair such as: not enough GPU computational power on the Jetson TX2 board, not enough space memory, and the distance estimation of object which should be done in real time. This is why in this section we will present only the distance measurement by camera. We compare the distance obtained through the camera with the ground truth data given by the Vicon motion capture system available in our ANL laboratory.
We have arranged the ANL in such a way that we have an indoor environment like indoor home of disabled people with reduced mobility, with rooms, corridors, doors, handles, etc. Figure  8  shows one example of ANL navigation environment for the wheelchair with doors, handles, and the Vicon ground truth system. Figure  9  shows the measured distances between objects and the camera, as well as the absolute distance measurement error, in comparison with millimetric ground truth from the Vicon motion capture system. The distribution of the relative distance measurement error is shown in Figure  10. In Table  2 , we present the different results of the depth estimation obtained on our own dataset. Because of the median filter applied to the distance measurements of the Realsense D435 camera, we obtain a significantly lower error than that indicated in the sensor datasheet, with a 3.
When the detection algorithm outputs a bounding box poorly centered on the ground truth, i. e. , The IoU has a low value, the median depth value can fall outside the object. In our dataset case, the doors are placed in the middle of our lab, instead of onto a wall. For this reason, when the IoU is too low, the depth measurement algorithm can returns a value corresponding to the wall behind the detected object. This is especially true for small objects such as handles. This explains the peaks noticeable at 2. 5 and 4. 5 m on Figure  11. For a future work, we plan on increasing the depth measurement precision by applying a temporal filter, in combination to the spatial filter, in an attempt to further refine the depth measurement accuracy.
Conclusions and Future Work In this paper, we have presented an object detection, depth estimation, location and tracking system for wheelchair healthcare smart mobility, specifically for indoor environ-ment, based on deep learning for object detection (YOLOv3), and various techniques and algorithms. We have measured the distance estimation error with the detected objects, by comparing the distance measured by the camera and the ground truth data given by the Vicon motion capture system. To validate the whole developments, our models have been re-trained using two datasets: the open MCIndoor20000 dataset and our own ESIGELEC dataset. By using our re-trained and fine-tuned model, our implementation offers good performance in inference on the wheelchair indoor environment. All developments are integrated on the platform via an Nvidia Jetson TX2 board as the main embedded computer for all algorithms (object detection, distance estimation, and tracking).
Among the added values of this work, we can mention the integration of an object detection and localization chain on the wheelchair while optimizing our algorithms so that they are compatible with real-time constraints. The use of two cameras, one for distance measurement and the other for localization of the wheelchair, also represents a strong added value. Finally, this work allowed us to validate our own database (866 images). As a future work, and in order to finalize our smart healthcare wheelchair platform, we plan on improving the quality of the dataset by extending out image dataset for the indoor environment, including more classes such as doors, handles, switches, corridors, etc. Updating the object detection neural network to a more recent version, i. e. , Yolov5, and refining the training using data augmentation should also increase the performance in term of speed, precision and recall.
The next step after object detection, localisation, and tracking chain is semantic segmentation in order to carry out a complete scene understanding. For this, we plan to develop a virtual dataset with which we will enable the training of a semantic segmentation deep learning model. The wheelchair needs to recognize its environment with high accuracy in order to execute different scenarios such as: (i) the system detects a door and automatically controls the wheelchair to navigate to it, and then detects the handle on the door to allow opening; (ii) the system detects a light switch and automatically drives the wheelchair to it to alow light activation, etc.
variables representing monthly temperature and precipitation estimates, past human population density data, and an extensive database of georeferenced presence points, we built a model of changing habitat suitability for chimpanzees at fine spatio-temporal scales dating back to the Last Interglacial (120,000 BP). Our models cover a spatial resolution of 0. 0467°(approximately 5. 19 km 2 grid cells) and a temporal resolution of between 1000 and 4000 years. Using our model, we mapped habitat stability over time using three approaches, comparing our modeled stability estimates to existing knowledge of Afrotropical refugia, as well as contemporary patterns of major keystone tropical food resources used by chimpanzees, figs  (Moraceae) , and palms (Arecacae). Results show habitat stability congruent with known glacial refugia across Africa, suggesting their extents may have been underestimated for chimpanzees, with potentially up to approximately 60,000 km 2 of previously unrecognized glacial refugia.
g. , last interglacial, last glacial maximum, and late holocene;  Hijmans et al. , 2005). Here, we address this limitation by modeling the historical habitat suitability of chimpanzees (Pan troglodytes) across the Afrotropics using a comprehensive database of recently collected georeferenced presence points spanning their entire range, paleoclimate reconstructions, and human density data since the Last Interglacial (120,000 BP). Chimpanzees are an appropriate focal species for our approach due to their wide geographical distribution and high detectability. This means that their presence in an area is unlikely to be overlooked even in places that are only sporadically surveyed, as is the case for many poorly known and remote field sites across the Afrotropics. Chimpanzees occur across 21 African countries, from dense moist forest to arid savannah, from sea level up to an altitude of 2800 m and varying levels of anthropogenic pressure  (Humle et al.
| General workflow To summarize our general workflow (Figure  1 ), we modeled contemporary chimpanzee habitat suitability using a database of presence points and six noncorrelated climatic and anthropogenic variables that could be expected to influence their distribution. As separate species distribution modeling algorithms may provide biased results because of their sensitivity to the data and the mathematical functions utilized, we built ensemble models which combine multiple replicates of several different modeling algorithms together  (Araújo & New, 2007). These models based on contemporary chimpanzee habitat suitability were then projected into the past at paleoclimate timescales (back to the Last Interglacial, 120,000 years BP) to assess how habitat suitability has changed through time.
| Reconstructing African paleoclimatic conditions back to the Last Interglacial We obtained data on paleoclimatic conditions in Africa since the Last Interglacial from  Bell et al. (2017)   (IPCC, 2007 (IPCC, , 2014)). These data represent 62 snapshots of global climatic conditions at 1000 year intervals from the present back to the Last Glacial Maximum (LGM, ~22,000 BP), at 2000 year intervals from the LGM to 80,000 BP, and at 4000 year intervals from 80,000 BP back to the Last Interglacial (120,000 BP). These data comprised of eight climate variables: mean annual temperature (bioclim_01), temperature seasonality (bioclim_04), mean temperature of the warmest (bioclim_10) and coldest quarters (bioclim_11), mean annual precipitation (bio-clim_12), precipitation seasonality (bioclim_15), and precipitation of the wettest (bioclim_16) and driest quarters (bioclim_17) for each snapshot.
We set the geographic extent of our models to encompass both the contemporary and potential historical habitat suitability of chimpanzees  (McBrearty & Jablonski, 2005) , from -18°to 32°l ongitude and -10°to 16°latitude. This extent includes the current distribution range of bonobos (P. paniscus), the sister species of chimpanzees, spatially separated from the latter by the Congo River. We included the bonobo distribution in our models due to previous detection of historic admixture between chimpanzees and bonobos based on genomic data (De  Manuel et al. , 2016; Kuhlwilm et al. , 2019) , suggesting that chimpanzees may have occasionally been found in these areas. To account for anthropogenic effects on chimpanzee habitat suitability, we also included a spatial layer available from the HyDE database  (Klein Goldewijk et al.
, 2011) , which provides information on modeled past human population density based on population and agricultural data. The calculation of this spatial layer is based on historical population, cropland, and pasture statistics combined with satellite information and specific allocation algorithms (which change over time) to create spatially explicit maps covering the last 12,000 years BP. We linearly extrapolated the human population densities back in time from 12,000 BP (i. e. , the last available data from the HyDe database) uniformly to 120,000 BP and downscaled it so that it matched the temporal and spatial scale of our paleoclimate reconstructions. We selected these climatic and anthropogenic variables to reflect biologically informative conditions likely to have influenced chimpanzee habitat suitability, and because these variables were available at paleoclimatic timescales.
| Species distribution modeling of chimpanzees We compiled a database of 139,902 georeferenced chimpanzee presence point records (nests, sightings, feces, and footprints) from our own fieldwork, from the IUCN SSC A. P. E. S. database, a collaborative initiative to centralize great ape population surveillance data (http://apes. eva. mpg. de/) and from the Pan African Programme: The Cultured Chimpanzee (http://panafrican. eva. mpg. de/). For each occurrence record, subspecies were determined by their geographical location. In total, this data, collected between 1984 and 2006, re- present over fifty collaborating wildlife research institutions across the entire distributional range of chimpanzees. We followed recommended guidelines for constructing SDMs  (Araujo et al. , 2019; Merow et al. , 2013 , Merow et al.
, 2014)  to minimize bias in our models. This included spatially rarefying the presence data so that occurrences were not overclustered together (minimum distance of 10 km between each point), resulting in 1677 unique presence points remaining that were used to build SDMs. We selected background points (pseudo-absences) from a 0. 5°buffer radius around presence points, to emphasize factors locally relevant in distinguishing suitable from unsuitable habitat, while adequately sampling the range of climatic conditions for chimpanzees  (VanDerWal et al. , 2009). Furthermore, to minimize spatial autocorrelation in our models we reduced our nine variables to six due to high co-correlations (Pearson's r > 0. 6). Our retained variables were: annual mean temperature, temperature seasonality, annual precipitation, precipitation of wettest quarter, precipitation of driest quarter, and human density.
These three output stability maps are then quantitatively compared with assumed forest refugia  (Maley, 1996) , as well as species richness of major keystone food resources for primates (figs,  Kissling et al. , 2007)  and palms  (Blach-Overgaard et al. , 2013 )  (Forester et al. , 2013). We constructed an ensemble species distribution model (SDM) representing the contemporary habitat suitability of chimpanzees (i. e. , all combined subspecies) using the "sdm" R package version 1. 0-46  (Naimi & Araújo, 2016). We used five crossvalidated replicates each of 14 different modeling algorithms available in the sdm R package, evaluating the performance of each modeling algorithm based on the Area Under the Curve of a Receiver Operating Characteristics plot (AUC,  Fielding & Bell, 1997)   and True  Skill Statistic (TSS,  Allouche et al.