dataesr/scientific-paragraphs-categorization huggingface dataset:
Deployment on Smart Glasses A ResNetv1-6 is integrated into Ellcie Healthy's smart glasses firmware version 6. 1. 2 using the C inference library generated by MicroAI. In this firmware version, only 77,604 B of Flash (for the inference code and the weights) and 40,572 B of RAM (for the intermediate computation and the layers' output buffers of the deep neural network) can be used. Therefore, these memory limitations constrain the neural network that can be executed on the microcontroller. For the 32-bit floating-point inference, the largest ResNetv1-6 that can be deployed only contains 32 filters per convolution. Since the 16-bit fixed-point quantization provides the best memory efficiency, we also deployed a 16-bit ResNetv1-6 with 48 filters per convolution to get the best possible accuracy on the smart glasses.
89% of Flash and 86. 75% of statically allocated RAM. The inference is performed after each time 64 samples are collected by the inertial measurement unit (IMU) whose sampling rate is 26 Hz. As the barometer sampling rate is 6. 66 Hz, this sensor is not used in these experiments since resampling the signal would be required. The power consumption of the smart glasses is measured using a Qoitech Otii Arc laboratory power supply, supplying 3. 75 V in place of the LiPo battery. Energy values are computed by the Otii software from the current and voltage over a one minute window starting from the beginning of an inference. Obtained measurement over one inference period is shown in Figure  9   In the Figure  9 , the inference task starts at the very beginning of the measurement. After the 173 ms of inference, 64 new samples are collected from the IMU.
Live Human Activity Recognition on Smart Glasses The ResNetv1-6 model with 48 filters per convolution, 16-bit fixed-point quantization and CMSIS-NN optimizations, has been trained using the UCA-EHAR dataset. This network has been then integrated onto the smart glasses firmware to perform live human activity recognition. Data are collected from the accelerometer and the gyroscope of the smart glasses when worn by a subject. The smart glasses' microcontroller performs the classification and sends the label of the recognized activity to a computer for visualization through a Bluetooth Low Energy communication. Additionally, the accelerometer and gyroscope data are also sent for visualization, even though the classification is not performed on the computer. A 30-second sample of such a live recognition has been extracted and can be seen in Figure  10.
Conclusions In this a novel dataset for human activity recognition called UCA-EHAR has been presented. This dataset gathers data collected from the accelerometer, the gyroscope and the barometer of smart glasses. UCA-EHAR is the first publicly available dataset dedicated to human activity recognition on activities of daily living using smart glasses. To provide a comparison baseline for a classification task, we evaluated the performance of a residual neural network on our dataset and we provided accuracy results as well as a confusion matrix. The accuracy for this dataset using a floating-point ResNetv1-6 with 80 filters per convolution is 80. 2%. However, such a floating-point implementation does not respect embedded constraints of the smart glasses. Therefore, the neural network has been quantized using 8-bit and 16-bit fixed-point inference in order to optimize the memory footprint and the inference time, thus the energy consumption.
Obtained results show that the 16-bit quantization provides the best accuracy vs. memory efficiency. To illustrate the energy that can be saved by quantization, we deployed a deep neural network onto the smart glasses using our MicroAI framework. We then measured the current and voltage during a human activity recognition task running on the smart glasses. Using the 16-bit quantized network with 48 filters per convolution we have shown that we can run human activity recognition for up to 24 h on the smart glasses. In the future, we will build a dataset including more classes such as transitions (SIT_TO_STAND, STAND_TO_SIT, SIT_TO_LIE, LIE_TO_SIT) or other activities (DRIVING). We would also like to explore unsupervised online learning using this dataset. To do so, collecting data for some subjects over a longer period of time will be required.
Example of Data from the Dataset float32 1096 4384 78. 08% 16 float32 3848 15,392 78. 99% 24 float32 8264 33,056 79. 14% 32 float32 14,344 57,376 79. 28% 40 float32 22,088 88,352 79. 48% 48 float32 31,496 125,984 79. 87% 64 float32 55,304 221,216 80. 24% 80 float32 85,768 343,072 80. 20% 8 int16 1096 2192 78. 08% 16 int16 3848 7696 79. 06% 24 int16 8264 16,528 79. 28% 32 int16 14,344 28,688 79. 21% 40 int16 22,088 44,176 79. 50% 48 int16 31,496 62,992 79. 79% 64 int16 55,304 110,608 79. 97% 80 int16 85,768 171,536 80. 16% 8 int8 1096 1096 75. 83% 16 int8 3848 3848 77. 69% 24 int8 8264 8264 78. 58% 32 int8 14,344 14,344 77. 90% 40 int8 22,088 22,088 77. 78% 48 int8 31,496 31,496 77. 94% 64 int8 55,304 55,304 77. 71% 80 int8 85,768 85,768 78. 27% Appendix C.
137) The first difficulty in computing characteristic sets for non linear differential systems is to differentiate the equations. Using a classical representation of data, the sizes of the successive derivatives are exponential in the order and it is well possible to saturate the available memory before starting any algebraic elimination. As Jacobi's method reduces, in the generic case, the number of requested differentiations to the minimum, it suggests many computational applications and easily implemented improvements of existing softwares such as Pardi! or Diffalg (see Boulier et al. [5, 6] ).
DEFINITION 111. - 111 The ideals ℐ  , ℐ ,  defined in the theorem and the quantities Δ ,  or ∇  , will be respectively denoted by ℐ    , ℐ   ,  , Δ   ,  or ∇    when we need to avoid ambiguities. As the data of  defines the cover ,  up to equivalence and those objects are uniquely defined by , "" does not appear in the above notations.
DEFINITION 19. -We call a cover for  the data of two vectors ( 1 , … ,   ) ∈ M  and ( 1 , … ,   ) ∈ M  , such that  , ≤   +. A cover ,  is a minimal cover if the sum ∑ If  ′ ,  ′ is the cover associated to the canon associated to a cover , , then  ′ ,  ′ and ,  are equivalent, with  ′ =  -min  =1. PROOF. -i) ⟹ We have, by hypothesis, ∑ ∑    +   ≥ ∑   ,(), hence the minimality of the cover ,. ⟸Letusassume that there is no such permutation. Then, by th. 17 the entries  , with  , =   +   belong to  rows and  - columns, with  < , that we may suppose to be rows 1, … ,  and columns 1, … ,  -. Let   +   ≥  , , so that ,  is a cover. Minimality is a consequence of i), remarking that, if  ,() + ℓ  form a maximal transversal sum in  + ℓ, then  ,() =   +. iii)Byi) there exists a permutation  such that  ,() =   +  (), so that  ,() + ℓ  =   +  () + ℓ  =  () +  =   ′ +  () + ℓ  ′ ≥   ′ ,() + ℓ  ′ for all 1 ≤  ′ ≤ , so that rows 𝑖 for which 𝜇 𝑖 is decreased is chosen to be minimal, such an algorithm has a polynomial complexity and is indeed very close to Kuhn's Hungarian method, 𝑛 but also to Jacobi's algorithm, as they will correspond to third class rows.
This mainly consists of i) an analysis of IoT traffic data acquisition methodologies and scenarios, as well as a classification of public datasets, ii) a literature evaluation of IoT traffic feature extraction, categorizing and comparing popular features, as well as describing open-source feature extraction tools, and iii) a comparison of ML approaches for IoT device classification and how they have been evaluated. The findings of the analysis are presented in taxonomies with statistics showing literature trends. This study also explores and suggests undiscovered or understudied research directions. 15 16 to a class with predetermined policies. 50 Note that the term device classification is often confused 51 with many similar tasks, namely i) traffic classification, 52 ii) intrusion detection, iii) device identification, and iv) device 53 fingerprinting.
• An in-depth examination of IoT traffic data collection strategies. This includes: i) a review of the devices used to represent a smart home setting, ii) a study of IoT traffic types (depending on device operation mode) and their utility for classifying devices, iii) a description of the architecture and different traffic collection points (depending on the traffic probe location) and a debate on how realistic they are, and iv) an evaluation of public datasets for IoT device classification. 1 A dynamic version of the taxonomy and websites is available at : https://gitlab. com/jmila/smart-home-iot-device-classification-usingmachinelearning-based-network-traffic-analysis network (blocked). IoT device classification can also be used to establish an automatic whitelisting system to ensure only authorized IoT devices can connect to the network, as proposed by Meidan et al.
A. THE CLASSIFIED DEVICES The input to the IoT device classification process is a list of devices to be classified. They can be both IoT and non-IoT devices, also referred to as single-purpose and multipurpose devices, since IoT devices are typically intended for a single specific task. An up-to-date list of the most common smart home IoT devices can be found on the website  [18]. Examples of non-IoT devices include laptops, cell phones, and Android tablets. In the literature, some approaches classify only IoT devices, and others classify both IoT and non-IoT devices. Taxonomy of data-acquisition approaches: the approaches are classified according to i) the devices under consideration: only IoT, or both IoT and non-IoT devices, ii) the operation mode of the devices, iii) the probe location, and iv) whether a public dataset is utilized or the traffic is collected by the authors.
D. PUBLIC DATASETS COMPARISION 57% of reviewed publications use public datasets, either completely or to complement or enrich their data. Most of the datasets we mention in this survey were created for IoT device classification. However, we include other datasets developed for other topics that contain IoT traffic and can be used for IoT device classification. Table  1  summarizes the datasets listed below. To compare them, we specify for each: i) the devices used to generate the traffic (IoT only, or both IoT and non-IoT), ii) the operation mode of the devices (i. e. setup, interaction, idle), iii) the probe location (i. e. , at or after the gateway), iv) the duration of the collection, v) the amount of traffic collected, and we provide vi) a direct access link to the dataset. 1) IoTSentinel DATASET  [14]  This dataset was collected to identify IoT devices based on their setup traffic.
To generate enough traffic, the typical device configuration process was repeated 20 times for each VOLUME 10, 2022  2) UNSW DATASET  [19] ,  [27]  This dataset was published by UNSW researchers and covers various IoT research areas. In addition to traffic for IoT device classification, the dataset includes IoT attack traces, IoT MUD profiles, and IoT IPFIX records that can be useful for other IoT-related research topics (the relevance of MUD profiles to device classification is discussed in Sec. VIII). In this paper, we focus on the traffic for IoT device classification. It was first published in  [19]  and has since evolved. The first version has been extensively used in the literature. The same authors published an updated and more elaborate version in  [27]. Recent articles now use the modified version.
This study focuses on the IoT traffic traces reported in  [27]. They were collected over 26 weeks, from October 1 st , 2016 to April 13 th , 2017, but only two weeks' worth of data is available for download. 3) IoTFinder  [31]  AND YourThings DATASETS  [29]  The IoTFinder dataset was created to explore IoT device identification using DNS fingerprints. Thus, the dataset con-     This dataset was created to identify IoT devices behind VPN 527 and NAT-enabled gateways in smart homes. Three collection 528 scenarios were developed: i) a single device environment in 529 which only one device is considered, ii) a noisy environment 530 in which various IoT and non-IoT devices are investigated. 531 Multiple devices may be operating simultaneously at any 532 given time, resulting in traffic aggregation, and iii) a VPN 533 environment where VPN is enabled.
In this case, traffic is 534 collected before and after the VPN. 535 7) IoT-deNAT  [25]  536 The dataset was collected to detect vulnerable IoT devices 537 behind a home NAT. The traffic is captured considering 538 only NetFlow's [42] statistical aggregations (i. e. , Netflow is 539 a flow-level aggregation of information, usually a 5-tuple 540 header and some counters) instead of the raw data to reduce 541 processing and storage. Feature extraction is defined in  [45]  as ''the process of defining a set of features (. ) which will most efficiently or meaningfully represent the information that is important for analysis and classification. '' In our case, the feature extraction step consists of describing the network traffic in the most appropriate way to retrieve the maximum amount of information about the device.
1) METHODS USING MULTI-CLASS CLASSIFIER Only one classifier is used for the multi-class classification. The trained classification model outputs a vector of class membership probabilities P s = p s i 1 i≤n denoting the likelihood that the inspected traffic sample s comes from device class c i. The traffic is labelled as originating from the device having the highest probability. To capture unknown devices, a threshold parameter tr can be defined and fine-tuned using the validation dataset. If one probability p s i exceeds the threshold parameter tr (p s i > tr), the traffic is classified as originating from the device class c i. Otherwise, it is classified as unknown. A device can also be considered as unknown if the feature vector matches more than one class with a low discriminative threshold (0.
2) METHODS USING ONE-CLASS CLASSIFIER (A CLASSIFIER PER DEVICE) A minority of reviewed papers (14%) use this classification approach. In the following, we describe how this strategy 97130  VOLUME 10, 2022  is employed in the literature. This consists of splitting the dataset into numerous binary classification problems (focusing on a single class, regardless of all other classes) and then a binary classifier is trained for each device. Each classifier provides either i) a probability p i that the traffic was generated by a device class c i or ii) a binary decision on whether the input matches the device type. In the first case, a threshold t (cutoff value) should be set. If p i > t, the traffic is labelled as originating from the device class c i. t is empirically set to maximize the classifier's accuracy  [57].
In the second situation, if a device is accepted by multiple classifiers, the conflict should be resolved, for example, by computing a distance-based metric between the sample to identify and a subset of samples from each class that it has a match for  [14] , or by applying majority votes  [15]  to break the tie between multiple matches. Note that using this strategy, classification accuracy can be increased by evaluating the classification results of more than one sample before choosing the device class. For example, in [57], the authors perform a majority vote on the classification results of several consecutive TCP sessions to determine, with an accuracy of 100%, if they were generated by a certain device. The optimal number of consecutive sessions is defined as the minimum number of sessions on which a majority vote provides zero false positives and zero false negatives on the test dataset.
However, the one-class classifier approaches are more computationally expensive since the results of more than one classifier should be computed. Moreover, managing conflicts might be time-consuming if a sample fits many device types. As reported in  [14] , most device type identification time is spent on tiebreaks. Moreover, unbalanced training datasets can affect classifier performance (there are generally fewer samples for one device type compared to the samples of all the remaining samples combined). This issue can be solved by utilizing under-sampling and over-sampling approaches  [86]. Unsupervised learning is more scalable since it minimizes 1153 human assistance, but it is harder to execute and its accu-1154 racy is likely to be lower than supervised approaches. Thus, 1155 only 16% of reviewed papers use unsupervised classification 1156 approaches.
For instance, accuracy gives equal 1184 weights to all classes, which is inappropriate if the dataset 1185 is unbalanced (e. g. you can have 90% of total accuracy but, 1186 in minority classes, most samples are misclassified). The 1187 performance of classifiers should be assessed in different 1188 scenarios and through diverse metrics and measures. Below, 1189 we describe some other metrics found in the literature to 1190 inspire other evaluation methodologies.
2) MEASURING CPU, MEMORY CONSUMPTION AND COMPUTATIONAL COMPLEXITY In  [14] , the authors measure the CPU used by the security gateway for the classification and for the enforcement mechanism. In  [87] , the authors calculate the computational 3) VARYING EVALUATION SCENARIOS Some papers measure the variation of performance metrics in different scenarios. For example, Huang et al. [51]  test the scalability of their approach and show that accuracy diminishes with many device types. Meidan et al. [15]  measure the classification accuracy as a function of the number of consecutive sessions needed for classification. Similarly, Song et al. [88]  examine the relationship between identification accuracy and the number of packets required for classification. Bai et al. [55]  measure the classification results under different time window sizes and over different ratios of training and testing datasets.
A. THE PROBLEM OF UNBALANCED DATASETS This is a common problem in many ML applications, but it is accentuated in IoT device classification due to the heterogeneous behavior of IoT devices: some devices, like plugs, generate sparse traffic, while others, like cameras, generate large amounts of traffic. This makes the detection of minority class devices difficult. Bai et al. [55]  report limited data for detecting hubs and Hsu et al. [92]  remark that it is difficult to distinguish smart plug traffic from IP cam traffic. Thus, having a balanced dataset is more important than the size of the dataset. Solutions based on data augmentation can be considered during the training phase  [48]. However, it is important to avoid introducing biases when over-representing minority classes. There is therefore a trade-off to consider to avoid overfitting the model.
D. DISCUSSING SCALABILITY Given the exponential growth of the number and types of IoT devices, it is crucial to design scalable solutions. Scalability must be considered at all stages of the solution design, as explained below. 1) Traffic collection: the collection must be quick, efficient, and non-exhaustive. For instance, data sampling  [104]  (i. e. , taking sufficiently representative samples rather than the entire dataset) can be used to improve scalability. However, the choice of the sampling solution must be well thought out as it may be inappropriate for minority and sparse traffic classes, which brings us back to the unbalanced dataset problem, discussed above in Sec. VIII-A. 2) Feature extraction: feature extraction should not be complex, long, or costly. It is important to choose a scalable method.
THE MON(IOT)R DATASET [38] 543 This data set examines IoT device information exposure. 544 It contains data from 81 IoT devices deployed in two labs 545 (one at Northeastern University in the United States and 546 the second at Imperial College London in the United King-547 dom) over 30 days between September 2018 and February 548 2019. Different types of traffic are provided: i) power traffic 549 97124 VOLUME 10, 2022.
IoT-23 DATASET [40] 559 IoT-23 is a dataset containing benign and malicious IoT net-560 work activity. The traffic was captured at the Czech Technical 561 University. The dataset contains 20 pcap files from infected 562 IoT devices, labeled by the malware that infected them, 563 and 3 pcap files containing benign network traffic generated 564 by 3 IoT devices: a smart lamp, a voice assistant, and a smart 565 door lock. The packet captures are labeled with the device 566 that generated the traffic. As done in [43], legitimate traffic 567 can be used for IoT device classification,. 568 10) HOW VALUABLE ARE PUBLIC DATASETS? 569 Public datasets enable comparing different solutions. Unfor-570 tunately, the available public datasets for IoT device clas-571 sification are scarce (only 5 of the surveyed papers shared 572 their datasets publicly) and not diversified: most provide idle 573 and interaction traffic, and capture at the gateway, when this 574 is not the most realistic scenario.
Since public datasets are 575 not diverse, researchers must collect their own data when 576 examining new scenarios. For instance, Yu et al. [44] identify 577 IoT devices based on passively receiving broadcast and multi-578 cast packets, and had to collect their own data from different 579 WiFi networks. In conclusion, additional datasets exploring 580 new classification scenarios should be released, and more 581 diversified IoT traffic needs to be collected, in order to boost 582 research on IoT device classification. As shown in Fig. 4, the 583 most used datasets are UNSW (30%), IoTSentinel (15%), and 584 YourThings (6%).
In supervised classification, labeled datasets are split into 1139 training, validation, and test datasets. Datasets can be 1140 separated chronologically or randomly. However, temporal 1141 partitioning better matches the real world scenario, when 1142 a classifier is trained on existing data and then tested on 1143 new data. Despite the cost of labeling and the difficulty of 1144 detecting new devices not included in training, supervised 1145 classification techniques are commonly employed in IoT 1146 device classification literature (84% of reviewed papers) due 1147 to their high accuracy and ease of implementation.
8 ) 8 THE MON(IOT)R DATASET [38] 543 This data set examines IoT device information exposure. 544 It contains data from 81 IoT devices deployed in two labs 545 (one at Northeastern University in the United States and 546 the second at Imperial College London in the United King-547 dom) over 30 days between September 2018 and February 548 2019. Different types of traffic are provided: i) power traffic 549 97124 VOLUME 10, 2022.
558 9 ) 9 IoT-23 DATASET [40] 559 IoT-23 is a dataset containing benign and malicious IoT net-560 work activity. The traffic was captured at the Czech Technical 561 University. The dataset contains 20 pcap files from infected 562 IoT devices, labeled by the malware that infected them, 563 and 3 pcap files containing benign network traffic generated 564 by 3 IoT devices: a smart lamp, a voice assistant, and a smart 565 door lock. The packet captures are labeled with the device 566 that generated the traffic. As done in [43], legitimate traffic 567 can be used for IoT device classification,. 568 10) HOW VALUABLE ARE PUBLIC DATASETS? 569 Public datasets enable comparing different solutions. Unfor-570 tunately, the available public datasets for IoT device clas-571 sification are scarce (only 5 of the surveyed papers shared 572 their datasets publicly) and not diversified: most provide idle 573 and interaction traffic, and capture at the gateway, when this 574 is not the most realistic scenario.