dataesr/scientific-paragraphs-categorization huggingface dataset:
Commonsense Reasoning. Following  Bang et al. (2023) , in this type of reasoning we evaluate three datasets: CommonsenseQA, PiQA, and Pep-3K using prompt engineering. The results are presented in Table  6. In terms of CommonsenseQA dataset,  Bang et al. (2023)  achieved the highest score (27/30), while our experiments with ChatGPT-3. 5, ChatGPT-4 and BARD obtained scores of 21/30, 25/30, and 20/30, respectively. It is worth noting that our 30 samples differ from the ones evaluated by  Bang et al. (2023) , but their samples are not publicly available. Interestingly, four of the answers (samples 11, 15, 24, and 28 in Table  13 ) were incorrectly answered by all three models. For example, in question 11, ''The cat carefully navigated the area, doing everything they can to avoid what?'', with answer options '  'A.
This is because a dining room could indeed have a sideboard on the counter, making the answer less straightforward. In the context of the PiQA dataset, our results surpassed the score achieved by  Bang et al. (2023)  (see Table  6 ). They obtained a score of 25 out of 30, while ChatGPT-4 almost achieved a perfect score of 29 out of 30. Furthermore, ChatGPT-3. 5 also performed competitively, scoring 24 out of 30. Comparatively, the BARD model scored 23 out of 30, making it a strong contender. Interestingly, the errors made by both GPT models were unique to each model. Contrarily, there were some common errors between both GPT models and the BARD model. Let us examine an example that illustrates an incorrect answer given by ChatGPT-3. 5 and BARD (sample 10). The question is, ''Complete the phrase with one of two answer options: To attach rope to latex, you can sol1: Rubber band the rope in place on the latex sol2: Glue the rope in place on the latex''.
Finally, in the Pep-3K dataset, we applied the prompt engineering technique suggested by  Bang et al. (2023). They achieved 28 out of 30 correct answers. However, when we used the same model (ChatGPT-3. 5) and sample size, the results yielded 15 correct answers. As expected, the newer model, ChatGPT-4, outperformed its predecessor, reaching 18 correct answers in our experiments. However, what was truly surprising was the performance of BARD, which surpassed even ChatGPT-4 by achieving 19 out of 30 correct answers. It is worth mentioning that there are eight incorrect responses common to all three models, indicating potential areas of improvement or challenges in handling those specific queries. To begin with, we concentrate on analyzing the shared errors in both GPT models. Our observation suggests that these errors arise due to their tendency to generalize.
For instance, let us consider the first sample listed in Table  13  under the Pep-3K dataset section. In this case, the models were tasked with assessing the plausibility of the statement ''dog ride cat''. Both models argued that dogs are generally larger than cats, and it would be unusual for a cat to be able to support the weight of a dog. However, we, as humans, are aware that there are dogs that are smaller or of equivalent size to cats, making it plausible for a dog to ride a cat. Furthermore, let us consider a sample where all three models failed in generating a suitable response (sample four). In this instance, the models were tasked with evaluating the plausibility of the sentence: ''clown catch baboon''. This specific sentence allows us to observe distinct reasoning approaches in each model.
ChatGPT-3. 5 confidently categorized it as implausible, attributing the decision to its training data. ChatGPT-4 shared a similar conclusion but recognized a possibility, albeit unlikely. In contrast, BARD not only labeled the sentence as implausible but also incorporated common knowledge of the world into its reasoning process. Causal Reasoning. In alignment with the research conducted by  Bang et al. (2023) , which explored causal reasoning using the E-Care dataset and the ChatGPT-3. 5 model, both their study and our own experiments with BARD yielded 24 correct answers out of 30 questions (see Table  7 ). However, during our experiments, we encountered challenges with ChatGPT-3. 5. Specifically, it faced challenges in accurately connecting premises with the appropriate hypotheses, providing only 14 correct answers out of 30.
Multi-hop Reasoning. To evaluate multi-hop reasoning skills, we utilized the HotpotQA dataset. In a study conducted by  Bang et al. (2023) , they achieved 8 correct samples out of 30 (refer to Table  8  for complete results). However, when we employed the same number of samples with ChatGPT-3. 5, we attained 16 correct answers out of 30. Using these identical samples, we compared the performance of ChatGPT-4, which outperformed ChatGPT-3. 5 with 17 correct answers out of 30. Since the difference is not significant, it led us to conclude that ChatGPT-4 still faces challenges when integrating information from multiple sources to determine an answer. We observed that both models struggle with answers when the context is extensive, particularly when it involves people related with specific dates.
To enhance our understanding of mathematical skills in ChatGPT-4 and ChatGPT-3. 5, we intend to conduct an exhaustive evaluation for each specific area, ensuring a deeper and more comprehensive analysis. • It would be interesting to enhance the scope of our research by evaluating our samples across more Language Models (LLMs). This will be helpful to have a better understanding of the data, and a wider comparative study of state-of-the-art models.
Dataset: bAbI (task 15) (continued on next page)      Based on the given facts, we know that mice are afraid of cats. Since Emily is a mouse, we can deduce that Emily is afraid of cats. Based on the given facts, we can use deductive reasoning to infer that Emily is afraid of cats. Here is the reasoning: -Mice are afraid of cats. -Emily is a mouse. -Therefore, Emily is afraid of cats. We can also infer that Emily is afraid of wolves, but this is not as certain as the first inference. This is because we only know that wolves are afraid of mice, not that they are afraid of mice specifically. However, it is reasonable to assume that Emily would be afraid of wolves because they are predators that could potentially harm her. Based on the given facts, we know that Gertrude is a mouse. It is stated that mice are afraid of wolves.
D Dataset: PiQA 1. Complete the phrase with one of two answer options: Use peroxide to lighten hair sol1: Put peroxide on the parts of your hair that you want to lighten. Leave on overnight or longer for best results sol2: Put peroxide on the parts of your hair that you want to lighten. Wait 30 min and rinse. Both solutions are commonly suggested methods for lightening hair using peroxide, but they differ in the duration of application. Solution 1: Put peroxide on the parts of your hair that you want to lighten. Leave on overnight or longer for best results. This solution suggests applying peroxide to the hair and leaving it on for an extended period, such as overnight, to allow for maximum lightening. However, this method may not be suitable for everyone, as leaving peroxide on the hair for too long can cause damage and dryness.
Marianne is not mentioned in any of the other relationships, so the most likely explanation is that she is Franklin's sister. In addition, the fact that Marianne is watch- ing a golf tournament with Stephanie suggests that they are close in age. This further sup- ports the theory that Marianne is Franklin's sister, as siblings are often close in age. Therefore, based on the information provided in the story, the most likely relationship between Marianne and Franklin is that they are siblings. Dataset: bAbI (task16) (continued on next page).
He was one of the leaders of the later Irish Home Rule movement and promoter of the Housing of the Working Classes (Ireland) Act 1908, known as the Clancy Act. Called to the Irish Table 15 Multi-hop reasoning prompts. Multi-hop Reasoning Prompt GPT-3. 5 An- GPT-4 BARD Gold swer Answer Answer Dataset: HotpotQA 1. Given the context: [John Franks (judge)] Franks was the second son of Thomas Franks (1729- It is Always Robbie Robbie It is 1787), of Ballymagooly, Bar in 1887 he became a KC (King's Counsel) in 1906. [Thomas O'Shaughnessy] O'Shaughnessy Sunny Philadelphia in Tucker has appeared in the series ''It Sunny in Philadel-phia,'' which follows a group of friends who run an Irish bar. is Always Tucker plays in the series It is Always Philadelphia that follows a group of friends who run an Irish bar.
[Phyteuma] Phyteuma is a genus of flowering plants in the family Campanulaceae, native to Europe and Morocco. [Zygotritonia] Zygotritonia is a genus of herbaceous, perennial and bulbous plants in the Iris family (Iridaceae). It contains four species distributed in Tropical Africa. The genus name is derived from the word zygomorphic (meaning bilabiate), and the apparent resemblance to some species in the genus ''Tritonia''. [List of plants by common name] This is a list of plants organized by their common names. However, the common names of plants often vary from region to region, which is why most plant encyclopedias refer to plants using their scientific names, in other words using binomials or ''Latin'' names. [International Plant Names Index] The International Plant Names Index (IPNI) describes itself as ''a database of the names and associated basic bibliographical details of seed plants, ferns and lycophytes.
Donald Wilson was a database pioneer and entrepreneur. He was also the first president and one of the principal creators of the Lexis legal information system, and Nexis. An attorney by training who became an information industry innovator and a venture capital consultant to numerous businesses, Mr. Wilson was also an internationalist and a conservationist. At the time of his death, he was chairman of Lessac Technologies Inc. , a text-to-voice software venture based on nearly fifty years of partnership with Arthur Lessac. [William Scott Wilson] William Scott Wilson (born 1944 in Nashville, Tennessee) is known for translating several works of Japanese literature, mostly those relating to the martial tradition of that country. He is recognized by The American Literary Translator's Association (ALTA) as ''today's foremost translator of classic Samurai texts.
This paper proposes an innovative graphical user interface to visualize the attitude of a sensing device in a threedimensional space, serving a wide-range of medical applications. Based on inertial measurement units (IMU) or on Magnetic, Angular Rate and Gravity sensors (MARG), the proposed tool estimates Euler angles using a predefined attitude filter to display the orientation of the device relative to the Earth frame in real-time. The device is schematized by linking six polygonal regions, and is subject to sequential rotations by updating the graph each 350 ms. Two public datasets were used to demonstrate the value of this contribution in fall risk assessment and body posture monitoring for elderly care.
Now, sensor fusion techniques have been developed to estimate the attitude of sensing devices  [7]. In other words, the orientation of the device relative to the Earth frame is measured by merging acquired data from a tri-axial accelerometer and a tri-axial gyroscope, or from these two sensors and a tri-axial magnetometer. These approaches are useful for human motion tracking  [8] , which is the base of several medical solutions. Two types of accelerometers exist. The first one concerns capacitive accelerometers, also known as Micro-Electro-Mechanical Systems (MEMS). These accelerometers depend on a change in electrical capacitance to measure acceleration. The second type refers to piezoelectric accelerometer, which is made of a quartz crystal. A force acting on the piezoelectric element is produced when the accelerometer is moving.
III. REAL-TIME ORIENTATION DISPLAY VIA GRAPHICAL USER INTERFACE A graphical user interface (GUI) tool was developed on Matlab to display the orientation of a sensing device based on its acquired data. The sensing device in Fig. 1  is symbolized by plotting three pairs of polygonal regions. Each of these six regions represents one face of this device. The color of the frontal zone is set to grey (  ), and that of the backward zone to white (  ), while the remaining faces are colored in dark blue (  ). The three hexadecimal color codes ('r', 'g', 'b') are as follows: {   → (. ,. ,. )   → (, , )   → (,. ,. ) Moreover, the six regions are labeled using the following letters: 'U' which stands for up, 'D' for down, 'L' for left, 'R' for right, 'F' for front, and 'B' for back. The color of these letters is set to yellow.
A. Fall risk assessment Falls are one of the main causes of mortality in elderly. While most of the proposed solutions target fall detection, the clinician may need to evaluate this fall by observing the successive movements of the wearable device during this fall. The initial state of the human body (standing, sitting, lying, walking, etc. ) which precedes the fall, the velocity of the impact, and the direction of the fall (forward, backward, lateral) are important factors from a medical point of view. Our tool answers all these questions and provides an addedvalue for the practitioner. To address the real world situation, we resorted to the FallAllD public dataset  [15]. This dataset contains acquired data from tri-axial accelerometers, tri-axial gyroscopes, and tri-axial magnetometers, including human falls.
B. Body posture monitoring Another medical application is the distribution of the body posture over the course of a day. This information is valuable for health monitoring and frailty prevention in elderly care. Hence, we resorted to another public dataset  [16] , available in the UCI Machine Learning Repository  [17]. The subjects have worn the smartphone on the waist and simulated six different activities. This dataset only contains data acquired from an accelerometer and a gyroscope. Hence, the considered sensing unit in this sub-section is IMU. The reliability of this type is sufficient for this application, since the estimation of the body posture is done while the subject is inactive. The targeted classes for this task are (i) Sitting, (ii) Standing, and (iii) Lying. Euler angles were estimated from 2.
VI. CONCLUSION AND PERSPECTIVES This paper discussed the attitude estimation of inertial sensors. Based on widely used orientation filters, a GUI tool was developed to visualize the orientation of a rigid body (a sensing device) in a three-dimensional space. Such tool is important for a wide-range of real world applications in elderly care, as seen in section IV while using public datasets. In a future work, the orientation of the human body (instead of the device) is going to be considered using wearable sensors. The output of the proposed tool is going to p. 5 Colloque JETSAN 2021 be exploited in order to constitute a monitoring system by developing a human activity recognition process.
IoT devices use different means of data transmission, which is the most energy-hungry process. Bluetooth Low Energy (BLE) is one of the most popular protocols; in fact, BLE is widely adopted for devices with limited memory and size constraints due to its low cost, high throughput, and low power communication compared to other existing short-range communication modules such as ZigBee, Bluetooth Classic (BR/EDR), and ANT as reported in  [10] [11] [12]. These features made it a suitable choice for IoT applications, and the fact that it is also available on all smartphones significantly influences its adoption. Recent research showed how edge computing for LTE terminals could reduce their energy consumption and communication latency compared to a centralized cloud  [13]. To the best of our knowledge, no study has evaluated the power consumption of smart wearable devices transmitting raw data to a remote/cloud deep learning model versus edge computing on an embedded AI model.
Therefore, we developed a HAR system capable of classifying seven activities using accelerometer data. We aim to prove that using edge AI on IoT devices instead of cloud AI can extend battery life and guarantee data security. • We first trained and evaluated a DCNN using accelerometer data from a public dataset. • The DCNN was then converted to a C++ Flatbuffer using TensorFlow Lite Micro (Tflite Micro)  [14]. Afterward, we compiled and ported it into an Arduino Nano 33 BLE Microcontroller. • We then measured and compared the power consumption and accuracy of the complete system in two configurations: -Microcontroller with no AI capabilities sending data at a frequency of 59 Hz to a remote server using BLE. -Microcontroller with an embedded DCNN model sending only the model classification output to a remote server using BLE.
Our paper makes the following contributions: (a) a quantitative analysis and evaluation of energy consumption in HAR systems utilizing cloud-based AI compared to edge AI, (b) using real measurements, we quantify energy usage in various scenarios, emphasizing the advantages and trade-offs of edge AI, and (c) we offer insights and recommendations for designing energy-efficient devices in real-world IoT deployments. The rest of the article is organized as follows. We review related existing work in Section 2. In Section 3, we present the materials and methods used to conduct this study, and we will discuss the dataset used, deep learning model architecture, and offdevice training results. Section 4 describes the hardware, signal acquisition methods, signal processing, and embedded deep learning model features.
Materials and methods This study used a publicly available dataset called the ''Sensor activity dataset". This dataset was released in 2014 and is available in the UTWENTE datasets  [15]. This dataset has 630,000 samples from ten participants aged 25 and 30. It combines seven basic daily activities labeled as follows: • Jogging • Biking • Standing • Sitting • Walking • Upstairs • Downstairs Participants had smartphones to collect data from five body positions: upper arm, wrist, belt, right, and left jeans pocket. According to the Nyquist theorem, capturing any signal requires that the sampling rate be at least double the frequency of the signal we want to acquire. The acceleration signal related to human motion is found below 15 Hz, and 98% of the signal power is below 10 Hz  [27]. Hence, a sampling frequency greater than 20 Hz would be enough for human activity data acquisition.
In this dataset, they collected data at a frequency of 50 Hz. Other datasets have shown that frequencies equal to or below 50 samples per second are sufficient for HAR applications  [16, 28]. The collected signal is then divided into small segments using a sliding window of two seconds and 50% overlap, that is, 100 samples per window. The dataset contains ten files; we began by shuffling the order of the files and concatenating them to create one single file. We then separated the data by position and created an additional feature corresponding to the magnitude of each sensor value. The magnitude is calculated using the following Eq. (  1 ). 𝑀𝑎𝑔𝑛𝑖𝑡𝑢𝑑𝑒 = √ (𝑥 2 + 𝑦 2 + 𝑧 2 ). (1) Type of data collected: • Triaxial acceleration • Triaxial angular velocity • Triaxial magnetic field The collected data are then reorganized into segments of 100 samples corresponding to a 2 s window, as previously described.
After this process, we split the data into 80% training and 20% testing; during the training phase, 20% of the training samples are used for validation. Fig. 1  is a data plot from accelerometer data of a sample of each activity collected from the jeans pockets. We can notice the visual similarity between activities like upstairs, downstairs and walking. Static activities (i. e. , sitting and standing) and dynamic activities can also be visually detected, except for biking which can easily be mistaken for a semi-static activity due to minor amplitude variations; this can be explained by the fact that an apartment bike was used in this dataset instead of biking outdoors. These variations also depend on the smartphone's position; that is, the acceleration from the 'belt' position will vary less than the 'pockets' and 'wrist' position during biking activity.
Model performance The performance of this model is evaluated with the following metrics: 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = 𝑇 𝑃 𝑇 𝑃 + 𝐹 𝑃 + 𝑇 𝑁 + 𝐹 𝑁 (2a) 𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = 𝑇 𝑃 𝑇 𝑃 + 𝐹 𝑃 (2b) 𝑅𝑒𝑐𝑎𝑙𝑙 = 𝑇 𝑃 𝑇 𝑃 + 𝐹 𝑁 (2c) 𝐹 1 -𝑆𝑐𝑜𝑟𝑒 = 2 × 𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 × 𝑅𝑒𝑐𝑎𝑙𝑙 𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑅𝑒𝑐𝑎𝑙𝑙 (2d) Here, TP stands for true positives, FP for false positives, TN for true negatives, and FN for false negatives. We then trained the DCNN model on the PC side. In this study, we used a PC with Windows 10 operating system, a dedicated graphics card, NVIDIA Quadro P100, Python 3. 9, and Tensorflow version 2. 6.
Training results Table  2  shows the training results on the test data. Our model performs better on data from jeans pockets (we combined the left and right jeans pockets) with an F1 score of 98. 8%. The next step is to evaluate our model and quantify the overall energy consumption on a microcontroller device. We selected an Arduino Nano 33 BLE board for this purpose. It is a TensorFlow Lite Micro (TFLM) compatible board with an Arm Cortex-M4 processor, BLE capabilities, and multiple sensors. We used the LSM9DS1 available on this development board to collect accelerometer data and perform inferences. To compare and discuss the energy cost between performing inference on an edge device (DCNN model embedded in the microcontroller) and performing inference on a PC or cloud, we implemented and studied four different use case scenarios.
Embedded AI on microcontroller Scenarios II, III, and IV use an AI model embedded in the microcontroller; to do this, we first converted our DCNN model to a C++ flatbuffer (array), capable of running directly on the Arduino board after compilation. We use a Python API of TensorFlow Lite Converter to do this conversion  [32]. However, not all the layers and operations available in TensorFlow and TensorFlow Lite libraries are available in Tensorflow Lite Micro. As presented in Table  4 , the converted model has more layers than the initial DCNN model; the Tflite Micro API translates the unavailable layers into equivalent layers, i. e. , the 2D-Convolution layer replaces the original 1D-Convolution layer used during training, but the classification accuracy and precision remain the same as the original model (presented on Table  5 ).
Data transmission to the remote server In Scenario I. , we used the Arduino to collect accelerometer samples and their transmission via BLE to a remote PC, which performs inference and outputs the highly probable activity. The model expects 100 samples of 4 features: acceleration [x, y, z] and its magnitude M. The BLE radio module is configured to minimize energy consumption with maximum throughput. The development board supports BLE version 5, which is on the LE 2M PHY, capable of transmitting data with a data rate of 2 Mbps in theory  [33] ; in practice, the data rate is less than the specifications but remains sufficient for our application. We used the most optimal configurations to minimize energy consumption and data loss. We use the notify and write without response operations to limit the wait for ACK packets (handshakes) and retransmission of failed packets.