dataesr/scientific-paragraphs-categorization huggingface dataset:
Data availability. C1999-SMA and J2014-SMA used and revised in this work and mapped at 0. 0833 • × 0. 0833 • horizontal resolution, as well as the size-distributed multiannual means of the atmospheric mass mineral fractions from the C1999 and J2014 experiments at different pressure levels, are available at Zenodo (Gonçalves Ageitos et al. , 2023, https://doi. org/10. 5281/zenodo. 8091828). Supplement. The supplement related to this article is available online at: https://doi. org/10. 5194/acp-23-8623-2023-supplement.
Service providers provision more and more Internet-of-Things (IoT) services in the cloud for dynamicity and cost-effectiveness purposes. This is made possible thanks to the introduction of edge computing that brings additional computing and resources for analytics close to the data sources and thus enables meeting the low latency requirement. Edge nodes should support (i) the heterogeneity of IoT devices (e. g. , sensor, actuator) and (ii) characteristics (e. g. , mobility, location awareness). IoT is already integrated to the hybrid cloud/edge environment. However, the ecosystem lacks of automation due to the previously mentioned characteristics. Indeed, edge nodes are often manually selected during deployment time, and most of the regular Quality Of Service (QoS) management procedures remain difficult to implement.
Introduction Internet-of-Things (IoT) service owners are more and more leaning on hyperscale cloud providers to provision IoT services, as well as, to store and to process the generated data. This trend is motivated by the several advantages that the cloud computing paradigm brings such as dynamicity, flexibility and cost-effectiveness. However, with the ever-growing workloads that is tied to 5G telco network, automation, real-time analytics and IoT, service owners started looking elsewhere for their computing needs. Notably, edge computing enables meeting these computing needs. In fact, edge computing is currently reshaping IT and business computing. It brings additional resources close to the end-users and data sources so that the latency-sensitive computing and data analytics could be processed within a low network latency  [24].
Requirements The evaluation of related ontologies is based on conceptual requirements identified as follows: • CR1: "Cloud" constitutes the servers accessible on the Internet as well as software and databases which run on these servers. The servers located in the cloud are hosted in data centers distributed around the world. • CR2: "Edge" constitutes a computer that acts as an end user portal for communication with other nodes. In this paper, edge node allows the communication between cloud and IoT device. • CR3: "Location" constitutes context location of an entity in general (e. g. , device, actuator, cloud, edge). A distinction is made between physical place and virtual place. • CR3. 1: "Physical place" constitutes an absolute position with geographic coordinates. • CR3. 2: "Virtual place" constitutes a relative position of an entity.
This can facilitate useful data identification and reduce packet loss and transmission delay. Some applications need real-time treatment, which can be guaranteed by edge computing. Second, edge node support data heterogeneity of IoT devices. Finally, it is an opportunity to know the localization of an object which can be physical or virtual, the quality of service offered and information's about time. To put it in a nut shell, an Iot ontology based on the use of edge computing should integrate other concepts useful in addition to the concepts already presented in the literature in relation to device, life cycle, consumption and other. The next section gives an overview about these concepts and the different requirement which should be respected on the one hand. On the other hand, it details reused ontologies for the design process of our EdgeOnto ontology.
). • DUL:InformationEntity: A piece of information, be it concretely realized or not. It is a catchall class, intended to bypass the ambiguities of many data or text that could denote either an expression or a concrete realization of that expression. • DUL:Object: Any physical, social, or mental object, or a substance. Following DOLCE Full, objects are always participating in some event (at least their own life), and are spatially located. • DUL:Quality: Any aspect of an Entity (but not a part of it), which cannot exist without that Entity. For example, the way the surface of a specific PhysicalObject looks like, or the specific light of a place at a certain time, are examples of Quality, while the encoding of a Quality into, e. g. , a PhysicalAttribute should be modeled as a Region. • DUL:Situation: A view, consistent with ('satisfying') a Description, on a set of entities.
EdgeOnto architecture and specifications This section firstly discusses the several modules (i. e. , IoT, location and time-awareness, and QoS management) that make up the EdgeOnto core domain. Our ontology is OWLbased (ontology web language). For each module, we detail the multiple defined concepts, as well as, the relations between them. Then, the second subsection formalizes the end user requests to query EdgeOnto. This is followed by the description of the associated semantic matching procedure.
) and edge computing (edge, cloud). • Time and location module: describes the main concepts related to time and location of each object in EdgeOnto (for example, location of an IoT-Thing in point of time, location of an edge node in instant t, location of a cloud node, etc. ) • QoS module: describes the quality of service between EdgeOnto objects (IoT devices, edge and clouds). A set of semantic relations are defined in EdgeOnto represented in Table  2.
IoT module This dimension encompasses four main concepts, namely EdgeOnto:Cloud, EdgeOnto:Edge, EdgeOnto:Resource and IoT-O:IoT-Thing (Fig. 3 ). -A node cloud (datacenter, database, server, etc. ) is related to cloud computing which is defined by NIST as "a model for enabling convenient, on demand network access to a shared pool of configurable computing resources (e. g. , networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction". -An edge node (switcher, router, small/macro base station, etc. ) is a node that acts as an end user portal for communication with other nodes (cloud node or device node). -Edge nodes are located between device (EdgeOnto:Device) and cloud (EdgeOnto:Cloud) and can facilitate computations nearer to the source of data (or where data is generated) and can incorporate strategies for remotely enhancing capabilities of front-end devices  [30].
QOS management module QoS is the description or measurement of the overall performance of a particularly the performance seen by the users of the network. Different measurements are taken into account to characterize and evaluate QoS like availability, performance, reliability, scalability, and security. The ontology proposed by  [31]  uses the DUL ontology and define the concept of Quality of Service (DO:QualityOfService) as a DUL:Information Object. In IoT-O, reference  [29]  reuses also the DUL:Information Object which "is a piece of information, be concretely realized or not. It is intended to bypass the ambiguities of many data or text". By aligning ontology of  [31]  and IoT-O, DO: QualityOfService is treated in this paper as an DUL:Information Object which is DUL:Information Entity (Fig.
Large Language Models (LLMs) have exhibited remarkable performance on various Natural Language Processing (NLP) tasks. However, there is a current hot debate regarding their reasoning capacity. In this paper, we examine the performance of GPT-3. 5, GPT-4, and BARD models, by performing a thorough technical evaluation on different reasoning tasks across eleven distinct datasets. Our paper provides empirical evidence showcasing the superior performance of ChatGPT-4 in comparison to both ChatGPT-3. 5 and BARD in zero-shot setting throughout almost all evaluated tasks. While the superiority of GPT-4 compared to GPT-3. 5 might be explained by its larger size and NLP efficiency, this was not evident for BARD. We also demonstrate that the three models show limited proficiency in Inductive, Mathematical, and Multi-hop Reasoning Tasks.
, 2018a,b; Ouyang et al. , 2022). It is considered as one of the most revolutionary LLMs because it has made a profound impression on the world with its remarkable ability to engage in coherent interactions with users. Unfortunately, ChatGPT-3. 5 faces significant challenges in certain areas, such as mathematics  (Davis, 2023; Frieder et al. , 2023a) , and commonsense tasks  (Guo et al. , 2023). To address this issue, OpenAI developed ChatGPT-4, an enhanced version of the chatbot that is based on GPT-4 model. While ChatGPT-3. 5 is built upon In-structGPT  (Ouyang et al. , 2022) , ChatGPT-4 is a large language model that employs fine-tuning and Reinforcement Learning from Human Feedback (RLHF)  (Christiano et al. , 2017). It is a robust multimodal model that can process both textual and visual inputs and generate text-based outputs.
After the successful release of ChatGPT-3. 5, Google took further strides in the AI landscape by introducing their innovative large language model, called BARD  (Manyika, 2023). Through an intensive process of gathering human feedback and rigorous evaluation, Google refined and optimized BARD to create a lightweight version of LaMDA. Currently, BARD remains in the experimental stage, yet it has already made a remarkable impact. Users can access it through an intuitive interface, enabling direct collaboration with a generative AI system. This paper aims to investigate the reasoning capabilities of ChatGPT-3. 5, BARD and ChatGPT-4. We notably build on previous work from the state of the art  (Bang et al. , 2023)  and propose the following contributions: • We manually evaluated the reasoning abilities of both ChatGPT models, and Google's BARD by conducting evaluations covering a range of reasoning domains, such as deductive, inductive, abductive, analogical, causal, and multi-hop reasoning, through question-answering tasks.
, 2021)  model that has been trained on both programming languages and natural language. InstructGPT is built on code-davinci-002, and text-davinci-002 is the most powerful GPT-3. 5 model when compared to previous GPT-3 models. This advanced model is renowned for its exceptional language generation quality and superior instruction-following abilities. Several studies have performed experiments to assess the reasoning capabilities of ChatGPT-3. 5 across various domains. For instance,  Bang et al. (2023)  have proposed a framework to assess zeroshot setting on multitask, multilingual and multimodal aspects of. Their research focused on zero-shot setting and revealed interesting findings regarding ChatGPT-3. 5's performance in reasoning tasks. For instance, the model demonstrated a stronger proficiency in deductive reasoning while struggling with inductive reasoning.
As well, their study highlighted limitations in ChatGPT-3. 5's ability to solve mathematical problems. To delve further into the mathematical reasoning skills of  ChatGPT-3. 5, Frieder et al. (2023b)  conducted experiments using a dataset called GHOSTS. This dataset consists of natural language prompts for graduate-level mathematics problems. Moreover,  Guo et al. (2023)  explored ChatGPT's performance across various domains, including finance, medicine, law, and psychology. ChatGPT-4 -It is an improved version of ChatGPT-3. 5. GPT-4 (Ope-nIA, 2023a) is the most powerful large multimodal model released by OpenAI. This model has the ability to process both textual and visual inputs and generate textual results. One of the key changes in this model compared to previous GPT models lies in the computation of the loss function.
The authors successfully predicted the final loss of GPT-4 in their internal codebase by incorporating a scaling law with an irreducible loss term. ChatGPT-4 is built upon the GPT-4 model, which has been trained using an unparalleled amount of computational power and data  (Bubeck et al. , 2023). Although GPT-4 has shown significant improvements compared to earlier GPT models, it still possesses limitations similar to its predecessors. These limitations encompass generating false information, providing harmful advice, having a restricted context window, and lacking the ability to learn from experience. Furthermore, the authors lead a set of assessments of the model's performance using diverse datasets, including academic exams. The results demonstrated that GPT-4 achieved a human-level performance on most of these tests.
Logical reasoning 4. 1. 1. Deductive reasoning EntailmentBank  (Dalvi et al. , 2021)  -it is a multi-step entailment dataset designed to generate explanations in the form of entailment trees. It consists of seven fields:  ID, context, question, answer, hypothesis, proof, meta. Additionally, the authors have defined three tasks that gradually increase in difficulty and simplify the inherent problems of the task. bAbI (task 15)  (Weston et al. , 2016)  -Basic Deduction. In this task, the machine learning model is presented with a set of premises and must deduce a conclusion based on those premises. The task tests the model's ability to perform deductive reasoning. It has 1000 questions for training, and 1000 for testing.
Commonsense reasoning CommonsenseQA  (Talmor et al. , 2019)  -it comprises 12,247 multiplechoice questions, each question has five possible answers. The questions are crafted to evaluate a system's capacity to reason through commonsense knowledge. The topics covered in the assessment are diverse and include cause-and-effect relationships, spatial reasoning, social norms, time and sequencing, and physical laws, applied to everyday scenarios and situations. PiQA  (Bisk et al. , 2019)  -It consists of 2,000 multiple-choice questions designed to evaluate common sense reasoning in NLP models. The questions cover a range of topics, including physics, biology, social behavior, and more. The dataset is divided into two parts: a training set with 1500 questions and a test set with 500 questions. Pep-3k  (Wang et al.
Causal reasoning e-Care (CAusal REasoning)  (Du et al. , 2022)  -It is a human-annotated dataset that explains causal reasoning. Given a premise and two hypotheses, the goal is to select the hypothesis that better explains the cause or effect of the premise. The dataset contains 21,324 causal questions (train set = 14928, dev set = 2132, test set = 4264), and 13,048 unique explanations (train set = 10491, dev set = 2102, test set = 3814).
Data selection We randomly selected 30 samples from each evaluation dataset, depending on their availability from either the train, dev, or test sets. For example, we chose 10 samples each from the training-easy, trainmedium, and train-hard sets of data in the Math dataset. Meanwhile, we selected 30 samples from the dev set for 𝛼NLI, CommonsenseQA, HotpotQA, PiQA, Pep-3k and E-Care. Finally, we took 30 samples from the test set for the bAbI (task 15), bAbI (task 16), CLUTRR, and EntailmentBank datasets. The exact set of samples selected in each dataset are in Appendix.
Prompt Engineering Since the LLMs have exhibited enhanced zero-shot reasoning abilities when provided with effective prompts  (Kojima and Shane Gu, 2022; Liu et al. , 2023) , we opted to employ a combination of wellestablished, high-performing prompts and our original proposals. In the following list, we outline the specific prompt used for each dataset. Prompts marked with a † symbol represent those that we have proposed. • HotpotQA - † Given the context: + context. Based on the given context above, answer the question: + question.
Results and discussion In this section, we present the scores obtained from our conducted experiments. It is important to mention that we compared our results with the findings of  Bang et al. (2023). However, since the samples from  Bang et al. (2023)  are not publicly available, we had to take measures to ensure a fair comparison. Therefore, we selected an equal number of samples and conducted evaluations using  BARD,  Additionally, not all engineering prompts used by  Bang et al. (2023)  are publicly accessible. The publicly available prompts are for the bAbI (task 15), bAbI (task 16), and Pep-3k datasets. For the remaining datasets, we utilize the prompt engineering approach we presented in Section 5. 2. Inductive Reasoning. According to the study conducted by  Bang et al. (2023) , it has been observed that although GPT-3.
5 is a highly advanced language model, it tends to struggle more with inductive reasoning than with deductive or abductive reasoning. Authors initially evaluated the model's basic reasoning abilities using the bAbI-task 16 dataset (Table  2 ), both with and without the use of prompt engineering. Their reported results indicate a significant improvement in scores when prompt engineering was employed (20/30), in contrary to not using prompt engineering (0/30). In our experiments, we used the same model (GPT-3. 5) as  Bang et al. (2023)  and the same number of samples. With prompt engineering, we obtained a score of 14/30, but without prompt engineering, our score decreased to 1/30. Nevertheless,  provided the correct answer by relying more on general knowledge of the real world rather than the specific given facts.
Here is the sample we tested: Brian is a frog. Julius is a swan. Bernhard is a rhino. Greg is a swan. Brian is green. Bernhard is green. Greg is gray. Julius is gray. Lily is a frog. What color is Lily? Correct answer: green ChatGPT-3. 5 answer: Based on the given information, Lily is a frog, and frogs are typically associated with the color green. Therefore, it can be inferred that Lily is green. Additionally, we evaluated the performance of ChatGPT-4 on the bAbI-task 16 dataset, both with and without prompt engineering. In this particular case, the impact of prompt engineering on the results was significant, with a notable improvement from a score of 2/30 without prompt engineering to an impressive score of 28/30 when using prompt engineering. Furthermore, the absence of prompt engineering highlighted the inability of  to deduce the answer solely from the given information.
Similarly, we evaluated bAbI-task 16 dataset using the Google's BARD model. Through prompt engineering, the model improved its performance, by achieving a score of 20/30 compared to 15/30 without prompt engineering. It is noteworthy to mention that BARD outperformed both GPT-3. 5 and GPT-4 when prompt engineering was not employed, indicating its ability to reason effectively without relying on strong guidance. However, even though BARD outperformed GPT-3. 5 using prompt engineering, it did not outperform GPT-4. This suggests that while BARD is promising, further advancements may be necessary to overcome the capabilities of  Furthermore, in line with the experiment conducted by  Bang et al. (2023) , we evaluated the improved logical thinking skills in inductive tasks using the CLUTRR dataset and prompt engineering (see Table  2 ).
Deductive Reasoning. Similarly to  Bang et al. (2023) , we investigate the basic reasoning skills using the bAbI -task 15 dataset, both with and without prompt engineering. Unlike inductive reasoning, ChatGPT-4 demonstrates significantly enhanced performance, particularly in tasks without prompt engineering. According to the findings reported by  Bang et al. (2023) , the use of prompt engineering proves to be advantageous in generating answers to questions that are based on the provided facts (see Table  3 ). However, in our experiments, when we employed the same number of samples and the same chat version (ChatGPT-3. 5), we achieved a higher score without prompt engineering (28/30) compared to using prompt engineering (26/30). One possible explanation for these results could be that the model encounters more questions without prompt engineering, thus having already learned to provide correct answers implicitly.
The initial statements claim that wolves are afraid of mice, but these are contradictory and not logical, as wolves are typically predators and not afraid of mice. Building on our exploration of advanced reasoning, we leveraged the EntailmentBank dataset along with prompt engineering techniques to further enhance our study. The results in Table  3  show that the research conducted by  Bang et al. (2023)  achieved better performance than our findings. They used the same model and number of samples but different samples, obtaining a score of 28 out of 30, while we scored 25 out of 30. Furthermore, our experiments revealed that ChatGPT-4 and BARD performed better than ChatGPT-3. 5, scoring 27 out of 30 compared to 25 out of 30, but this difference was not statistically significant. In this specific case, we observed that both ChatGPT-3.
Despite the advancements exhibited by GPT-4, there remains a need to further refine its capability to address specific queries based on provided texts. In comparison with ChatGPT, BARD tends to produce more extensive answers. Remarkably, when the model's intention shifts towards generating images rather than providing textual answers, it tends to focus on the topic of the moon. Abdutive Reasoning. To evaluate the abductive reasoning capabilities of GPT-4 and BARD, we utilized the 𝛼NLI dataset with prompt engineering. The scores reported by  Bang et al. (2023)  demonstrated superior performance compared to our results using , , and BARD (22/30) (see Table  4 ). In our experiments, ChatGPT-4 demonstrated superior performance compared to ChatGPT-3. 5 and Google's BARD. It is worth noting that they failed in different examples, with only one instance in common (sample 28): Input: Given the following two observations: obs1: Ryan climbed the long set of stairs to get to the water slide.
𝛼NLI dataset w/ prompt eng. Bang et al. (2023)  26/30  19/30 GPT-4 25/30 BARD 22/30 or fear to climb the stairs, which can lead to think Ryan might be happy to climb. On the other hand, the second observation shows that Ryan had expectations about the water, that might be negative, but what he found was better than his expectations. The latter observation has a tiny more impact on the phrase than the first one, and the nuance between the two observations was not detected by any model. Mathematical Reasoning. To compare the mathematical skills of Chat-GPT-4, BARD and ChatGPT-3. 5, we utilize the Math dataset. While the scores presented in Table  5  reflect the performance of  Bang et al. (2023) , it is important to note that our experiments utilized different samples. Notably, the discrepancy between our scores and theirs can vary significantly due to the varying difficulty levels of the mathematical problem samples.
Mathematical reasoning tasks Math dataset w/ prompt eng. Bang et al. (2023)  13/30 GPT-3. 5 15/30 GPT-4 20/30 BARD 16/30 these categories. On the other hand, when it comes to resolving difficult problems, BARD remains ahead. While ChatGPT-4 boasts a superior overall score, its specialized strengths differ from those of BARD, each excelling in distinct problem-solving areas. Furthermore, GPT-4 faces challenges when it comes to finding the prime factors of a given number and converting numbers to different bases. Additionally, while it provides accurate answers in certain instances, it does not always simplify the responses to the minimum expression. However, there is significant room for improvement in the mathematical skills. In this paper, we have only presented thirty samples, which do not encompass all the diverse areas of mathematics.