dataesr/scientific-paragraphs-categorization huggingface dataset:
CONCLUSION AND PERSPECTIVES In this paper, we presented the context of industrial cybersecurity and the need for dedicated automated tools. To this end, we proposed a configuration file used to build a database of industrial communication protocols and devices which can be customized and incremented by modelers. Then, we presented our tool which automatically generates a MTMT template based on the data flows and devices defined in the configuration file. In addition, we explained our methodology for gathering component CVEs and matching them to STRIDE and thus integrating the CVEs within MTMT. These tools allow companies to quickly build a large database of industrial components to efficiently design threat models dedicated to their project and thus better integrate cybersecurity into their systems.
As a side note, we think that our contributions could fit in other fields than industrial systems. However, as ICS are our primary focus, we left the validation of this statement for future work and presently claim our contributions mainly regarding ICS. In a future work, we plan to further improve the filtering of CVE based on stencil characteristics, such as the firmware version of the device, the version of the software or the configuration parameters like the activation or deactivation of some network protocols. Moreover, we would like to provide a mapping method between protection profile and STRIDE. Finally, we will extend our ICS template with the other ten protection profiles provided by ANSSI to permit model more complex industrial architectures as DCS or SCADA.
JWST As part of the JWST's ERO of SMACS J0723, the observatory also acquired spectroscopic data with the Micro Shutter Array (MSA) NIRSpec of 58 individual galaxies, as well as spectra of all objects in the entire field with NIRISS in wide-field slitless mode. The total on-source exposure time ranged from 1. 5 to 5 hr. Our first analysis presented here uses primarily NIRSpec MSA data (reduced two-dimensional spectra and one-dimensional extracted spectra of various multiple images) directly available from the ERO data release.
X-Ray Imaging Spectroscopy SMACS J0723 was observed with the Advanced CCD Imaging Spectrometer (ACIS-I) on board the Chandra X-ray Observatory on 2014 April 14. The observations (Sequence Number 801329; ObsID 15296; PI: Murray) were performed in VFAINT mode for a total duration of 19. 8 ks. We performed a standard reduction of the data using the CIAO 17  v4. 13  (Fruscione et al. 2006 ) and CALDB v4. 9. 6 packages. We removed point sources detected either automatically by the WAVEDETECT routine or by visual inspection. Periods of background flares were removed by running the DEFLARE tool in the 9. 5-12 keV band and for the whole energy range. We used the blank-sky background data associated with the observation as provided by the standard data-reduction pipeline.
We perform all modeling within the SHERPA fitting environment  (Freeman et al. 2001 ) combined with the Python wrapper of the MULTINEST nested-sampling package  (Buchner et al. 2014; Feroz et al. 2019)  to explore the parameter space of our model in the 0. 5-8 keV energy band. As appropriate for the mostly low photon statistics per bin, we use a Poisson likelihood similar to CSTAT. 21  Depending on the fit, not all emission model parameters are left free to vary. We consider the background normalization a nuisance parameter and marginalize over it in our best-fit estimates for all physical model parameters.
Cluster Galaxies In order to obtain an independent assessment of the dynamical state of SMACS J0723 as probed by the spatial and velocity distribution of the system's member galaxies, we Figure  3. Top: color-magnitude diagram of galaxies in the field of view of SMACS J0723. The red sequence of cluster-member galaxies is clearly visible since the two filters used, F606W and F814W, straddle the Balmer break of massive elliptical galaxies at that redshift. The 26 spectroscopically confirmed cluster members from MUSE spectroscopy (open red circles) are overplotted. The rectangular shape shows the selection of the 144 cluster-member galaxies used in our lens model. Additionally, the four spectroscopically identified cluster members are included in our final cluster-member catalog. Bottom: histogram of the redshifts of the 26 cluster members identified spectroscopically within the MUSE data cube ranging from 0.
3727 to 0. 3981. Overlaid is the best Gaussian model which determines the cluster velocity dispersion. The vertical dashed line marks the location of the BCG in redshift space, z BCG = 0. 3912, which is displaced from the systemic redshift of the cluster, z = 0. 3877, corresponding to the mean redshift (see Section 5. 1). examine the MUSE data cube and extract a catalog of 26 spectroscopically confirmed cluster members (Table  1 ). Using the ROSTAT package of  Beers et al. (1990)  we derive an improved cluster redshift of z = 0. 3877 for SMACS J0723 and determine the cluster velocity dispersion as s = -+ 1180 180 160 km s -1. We show the corresponding redshift histogram in Figure  3. Within the statistical uncertainties set by the small sample size, the radialvelocity distribution exhibits no significant substructure indicative of an active merger along an axis that lies close to our line of sight.
Strong-lensing Constraints The strong-lensing constraints for our lens models are given by the image-plane locations of multiple images of lensed sources, identified either in previous HST images or in the new JWST observations. Golubchik et al. (2022)  identify five arc candidates in the field of SMACS J0723 and report three multiple-image systems that have spectroscopic redshifts. We confirm all of these in our examination of all available data and identify 16 additional multiple-image systems. Through careful inspection of the MUSE data cube (Section 3. 2), we also secure an additional spectroscopic redshift for one of the systems photometrically identified by  Golubchik et al. (2022) : System 3 (at z = 1. 9914). Initial inspection of the NIRSpec MSA spectroscopic data yields an additional spectroscopic constraint by confirming a star-forming region in image 7.
Comparison with Other Mass Models We compare the results of our improved strong-lensing analysis with models from previous works on SMACS J0723. Two of these are from the public release of RELICS cluster models  (Coe et al. 2019) , derived using the GLAFIC lensmapping package and a Lenstool model detailed in  Sharon et al. (2023) , respectively. In addition, we compare our results with those from the recent analysis by  Golubchik et al. (2022) , performed using the Light Traces Mass (LTM) software. Table  4  lists and compares the masses from all existing lens models for SMACS J0723 at three different radii: 128, 200, and 400 kpc. Here, 128 kpc corresponds to the largest cluster-centric distance of the strong-lensing constraints commonly used by all models (this multiple-image system is labeled System 4 in our analysis).
Gas Mass We perform a multiscale deprojection of the gas density and gas mass using the PYPROFFIT Python package, developed by  Eckert et al. (2020). The analysis uses counts and background maps in the 0. 5-2 keV energy band, an associated monochromatic Energy band f X (10 -13 erg s -1 cm -2 ) L X (10 44 erg s -1 ) 0. 1-2. 4 keV Note. Unabsorbed fluxes and total luminosities are both point-source corrected. exposure map for an energy of 1. 2 keV, as well as the values from our best-fit spectral model. The resulting profiles of the ICM density and the cumulative gas mass are shown in Figure  12  and place the total gas mass of SMACS J0723 at almost 10 14 M e. A comparison with the total gravitational mass derived from our lens model (Figure  6  and Table  4 ) yields a gas-mass fraction of just under 10% for the cluster core, typical of massive clusters in general.
Discussion and Conclusion We create and make available to the scientific community a robust strong-lensing mass model of the galaxy cluster SMACS J0723 at z = 0. 39, the first strong-lensing cluster to be observed with JWST. Our model uses JWST ERO data, as well as archival, multiwavelength data of the cluster, from optical to X-ray wavelengths, and combines both imaging and spectroscopic observations. We identify 17 new multiple-image systems. We report a total number of 30 candidate multipleimage systems, two of which are isolated galaxy-galaxy lensed sources. Of the final 28 cluster-wide multiple-image systems, we use 21, namely 19 robust systems and two additional candidates located near the intracluster light concentrations identified by us. Our best-fit mass model contains only one large cluster-scale halo and includes one diffuse large-scale halo that accounts for mass traced by the cluster ICL.
We propose the use of controlled natural language as a target for knowledge graph question answering (KGQA) semantic parsing via language models as opposed to using formal query languages directly. Controlled natural languages are close to (human) natural languages, but can be unambiguously translated into a formal language such as SPARQL. Our research hypothesis is that the pretraining of large language models (LLMs) on vast amounts of textual data leads to the ability to parse into controlled natural language for KGQA with limited training data requirements. We devise an LLMspecific approach for semantic parsing to study this hypothesis. To conduct our study, we created a dataset that allows the comparison of one formal and two different controlled natural languages. Our analysis shows that training data requirements are indeed substantially reduced when using controlled natural languages, which is relevant since collecting and maintaining high-quality KGQA semantic parsing training data is very expensive and time-consuming.
Introduction Over the past decades, large amounts of structured knowledge in the form of knowledge graphs have been published  [11, 29, 32]. To access the information in such knowledge graphs, query languages like SPARQL 1 are used. The use of formal queries to access knowledge graph poses difficulties for non-expert users as they require the user to understand the syntax of the formal query language, as well as the underlying structure of entities and their relationships. KG question answering (KGQA) systems, therefore, aim to provide the users with an interface to ask questions in natural language, using their own terminology, to which they receive a concise answer generated by querying the KG. Recently, KGQA systems have also been used as an auxiliary system (commonly called a "tool" or "plugin") for LLMs in chatbots, since they allow LLMs to ground their responses in factual knowledge stored in a knowledge graph  [16, 34]  to reduce LLM hallucinations.
KGQA systems typically use semantic parsing approaches in which natural language questions are transformed into formal language. Such approaches are often trained on large datasets of (natural language, formal query language) pairs. Creating such datasets can be very expensive since handwriting formal queries for natural language input queries is a time-consuming task. At the same time, machine learning techniques for semantic parsing are typically not sufficiently accurate when trained on smaller datasets. To enable the creation of large datasets, various semi-automated approaches have been used  [5, 26, 31]. Although those datasets were helpful to advance the state-of-the-art, they are still largely template-based, do not generalise well outside of the scope of those templates and often do not reflect the traffic seen in QA systems in production.
Therefore, we argue that it is important to investigate KGQA semantic parsing systems which require smaller amounts of training data. As a step towards achieving this, we propose using a controlled natural language (CNL) as the target formal language. CNL expressions are close to (human) natural language while they can still be unambiguously translated into a formal language. Our research hypothesis is that using controlled natural language is more suitable than a formal query language for knowledge graphs, since CNLs are a) closer to the input query in natural language and b) closer to the high amounts of textual pre-training data of language models, that can be employed for semantic parsing. To the best of our knowledge, this research hypothesis has so far not been investigated for knowledge graph query languages.
To verify our research hypothesis, we conducted a study using the Mintaka dataset  [25]  based on Wikidata. Unlike most KGQA datasets, Mintaka contains very natural questions, as they were collected manually in a high-quality data collection process. However, due to the significant effort that would be required to collect (natural language, logical form) pairs, Mintaka only contains (natural language, response) pairs, i. e. a so called weak supervision setting. We prefer to investigate the strong supervision setting because of its widespread use in semantic parsing and its simplicity, which only requires LLM fine-tuning. To be able to investigate our research hypothesis, we labelled a subset of 550 questions of Mintaka with their corresponding logical forms. We provide three variants of those logical forms: SPARQL  [10] , Sparklis  [7]  and SQUALL  [6].
The latter two are controlled natural languages. We then used several language models, specifically BLOOM  [24] , GPT Neo 2  , GPT-2  [18] , GPT-3  [3] , T5  [19]  and Llama 2  [30]  on this dataset. We employ the state-of-the-art ReFinED  [1]  entity linker to enrich the LLM prompt with entity information. The specific prompting and fine-tuning techniques that we used are described in Section 4. We generally focused on comparing results across target languages rather than optimising the performance of the approach itself. Overall, we make the following contributions: • Proposing controlled natural languages as target for KGQA semantic parsing, opposed to using formal query languages directly. • Provision of a dataset containing 550 (NL question, formal KG query) pairs to the community. 3  While larger (semi-)automatically created datasets exist, this is one of the largest publicly available strong supervision datasets using a manually collected set of questions (similar to, e.
g. [17] ) and the largest for controlled natural language KGQA. • Development of an LLM-based semantic parsing approach that builds on an external entity linker and leverages the capabilities of LLMs to hallucinate for relation linking. • An evaluation of semantic parsing accuracy for three target languages on six different language models using different metrics. • An analysis of the relationship between fine-tuning scale and semantic parsing accuracy across the three target languages. • A qualitative result analysis on our test set of 150 queries.
Those approaches have shown to be accurate when trained on large amounts of high quality training data. However, as described in the introduction obtaining training data is very challenging and updating the training set to incorporate new features or cover more domains requires substantial effort. For this reason, there was an increasing interest in training semantic parsers with less data. For example,  [22]  explores an unsupervised approach for semantic parsing. While achieving promising results, such approaches do not achieve the performance levels required for production level systems and cannot easily be tuned to do so. Large language models such as GPT-3  [3] , BLOOM  [24]  or Llama 2  [30]  also support semantic parsing with zero or few examples and are a promising avenue towards reducing the need for high amounts of training data, in particular after several improvements have been made to reduce the inference costs of such models.
Controlled Natural Languages for Semantic Parsing Controlled natural languages (CNLs) have been used as target for semantic parsing in applications outside of knowledge graph question answering. For example,  [20]  has already investigated the idea of using a special-purpose CNL to reduce training data requirements and showed improvements on the Overnight dataset  [33]  in several smaller domains. On the same (and other) datasets,  [28]  used a constraint decoding procedure for semantic parsing that employs the GPT-3 and Codex models. They compared a controlled natural language and a standard logical form language and could show accuracy benefits of using a controlled natural language. Although these previous studies are an indicator and motivation for our investigation, they were performed on datasets with a smaller scale and not on knowledge graphs.
For example, an Overnight domain has at most 45 relations compared to more than 10,000 relations (and more than 100 million entities) in the Wikidata knowledge graph  [32]  that we use. Similarly,  [27]  investigated the use of language models (BART, GPT-2, GPT-3) for constrained decoding of synchronous context-free grammars (SCFG) following an idea similar to ours by first converting an NL question into a canonical human-like utterance which is then translated into the target logical form. These are again evaluated on Overnight and related datasets. [14]  introduces a new language Graph-IR with the aim of unifying the semantic parsing of graph query languages with one intermediate representation. The principles of their intermediate representation language are similar to CNL but with less emphasis on naturalness compared to the SQUALL language which we are investigating.
Knowledge Graphs In this paper, we use hyperrelational knowledge graphs as a base. Hyperrelations are common, e. g. , they are required to model the relationship properties in graph databases and the DBpedia  [11]  or Wikidata  [32]  knowledge graphs use variants of this data model. Using hyperrelations via so called qualifiers in Wikidata is indeed required for several questions in the Mintaka dataset which we evaluate on. Let E = {e 1. e n e } be a set of entities, L be the set of all literal values, and P = {p 1. p n r } be a set of properties (also called relations). A statement t = (s, p, o, Q) comprises a subject s ∈ E, a property p ∈ P, an object o ∈ (E ∪ L), and a set of qualifiers Q ⊆ P × (E ∪ L). The property p relates the subject s to the object o, and the set of qualifiers Q further describes the relationship.
Controlled Natural Languages for SPARQL Generally, Controlled Natural Languages aim at bridging the highlevel and natural syntax of natural languages, and the lack of ambiguity of formal languages such as SPARQL  [10]. CNLs allow for abstracting from the low-level aspects of formal languages (such as bindings and relational algebra). Sparklis  [7]  and SQUALL  [6]  are examples of SPARQL-oriented CNLs for querying and updating knowledge graphs. SQUALL has a very high coverage of SPARQL queries and updates. It therefore combines the expressivity of SPARQL, and at the same time the readability of natural languages. Its main limit is that the content words (e. g. , proper nouns, verbs) must be non-ambiguous references to entities and relations. SQUALL is based on Montague grammars  [13]  that combine context-free grammars, first order logic, and λ-calculus.
Sparklis acts as an interactive SPARQL endpoint explorer that completely hides SPARQL behind a CNL. Sparklis' CNL is slightly less expressive and less natural than SQUALL, and it only covers SELECT queries. In counterpart, queries can be built incrementally through user interaction: at each step, Sparklis suggests some query refinements, among which the user selects the refinement that fits her information needs. This allows the user to discover the data model on-the-fly, and this prevents ill-formed queries. The main limitation is that, in case of a complex data model (like in Wikidata), it may be difficult for the user to find a valid sequence of refinements. Figure  1  shows an example natural language query that is taken from the Mintaka dataset. We showcase the corresponding SPARQL, Sparklis and SQUALL versions of this question.
Prompting In our approach, the LLM prompt consists of three parts: the instruction, the input question, and the list of entities. As instruction, we use "Given the question and the entities generate a $language query!" where $language ∈ {SPARQL, Sparklis, SQUALL} is the target language. The question is passed to the LLM as is. For the entities, we pass the quadruples (entity ID, probability, label, description) as described in the previous section. An example of a prompt is given below. We did not perform prompt engineering since optimising performance is not the main scope of the experiments. While this may affect absolute ceiling performance, it is unlikely to substantially affect the performance difference between the different target languages that is our focus here. Given the question and the entities generate a SQUALL query! Question: Which actor was the star of Titanic and born in Los Angeles?.
The Mintaka-CNL Dataset Mintaka  [25]  is a natural and multilingual question answering dataset containing 20k manually written NL queries in English. We used Mintaka as a base for our studies since it is the only KGQA dataset we are aware of that contains a high number of queries and does not contain questions that are, at least in part, automatically generated (e. g. "Is the WOEID of Tuscaloosa 14605?" in KQA Pro  [26]  or "1520. 0 is the minimum width for which size rail gauge?" in GrailQA  [8] ). To create our dataset, we sampled questions from Mintaka using the following criteria: a) diversity in question themes, b) diversity in question complexity, c) preference of questions with more than one entity, and d) having at least three examples of the same type of question in each category.
g. selecting the nth element according to an order or comparisons of relation values (e. g. height, length). SQUALL also supports a direct conversion into SPARQL such that we could verify that both queries return the same output modulo differences in language expressivity. Mintaka has seven categories of questions including (1) count, (2) intersection, (3) superlative, (4) difference, (5) comparative, (6) generic and (  7 ) ordinal. Our questions are spread over the different categories as follows: 92, 85, 57, 49, 56, 74 and 137. The original Mintaka dataset also has another category named of yes/no questions which we did not consider in this work, since Sparklis does not support SPARQL ASK queries. In total, we sampled 550 questions and created three different logical forms for each. This dataset is split into 400 train and 150 test questions.
Models and Fine-Tuning We used the language models shown in Table  1. The models were selected to cover a range of different sizes suitable for generative NLP tasks. For all models, we used a fine-tuning procedure using the prompt as described in Section 4 and the actual target query as the completion target. For models outside of the GPT-3 family, we updated the tokenizer to include special tokens and re-sized the model's token embeddings. During the fine-tuning phase, the models were trained following parameters of  [23]  which states 25 epochs with a constant learning rate of 5e-05 and a batch size of 4. The maximum sequence length during training was set at 120. No extensive hyperparameter optimization was performed. The same settings are used for all target languages. For the decoding step, we use a greedy search strategy.
Special tokens are excluded from the LLM output before evaluation. The threshold for entity linking was set to 0. 1. The fine-tuning was done on 2 Nvidia A100 GPUs on Linux and on Ope-nAI servers for GPT-3. Code was written in Python 3. 10. The fine tuning for Llama 2  [30]  was done for 20 epochs, with constant learning rate of 1e-04 and batch size of 1. No extensive hyper-parameter optimization was performed.