dataesr/scientific-paragraphs-categorization huggingface dataset:
Otherwise, we exclude "Port Scan" reports and consider the category counting at least 75% of the remaining reports. Subsequently, each probing flow is labeled with the most frequent category in the reports on their corresponding source IP addresses. The third step consists of extracting network flow graphs following the method presented in Section V. It is to note that our network telescopes record activities of a daily average of 700,000 unique IP addresses. Thus, to reduce the computation complexity, we consider only probing flow graphs comprising at least 3 nodes. This eliminates probing flows exhibiting noncomplex graph structures and thus easily recognizable visually or using rule-based inference. As a result, we obtain two evaluation datasets that we denote ED FR and ED JP , consisting of labeled graphs of probing flows collected by the French and the Japanese darknets, respectively.
E. EVALUATION APPROACH The goal of our evaluation experiment is to assess the performance of the embedding models AWE, GAE and VGAE in inferring robust latent representations in an self-supervised fashion. Hence, for an accurate performance measurement, our evaluation approach should completely dissociate the embedding learning process and the downstream classification task. The conducted experiment is then as follows. First, we construct two separate probing flow graph datasets, D FR and D JP , each consisting of 100, 000 probing flow graphs extracted from the probing flows collected by the French and the Japanese darknets, respectively. The embedding models GAE and VGAE are then trained on D FR and D JP. Let's denote these trained models GAE D FR , GAE D JP , VGAE D FR and VGAE D JP. Second, using GAE D FR (respectively GAE D JP ), we infer latent representations of the evaluation probing flow graphs ED JP (respectively ED FR ).
We do the same using the VGAE models. As a result, we obtain four new datasets that we denote ED. For example, the notation ED (GAE D FR ) JP corresponds to the evaluation (labeled) dataset consisting of the embeddings of the probing flow graphs ED JP inferred using the model GAE D FR that is trained on the probing flow graphs D FR. Then, for each dataset, we set aside 25% of the data to serve as a testing set. Finally, we evaluate the performance of the embedding models via a downstream task which consists of classifying the probing flows using their inferred latent representations. To classify these probing flows, we use a single-layer neural network taking as input the obtained embeddings and outputting the probabilities of each class of abuse (cf. Section VIII-C) through a Softmax layer.
IX. RESULTS AND DISCUSSION In this section, we report and discuss the performance of the embedding models. The reported evaluation metrics are the micro-averages of the precision, recall and F1, as well as the accuracy, all expressed as follows: P = 1 m c∈C m c P c , (15) R = 1 m c∈C m c R c , (16) F1 = 1 m c∈C m c F1 c , (17) A = number of correct predictions total number of predictions ,  F1 c = 2 × P c × R c P c + R c (21) Table  4  gives an overview of the obtained classification scores related to the best scenario when we use all of the labeled data to train our models. Table  5  and 6  (in the Appendices Section) provide details per class of abuse measured on the evaluation datasets presented in Section VIII-C. The given scores are micro-averages taking into account the number of instances of each class of abuse.
This can be of practical implications for security experts, for instance, it could be used to dynamically make operational decisions about a potential attacker based on a sequence of graphs representing his behavior within a network. Finally, to extend the evaluation of our approach, we intend to adapt the features encoded in the network flow graphs to tackle the issue of encrypted traffic classification. consisting of latent representations of the probing flows ED FR are inferred using AWE, GAE D JP and VGAE D JP , respectively. Single layer neural networks are trained on these latter datasets' training sets, respectively. The scores are reported on the testing sets, respectively. The Top N score corresponds to the actual class of abuse belonging to the N most probable categories predicted by the model.
Classification Accuracy by Training Set Size (ED FR ) GCN (Fully-supervised) AWE (Self-supervised) GAE (Self-supervzised) VGAE (Self-supervised) (a) The training is performed on the French Evaluation Dataset (ED FR ) and the scores are reported on the testing set of the Japanese Evaluation Dataset (ED JP ). Classification Accuracy by Training Set Size (ED JP ) GCN (Fully-supervised) AWE (Self-supervised) GAE (Self-supervzised) VGAE (Self-supervised) (b) The training is performed on the Japanese Evaluation Dataset (ED JP ) and the scores are reported on the testing set of the French Evaluation Dataset (ED FR ).
Cyber-physical systems (CPSs) are embedded systems that are tightly integrated with their physical environment. The correctness of a CPS depends on the output of its computations and on the timeliness of completing the computations. The increasing use of high-performing and low-power multi-core processors in embedded systems is pushing embedded programmers to be parallel programming experts. Parallel programming is challenging because of the skills, experiences, and knowledge needed to avoid common parallel programming traps and pitfalls. This paper proposes the ForeC language for the deterministic, parallel, and reactive programming of embedded multi-cores. The synchronous semantics of ForeC is designed to greatly simplify the understanding and debugging of parallel programs. ForeC allows programmers to express many forms of parallel patterns while ensuring that ForeC programs can be compiled efficiently for parallel execution and be amenable to static timing analysis.
ForeC's main innovation is its shared variable semantics that provides thread isolation and deterministic thread communication. All ForeC programs are correct by construction and deadlock-free because mutual exclusion constructs are not needed. Through benchmarking, we demonstrate that ForeC can achieve better parallel performance than Esterel, a widely used synchronous language for concurrent safety-critical systems, and OpenMP, a popular desktop solution for parallel programming. We demonstrate that the worst-case execution time of ForeC programs can be estimated to a high degree of precision.
Central to synchronous languages is the synchrony hypothesis  [13] , which states that the execution of each reaction is considered to be atomic and instantaneous. The sampling of inputs avoids the need to use interrupts which are sources of unpredictable delays that degrade the system's timing predictability. Concurrent threads communicate instantaneously with each other (dashed arrows in Figure  2 ) due to the synchrony hypothesis. Once the embedded system is implemented, the synchrony hypothesis has to be validated. That is, the worst-case execution time  [144]  of any global tick must not exceed the minimal inter-arrival time of the inputs. We use the Esterel synchronous language  [18]  to illustrate some features of the synchronous paradigm in Figure  3a. It contains two threads (starting from lines 4 and 7 respectively), scoped between the square brackets and separated by the parallel operator (line 6).
Synchronous programs are considerably difficult to parallelize  [48, 66, 153]  due to the need to resolve instantaneous thread communication and associated causality issues. At runtime, all potential signal emitters must be executed before all testers of a signal. If this is not possible, then a causality issue arises. Thus, concurrency is typically compiled away to produce only sequential code  [44]. The common approach for parallelizing synchronous programs is to automatically parallelize an intermediate representation of the program  [48, 66, 153, 10]. The techniques differ in the heuristics used to partition the program to achieve sufficient parallelism. Esterel only supports basic data computations and delegates complex data computations to a host language, for instance C. Consequently, C-based synchronous languages have been developed to provide data handling at the language level.
These languages extend C with a range of synchronous constructs to support concurrency, preemption, and thread communication. C-based synchronous languages appeal to C programmers because the learning barrier for synchronous languages is reduced. PRET-C  [4]  is one such example and Figure  3b  is the PRET-C version of the Esterel example (Figure  3a ). The two threads (t0 and t1, defined on lines 5 and 9) are arguments to the parallel operator PAR (line 3). The EOT statement demarcates the end of a thread's local tick (e. g. , lines  7, 11, and 14). Unlike Esterel, threads in the PAR's argument are executed in a left-to-right (static) order. The thread's local tick must be executed entirely before the next thread can be executed. In the example program, t0 always executes its local tick before t1.
Contributions We propose the ForeC parallel programming language for simplifying the deterministic parallel programming of embedded multi-core systems. Execution platforms have evolved from singlecores to multi-cores. Hence, all the synchronous languages designed for the single-core era must be reinvented to address the multi-core challenges. To this end, ForeC is a C-based synchronous language designed specifically for the programming of multi-cores. ForeC brings together the formal deterministic semantics of synchronous languages and the benefits of C's control and data structures. A key innovation is ForeC's shared variable semantics that provides thread isolation and deterministic thread communication. Moreover, many forms of parallel patterns can be expressed in ForeC. We show that ForeC programs are reactive and deterministic by construction.
In the parallel phase, threads execute in parallel until they all reach one of the following synchronization points: a lock, memory access, or statically defined number of executed instructions. Then, in the serial phase, threads take turns to resolve their memory accesses or lock acquisitions. Threads in CoreDet and Dthreads also maintain their own version of the shared memory state, which is resynchronized in every serial phase. This concept is used and formally defined in concurrent revisions  [27]. DOMP and Grace differ in that the resynchronization only occurs when threads reach a synchronization construct. However, understanding the program's behavior at compile time remains difficult because the determinism is only enforced at runtime. Thus, if the program is modified, e. g. , to fix a bug, then a vastly different runtime behavior is possible.
The Synchronized Distributed Executive (SynDEx)  [111]  approach considers the cost of communication when distributing code to each processing element. When distributing a synchronous program, some desynchronization  [12, 49, 24]  is needed among the concurrent threads. That is, the concurrent threads execute at their own pace, but sufficient inter-thread Inria communication is used to preserve the original synchronous semantics. The use of futures has been proposed as a method for desynchronizing long computations in Lustre  [34]. A future is a proxy for a result that is initially unknown but becomes known at a later time and can be computed in parallel with other computations. Once a synchronous program is implemented, it is necessary to validate the synchrony hypothesis. That is, the worst-case execution time  [145, 144]  (WCET) of any global tick must not exceed the minimal inter-arrival time of the inputs.
The ForeC Language Execution platforms have evolved from single-cores to multi-cores. Hence, all the synchronous languages designed earlier (e. g. , Esterel  [18] , Lustre  [53] , Signal  [52] , Esterel C Language  [77] , Reactive Shared Variables  [23] , and PRET-C  [4] ) must be remodeled to address the challenges raised by multi-cores. Over 30 years of synchronous programming languages have demonstrated that they are very well suited to the design of safety-critical real-time systems  [25, 127]. Moreover, the ideal modeling of time brought by the synchrony hypothesis makes them good candidates for PRET programming. This motivates our proposed ForeC language that is dedicated to the programming of multi-cores. ForeC inherits the benefits of synchrony, such as determinism and reactivity, along with the benefits and power of the C language, such as control and data structures.
Overview and Syntax ForeC is a synchronous language that extends a safety-critical subset of C  [20, 65]  (see Section 1. 3) with a minimal set of synchronous constructs. We briefly describe the statements, type specifiers, and type qualifiers allowed in the C subset: C statements (c_st): Expressions in a statement can only be constants, variables, pointers, and arrays that are composed with the logical, bitwise, relational, and arithmetic operators of C. Although the use of pointers and arrays is allowed, they can make static dataflow analysis difficult  [28]  because of pointer aliasing. Thus, we assume that pointers are never reassigned to point to other variables. All C control statements, except goto, can be used. weak? abort st when immediate? (exp): Preempts its body st when the expression exp evaluates to a non-zero value.
Like C, extra properties can be specified for variables using type qualifiers. A type qualifier (tq) in ForeC is a traditional C type qualifier (c_tq), an environment interface (input and output), or a shared variable amongst threads (shared). The input, output, and shared type qualifiers precede the C type qualifiers in variable declarations. As a running example to illustrate the ForeC language, we describe the design of an unmanned aerial vehicle (UAV) inspired by the Paparazzi project  [95]. A UAV is a remotely controlled aerial vehicle commonly used in surveillance operations. Figure  8  is a ForeC implementation of the UAV example given in Figure  7. Figure  9  is a possible execution trace of Figure  8  to help illustrate the execution of ForeC programs. Sections 1. 2 described the execution behavior of synchronous programs.
Hierarchical Preemption Inspired by Esterel  [18] , the abort st when (exp) statement provides preemption  [17] , which is the termination of the abort body st when the condition exp evaluates to true. Preemption can be used to model state machines succinctly. The condition exp must be a side-effect free expression produced from the syntax shown in Figure  11. In Figure  12 , the main function of the UAV has been extended to respond to external commands through the input comms (line 2). The value of comms can be OK, ERROR, WARN, or TERM (line 1). The abort statement on line 7 preempts the execution of all the UAV tasks when TERM is received. A possible execution trace of the program of Figure  12  is given in Figure  13. The italicized line numbers in Figure  13  refer to the line numbers in Figure  8 , while the non-italicized line numbers refer to the line numbers in Figure  12.
Esterel threads communicate instantaneously by emitting and receiving pure or valued signals during each tick. Pure signals are either present or absent and carry no value. All potential signal emissions must be performed by the concurrent threads before the signal status (present or absent) can be read. Compilers  [48, 44, 153]  typically compile away the parallelism to create a sequential program that resolves the causality, but this restricts the execution onto single cores. Valued signals are like pure signals except that each emission has an associated value. Before a valued signal can be read, all the emitted values are combined using a programmer-specified commutative and associative combine function. The combine function only considers the emitted values, i. e. , equivalent to ForeC's combine policy mod.
Discussion This section has introduced the ForeC language that enables the deterministic parallel programming of multi-cores. The language features of ForeC help bridge the differences between synchronous-reactive programming and general-purpose parallel programming. ForeC makes deterministic parallelism accessible to traditional embedded C programmers. ForeC offers shared variable semantics that removes the burden of ensuring mutual exclusion from the programmer and guarantees deadlock freedom. Thread isolation is guaranteed by stipulating that threads work on local copies of the shared variables. Resynchronizing the shared variables when the threads have finished their respective local ticks makes program behavior agnostic to scheduling decisions. These features simplify the understanding and debugging of ForeC programs.
Overview The ForeC compiler can generate code for direct (bare metal) execution on the Xilinx MicroBlaze embedded multi-core (Section 3. 1) processor. Later in Section 5. 9, we extend the compiler to generate code for execution on an operating system. Figure  22  is an overview of the compilation process. The first step checks the syntax of the ForeC source code. This includes checking whether all threads have been defined and whether all variables accessed by multiple threads have been declared with the shared qualifier. The second step translates the ForeC statements into equivalent C code. Bootup and thread scheduling routines are generated for each core. The ForeC threads are statically allocated and statically scheduled on each core. The final step is to compile the generated C program with the GNU C compiler (GNU's computed goto extension is used to implement fast context-switching).
sJoin: Uses a non-blocking send to notify the master core whether or not its child threads have terminated. Then, it blocks until it receives whether the parent thread has resumed. mSync: Synchronizes with all the cores, performs the housekeeping tasks, and then synchronizes with all the cores again to start the next global tick. sSync: Synchronizes with all the cores and waits for the next synchronization to start the next global tick. mAbort and sAbort: Evaluates the preemption condition of an abort. • The global declarations and functions from the ForeC program (lines 4-6). • The global declarations for storing the execution states of the threads and implementing the shared variables (lines 9-13). • The main function (line 16) with the bootup routine (lines 34-41), the synchronization routines (lines , and the threads (lines 141-186).
Recall that the slave cores have an sFork routine in their initial linked list. Lines 52-59 is an example of the C code that is generated for an sFork routine. Line 53 blocks until it receives whether the parent thread has forked. If the parent thread has forked, then line 55 inserts the allocated child threads and an sJoin routine into the linked list. The sFork routine is removed from the linked list. The end of line 57 is a context-switch to the first node that was inserted. Otherwise, if the parent thread has not forked, then line 58 is a context-switch to the next node in sequence. Lines 180-183 in Figure  25  is an example of the C code that is generated for the end of a child thread to handle thread termination. Line 181 sets the thread's execution state to TERM. Line 182 removes the thread from the linked list.
Line 137 are barrier synchronizations for the end of the tick and the start of the next tick. This is followed by a context-switch to the first node in the linked list. Inria code for each ForeC construct. Light-weight synchronization routines are generated to manage the forking and joining of threads across the cores. The use of linked lists to manage the scheduling of threads and routines is inspired by that of the Columbia Esterel Compiler  [44]. The code generation is structural, meaning that a nesting of ForeC constructs is compiled into a nesting of each construct's generated code. The advantages with our static scheduling approach include: (1) a light-weight scheduling of ForeC threads, and (2) analysis is easier because all scheduling decisions are known beforehand. However, the disadvantages include: (1) the inability to dynamically load balance the ForeC threads to utilize the idle cores, and (2) the need to recompile the program to target a different number of cores.
SynDEx  [111]  is a tool that automatically distributes synchronous programs and considers the cost of communication between the processors. In contrast, ForeC is significantly easier to compile because thread communication is delayed with shared variables (Section 4. 1. 3). Causality analysis is not required and ForeC threads can be distributed directly to the available cores. The parallelism specified by the programmer is preserved by the ForeC compiler and a sequential intermediate code is not required. With the advent of multi-cores, the distribution of synchronous programs is motivated by the desire to improve their execution performance. The distribution of synchronous programs over multi-threaded and multi-core reactive processors has been studied extensively  [81, 152, 37, 120, 150].
Inria 6 ForeC Benchmarking This section quantitatively assesses ForeC's parallel execution performance on a mixture of data and control dominated benchmark programs. ForeC's execution performance is compared with that of Esterel, a widely used synchronous language for concurrent safety-critical systems that has inspired some features of ForeC, and that of OpenMP, a popular desktop solution for parallel programming. The static timing analysis of ForeC using the reachability technique is described in a previous paper  [149]. The benchmark results  [149]  showed that the worst-case reaction time  [22]  (WCRT) of ForeC programs could be estimated to a high degree of precision, which is very useful for implementing real-time embedded systems in general, and time-predictable systems in particular.
A core makes a context-switch by adding the executing thread back to the queue and retrieving a different thread from the queue. Threads are added to the queue when they are forked by other threads. Threads are removed from the queue when they terminate. All the cores can access the FIFO queue in parallel and each access takes two clock cycles to complete. For benchmarking, the MicroBlaze multi-core simulator described in Section 3. 1 is extended with a hardware queue to support the dynamic scheduling. The configuration of the simulator is shown in Figure  29. Table  9  shows the implementation details of the ForeC and Esterel versions of the benchmark programs. Esterel is suited for specifying control concurrency, but is not so for specifying data dominated computations. Esterel allows data computations to be delegated to external host functions, defined in a host language such as C.
Compared to GRC, for most programs, the ForeC compiler (see Section 5) can generate more efficient code that require less context-switching. The same input vector is given to the ForeC and Esterel versions of a benchmark program to ensure that the same computations are performed. When a program terminates, the simulator returns the execution time in clock cycles. Figure  30  shows the speedups achieved by ForeC and Esterel when the benchmark programs execute on four cores. Apart from MatrixMultiply, ForeC shows superior performance compared to Esterel, even though Esterel uses dynamic scheduling with hardware acceleration. The need to resolve instantaneous signal communication in Esterel can lead to significant runtime overheads. All possible signal emitters must execute before any signal consumers can execute and this invariant is achieved using a signal locking protocol  [151]  that is costly.