dataesr/scientific-paragraphs-categorization huggingface dataset:
B. New Attack Vectors Software side channels through procfs interfaces can be mitigated by restricting access to them  [10] ,  [60]. However, such restrictions do not prevent keystroke timing attacks. We demonstrate two new attacks to infer keystroke timings: the first one exploits interrupt timings to detect keystrokes, and the second one relies on Prime+Probe to attack a kernel module. Table  I  compares the novel attacks we describe in the following  with the state-of-the-art attack vectors (cf. Section II-C) in terms of attack techniques and the exploited attack surface. Algorithm 1: Recording interrupt timing for i ∈ {1,. , N } do tsc[i] ← rdtsc(); if tsc[i] -tsc[i -1] > threshold then events[i] ← tsc[i]; diff [i] ← tsc[i] -tsc[i -1]; Low-Requirement Interrupt Timing Attack. We propose a new timing-based attack that only requires unprivileged sandboxed code execution on the targeted platform and an accurate timing source, e.
C. Requirements for Elimination of Keystroke Timing Attacks As demonstrated in the previous section, we are able to craft new attacks with fewer requirements than state-of-theart attacks. Hence, countermeasures against keystroke timing attacks must be designed in a generic way, in all affected layers of the software stack, covering known and unknown attacks. Attack Model. We assume that an attacker can run an unprivileged program on the target machine, with a recently updated system. As sensor-based attacks  [7]  are already addressed in  [48] , and Android O  [14] ,  [25]  also mitigates various procfs attacks, we consider them out of scope for this paper. The attacker is able to continuously monitor a side channel to obtain traces for all user input. We assume the (hypothetical) countermeasure against keystroke timing attacks was already installed when the attacker gained unprivileged access to the machine.
The second layer protects the library handling the user input against Flush+Reload attacks, including cache template attacks, and Prime+Probe attacks. For every keystroke event received from the kernel, a random keystroke is sent to a hidden window. The library cannot distinguish between real and fake keystrokes and thus both have the same execution path. Note that this also triggers screen redraw events, hence, the screen-redraw interrupt side channel is also covered by KeyDrown. In the third layer, the actual password entry field is protected against Prime+Probe attacks by accessing the underlying buffer whenever a real or a fake keystroke is received. Combining the three layers, the system-wide set of cache lines that are touched by the code paths through the entire software stack for real and fake keystrokes, are identical.
A. First Layer Basic Concept. Figure  6  shows the program flow for the kernel part of KeyDrown for both x86 and ARM. We use a nonperiodic one-shot timer interrupt with a random delay to inject a fake keystroke. 3 This leads to a uniform random distribution of keystrokes over time. The kernel module handles two types of events: Hardware interrupts from the input device, and the timer interrupts. If the 3 Timer interrupts are often known as periodic interrupts triggering regular operations, e. g. , scheduling. However, on modern systems there are significantly more features to timer interrupts, such as non-periodic one-shot timers  [22]. One-shot timers are architectural features that can be used through legitimate kernel interfaces and have no side effects on any system timers. kernel module receives one of our timer interrupts, it injects a keyboard interrupt.
For the fake keystrokes, the kernel uses a typically unused key value. The kernel does not have varying code paths and data accesses based on the key value, hence, the same code is executed for both real and fake keystrokes. In both cases, the keystroke handler is delayed by a small random delay to hide timing differences from interrupt runtimes. Finally, all keystrokes are passed to the library through the same data structures (cf. Figure  5 ). Consequently, the attacker cannot use a Prime+Probe or Multi-Prime+Probe attack on the kernel to distinguish real and fake keystrokes. Implementation Details. The first layer of KeyDrown is implemented as a Linux kernel module that aims to prevent interrupt-based attacks on keystrokes. We do not require a custom kernel or any patches to the Linux kernel itself, but only the Linux kernel header files for the running kernel.
All functionality is implemented in one generic kernel module that can be loaded into any Linux kernel from version 3. 4 to 4. 10, the newest release at the time of writing. The interrupt hardware and handling mechanism is compatible with all personal computers; thus, there is no further limitation on PC hardware or Linux distributions. Figure  7  shows the implementation details of the KeyDrown kernel module. The non-periodic one-shot timer interrupts are implemented using the Linux platform-independent highresolution timer API  [32]. On Linux, a driver can register an interrupt handler for a specific interrupt which is called whenever the CPU receives the interrupt. The interrupt service routine do_IRQ calls the general handle_irq function which subsequently calls generic_handle_irq_desc to execute the correct handler for every interrupt.
To receive all hardware interrupts, we change the input device's interrupt handler to a function within our kernel module. Afterwards, we forward the interrupt to the actual input device driver (i. e. , i8042_interrupt on x86, and irq_touch_handler on the Nexus 5 (ARM)). Every time the kernel receives one of the non-periodic timer interrupts or a real hardware interrupt, we restart the non-periodic timer with a new random delay to maintain the uniform random distribution over time. The kernel module triggers a hardware interrupt for every non-periodic timer interrupt. On x86, we can simply execute the int assembly instruction with the corresponding interrupt number. This spurious keyboard interrupt travels up until the point where the keyboard driver tries to read the scancode from the hardware.
As the driver does not execute the entire i8042_interrupt function for spurious interrupts, we access the remaining function to fetch it into the cache as if it was executed. In contrast, for real keys, we access the code that injects the keys to fetch it into the cache as if it was executed. From an attacker's point of view, there is no difference in cache activity between a data fetch and a code fetch, i. e. , a Prime+ Probe attack cannot determine the difference. We inject a scancode of a typically unused key, such as F16 or a Windows multimedia key using the standard serio_interrupt interface. Thus, from this point on the only difference between real and fake keystrokes is the scancode. Finally, all scancodes are sent to the upper software layers and run through the same execution path.
However, we duplicate the key event in order to trigger key value processing and key drawing in the library and the hidden window for both fake and real keystrokes. Consequently, we cannot distinguish real and fake keystrokes on the library layer using a side channel anymore. Implementation Details. One of the most popular user interface libraries for Linux is GTK+  [57]. The GTK+ library handles the user input for many desktop environments and is included in most Linux distributions  [51]. As we cannot hide cache activity, we generate artificial cache activity for the same cache lines that are active when handling real user inputs. The kernel provides all events, such as keyboard inputs, through the /dev/input/event * pseudo-files to the user space. The X Window System uses these files to provide all events to the GTK+ event queue.
This is also the case for cache template attacks, as the injected events induce a significant amount of noise in both the profiling and the exploitation phase. Therefore, the third layer protects against attacks that are mounted against the Android keyboard as shown by Lipp et al. [30] , or Multi-Prime+Probe attacks directly on the input field buffer (cf. Section III-B). Implementation Details. In GTK+, the GtkEntry widget implements the GtkEditable interface, which describes a text-editing widget, used as a single-line text and password entry field. By setting its visibility flag, entered characters are shown as a symbol and, thus, hidden from the viewer. Implementing the countermeasure directly in the GTK+ library would require rebuilding the library and all of its dependencies. As this is highly impractical, we chose a different approach: LD_PRELOAD allows listing shared objects that are loaded before other shared objects on the execution of the program  [28].
92, yielding an F-score of 0. 81, which is an advantage over an always-one oracle of +440. 0 %. Memory accesses to one of the cache sets by any other application cannot be distinguished from a cache set access by the keyboard interrupt handler, causing a high number of false positives. If we enable KeyDrown, the precision drops to 0. 06, as the attacker additionally measures the noise generated by the injected keystrokes. The F-score is then 0. 11, which is a (negative) advantage over an always-one oracle of -27. 7 %. Figure  10  shows the results of mounting a Multi-Prime+ Probe attack on the buffer of a password field within a GTK+ application. Although there is more noise visible in the traces, we achieve the same precision and F-score as for the attack on the kernel module when KeyDrown is disabled.
C. Requirement R3 As KeyDrown fulfills R1 and R2, we can be assured that the underlying technique is a working countermeasure. However, as the implementation of a countermeasure itself can leak information, we need to ensure that KeyDrown does not create a new software-based side channel in order to satisfy R3. First Layer. The first layer runs in the kernel and can thus only be attacked using Prime+Probe. Figure  7  shows that, in general, we have the same execution flow and data accesses. For the few deviations, we prevent any potential cache leakage from non-executed code paths by performing the same memory accesses as if they were executed. As an attacker cannot distinguish if a cache activity is caused by an execution or a memory read, the module's cache activity does not leak additional information to an attacker.
We investigated the cache activity on the cache sets used by the KeyDrown kernel module in a Prime+Probe attack and found no leakage from our module. Second Layer. To make use of the same noise as in the first layer, the second layer listens to the /dev/input/event0 pseudo-file containing all keyboard events. This file is not world-readable but only readable by members of the input group. Thus, this layer runs as a separate keydrown user with default limited privileges and additional access to this file. As the second layer is a user space binary, an attacker could theoretically mount a Flush+Reload attack against the second layer. However, attacking the second layer does not result in any additional information. The second layer does not know whether an event is generated from a real or an injected keystroke.
The DRAMA side-channel attack presented by Pessl et al. [43]  results from a massive number of secret-dependent memory accesses that lead to heavy cache thrashing, i. e. , the victim program accesses lots of memory locations that are mapped to the same cache lines. It is therefore unclear whether or not KeyDrown protects against DRAMA. In particular, it does not protect against the specific attack against keystrokes in the Firefox address bar (cf. Section VI). However, we observe that KeyDrown adds significant amounts of noise to the attack. To our surprise, we found that KeyDrown also mitigates the keystroke timing attack based on the event queue of the Chrome browser by Vila et al. [53]  (USENIX Sec  '17). They state that the leakage is due to the time it takes Chrome to enqueue and dispatch every keystroke event.
However, we investigated their attack and were able to reproduce it on MacOS systems reliably, but not on other operating systems, indicating that this effect is not purely Chrome-specific, but also has other influences. We believe that their attack exploits multiple effects in combination: the Chrome event queue and the interruption by the hardware interrupts as in our rdtsc-based attack, which is additionally amplified by the significantly higher I/O latency caused by the atypical MacOS design for interrupt handling  [1]. 5  preliminary version of our rdtsc-based interrupt timing attack was the basis for the same attack in JavaScript  [29]. They were able to identify the user typed URL, and distinguish different users based on this attack. As they report, KeyDrown successfully mitigates their attack in JavaScript as well.
Zhang et al. [60]  speculated that this attack vector could also be exploited through /proc/net, which might still be available in Android O. However, this is not a local software-based attack but a side channel for a remote attacker. Hence, dedicated countermeasures beyond KeyDrown should be implemented to prevent this attack. Some software-based side channel attacks may be unaffected by KeyDrown, e. g. , the sensor-based attacks exploiting the accelerometer  [7] , but these attacks can be thwarted by introducing noise  [48]. KeyDrown protects against software-based attacks on keystrokes as well as touch events. However, swipe movements are not protected as their interrupt rate is too high. While this is not a problem in the case of a password input -if a password can be swiped and thus pasted from a dictionary, there is little to protect -it is future work to investigate how to extend KeyDrown to protect swipe movements.
ACKNOWLEDGMENT We would like to thank our anonymous reviewers for their valuable feedback and  Johannes Winter  for insights on ARM interrupt handling. This work has been supported by the  Austrian Research Promotion Agency (FFG)  via the Kproject DeSSnet, which is funded in the context of  COMET -Competence Centers for Excellent Technologies by BMVIT, BMWFW, Styria and Carinthia. This project has received funding from the  European Research Council (ERC)  under the European Union's  Horizon 2020 research and innovation programme  (grant agreement No  681402 ). This project has received funding from the  European Union 's  Horizon 2020 research and innovation programme  under grant agreement No  644052  (HECTOR). This work was partially supported by the  TU   Graz LEAD project  " Dependable Internet of Things in Adverse Environments ".
We present SpatialVOC2K, the first multilingual image dataset with spatial relation annotations and object features for imageto-text generation, built using 2,026 images from the PASCAL VOC2008 dataset. The dataset incorporates (i) the labelled object bounding boxes from VOC2008, (ii) geometrical, language and depth features for each object, and (iii) for each pair of objects in both orders, (a) the single best preposition and (b) the set of possible prepositions in the given language that describe the spatial relationship between the two objects. Compared to previous versions of the dataset, we have roughly doubled the size for French, and completely reannotated as well as increased the size of the English portion, providing single best prepositions for English for the first time. Furthermore, we have added explicit 3D depth features for objects.
Introduction Research in image labelling, description and understanding has a long tradition, but has recently seen explosive growth. Work in this area is most commonly motivated in terms of accessibility and data management, and has a range of different specific application tasks. One current research fo-cus is detection of relations between objects, in particular for image description generation, and the research presented here contributes to this line of work with a new dataset, SpatialVOC2K, 1  in which object pairs in images have been annotated with spatial relations encoded as sets of prepositions, specifically for image-to-text generation. We start below with the source datasets from which we obtained the images, bounding boxes, and candidate prepositions (Section 2), followed by an overview of directory structure and file schemas (Section 3), and a summary of the annotation process (Section 4) and spatially relevant features (Section 5).
Source Data Our main data source for SpatialVOC2K was the PASCAL VOC2008 image dataset  (Everingham et al. , 2010)  in which every object belonging to one of 20 object classes is annotated with class label, bounding box (BB), viewpoint, truncation, occlusion, and identification difficulty  (Everingham et al. , 2010). Of these annotations we retain just the BB geometries and the class labels  (aeroplane, bird, bicycle, boat, bottle, bus, car, cat, chair, cow, dining table, dog, horse, motorbike, person, potted plant, sheep, sofa, train, tv/monitor). We also used  Rashtchian et al. 's VOC'08 1K corpus (2010) , which has 5 descriptions per im-age obtained via Mechanical Turk for 50 images from each VOC2008 class, in order to determine an initial set of candidate prepositions for our annotations (for details see Section 4).
Due to quality control measures, the VOC'08 1K descriptions are of relatively high quality with few errors. For SpatialVOC2K, we selected all images from the VOC2008 data that had two or three object bounding boxes (BBs), meaning that images contained exactly two and three objects from the VOC2008 object classes, respectively. We also selected all images with four and five BBs where three were of normal size and the remainder very small (bearing the VOC2008 label 'difficult'). This selection process resulted in a set of 2,026 images with 9,804 unique object pairs. Numbers of BBs in images were distributed as follows: For each image, we then (i) collected additional annotations (Section 4) which list, for each ordered object pair, (a) the single best, and (b) all possible prepositions that correctly describe the spatial relationship between the objects; and (ii) computed a range of spatially relevant features from the image and BB geometries, BB labels, and image depth maps (Section 1).
SpatialVOC2K Structure and Schemas The overall structure and file conventions of the SpatialVOC2K dataset mirror those of the VOC2008 dataset where possible: In the All directory, files have the same structure except that in the preposition lines, instead of a single preposition, there are as many prepositions as were selected by the human annotators as possible for a given ordered object pair. The Spatial Features files also have the same basic structure, except that instead of prepositions, there are 19 feature-value pairs (explained in Section 5) for each ordered object pair (some feature values differ depending on object order), e. g. : In the following three sections, we explain how we obtained the preposition annotations and spatial features, and how the metrics encoded by the evaluation tools are defined.
Spatially Relevant Features Table  1  provides an overview of the 19 features included in SpatialVOC2K: F0, F1, F15 and F16 are language features. F0 is the class label of the first object, F1 of the second (e. g. person). F15 and F16 are GloVe word vectors of length 50  (Pennington et al. , 2014)  for the object labels. 6 F2-F14 are visual features measuring various aspects of the geometries of the image and two bounding boxes (BBs). Most features express a property of just one of the objects, but F4-F9 express a property of both objects jointly, e. g. F6 is the normalized BB overlap. F17 and F18 are the average pixel-level depth value within the BB of Obj s and Obj o , respectively. Pixel-level depth values were computed via the method described in  (Birmingham et al. ,  6 Evaluation Tools SpatialVOC2K includes two evaluation tools which we have used in all previous work involving similar data.
The two tools, systemAccuracy and relationPrecision implement the following two methods, respectively. System-level Accuracy: There are four different variants of system-level Accuracy, denoted Acc(n), n ∈ {1, 2, 3, 4}. Each variant returns Accuracy rates for the top n outputs returned by systems, in the sense that a system output is considered correct if at least one of the reference prepositions (the human-selected prepositions from the dataset annotations) can be found in the top n prepositions returned by the system (for n = 1 this yields standard Accuracy). Weighted Average Per-preposition Precision: This measure, denoted Acc P , computes the weighted mean of individual per-preposition precision scores. The individual per-preposition precision for a given system and a given preposition p is the proportion of times that p is among the corresponding human-selected prepositions out of all the times that p is returned as the top-ranked preposition by the system.
Related Datasets A number of datasets are available that incorporate annotations representing relations between objects in images. Types of relationships that have been annotated include actions (e. g. person kicks ball), other verbal relations (person wears shirt), spatial relations (person on horse), and comparative relations (one car bigger than another). In this section, we provide a brief overview of available datasets with relation annotations, in terms of their stated purpose (application task), the types of relations included, the range of spatial prepositions included, as well as size and other properties of the dataset. Table  2  has a summary of the datasets. Visual Phrases  (Sadeghi and Farhadi, 2011)  was the first image dataset with object relation annotations, and used the concept of a visual phrase (VP) which is defined as a bounding box that surrounds two objects in an image.
Out of 17 different types of VPs annotated in the data set, 13 comprise 2 objects, and 4 comprise one object. However, there are 120 predicates per object category. Visual and Linguistic Treebank  (Elliott and Keller, 2013)  contains 341 images that are annotated with regions (362 in total) and visual dependency representations, which unfold to a total of 5,748 spatial relations (from a set of 8) and are aligned to the dependency parse of the image description. This setup allows for the prediction of actions as well as spatial relations (using a set of 8 manual created rules). Scene Graphs  (Johnson et al. , 2015)  is a dataset of 5,000 human-generated scene graphs grounded to images; scene graphs describe objects and their relationships. ViSen  (Ramisa et al. , 2015)  associates sets of (object 1, preposition, object 2) triples with images, where the triples have been extracted from parses of the image descriptions in MSCOCO  (Lin et al.
, 2014)  and Flickr30k  (Young et al. , 2014). Prepositions covered include all those extracted from the image descriptions including non-spatial ones. By far not all descriptions contain prepositions so not all images have spatial relation annotations; the task addressed is preposition prediction, not spatial relation prediction. Visual Relationships Dataset (VRD)  (Lu et al. , 2016)  contains 5,000 images, 100 object categories, 6,672 unique relationships, and 24. 25 relations per object category. Scant information is available about how the dataset was created other than that relations broadly fit into the categories action, verbal, spatial, preposition and comparative. Visual Genome  (Krishna et al. , 2017)  contains 108K images, split into 4M regions, corresponding to 108K scene graphs and about 4K region graphs, 1.