dataesr/scientific-paragraphs-categorization huggingface dataset:
Our results show that the meanings of recognized words are often complementary to the visual information, resulting in the significant improvements on finegrained classification. 2) We develop an end-to-end trainable neural network with the attention mechanism for implicitly exploiting the semantic relationships between textual and visual cues. Such a unified framework is more elegant and efficient than the existing alternatives designed for cross-domain representations. 3) We collect and will release a new image dataset consisting of various types of drink bottle images. Our proposed method works well on drink bottle image classification and retrieval, demonstrating its potential in product search. The remainder of the paper is organized as follows. Section II briefly reviews some related works.
D. Attention Mechanism Attention mechanism is widely used in natural language processing, such as speech recognition  [63] , machine translation  [64] , and dialog management  [65]  to improve the network's ability of finding relevant features from corresponding input parts. Recently, visual attention attracts much attention in computer vision society. Ba et al. [66]  presented an attentionbased model for recognizing multiple objects in images. In  [67] , the authors proposed a novel network which combines the attention mechanism and an auto-encoding framework for iterative image construction. The attention mechanism is applied to video description in  [68] , and  [69] ,  [70]  studied the spatial attention for image captioning. In our case, even though a powerful text reading system TextBoxes  [4]  is used to spot words in natural images, there are still some errors (see Figure  1  due to large variations in both foreground texts and background objects, and uncontrollable lighting conditions).
B. Visual Representation In computer vision tasks, convolutional neural networks have become a popular method to extract image features. In  [71] , it has been shown that CNN models trained on a large image dataset can learn common features, which can be shared in many different tasks. In  [3] , the authors have also shown that fine-tunning from a trained model can obtain better performance than training from scratch in some tasks. Based on this property, we use GoogleNet  [5]  trained on millions of ImageNet  [72]  images as initial CNN model and fine tune its parameters with other parts of our model in an end-to-end way. Given an image I, we use the output of the last average pooling layer of the fine tuned GoogLeNet as the representation of visual features f v , a 1024-dimensional feature vector.
IV. EXPERIMENTAL RESULTS A. Experimental setup 1) Datasets: We evaluate the proposed method on two datasets: Con-Text dataset proposed in  [20]  and our selfcollected Drink Bottle dataset. The Con-Text dataset is dedicated for fine-grained classification of business places, e. g. , Cafe, Bookstore, and Pharmacy. It is composed of 24,255 images classified into 28 categories. All the images in this dataset are split into three folds, where two folds are used for training and the other one is used for testing. Each type of experiment on this dataset are repeated three times using respectively the three possible training and testing settings. The average performance is used as the reported results. To further verify the generality of the proposed method, we have collected a new image dataset named Drink Bottle dataset which will be released soon.
It consists of various types of drink bottle images contained in soft drink and alcoholic drink sets in ImageNet dataset  [72]. The dataset has 18,488 images divided into 20 categories: Root Beer, Ginger ale, Coca Cola, Pepsi Cola, Cream Soda, Egg Cream, Birch Beer, Tonic, Sarsaparilla, Orange Soda, Pulque, Kvass, Bitter, Guinness, Ouzo, Slivovitz, Drambuie, Vodka, Chablis and Sauterne. The other settings follow Con-Text dataset. As shown in some image samples illustrated in Figure  4 , the images in this dataset have low-resolutions and dense words. Moreover, a lot of fonts are in exotic styles. All these aspects make the fine-grained classification on this dataset more challenge than on the Con-Text dataset. 2) Evaluation measures: The Average Precision (AP) of each category and the mean of AP (mAP) over all categories are used to measure the performance.
Because most images in this dataset do not have text or the text reading system fails to spot words in images. Consequently, we train and test the textual baseline using 28 categories on a subset of images having spotted words. Since only text information is involved in the textual baseline, instead of using a weighted sum-pooling of word vectors described in Section III-D, we take the classical average of the word vectors given by all spotted words as text feature for this baseline. For a fair comparison, we also train a visual baseline model and a model of the proposed method on this subset.
Because the words having less than 3 letters are more likely to be false positives or represent irrelevant information for fine-grained classification. The loss weights in Eq. (  4 ) are set respectively to β 1 = 1. 0, β 2 = 0. 5, and β 3 = 0. 5. In the training stage, we use classical dataset augmentation strategy  [5]  with different scales and aspect ratios. More precisely, we re-size each training image to 255 × 255. Then, we randomly crop it with re-size scale in the interval [0. 08, 1] and aspect ratio in [3/4, 4/3]. Finally, the cropped patch is re-sized to 224 × 224. Then randomly horizontal flip is also used with a probability of 0. 5. In the testing stage, images are directly re-sized to 224 × 224 and the standard ten-crop testing  [5]  is adopted. The proposed method is implemented using Torch  [79]  deep learning framework.
All the experiments are conducted on a NVIDIA GTX TitanX GPU. The materials to reproduce all the reported results will be released soon. B. Importance of attention unit and multi-supervisions 1) Importance of attention unit: The state-of-the-art text reading system may result in some noisy texts including words that are irrelevant to the scene and incorrectly spotted words. These words may be similar in appearance but semantically far away, such as 'hotpot' and 'hotspot', 'barbera' and 'barber', and etc. We propose to use the attention mechanism to assign appropriate weights to different spotted words. The weighted sum-pooling of word vectors is then used to represent the textual information related to the scene. As depicted in Table I on the Con-Text dataset, the proposed attentionbased method achieves nearly 1% performance gain over the classical average-pooling scheme.
Without adjusting carefully the weights over different supervisions in Eq. (  4 ), we simply set the loss weights for combined text and visual classifier, visual classifier, and text classifier as described in Section IV-A4. As depicted in Table  II  on the Con-Text dataset, the multi-supervisions provide about a 0. 7% performance gain compared to single-supervision. Note that the two extra supervision classifiers are only used in training stage. During test stage, we simply use the classifier based on combined visual and text feature f c. Consequently, the 0. 7% performance gain thanks to multi-supervisions is achieved without any loss in runtime during the test.
C. Comparison with baselines We compare the proposed method with several baseline methods described in Section IV-A3. We have conducted two types of experiments using two different dataset settings: 1) Visual baseline and the proposed method trained on all training images and tested on all testing images; 2) Visual baseline, textual baseline, and the proposed method trained on the subset of training images and tested on the subset of testing images having spotted texts in each dataset. We add * after corresponding name to distinguish these experiments from previous ones. Table  III  displays the quantitative comparisons on the Con-Text dataset. In general, the textual baseline achieves results on par with visual baseline on images having spotted texts in this dataset. This shows that the text feature is as useful as visual feature for fine-grained classification on images containing texts.
The proposed method using the combination of text and visual features significantly outperforms the baseline based only on text or visual feature. A large improvement is observed particularly for the images having similar visual layout but discriminative textual cues, e. g. , Dry Cleaner, Discount House,Steak House, and Pawn Shop. For the images having fewer texts or no texts, the improvement is less significant. The observations on the Con-Text dataset also hold for the Drink Bottle dataset. As depicted in Table  IV , the proposed method also significantly outperforms the visual and textual baselines. Yet, the improvement over visual baseline (about 10%) is less important as compared to the improvement (about 20%) on the Con-Text dataset. The main reason is likely to be that recognition of the exotic texts in the low-resolution drink bottle images is more challenge.
D. Comparison with other methods We compare the proposed method with the state-of-the-art methods  [3] ,  [20]  on the Con-Text dataset. In  [3] ,  [20]. As shown in Table  III , the proposed method improves the stateof-the-art results from 77. 3% to 79. 6% mAP. Besides, the method in  [3]  relies on both global visual information and local object information given by 100 EdgeBoxes  [21]. The proposed method only relies on global visual information without the additional local part-based visual features. Moreover, the proposed method is end-to-end trainable, which is more elegant and efficient. Some fine-grained classification results of the proposed method are given in Figure  5. The proposed attention mechanism accurately selects the most relevant words related the scene. From (a) to (h), the proposed method makes correct predictions regardless the very close visual appearances of different fine-grained classes.
E. Application to Product Retrieval we present an application of the proposed framework for Product Retrieval to further demonstrate its potential. Compared to image classification, image retrieval attracts more attention in e-commerce. Given a query image I, the objective  of product retrieval is to select all product images from an image set withing the same category as I. We have conducted this experiment on the Drink Bottle dataset. We simply extract the input vector of the softmax-based classifier in fine-grained classification model. This input vector is considered as product feature for retrieval. The cosine similarity is used to rank images during matching. Following FlickrLogos-32's evaluation settings  [80] , the test set and train set are used as queries and respectively search set.
V. CONCLUSION In this paper, we have introduced a unified, end-to-end trainable framework which integrates visual and text cues for fine-grained image classification. We have also incorporated the attention mechanism to better exploit the extracted text in images by only selecting the most relevant words. Using several state of the art components in an effective pipeline, our experimental results on two datasets have demonstrated both the value and performance of the proposed method. Furthermore, the product retrieval experiment shows that the learned representation provides a potentially powerful tool for product search. It is worth noting that while it certainly makes sense to employ the state of the art in scene text and visual recognition as the components, our main motivation and contribution are to fill a gap in fine-grained image classification by exploiting multi-modal complementary information when present, and to do so with an end-to-end trainable neural network with the attention mechanism for implicitly exploiting the semantic relationships between textual and visual cues.
Classification performance for baselines, our method, and two state-of-the art methods on the Con-Text dataset. Models with * are trained and tested on the subset of images containing spotted texts. For clearness, the words in the first row are massage parlor, pet shop, restaurant, computer store, theatre, repair shop, bookstore, bakery, medical center, barbershop, pizzeria, diner, hotspot, bistro, teahouse, school, pharmacy, funeral, country store, tavern, motel, packinghouse, cafe, tobacco shop, dry cleaner, discount house, steakhouse, and pawnshop respectively. Method r. b. g. a. coke pepsi c. s. e. c. b. c. tonic sarsap. o. s. pulque kvass bitter guin. ouzo sliv. dramb. vodka chabl. saut. mAP.
Classification performance for baselines and the proposed method on the Drink Bottle dataset. The category of every logogram in this table is depicted in Section IV-A1. Similar to TableIII, models with * are trained and tested on the subset of images containing spotted texts. compare the performance of the proposed method and visual baseline on the subset of testing images having spotted texts and subset without texts separately in Con-Text dataset. In this dataset, about 42% images contain texts. The quantitative comparison is given in TableV. The proposed method improves dramatically the performance on images having texts as observed in the above experiments. Surprisingly, It also boosts slightly the performance for images without texts. One reason for this improvement may lie on the multi-supervisions of the network using shared parameters.
The synchronicity abstraction eliminates the non-determinism from the interleaving of concurrent behaviors. The synchronous languages lack appropriate concepts for task parallelism and timing-predictable scheduling on multiprocessors, whereas the streaming models do not support reactive behavior. The Fixed Priority Process Network (FPPN) model of computation has been proposed as a trade-off between streaming and reactive control processing, for task parallel programs. In FPPNs, task invocations depend on a combination of periodic data availability (similar to streaming models) and sporadic control events. Static scheduling methods for FPPNs  [20]  have demonstrated a predictable timing on multi-cores. A first implementation of the model  [22]  in an executable formal specification language called BIP (Behavior, Interaction, Priority) exists, more specifically in its real-time dialect  [3]  extended to tasks  [10].
Real-Time Semantics for the FPPN Model In the real-time semantics, job executions last for some physical time and can start concurrently with each other at any time after their invocation. Certain precedence constraints are respected which for certain jobs impose the same relative order of execution as in the zero-delay semantics, so that non-deterministic updates of the states of processes and channels are excluded. To ensure timeliness, the jobs should complete their execution within the deadline after their invocation. The semantics specifies the entities for communication, synchronization, scheduling and is defined by compilation to an executable formal specification language. Our approach is based on (real-time) 'BIP'  [3]  for modeling networks of connected timed automata components  [24].
We adopt the extension in  [10] , which introduces the concept of continuous (asynchronous) automata transitions, which, unlike the default (discrete) transitions take a certain physical time. Next to support of tasks (via continuous transitions), BIP supports the urgency in timing constraints, and those are timed-automata features required for adequate modeling and timing verification of dataflow languages  [9]. An important BIP language feature for implementing the functional code of tasks is the possibility to specify data actions in imperative programming language (C/C++). Figure  3  illustrates how an FPPN process is compiled to a BIP component. The source code is parsed, searching for primitives that are relevant for the interactions of the process with other components. The relevant primitives are the reads and writes from/to the data channels.
We assume that these tasks are encapsulated into software-architecture functional blocks, corresponding to FPPN processes. Before being integrated into a single architectural model they can be compiled and tested separately by functional simulation or by running on embedded platform. The high-level architecture description framework of our choice is the TASTE toolset  [14, 19] , whose front-end tools are based on the AADL (Architecture Analysis & Design Language) syntax  [7]. An architecture model in TASTE consists of functional blocks -so-called 'functions' -which interact with each other via pairs of interfaces (IF) 'required IF'/'provided IF', where the first performs a procedure call in the second one. In TASTE, the provided interfaces can be explicitly used for task invocations, i. e.
, they may get attributes like 'periodic'/'sporadic', 'deadline' and 'period'. The FPPN processes are represented by TASTE 'functions' that 'provide' such interfaces, implementing job execution of the respective in C/C++. Our TASTE-to-BIP framework is available for download at  [2]. The first step is plugging the data channels for explicit communication between the processes. The data channels are also modeled as TASTE functions, whereas reads and writes are implemented via interfaces. We have The second and final refinement step is scheduling. To schedule on multicores while respecting the real-time semantics of FPPN this step is preceded by transformation from TASTE architectural model into BIP FPPN model. The transformation process implements the FPPN-to-BIP 'compilation' sketched in the previous section, and we believe it could be formalized by a set of transformation rules.
Case Study: Guidance, Navigation and Control Application Our design flow was applied to a Guidance Navigation & Control (GNC) onboard spacecraft application that was ported onto ESA's NGMP, more specifically the quad-core LEON4FT processor  [1]. In the space industry, multi-cores provide a means for integrating more software functions onto a single platform, which contributes to reducing size, weight, cost, and power consumption. Onboard software has to efficiently utilize the processor resources, while retaining predictability. A GNC application affects the movement of the vehicle by reading the sensors and controlling the actuators. We estimated the WCETs of all tasks, W p , by measurements. There are four tasks: the Guidance Navigation Task (T p = 500 ms, d p = 500 ms, W p = 22 ms), the Control Output Task (T p = 50 ms, d p = 50 ms, W p = 3ms) that sends the outputs to the appropriate spacecraft unit, the Control FM Task (T p = 50 ms, d p = 50 ms, W p = 8 ms) which runs the control and flight management algorithms, and the Data Input Dispatcher Task (T p = 50 ms, d p = 50 ms, W p = 6 ms), which reads, decodes and dispatches data to the right destination whenever new data from the spacecraft's sensors are available.
Conclusion We presented the formal semantics of the FPPN model, at two levels: zero-delay semantics with precedence constraints on the job execution order to ensure functional determinism, and real-time semantics for scheduling. The semantics was implemented by a model transformational framework. Our approach was validated through a spacecraft on-board application running on a multi-core. In future work we consider it important to improve the efficiency of code generation, formal proofs of equivalence of the scheduling constraints (like the task graph) and the generated BIP model. The offline and online schedulers need to be enhanced to a wider spectrum of online policies and a better awareness of resource interference.
MePRiSIA design MePRiSIA is a methodology designed for the Information Technology (IT) staff of educational institutions. This methodology provides a basis for the development of an effective risk prevention program and contains a practical guidance to identify, assess and prevent the risks encountered in an academic information system. MePRiSIA is structured in 4 steps (see Figure  1 ) that include the human factor. The complete description of MePRiSIA is in  [20].
For example: * Servers of the system. * Devices used by users to access the system (mobile phones, PCs, laptops, etc. ). -Software: Includes the applications used to make use of the system. For example: * Authentication application * Database of academic information * Web browser or application used by users to access the system • Network: Includes the communication channel and network devices (switches, routers, etc. ). For example: -Client/server channel -Border router -Data center -Place where users access the system (internet cafe, home, university). • Organization: Includes assets that are responsibility of the institution. For example: -Image and reputation of the institution. -Policies of the system 2. What is the role of each asset? To determine the functions that each asset has within the system, according to the identified processes in the previous item.
What is the confidential information of the system? and what should be the level of privacy of the information? First, the evaluator identifies the personal information that is stored in the system, such as grades of students, financial information, etc. Then, he/she must determine what degree of confidentiality this information should have (low (public information), medium (internal use information), high (confidential information)). Finally, the evaluator must identify the assets that store or transport this information. 5. What are the security laws that can be applied to the system at a national and regional level? Government regulations awareness on the management of databases and personal information can be a valuable guideline to manage adequately the system information. 6. What are the institutional security policies applicable to the system assets? The evaluator must identify what institutional security policies talk about the assets of the system.
We infer global emissions of 32. 5-129 Tg C yr -1 (23. 2-91. 9 TgC yr -1 ) of organic vapors from the oceans into the marine atmosphere. Moreover, our results suggest a potential contribution to organic aerosol mass of more than 60% over the remote ocean. In addition, to support planning and design of future field studies with a focus on VOC emissions from interfacial photochemistry, we provide global distributions of VOC emission potentials as well as a small calculation tool based on Matlab, to estimate photochemical VOC emissions depending on location and season.
Methods Software and data processing. All data processing was conducted using Matlab (version 8. 6, R2015b, Mathworks Inc. ), including the m_map tools package (v1. 4h, https://www. eoas. ubc. ca/~rich/map. html) for the production of global maps. A general overview on all data sources and calculations is given in Supplementary Fig. 7. Moreover, the Matlab code as well as a stand-alone version of the Photochemistry At Liquid/Air interfaces containing Surfactants (PhotochemAtLAS) model, including instructions, is publicly available on Bitbucket, an online community repository. Production of organic vapors from interfacial photochemistry. For all calculations on photochemical VOC production presented here, we used the following three major assumptions. First, for the total surface area that is available for photochemistry, the air/water interface of the ocean is treated as a flat surface.