dataesr/scientific-paragraphs-categorization huggingface dataset:
For the rest of the paper, we will use the following notation when needed (see Table  1  for an example). A rule-type , similar to a database schema, is a structure  = ⟨, , ⟩, where  is a set of criteria (e. g. , Airport, Time frame, Region, and Terminal); , a set of rules; and , a set of decisions (e. g. , 25, 40, 45, and 90 minutes). A criterion  ∈  is a structure  = ⟨A, ,   ⟩ where A, is a set of values -or alphabet (e. g. , Schengen and International for criterion Region);  : A 2 → 0, 1 a matching function (e. g. , a numerical range check between the Time frame and the query value); and   , a precision weight (i. e. , how generic or specific the criterion is). A rule  ∈  is a structure  = ⟨, ,   ⟩ where  :  → ⊥ A, is a partial assignment function from criteria to values (i. e, a unique combination of values);  ∈ ; and   =   , the precision weight of the rule.
The engine is divided into several components. The Domain Explorer searches the flight domain, exploring all possible connecting airports, carriers and flights combinations; and pre-selects a number of potential routes. These routes are fed into the Route Selection component, which uses heuristics and a decision tree ensemble  [21]  (module Route Scoring) to reduce further the set of potential routes most likely to be bought. The reduced set is passed on to the Pricing Engine, which then computes the price for each flight combination. The engine is currently implemented in a data centre architecture, where the different components run on different machines, and many machines are used to run the components in parallel to reach the necessary throughput. The system is large, with each one of the components using several hundred machines.
In recent work, we have explored the complex interplay between the number of routes being considered and the overall throughput/latency constraints, and we refer to  [21]  for more details on that aspect of the system and how we successfully used hardware acceleration to optimise the Route Scoring module. The MCT module is part of the Domain Explorer and runs on the same machines. This design is necessary to cut on the number of network hops needed to process a query, but imposes restrictions on the rule engine. For instance, Drools  [4]  requires 9. 5 GB of main memory to run the hundreds of thousands MCT rules. Such a high resource consumption makes it impossible to embed it within the Domain Explorer given the existing architecture. Deploying it stand-alone in its own set of servers is not an option either, as it would cause network overhead.
During the Execution phase, the engine is in its working state and performs query evaluation. The NFA Consistency module automatically detects the NFA version of incoming queries and turns the engine into Setup state when necessary. Externally, two AXI-controllers, NFA/Query and Results, are responsible for loading the NFA data during setup, fetching queries and dispatching results. An AXI-lite controller manages control registers for synchronisation between host CPU and FPGA kernel. AXI-controllers are modules provided by Xilinx Vitis 1 to manage accesses to either the FPGA's DDR or to the streaming interface, as well as to memory registers. q i,c1 q i,c2 q i,c3 MCT.
Current MCT Implementation. The current MCT module is a C++ implementation that uses neither an FA nor any special processing technique beyond ordering of the criteria and data partitioning. The static rule set is stored in main memory, where it takes about 130 MB. The downtime required to upload an updated set of rules is in the order of hundreds of milliseconds. The MCT module is embedded in the Domain Explorer of the flight search engine, which maps each user query to a thread that processes this one in its entirety while at the Domain Explorer component. The MCT module is therefore also single threaded. Scalability is accomplished by replicating the MCT module within each processing thread. Since a user query arriving to the Domain Explorer is mapped to a thread, it is easy to see how the requests to the MCT module are batched.
CPU implementations. We have developed two CPU versions in C++ using a DFA and an NFA similar to the one used in the FPGA. Both use Heuristic 2 with descending order to optimise and reduce the size of the FAs. Both index attributes as needed to speed up access. Five small data sets, corresponding to rules from individual airports, were used for the DFA vs. NFA comparison. We have tried to generate DFAs for larger subsets of rules, but we were not able to do so even using a very large server. Our analysis show that, for 136K rules, the intermediate DFA would reach about 3. 5 billions of transitions, and an optimistic estimate of the size after suffix merging would be about 1 billion transitions. Figure  8  shows that the CPU-DFA is about 10% to 47% faster than the CPU-NFA, at the cost of having a larger memory footprint.
This cost is about 300% to 500% bigger than the NFA-equivalent data structure, and is likely to increase as the number of rules increase. The CPU-NFA implementation follows the model used in the current system: each MCT module runs in a Domain Explorer thread, and each one keeps its own copy of the NFA. This makes the design compute-bound. Each thread navigates the NFA using a Depth-First-Search strategy. Since the NFA takes less than 2 MB (Figure  3 ), it largely fits in the cache. Furthermore, the NFA data is read-only during evaluation, eliminating any sort of synchronisation and cacheinvalidation among different cores. This design sacrifices parallelism within the MCT module, but allows to scale the Domain Explorer by spawning a thread per user query. The CPU version of the NFA is an important reference, as it is faster than the current MCT implementation, but not as fast as Drools (Figure  11 ).
DISCUSSION There are several ways to take advantage of the design just described as a deployment solution within the Amadeus flight search engine. We discuss them here to illustrate the interplay between hardware acceleration and existing systems. By running MCT on an FPGA, we could free up to 40% of CPU time in the Domain Explorer, which could then, theoretically, accept 40% more requests. However, even the higher number of requests would barely saturate the FPGA-based engine, given that it provides performance that is orders of magnitude higher than the current CPU implementation. While it is not possible to choose a different board on Amazon  Alternatively, the extra capacity of a big FPGA board could be used to combine the MCT module with Route Scoring, another part of the flight search engine successfully accelerated using FPGAs  [21].
The Route Scoring would move earlier in the flow, directly inside the Domain Explorer, to score the routes during the flight domain exploration (Figure  13 ). By doing so, the Route Scoring would be able to process several tens of thousands of routes in the Domain Explorer, instead of only few hundreds inside the Route Selection, while respecting the same response time constraint. Both MCT and Route Scoring would then be on the same FPGA, and pipelined together to minimise the back and forth with the CPU. Such pipeline would require an additional refactoring of the Domain Explorer, since the notions of route for MCT and Route Scoring are not exactly the same, but such deployment would optimise the FPGA occupancy and its usage, while combining the potential business benefits of both modules.
In terms of the resulting system, what we propose is narrower than a general purpose business rule engine. A good analogy is a key value store (KVS) versus a full relational database engine. On one hand, a KVS is faster and more scalable for a narrower set of use cases and operations. On the other hand, a database supports a far wider set of use cases, but the price of generality is reduced performance and elasticity. One way to look at what we propose here is to understand it as a form of "key value store" for business rules that significantly improves performance and resource utilisation for a narrower set of use cases. As with KVS, the advantages of a specialised system are significant enough to be considered a replacement to more general solutions.
However, this is not true when dealing with highly heterogeneous datasets and multiple modalities such as in the case of SAR-optical image matching. Although remote sensing imagery often contains geographical coordinates for 2004), by adapting the gradient operator and scale-space to be more suited to the properties of SAR imagery  (Dellinger et al. , 2015; Gong et al. , 2014; Suri et al. , 2010). These approaches were relatively successful in matching images in the SAR domain, however, they failed to match across modalities as the detected and described features were independent of those features detected in the optical domain  Ma et al. (2017). This is partially due to the vast radiometric differences between SAR and optical imagery. To address this,  (Ye and Shen, 2016)  proposed the histogram of oriented phase congruency (HOPC) descriptor whereby phase congruency was used as a proxy for gradient information.
For instance, in classical computer vision, many handcrafted approaches break down under large baselines or strong radiometric differences. For this reason, and with the advent of modern deep learning techniques, there has been a strong movement towards deep matching to solve the image matching problem directly from data  (Kuppala et al. , 2020). Fischer et al. (2014)  demonstrated that features extracted from the last layer of a Convolutional Neural Network (CNN), pretrained on ImageNet, can outperform the SIFT descriptor in image matching tasks. This lead to the development of several CNN-based descriptors, which learned similarity metrics directly from corresponding image patch pairs. Simo-Serra et al. (2015)  proposed the use of a siamese network trained with pairs of corresponding and non-corresponding patches, and a Euclidean distance metric to learn a 128-dimensional descriptor for image matching.
The function   is a spatial softmax operation which is applied to the predicted heatmap in order to convert the matching scores into a probability distribution with the peak at the point of correspondence. The softmax activation relates all points in the heatmap and thus to obtain a strong peak it encourages the suppression of the matching score in other regions. As the ground truth map contains only a single non-zero value we make use of a weighting vector,   , to ensure the loss at the peak is given the same importance as the loss created by all non-corresponding points in the heatmap. This further exaggerating the requirement for a strong peak in the heatmap. Thus  1 and  0 represent the count of the number of zero-and non-zero pixels in. Due to the spatial softmax operation   , which normalizes ∑ , ỹ = 1, and the loss function which prioritizes peakiness, the network tends to overfit the training dataset.
Datasets and workflow While the logical structure of the framework follows from the goodness network via the correspondence network to the outlier reduction network, as depicted in Fig. 1 , this is not the case for training. As the training of goodness and outlier reduction networks rely on a trained correspondence network, we start by describing the dataset for the correspondence network followed by the description of the datasets, derived from the correspondence network outputs, which are used for training the goodness and outlier reduction networks. We further provide insights into the assumptions which were made and outline the way in which training, validation and testing samples were selected.
SAR and optical correspondence To train the correspondence network we require a large dataset of salient candidate search and template patches with known points of correspondence. Due to the complexity of creating such a dataset, and the intractability of manually annotating correspondence across heterogeneous domains, we rely on simplifying assumptions (such as the correspondence of points at ground level in co-registered imagery) and the Urban Atlas dataset  (Schneider et al. , 2010)  to generate our training and validation data. The Urban Atlas dataset consists of manually co-registered, high resolution SAR and optical imagery acquired over 23 European cities. The imagery covers types of scenes including rural, industrial, suburban and urban environments, and has a total coverage area of approximately 20,000 km 2.
The optical imagery was acquired using the Panchromatic Remote-sensing Instrument for Stereo Mapping (PRISM) sensor with a spatial resolution of 2. 5 m, while the SAR imagery is based off of Enhanced Ellipsoid Corrected (EEC) TerraSAR-X data products and thus have square pixels with a pixel spacing of 1. 25 m. Within the framework of the Urban Atlas project  (Schneider et al. , 2010) , these SAR and optical images were accurately co-registered using hundreds of manually selected tie points to align each image pair. Based on this manual alignment it is reported that the residual co-registration error is within the range of 3 to 5 m  (Merkle et al. , 2017). To reduce the complexity of the matching problem, as well as to allow larger batch sizes during training, we downsample the TerraSAR-X imagery using bi-linear interpolation, to a 2.
5 m pixel spacing. The 23 cities are then divided into three groups for training, validation and testing. The process of dividing the cities between the three datasets is formulated as a discrete optimization problem over the total area of the imagery for each scene. Within this formulation the objective is to assign the cities to the three datasets such that the final dataset exhibits an approximately 80/20 split between the number of training, and testing samples. The final assignment and the spatial distribution of the cities can be seen in Fig. 8. We then apply a Harris corner detector to the optical images to select points which are salient in at least one modality. Using these points, and the knowledge that the SAR and optical data in the Urban Atlas data set has been accurately co-registered  (Schneider et al.
Goodness As no goodness dataset exists, and the creation of such a dataset is non-obvious for manual annotation, we rely on the trained correspondence network to identify patches which can act as positive and negative samples for training and evaluating the goodness network. To do this we make use of the SAR and optical patches from the correspondence datasets, as well as the recorded matching loss,   , and an  2 correspondence point error for each sample . We then create binary goodness labels for each sample by thresholding -log(  ) and the  2 error. The negative log loss is used to invert the loss and reduce the dynamic range, which makes the task of selecting thresholds easier. We label the patch pairs such that, 𝑦 𝑚𝑖 = { 1 if -log( 𝑚𝑠𝑒 ) ≥ 1. 2 and  𝑒 ≤ 1 0 if -log( 𝑚𝑠𝑒 ) ≤ 1 or  𝑒 ≥ 2.
5 (8) where   is the label for the th patch pair, and label   = 1 represents a patch pair which is good for matching, while 0 represents patch pairs which lead to inaccurate or unsuccessful matching. The values for the thresholds were chosen based on the training dataset such that we avoid possibly ambiguous samples, this process is depicted in Fig. 10. The negative log loss allows for easier selection of patches which produce correspondence heatmaps with desirable properties (low matching loss), such as a single peak with a narrow spread and small values everywhere else, while the 2 threshold ensures that these heatmaps actually correspond to positive matches. As the correspondence dataset was created with no guarantees of mutually visible features, there is a large imbalance in the final goodness dataset with many more negative examples being present.
Outlier reduction To train the outlier removal network we make use of the valid region of the heatmaps generated from the correspondence network. These heatmaps are used as inputs to the outlier reduction network and the binary training labels indicate whether they were the result of a successful or unsuccessful matching result. The generation of the heatmap labels follows the same approach to labeling as the previously described goodness dataset. However, we only apply the  2 threshold as the label relies solely on whether the patch was accurately matched. Some labeled examples from the training dataset are shown in Fig. 11.
, 2019)  to implement all aspects of our proposed pipeline. The various sub-networks were randomly initialized using the method proposed by  He et al. (2015) , and are trained using the Adam solver  (Kingma and Ba, 2014). The hyperparameters used for the solver are specified in Table  1. For each of the sub-networks the optimal learning rate was determined using the search method proposed by  Smith (2017). We make use of a fixed batch size of 16 samples, which constitutes the maximum batch size that could be used to train the correspondence network on a Nvidia GTX1080Ti GPU. This batch size further allowed for both goodness networks and the outlier reduction network to be trained simultaneously on the same GPU. The correspondence network was trained for 50 epochs, while the remaining sub-networks were trained for 1000 epochs due to the relatively small dataset size in comparison to the correspondence dataset size.
Data augmentation was used to improve generalization and reduce the risks of overfitting. This step was found to be of increased importance when training the goodness and outlier reduction networks due to the reduced dataset size. The data augmentation pipeline consisted of horizontal (HF) and vertical flipping (VF), image scaling (IS) by a factor of ±0. 1, intensity scaling (CS) by a random value between (0. 7, 1. 3), and coarse image dropout (CD) of between (1%, 5%) of the image pixels, taken on a version of the image which is downsampled to between (80%, 98%) of the original size. Each of these augmentations is applied during training with a certain probability, as described in Table  2. To aid future development and in the interest of openness in science, a full implementation of the framework has been released.
Ablation study To aid the design of the correspondence network described in Section 2. 2 we performed an ablation study to compare the performance of the network as various architectural and regularization elements were added. We tested four variants of the correspondence network which are detailed in Table  3. The networks were trained as previously described, and the random elements in the training process were made deterministic such that all networks were trained on the same data and augmentations. Finally, we evaluated the performance of the various networks using the validation dataset to prevent biasing our architecture selection to the test data. We evaluate performance in terms of matching accuracy and precision. Whereby, matching accuracy is defined by the percentage of matches which have an 2 distance to the ground truth point of correspondence of at most one pixel, as well as the mean 2 error.
Matching results As the correspondence network plays a vital role in training the goodness and outlier reduction networks, it is imperative that we evaluate its performance relative to existing methods. To do so we make use of two relevant and available methods: Normalized Cross Correlation (NCC)  (Burger and Burge, 2009) , as well as the pseudo-Siamese matching approach (PSiam) presented in  Hughes et al. (2018). To ensure a fair comparison we retrained the pseudo-siamese approach on the same dataset, and under the same data augmentations and pre-processing as our correspondence network. As the pseudosiamese network requires corresponding and non-corresponding SARoptical patch pairs, for training, we applied random offsets for the creation of the non-corresponding pairs. Furthermore, both the SAR and optical pairs were cropped to an extent of 128 × 128 pixels.
During the evaluation phase we apply the pseudo-siamese network over the full extent of the SAR search patch, using a sliding window approach, to generate a correspondence heatmap. Table  5  shows the matching accuracy and precision for the baseline methods compared to the proposed method when assessed on our ground-level Harris corner derived test dataset. From Table  5  it is clear that our proposed matching architecture provides a significant improvement in matching accuracy as well as precision over the selected baseline methods. The discrepancy between the test precision in and the validation precision reported in Table  4  is most likely due to a wider diversity of scenes being used for testing. In Fig. 12  we evaluate the peakiness and smoothness of correspondence heatmaps generated by the various methods.
Goodness results To gain an understanding for the performance of the domain specific goodness networks, as well as the effects of minimum or maximum fusion on the cross-domain goodness, we assess the binary classification performance with respect to the test dataset. This assessment is done in terms of the classification accuracy, precision and recall, which are defined as, 𝑎𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = 𝑇 𝑃 + 𝑇 𝑁 𝑇 𝑃 + 𝑇 𝑁 + 𝐹 𝑃 + 𝐹 𝑁 , ( 9 ) 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = 𝑇 𝑃 𝑇 𝑃 + 𝐹 𝑃 , ( 10 ) 𝑟𝑒𝑐𝑎𝑙𝑙 = 𝑇 𝑃 𝑇 𝑃 + 𝐹 𝑁 , ( 11 ) where TP, TN, FN and FP represent the total number of true positives, true negatives, false negatives and false positives results, respectively. The results for this investigation are described in Table  6. The results presented in Table  6  represent the binary classification accuracy when predicting the likelihood of a SAR-optical patch pair for generating an accurate correspondence.
The same properties can be seen in the false negative results. While remote sensing imagery is highly calibrated, clouds and varying daylight conditions can lead to illumination variations in the optical imagery. For this reason, we evaluated the repeatability of the jointly identified points of high goodness under differing optical image brightness conditions. To do so, the optical image brightness was adjusted by a factor between [-50%, 50%] in 10% increments. The goodness network was then used to determine points of high goodness between these adjusted optical images and the original SAR image. The repeatability of the detected points, for a specific variation, was then calculated with respect the original set of points detected in the nonbrightness adjusted image. It was found that the goodness network can repeatably detect between 45% to 60% of the original points under The percentage of the dataset used for matching (# Regions), and matching performance obtained when pre-selecting search and template patches based on their domain specific and cross-domain goodness scores.
Goodness As the purpose of the goodness network is to improve the matching accuracy of the correspondence network, by pre-selecting regions which have a higher probability of being correctly matched, we further evaluate the goodness network through the process of matching. Table  7 , presents the matching performance when we match against the test patches which have been identified as having a high domain specific, or cross-modality goodness. The proportion of the original test dataset which was identified to have high goodness is described as the number of regions (# Regions). From the results presented in Table  7  it is evident that the prefiltering of regions, based on goodness, leads to improved matching accuracy and precision over the baseline (Table  5 ). Furthermore, the low percentage of good regions found in the evaluation dataset hints to the non-optimal choice of using optical domain Harris corners to create the dataset.