OpenRaiser/Intern-Atlas huggingface dataset:
A next-generation platform for Cyber Range-as-a-Service
Venue: arXiv cs.CR 2021
Year: 2021

In the last years, Cyber Ranges have become a widespread solution to train professionals for responding to cyber threats and attacks. Cloud computing plays a key role in this context since it enables the creation of virtual infrastructures on which Cyber Ranges are based. However, the setup and management of Cyber Ranges are expensive and time-consuming activities. In this paper, we highlight the novel features for the next-generation Cyber Range platforms. In particular, these features include the creation of a virtual clone for an actual corporate infrastructure, relieving the security managers from the setup of the training scenarios and sessions, the automatic monitoring of the participants' activities, and the emulation of their behavior.
Improving Robustness with Image Filtering
Venue: arXiv cs.CV 2021
Year: 2021

Adversarial robustness is one of the most challenging problems in Deep Learning and Computer Vision research. All the state-of-the-art techniques require a time-consuming procedure that creates cleverly perturbed images. Due to its cost, many solutions have been proposed to avoid Adversarial Training. However, all these attempts proved ineffective as the attacker manages to exploit spurious correlations among pixels to trigger brittle features implicitly learned by the model. This paper first introduces a new image filtering scheme called Image-Graph Extractor (IGE) that extracts the fundamental nodes of an image and their connections through a graph structure. By leveraging the IGE representation, we build a new defense method, Filtering As a Defense, that does not allow the attacker to entangle pixels to create malicious patterns. Moreover, we show that data augmentation with filtered images effectively improves the model's robustness to data corruption. We validate our techniques on CIFAR-10, CIFAR-100, and ImageNet.
Sports Video: Fine-Grained Action Detection and Classification of Table Tennis Strokes from Videos for MediaEval 2021
Venue: arXiv cs.CV 2021
Year: 2021

Sports video analysis is a prevalent research topic due to the variety of application areas, ranging from multimedia intelligent devices with user-tailored digests up to analysis of athletes' performance. The Sports Video task is part of the MediaEval 2021 benchmark. This task tackles fine-grained action detection and classification from videos. The focus is on recordings of table tennis games. Running since 2019, the task has offered a classification challenge from untrimmed video recorded in natural conditions with known temporal boundaries for each stroke. This year, the dataset is extended and offers, in addition, a detection challenge from untrimmed videos without annotations. This work aims at creating tools for sports coaches and players in order to analyze sports performance. Movement analysis and player profiling may be built upon such technology to enrich the training experience of athletes and improve their performance.
Supervised Graph Contrastive Pretraining for Text Classification
Venue: arXiv cs.CL 2021
Year: 2021

Contrastive pretraining techniques for text classification has been largely studied in an unsupervised setting. However, oftentimes labeled data from related tasks which share label semantics with current task is available. We hypothesize that using this labeled data effectively can lead to better generalization on current task. In this paper, we propose a novel way to effectively utilize labeled data from related tasks with a graph based supervised contrastive learning approach. We formulate a token-graph by extrapolating the supervised information from examples to tokens. Our formulation results in an embedding space where tokens with high/low probability of belonging to same class are near/further-away from one another. We also develop detailed theoretical insights which serve as a motivation for our method. In our experiments with $13$ datasets, we show our method outperforms pretraining schemes by $2.5\%$ and also example-level contrastive learning based formulation by $1.8\%$ on average. In addition, we show cross-domain effectiveness of our method in a zero-shot setting by $3.91\%$ on average. Lastly, we also demonstrate our method can be used as a noisy teacher in a knowledge distillation setting to significantly improve performance of transformer based models in low labeled data regime by $4.57\%$ on average.
Automated Drug-Related Information Extraction from French Clinical Documents: ReLyfe Approach
Venue: arXiv cs.CL 2021
Year: 2021

Structuring medical data in France remains a challenge mainly because of the lack of medical data due to privacy concerns and the lack of methods and approaches on processing the French language. One of these challenges is structuring drug-related information in French clinical documents. To our knowledge, over the last decade, there are less than five relevant papers that study French prescriptions. This paper proposes a new approach for extracting drug-related information from French clinical scanned documents while preserving patients' privacy. In addition, we deployed our method in a health data management platform where it is used to structure drug medical data and help patients organize their drug schedules. It can be implemented on any web or mobile platform. This work closes the gap between theoretical and practical work by creating an application adapted to real production problems. It is a combination of a rule-based phase and a Deep Learning approach. Finally, numerical results show the outperformance and relevance of the proposed methodology.
Multi-Modality Distillation via Learning the teacher's modality-level Gram Matrix
Venue: arXiv cs.AI 2021
Year: 2021

In the context of multi-modality knowledge distillation research, the existing methods was mainly focus on the problem of only learning teacher final output. Thus, there are still deep differences between the teacher network and the student network. It is necessary to force the student network to learn the modality relationship information of the teacher network. To effectively exploit transfering knowledge from teachers to students, a novel modality relation distillation paradigm by modeling the relationship information among different modality are adopted, that is learning the teacher modality-level Gram Matrix.
Self-Supervised Learning based Monaural Speech Enhancement with Multi-Task Pre-Training
Venue: arXiv cs.SD 2021
Year: 2021

In self-supervised learning, it is challenging to reduce the gap between the enhancement performance on the estimated and target speech signals with existed pre-tasks. In this paper, we propose a multi-task pre-training method to improve the speech enhancement performance with self-supervised learning. Within the pre-training autoencoder (PAE), only a limited set of clean speech signals are required to learn their latent representations. Meanwhile, to solve the limitation of single pre-task, the proposed masking module exploits the dereverberated mask and estimated ratio mask to denoise the mixture as the second pre-task. Different from the PAE, where the target speech signals are estimated, the downstream task autoencoder (DAE) utilizes a large number of unlabeled and unseen reverberant mixtures to generate the estimated mixtures. The trained DAE is shared by the learned representations and masks. Experimental results on a benchmark dataset demonstrate that the proposed method outperforms the state-of-the-art approaches.
Sentence Embeddings and High-speed Similarity Search for Fast Computer Assisted Annotation of Legal Documents
Venue: arXiv cs.CL 2021
Year: 2021

Human-performed annotation of sentences in legal documents is an important prerequisite to many machine learning based systems supporting legal tasks. Typically, the annotation is done sequentially, sentence by sentence, which is often time consuming and, hence, expensive. In this paper, we introduce a proof-of-concept system for annotating sentences "laterally." The approach is based on the observation that sentences that are similar in meaning often have the same label in terms of a particular type system. We use this observation in allowing annotators to quickly view and annotate sentences that are semantically similar to a given sentence, across an entire corpus of documents. Here, we present the interface of the system and empirically evaluate the approach. The experiments show that lateral annotation has the potential to make the annotation process quicker and more consistent.
Practical Active Learning with Model Selection for Small Data
Venue: arXiv cs.LG 2021
Year: 2021

Active learning is of great interest for many practical applications, especially in industry and the physical sciences, where there is a strong need to minimize the number of costly experiments necessary to train predictive models. However, there remain significant challenges for the adoption of active learning methods in many practical applications. One important challenge is that many methods assume a fixed model, where model hyperparameters are chosen a priori. In practice, it is rarely true that a good model will be known in advance. Existing methods for active learning with model selection typically depend on a medium-sized labeling budget. In this work, we focus on the case of having a very small labeling budget, on the order of a few dozen data points, and develop a simple and fast method for practical active learning with model selection. Our method is based on an underlying pool-based active learner for binary classification using support vector classification with a radial basis function kernel. First we show empirically that our method is able to find hyperparameters that lead to the best performance compared to an oracle model on less separable, difficult to classify datasets, and reasonable performance on datasets that are more separable and easier to classify. Then, we demonstrate that it is possible to refine our model selection method using a weighted approach to trade-off between achieving optimal performance on datasets that are easy to classify, versus datasets that are difficult to classify, which can be tuned based on prior domain knowledge about the dataset.
DarkGates: A Hybrid Power-Gating Architecture to Mitigate the Performance Impact of Dark-Silicon in High Performance Processors
Venue: arXiv cs.AR 2021
Year: 2021

To reduce the leakage power of inactive (dark) silicon components, modern processor systems shut-off these components' power supply using low-leakage transistors, called power-gates. Unfortunately, power-gates increase the system's power-delivery impedance and voltage guardband, limiting the system's maximum attainable voltage (i.e., Vmax) and, thus, the CPU core's maximum attainable frequency (i.e., Fmax). As a result, systems that are performance constrained by the CPU frequency (i.e., Fmax-constrained), such as high-end desktops, suffer significant performance loss due to power-gates. To mitigate this performance loss, we propose DarkGates, a hybrid system architecture that increases the performance of Fmax-constrained systems while fulfilling their power efficiency requirements. DarkGates is based on three key techniques: i) bypassing on-chip power-gates using package-level resources (called bypass mode), ii) extending power management firmware to support operation either in bypass mode or normal mode, and iii) introducing deeper idle power states. We implement DarkGates on an Intel Skylake microprocessor for client devices and evaluate it using a wide variety of workloads. On a real 4-core Skylake system with integrated graphics, DarkGates improves the average performance of SPEC CPU2006 workloads across all thermal design power (TDP) levels (35W-91W) between 4.2% and 5.3%. DarkGates maintains the performance of 3DMark workloads for desktop systems with TDP greater than 45W while for a 35W-TDP (the lowest TDP) desktop it experiences only a 2% degradation. In addition, DarkGates fulfills the requirements of the ENERGY STAR and the Intel Ready Mode energy efficiency benchmarks of desktop systems.
Convolutional neural network based on transfer learning for breast cancer screening
Venue: arXiv cs.CV 2021
Year: 2021

Breast cancer is the most common cancer in the world and the most prevalent cause of death among women worldwide. Nevertheless, it is also one of the most treatable malignancies if detected early. In this paper, a deep convolutional neural network-based algorithm is proposed to aid in accurately identifying breast cancer from ultrasonic images. In this algorithm, several neural networks are fused in a parallel architecture to perform the classification process and the voting criteria are applied in the final classification decision between the candidate object classes where the output of each neural network is representing a single vote. Several experiments were conducted on the breast ultrasound dataset consisting of 537 Benign, 360 malignant, and 133 normal images. These experiments show an optimistic result and a capability of the proposed model to outperform many state-of-the-art algorithms on several measures. Using k-fold cross-validation and a bagging classifier ensemble, we achieved an accuracy of 99.5% and a sensitivity of 99.6%.
Latent Space Simulation for Carbon Capture Design Optimization
Venue: arXiv cs.LG 2021
Year: 2021

The CO2 capture efficiency in solvent-based carbon capture systems (CCSs) critically depends on the gas-solvent interfacial area (IA), making maximization of IA a foundational challenge in CCS design. While the IA associated with a particular CCS design can be estimated via a computational fluid dynamics (CFD) simulation, using CFD to derive the IAs associated with numerous CCS designs is prohibitively costly. Fortunately, previous works such as Deep Fluids (DF) (Kim et al., 2019) show that large simulation speedups are achievable by replacing CFD simulators with neural network (NN) surrogates that faithfully mimic the CFD simulation process. This raises the possibility of a fast, accurate replacement for a CFD simulator and therefore efficient approximation of the IAs required by CCS design optimization. Thus, here, we build on the DF approach to develop surrogates that can successfully be applied to our complex carbon-capture CFD simulations. Our optimized DF-style surrogates produce large speedups (4000x) while obtaining IA relative errors as low as 4% on unseen CCS configurations that lie within the range of training configurations. This hints at the promise of NN surrogates for our CCS design optimization problem. Nonetheless, DF has inherent limitations with respect to CCS design (e.g., limited transferability of trained models to new CCS packings). We conclude with ideas to address these challenges.
New metal-plastic hybrid additive manufacturing strategy: Fabrication of arbitrary metal-patterns on external and even internal surfaces of 3D plastic structures
Venue: arXiv cs.RO 2021
Year: 2021

Constructing precise micro-nano metal patterns on complex three-dimensional (3D) plastic parts allows the fabrication of functional devices for advanced applications. However, this patterning is currently expensive and requires complex processes with long manufacturing lead time. The present work demonstrates a process for the fabrication of micro-nano 3D metal-plastic composite structures with arbitrarily complex shapes. In this approach, a light-cured resin is modified to prepare an active precursor capable of allowing subsequent electroless plating (ELP). A multi-material digital light processing 3D printer was newly developed to enable the fabrication of parts containing regions made of either standard resin or active precursor resin nested within each other. Selective 3D ELP processing of such parts provided various metal-plastic composite parts having complicated hollow micro-nano structures with specific topological relationships on a size scale as small as 40 um. Using this technique, 3D metal topologies that cannot be manufactured by traditional methods are possible, and metal patterns can be produced inside plastic parts as a means of further miniaturizing electronic devices. The proposed method can also generate metal coatings exhibiting improved adhesion of metal to plastic substrate. Based on this technique, several sensors composed of different functional nonmetallic materials and specific metal patterns were designed and fabricated. The present results demonstrate the viability of the proposed method and suggest potential applications in the fields of smart 3D micro-nano electronics, 3D wearable devices, micro/nano-sensors, and health care.
Domain Adaptation with Pre-trained Transformers for Query Focused Abstractive Text Summarization
Venue: arXiv cs.CL 2021
Year: 2021

The Query Focused Text Summarization (QFTS) task aims at building systems that generate the summary of the text document(s) based on the given query. A key challenge in addressing this task is the lack of large labeled data for training the summarization model. In this paper, we address this challenge by exploring a series of domain adaptation techniques. Given the recent success of pre-trained transformer models in a wide range of natural language processing tasks, we utilize such models to generate abstractive summaries for the QFTS task for both single-document and multi-document scenarios. For domain adaptation, we apply a variety of techniques using pre-trained transformer-based summarization models including transfer learning, weakly supervised learning, and distant supervision. Extensive experiments on six datasets show that our proposed approach is very effective in generating abstractive summaries for the QFTS task while setting a new state-of-the-art result in several datasets across a set of automatic and human evaluation metrics.
Supporting RISC-V Performance Counters through Performance analysis tools for Linux (Perf)
Venue: arXiv cs.PF 2021
Year: 2021

Increased attention to RISC-V in Cloud, Data Center, Automotive and Networking applications, has been fueling the move of RISC-V to the high-performance computing scenario. However, lack of powerful performance monitoring tools will result in poorly optimized applications and, consequently, a limited computing performance. While the RISC-V ISA already defines a hardware performance monitor (HPM), current software gives limited support for monitoring performance. In this paper we introduce extensions and modifications to the Performance analysis tools for Linux(perf/perf_events), Linux kernel, and OpenSBI, aiming to achieve full support for the RISC-V performance monitoring specification. Preliminary testing and evaluation was carried out in Linux 5.7 running on a FPGA booted CVA6 CPU, formerly named Ariane, showing a monitoring overhead of 0.283%.
On the Reverse-Complement String-Duplication System
Venue: arXiv cs.IT 2021
Year: 2021

Motivated by DNA storage in living organisms, and by known biological mutation processes, we study the reverse-complement string-duplication system. We fully classify the conditions under which the system has full expressiveness, for all alphabets and all fixed duplication lengths. We then focus on binary systems with duplication length $2$ and prove that they have full capacity, yet surprisingly, have zero entropy-rate. Finally, by using binary single burst-insertion correcting codes, we construct codes that correct a single reverse-complement duplication of odd length, over any alphabet. The redundancy (in bits) of the constructed code does not depend on the alphabet size.
Binary Image Skeletonization Using 2-Stage U-Net
Venue: arXiv cs.CV 2021
Year: 2021

Object Skeletonization is the process of extracting skeletal, line-like representations of shapes. It provides a very useful tool for geometric shape understanding and minimal shape representation. It also has a wide variety of applications, most notably in anatomical research and activity detection. Several mathematical algorithmic approaches have been developed to solve this problem, and some of them have been proven quite robust. However, a lesser amount of attention has been invested into deep learning solutions for it. In this paper, we use a 2-stage variant of the famous U-Net architecture to split the problem space into two sub-problems: shape minimization and corrective skeleton thinning. Our model produces results that are visually much better than the baseline SkelNetOn model. We propose a new metric, M-CCORR, based on normalized correlation coefficients as an alternative to F1 for this challenge as it solves the problem of class imbalance, managing to recognize skeleton similarity without suffering from F1's over-sensitivity to pixel-shifts.
Multimodal Analysis of memes for sentiment extraction
Venue: arXiv cs.CL 2021
Year: 2021

Memes are one of the most ubiquitous forms of social media communication. The study and processing of memes, which are intrinsically multimedia, is a popular topic right now. The study presented in this research is based on the Memotion dataset, which involves categorising memes based on irony, comedy, motivation, and overall-sentiment. Three separate innovative transformer-based techniques have been developed, and their outcomes have been thoroughly reviewed.The best algorithm achieved a macro F1 score of 0.633 for humour classification, 0.55 for motivation classification, 0.61 for sarcasm classification, and 0.575 for overall sentiment of the meme out of all our techniques.
Startup Ecosystem Rankings
Venue: arXiv cs.DL 2021
Year: 2021

The number, importance, and popularity of rankings measuring innovation performance and the strength and resources of ecosystems that provide its spatial framework are on an increasing trend globally. In addition to influencing the specific decisions taken by economic actors, these rankings significantly impact the development of innovation-related policies at regional, national, and international levels. The importance of startup ecosystems is proven by the growing scientific interest, which is demonstrated by the increasing number of related scientific articles. The concept of the startup ecosystem is a relatively new category, the application of which in everyday and scientific life has been gaining ground since the end of the 2000s. In parallel, of course, the demand for measurability and comparability has emerged among decision-makers and scholars. This demand is met by startup ecosystem rankings, which now measure and rank the performance of individual ecosystems on a continental and global scale. However, while the number of scientific publications examining rankings related to higher education, economic performance, or even innovation, can be measured in the order of thousands, scientific research has so far rarely or tangentially addressed the rankings of startup ecosystems. This study and the related research intend to fill this gap by presenting and analysing the characteristics of global rankings and identifying possible future research directions.