OpenRaiser/Intern-Atlas huggingface dataset:
De-specializing an HLS library for Deep Neural Networks: improvements upon hls4ml
Venue: arXiv cs.AR 2021
Year: 2021

Custom hardware accelerators for Deep Neural Networks are increasingly popular: in fact, the flexibility and performance offered by FPGAs are well-suited to the computational effort and low latency constraints required by many image recognition and natural language processing tasks. The gap between high-level Machine Learning frameworks (e.g., Tensorflow, Pytorch) and low-level hardware design in Verilog/VHDL creates a barrier to widespread adoption of FPGAs, which can be overcome with the help of High-Level Synthesis. hls4ml is a framework that translates Deep Neural Networks into annotated C++ code for High-Level Synthesis, offering a complete and user-friendly design process that has been enthusiastically adopted in physics research. We analyze the strengths and weaknesses of hls4ml, drafting a plan to enhance its core library of components in order to allow more advanced optimizations, target a wider selection of FPGAs, and support larger Neural Network models.
TRADE: TRusted Anonymous Data Exchange: Threat Sharing Using Blockchain Technology
Venue: arXiv cs.CR 2021
Year: 2021

Cyber attacks are becoming more frequent and sophisticated, introducing significant challenges for organizations to protect their systems and data from threat actors. Today, threat actors are highly motivated, persistent, and well-founded and operate in a coordinated manner to commit a diversity of attacks using various sophisticated tactics, techniques, and procedures. Given the risks these threats present, it has become clear that organizations need to collaborate and share cyber threat information (CTI) and use it to improve their security posture. In this paper, we present TRADE -- TRusted Anonymous Data Exchange -- a collaborative, distributed, trusted, and anonymized CTI sharing platform based on blockchain technology. TRADE uses a blockchain-based access control framework designed to provide essential features and requirements to incentivize and encourage organizations to share threat intelligence information. In TRADE, organizations can fully control their data by defining sharing policies enforced by smart contracts used to control and manage CTI sharing in the network. TRADE allows organizations to preserve their anonymity while keeping organizations fully accountable for their action in the network. Finally, TRADE can be easily integrated within existing threat intelligence exchange protocols - such as trusted automated exchange of intelligence information (TAXII) and OpenDXL, thereby allowing a fast and smooth technology adaptation.
Language learnability in the limit for general metrics: a Gold-Angluin result
Venue: arXiv cs.CL 2021
Year: 2021

In his pioneering work in the field of Inductive Inference, Gold (1967) proved that a set containing all finite languages and at least one infinite language over the same fixed alphabet is not learnable in the exact sense. Within the same framework, Angluin (1980) provided a complete characterization for the learnability of language families. Mathematically, the concept of exact learning in that classical setting can be seen as the use of a particular type of metric for learning in the limit. In this short research note we use Niyogi's extended version of a theorem by Blum and Blum (1975) on the existence of locking data sets to prove a necessary condition for learnability in the limit of any family of languages in any given metric. This recovers Gold's theorem as a special case. Moreover, when the language family is further assumed to contain all finite languages, the same condition also becomes sufficient for learnability in the limit.
Structure-Aware Face Clustering on a Large-Scale Graph with $\bf{10^{7}}$ Nodes
Venue: arXiv cs.CV 2021
Year: 2021

Face clustering is a promising method for annotating unlabeled face images. Recent supervised approaches have boosted the face clustering accuracy greatly, however their performance is still far from satisfactory. These methods can be roughly divided into global-based and local-based ones. Global-based methods suffer from the limitation of training data scale, while local-based ones are difficult to grasp the whole graph structure information and usually take a long time for inference. Previous approaches fail to tackle these two challenges simultaneously. To address the dilemma of large-scale training and efficient inference, we propose the STructure-AwaRe Face Clustering (STAR-FC) method. Specifically, we design a structure-preserved subgraph sampling strategy to explore the power of large-scale training data, which can increase the training data scale from ${10^{5}}$ to ${10^{7}}$. During inference, the STAR-FC performs efficient full-graph clustering with two steps: graph parsing and graph refinement. And the concept of node intimacy is introduced in the second step to mine the local structural information. The STAR-FC gets 91.97 pairwise F-score on partial MS1M within 310s which surpasses the state-of-the-arts. Furthermore, we are the first to train on very large-scale graph with 20M nodes, and achieve superior inference results on 12M testing data. Overall, as a simple and effective method, the proposed STAR-FC provides a strong baseline for large-scale face clustering. Code is available at \url{https://sstzal.github.io/STAR-FC/}.
Web Mining for Estimating Regulatory Blockchain Readiness
Venue: arXiv cs.IR 2021
Year: 2021

The regulatory framework of cryptocurrencies (and, in general, blockchain tokens) is of paramount importance. This framework drives nearly all key decisions in the respective business areas. In this work, a computational model is proposed for quantitatively estimating the regulatory stance of countries with respect to cryptocurrencies. This is conducted via web mining utilizing web search engines. The proposed model is experimentally validated. In addition, unsupervised learning (clustering) is applied for better analyzing the automatically derived estimations. Overall, very good performance is achieved by the proposed algorithmic approach.
Are Multilingual Models Effective in Code-Switching?
Venue: arXiv cs.CL 2021
Year: 2021

Multilingual language models have shown decent performance in multilingual and cross-lingual natural language understanding tasks. However, the power of these multilingual models in code-switching tasks has not been fully explored. In this paper, we study the effectiveness of multilingual language models to understand their capability and adaptability to the mixed-language setting by considering the inference speed, performance, and number of parameters to measure their practicality. We conduct experiments in three language pairs on named entity recognition and part-of-speech tagging and compare them with existing methods, such as using bilingual embeddings and multilingual meta-embeddings. Our findings suggest that pre-trained multilingual models do not necessarily guarantee high-quality representations on code-switching, while using meta-embeddings achieves similar results with significantly fewer parameters.
Network Reconfiguration Impact on Renewable Energy System and Energy Storage System in Day-Ahead Scheduling
Venue: arXiv cs.NI 2021
Year: 2021

Renewable energy sources (RES) has gained significant interest in recent years. However, due to favourable weather conditions, the RES is installed in remote locations with limited transmission capacity. As a result, it can lead to major curtailments of the free resource when the network is congested. Therefore, energy storage system (ESS) is considered as a viable solution to store energy and address the intermittent nature of RES though ESS is often distributed and may not be geographically close to RES. Therefore, ESS may also suffer from limited transmission capacity due to network congestion. Currently, grid operators overlook network flexibility as a congestion management tool in day-ahead scheduling. This paper addresses these issues and studies the benefits of introducing network reconfiguration (NR) as a preventive and corrective action for transmission flexibility in day-ahead stochastic security-constrained unit-commitment (SSCUC-PC) while considering a multi-scenario RES output. Simulation results demonstrate that NR can lower total system cost, reduce RES curtailments and utilize ESS for better impact by alleviating network congestion in both base-case and post-contingency networks.
DNN Quantization with Attention
Venue: arXiv cs.CV 2021
Year: 2021

Low-bit quantization of network weights and activations can drastically reduce the memory footprint, complexity, energy consumption and latency of Deep Neural Networks (DNNs). However, low-bit quantization can also cause a considerable drop in accuracy, in particular when we apply it to complex learning tasks or lightweight DNN architectures. In this paper, we propose a training procedure that relaxes the low-bit quantization. We call this procedure \textit{DNN Quantization with Attention} (DQA). The relaxation is achieved by using a learnable linear combination of high, medium and low-bit quantizations. Our learning procedure converges step by step to a low-bit quantization using an attention mechanism with temperature scheduling. In experiments, our approach outperforms other low-bit quantization techniques on various object recognition benchmarks such as CIFAR10, CIFAR100 and ImageNet ILSVRC 2012, achieves almost the same accuracy as a full precision DNN, and considerably reduces the accuracy drop when quantizing lightweight DNN architectures.
Discovering Multiple Design Approaches in Programming Assignment Submissions
Venue: arXiv cs.SE 2021
Year: 2021

In this paper, we present a novel approach of automated evaluation of programming assignments~(AEPA) the highlight of which is that it automatically identifies multiple solution approaches to the programming question from the set of submitted solutions. Our approach does not require the instructor to foresee all the possible solution approaches and accomplishes this task with little or no human intervention. This paves the way to multiple fundamental improvements in the way automated evaluation of programming assignments is done today. We have applied our method on multiple data sets of practical scale. In all cases, our method was able to detect the solution approaches employed by the students.
Foreground color prediction through inverse compositing
Venue: arXiv cs.CV 2021
Year: 2021

In natural image matting, the goal is to estimate the opacity of the foreground object in the image. This opacity controls the way the foreground and background is blended in transparent regions. In recent years, advances in deep learning have led to many natural image matting algorithms that have achieved outstanding performance in a fully automatic manner. However, most of these algorithms only predict the alpha matte from the image, which is not sufficient to create high-quality compositions. Further, it is not possible to manually interact with these algorithms in any way except by directly changing their input or output. We propose a novel recurrent neural network that can be used as a post-processing method to recover the foreground and background colors of an image, given an initial alpha estimation. Our method outperforms the state-of-the-art in color estimation for natural image matting and show that the recurrent nature of our method allows users to easily change candidate solutions that lead to superior color estimations.
A Framework for 3D Tracking of Frontal Dynamic Objects in Autonomous Cars
Venue: arXiv cs.RO 2021
Year: 2021

Both recognition and 3D tracking of frontal dynamic objects are crucial problems in an autonomous vehicle, while depth estimation as an essential issue becomes a challenging problem using a monocular camera. Since both camera and objects are moving, the issue can be formed as a structure from motion (SFM) problem. In this paper, to elicit features from an image, the YOLOv3 approach is utilized beside an OpenCV tracker. Subsequently, to obtain the lateral and longitudinal distances, a nonlinear SFM model is considered alongside a state-dependent Riccati equation (SDRE) filter and a newly developed observation model. Additionally, a switching method in the form of switching estimation error covariance is proposed to enhance the robust performance of the SDRE filter. The stability analysis of the presented filter is conducted on a class of discrete nonlinear systems. Furthermore, the ultimate bound of estimation error caused by model uncertainties is analytically obtained to investigate the switching significance. Simulations are reported to validate the performance of the switched SDRE filter. Finally, real-time experiments are performed through a multi-thread framework implemented on a Jetson TX2 board, while radar data is used for the evaluation.
A Simple and Efficient Stochastic Rounding Method for Training Neural Networks in Low Precision
Venue: arXiv cs.LG 2021
Year: 2021

Conventional stochastic rounding (CSR) is widely employed in the training of neural networks (NNs), showing promising training results even in low-precision computations. We introduce an improved stochastic rounding method, that is simple and efficient. The proposed method succeeds in training NNs with 16-bit fixed-point numbers and provides faster convergence and higher classification accuracy than both CSR and deterministic rounding-to-the-nearest method.
Symmetry-Preserving Paths in Integrated Gradients
Venue: arXiv cs.LG 2021
Year: 2021

We provide rigorous proofs that the Integrated Gradients (IG) attribution method for deep networks satisfies completeness and symmetry-preserving properties. We also study the uniqueness of IG as a path method preserving symmetry.
On Evolving Attention Towards Domain Adaptation
Venue: arXiv cs.CV 2021
Year: 2021

Towards better unsupervised domain adaptation (UDA). Recently, researchers propose various domain-conditioned attention modules and make promising progresses. However, considering that the configuration of attention, i.e., the type and the position of attention module, affects the performance significantly, it is more generalized to optimize the attention configuration automatically to be specialized for arbitrary UDA scenario. For the first time, this paper proposes EvoADA: a novel framework to evolve the attention configuration for a given UDA task without human intervention. In particular, we propose a novel search space containing diverse attention configurations. Then, to evaluate the attention configurations and make search procedure UDA-oriented (transferability + discrimination), we apply a simple and effective evaluation strategy: 1) training the network weights on two domains with off-the-shelf domain adaptation methods; 2) evolving the attention configurations under the guide of the discriminative ability on the target domain. Experiments on various kinds of cross-domain benchmarks, i.e., Office-31, Office-Home, CUB-Paintings, and Duke-Market-1510, reveal that the proposed EvoADA consistently boosts multiple state-of-the-art domain adaptation approaches, and the optimal attention configurations help them achieve better performance.
Improving Online Forums Summarization via Hierarchical Unified Deep Neural Network
Venue: arXiv cs.CL 2021
Year: 2021

Online discussion forums are prevalent and easily accessible, thus allowing people to share ideas and opinions by posting messages in the discussion threads. Forum threads that significantly grow in length can become difficult for participants, both newcomers and existing, to grasp main ideas. To mitigate this problem, this study aims to create an automatic text summarizer for online forums. We present Hierarchical Unified Deep Neural Network to build sentence and thread representations for the forum summarization. In this scheme, Bi-LSTM derives a representation that comprises information of the whole sentence and whole thread; whereas, CNN captures most informative features with respect to context from sentence and thread. Attention mechanism is applied on top of CNN to further highlight high-level representations that carry important information contributing to a desirable summary. Extensive performance evaluation has been conducted on three datasets, two of which are real-life online forums and one is news dataset. The results reveal that the proposed model outperforms several competitive baselines.
Directed Data-Processing Inequalities for Systems with Feedback
Venue: arXiv cs.IT 2021
Year: 2021

We present novel data-processing inequalities relating the mutual information and the directed information in systems with feedback. The internal blocks within such systems are restricted only to be causal mappings, but are allowed to be non-linear, stochastic and time varying. These blocks can for example represent source encoders, decoders or even communication channels. Moreover, the involved signals can be arbitrarily distributed. Our first main result relates mutual and directed informations and can be interpreted as a law of conservation of information flow. Our second main result is a pair of data-processing inequalities (one the conditional version of the other) between nested pairs of random sequences entirely within the closed loop. Our third main result is introducing and characterizing the notion of in-the-loop (ITL) transmission rate for channel coding scenarios in which the messages are internal to the loop. Interestingly, in this case the conventional notions of transmission rate associated with the entropy of the messages and of channel capacity based on maximizing the mutual information between the messages and the output turn out to be inadequate. Instead, as we show, the ITL transmission rate is the unique notion of rate for which a channel code attains zero error probability if and only if such ITL rate does not exceed the corresponding directed information rate from messages to decoded messages. We apply our data-processing inequalities to show that the supremum of achievable (in the usual channel coding sense) ITL transmission rates is upper bounded by the supremum of the directed information rate across the communication channel. Moreover, we present an example in which this upper bound is attained. Finally, ...
Predicting Directionality in Causal Relations in Text
Venue: arXiv cs.CL 2021
Year: 2021

In this work, we test the performance of two bidirectional transformer-based language models, BERT and SpanBERT, on predicting directionality in causal pairs in the textual content. Our preliminary results show that predicting direction for inter-sentence and implicit causal relations is more challenging. And, SpanBERT performs better than BERT on causal samples with longer span length. We also introduce CREST which is a framework for unifying a collection of scattered datasets of causal relations.
Multi-Execution Lattices Fast and Slow
Venue: arXiv cs.CR 2021
Year: 2021

Methods for automatically, soundly, and precisely guaranteeing the noninterference security policy are predominantly based on multi-execution. All other methods are either based on undecidable theorem proving or suffer from false alarms. The multi-execution mechanisms, meanwhile, work by isolating security levels during program execution and running multiple copies of the target program, once for each security level with carefully tailored inputs that ensure both soundness and precision. When security levels are hierarchically organised in a lattice, this may lead to an exponential number of executions of the target program as the number of possible ways of combining security levels grows. In this paper we study how the lattice structure for security levels influences the runtime overhead of multi-execution. We additionally show how to use Galois connections to gain speedups in multi-execution by switching from lattices with high overhead to lattices with low overhead. Additionally, we give an empirical evaluation that corroborates our analysis and shows how Galois connections have potential to speed up multi-execution.
MCTSteg: A Monte Carlo Tree Search-based Reinforcement Learning Framework for Universal Non-additive Steganography
Venue: arXiv cs.MM 2021
Year: 2021

Recent research has shown that non-additive image steganographic frameworks effectively improve security performance through adjusting distortion distribution. However, as far as we know, all of the existing non-additive proposals are based on handcrafted policies, and can only be applied to a specific image domain, which heavily prevent non-additive steganography from releasing its full potentiality. In this paper, we propose an automatic non-additive steganographic distortion learning framework called MCTSteg to remove the above restrictions. Guided by the reinforcement learning paradigm, we combine Monte Carlo Tree Search (MCTS) and steganalyzer-based environmental model to build MCTSteg. MCTS makes sequential decisions to adjust distortion distribution without human intervention. Our proposed environmental model is used to obtain feedbacks from each decision. Due to its self-learning characteristic and domain-independent reward function, MCTSteg has become the first reported universal non-additive steganographic framework which can work in both spatial and JPEG domains. Extensive experimental results show that MCTSteg can effectively withstand the detection of both hand-crafted feature-based and deep-learning-based steganalyzers. In both spatial and JPEG domains, the security performance of MCTSteg steadily outperforms the state of the art by a clear margin under different scenarios.
AttrLostGAN: Attribute Controlled Image Synthesis from Reconfigurable Layout and Style
Venue: arXiv cs.CV 2021
Year: 2021

Conditional image synthesis from layout has recently attracted much interest. Previous approaches condition the generator on object locations as well as class labels but lack fine-grained control over the diverse appearance aspects of individual objects. Gaining control over the image generation process is fundamental to build practical applications with a user-friendly interface. In this paper, we propose a method for attribute controlled image synthesis from layout which allows to specify the appearance of individual objects without affecting the rest of the image. We extend a state-of-the-art approach for layout-to-image generation to additionally condition individual objects on attributes. We create and experiment on a synthetic, as well as the challenging Visual Genome dataset. Our qualitative and quantitative results show that our method can successfully control the fine-grained details of individual objects when modelling complex scenes with multiple objects. Source code, dataset and pre-trained models are publicly available (https://github.com/stanifrolov/AttrLostGAN).
Latus Incentive Scheme: Enabling Decentralization in Blockchains based on Recursive SNARKs
Venue: arXiv cs.CR 2021
Year: 2021

In the Zendoo whitepaper we introduced a novel SNARK-based construction that allows Bitcoin-like blockchains to create and communicate with sidechains of different types without knowing their internal structure. We also introduced a specific construction, called Latus, allowing creation of fully verifiable sidechains. In the paper we omitted a detailed description of an incentive scheme for Latus that is an essential element of a real decentralized system. This paper fills the gap by introducing details of the incentive scheme for the Latus sidechain. The represented ideas can also be adopted by other SNARK-based blockchains to incentivize decentralized proofs creation.