OpenRaiser/Intern-Atlas huggingface dataset:
Using data compression and randomization to build an unconditionally secure short key cipher
Venue: arXiv cs.CR 2021
Year: 2021

We consider the problem of constructing an unconditionally secure cipher for the case when the key length is less than the length of the encrypted message. (Unconditional security means that a computationally unbounded adversary cannot obtain information about the encrypted message without the key.) In this article, we propose data compression and randomization techniques combined with entropically-secure encryption. The resulting cipher can be used for encryption in such a way that the key length does not depend on the entropy or the length of the encrypted message; instead, it is determined by the required security level.
LUC at ComMA-2021 Shared Task: Multilingual Gender Biased and Communal Language Identification without using linguistic features
Venue: arXiv cs.CL 2021
Year: 2021

This work aims to evaluate the ability that both probabilistic and state-of-the-art vector space modeling (VSM) methods provide to well known machine learning algorithms to identify social network documents to be classified as aggressive, gender biased or communally charged. To this end, an exploratory stage was performed first in order to find relevant settings to test, i.e. by using training and development samples, we trained multiple algorithms using multiple vector space modeling and probabilistic methods and discarded the less informative configurations. These systems were submitted to the competition of the ComMA@ICON'21 Workshop on Multilingual Gender Biased and Communal Language Identification.
Symbolic Specialization of Rewriting Logic Theories with Presto
Venue: arXiv cs.LO 2021
Year: 2021

This paper introduces Presto, a symbolic partial evaluator for Maude's rewriting logic theories that can improve system analysis and verification. In Presto, the automated optimization of a conditional rewrite theory R (whose rules define the concurrent transitions of a system) is achieved by partially evaluating, with respect to the rules of R, an underlying, companion equational logic theory E that specifies the algebraic structure of the system states of R. This can be particularly useful for specializing an overly general equational theory E whose operators may obey complex combinations of associativity, commutativity, and/or identity axioms, when being plugged into a host rewrite theory R as happens, for instance, in protocol analysis, where sophisticated equational theories for cryptography are used. Presto implements different unfolding operators that are based on folding variant narrowing (the symbolic engine of Maude's equational theories). When combined with an appropriate abstraction algorithm, they allow the specialization to be adapted to the theory termination behavior and bring significant improvement while ensuring strong correctness and termination of the specialization. We demonstrate the effectiveness of Presto in several examples of protocol analysis where it achieves a significant speed-up. Actually, the transformation provided by Presto may cut down an infinite folding variant narrowing space to a finite one, and moreover, some of the costly algebraic axioms and rule conditions may be eliminated as well. As far as we know, this is the first partial evaluator for Maude that respects the semantics of functional, logic, concurrent, and object-oriented computations. Under consideration in Theory and Practice of Logic Programming (TPLP).
Integrating Knowledge in End-to-End Automatic Speech Recognition for Mandarin-English Code-Switching
Venue: arXiv cs.CL 2021
Year: 2021

Code-Switching (CS) is a common linguistic phenomenon in multilingual communities that consists of switching between languages while speaking. This paper presents our investigations on end-to-end speech recognition for Mandarin-English CS speech. We analyse different CS specific issues such as the properties mismatches between languages in a CS language pair, the unpredictable nature of switching points, and the data scarcity problem. We exploit and improve the state-of-the-art end-to-end system by merging nonlinguistic symbols, by integrating language identification using hierarchical softmax, by modeling sub-word units, by artificially lowering the speaking rate, and by augmenting data using speed perturbed technique and several monolingual datasets to improve the final performance not only on CS speech but also on monolingual benchmarks in order to make the system more applicable on real life settings. Finally, we explore the effect of different language model integration methods on the performance of the proposed model. Our experimental results reveal that all the proposed techniques improve the recognition performance. The best combined system improves the baseline system by up to 35% relatively in terms of mixed error rate and delivers acceptable performance on monolingual benchmarks.
Inverse deep learning methods and benchmarks for artificial electromagnetic material design
Venue: arXiv cs.LG 2021
Year: 2021

Deep learning (DL) inverse techniques have increased the speed of artificial electromagnetic material (AEM) design and improved the quality of resulting devices. Many DL inverse techniques have succeeded on a number of AEM design tasks, but to compare, contrast, and evaluate assorted techniques it is critical to clarify the underlying ill-posedness of inverse problems. Here we review state-of-the-art approaches and present a comprehensive survey of deep learning inverse methods and invertible and conditional invertible neural networks to AEM design. We produce easily accessible and rapidly implementable AEM design benchmarks, which offers a methodology to efficiently determine the DL technique best suited to solving different design challenges. Our methodology is guided by constraints on repeated simulation and an easily integrated metric, which we propose expresses the relative ill-posedness of any AEM design problem. We show that as the problem becomes increasingly ill-posed, the neural adjoint with boundary loss (NA) generates better solutions faster, regardless of simulation constraints. On simpler AEM design tasks, direct neural networks (NN) fare better when simulations are limited, while geometries predicted by mixture density networks (MDN) and conditional variational auto-encoders (VAE) can improve with continued sampling and re-simulation.
GPU optimization of the 3D Scale-invariant Feature Transform Algorithm and a Novel BRIEF-inspired 3D Fast Descriptor
Venue: arXiv cs.CV 2021
Year: 2021

This work details a highly efficient implementation of the 3D scale-invariant feature transform (SIFT) algorithm, for the purpose of machine learning from large sets of volumetric medical image data. The primary operations of the 3D SIFT code are implemented on a graphics processing unit (GPU), including convolution, sub-sampling, and 4D peak detection from scale-space pyramids. The performance improvements are quantified in keypoint detection and image-to-image matching experiments, using 3D MRI human brain volumes of different people. Computationally efficient 3D keypoint descriptors are proposed based on the Binary Robust Independent Elementary Feature (BRIEF) code, including a novel descriptor we call Ranked Robust Independent Elementary Features (RRIEF), and compared to the original 3D SIFT-Rank method\citep{toews2013efficient}. The GPU implementation affords a speedup of approximately 7X beyond an optimised CPU implementation, where computation time is reduced from 1.4 seconds to 0.2 seconds for 3D volumes of size (145, 174, 145) voxels with approximately 3000 keypoints. Notable speedups include the convolution operation (20X), 4D peak detection (3X), sub-sampling (3X), and difference-of-Gaussian pyramid construction (2X). Efficient descriptors offer a speedup of 2X and a memory savings of 6X compared to standard SIFT-Rank descriptors, at a cost of reduced numbers of keypoint correspondences, revealing a trade-off between computational efficiency and algorithmic performance. The speedups gained by our implementation will allow for a more efficient analysis on larger data sets. Our optimized GPU implementation of the 3D SIFT-Rank extractor is available at https://github.com/CarluerJB/3D_SIFT_CUDA.
Wiener Guided DIP for Unsupervised Blind Image Deconvolution
Venue: arXiv cs.CV 2021
Year: 2021

Blind deconvolution is an ill-posed problem arising in various fields ranging from microscopy to astronomy. The ill-posed nature of the problem requires adequate priors to arrive to a desirable solution. Recently, it has been shown that deep learning architectures can serve as an image generation prior during unsupervised blind deconvolution optimization, however often exhibiting a performance fluctuation even on a single image. We propose to use Wiener-deconvolution to guide the image generator during optimization by providing it a sharpened version of the blurry image using an auxiliary kernel estimate starting from a Gaussian. We observe that the high-frequency artifacts of deconvolution are reproduced with a delay compared to low-frequency features. In addition, the image generator reproduces low-frequency features of the deconvolved image faster than that of a blurry image. We embed the computational process in a constrained optimization framework and show that the proposed method yields higher stability and performance across multiple datasets. In addition, we provide the code.
Distributed Adaptive and Resilient Control of Multi-Robot Systems with Limited Field of View Interactions
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we consider two coupled problems for distributed multi-robot systems (MRSs) coordinating with limited field of view (FOV) sensors: adaptive tuning of interaction gains and rejection of sensor attacks. First, a typical shortcoming of distributed control frameworks (e.g., potential fields) is that the overall system behavior is highly sensitive to the gain assigned to relative interactions. Second, MRSs with limited FOV sensors can be more susceptible to sensor attacks aimed at their FOVs, and therefore must be resilient to such attacks. Based on these shortcomings, we propose a comprehensive solution that combines efforts in adaptive gain tuning and attack resilience to the problem of topology control for MRSs with limited FOVs. Specifically, we first derive an adaptive gain tuning scheme based on satisfying nominal pairwise interactions, which yields a dynamic balancing of interaction strengths in a robot's neighborhood. We then model additive sensor and actuator attacks (or faults) and derive H infinity control protocols by employing a static output-feedback technique, guaranteeing bounded L2 gains of the error induced by the attack (fault) signals. Finally, simulation results using ROS Gazebo are provided to support our theoretical findings.
DXML: Distributed Extreme Multilabel Classification
Venue: arXiv cs.DC 2021
Year: 2021

As a big data application, extreme multilabel classification has emerged as an important research topic with applications in ranking and recommendation of products and items. A scalable hybrid distributed and shared memory implementation of extreme classification for large scale ranking and recommendation is proposed. In particular, the implementation is a mix of message passing using MPI across nodes and using multithreading on the nodes using OpenMP. The expression for communication latency and communication volume is derived. Parallelism using work-span model is derived for shared memory architecture. This throws light on the expected scalability of similar extreme classification methods. Experiments show that the implementation is relatively faster to train and test on some large datasets. In some cases, model size is relatively small.
NavP: Enabling Navigational Programming for Science Data Processing via Application-Initiated Checkpointing
Venue: arXiv cs.DC 2021
Year: 2021

Science Data Systems (SDS) handle science data from acquisition through processing to distribution. They are deployed in the Cloud today, and the efficiency of Cloud instance utilization is critical to success. Conventional SDS are unable to take advantage of a cost-effective Amazon EC2 spot market, especially for long-running tasks. Some of the difficulties found in current practice at NASA/JPL are: a lack of mechanism for app programmers to save valuable partial results for future processing continuation, the heavy weight from using container-based (Singularity) sandboxes with more than 200,000 OS-level files; and the gap between scientists developing algorithms/programs on a laptop and the SDS experts deploying software in Cloud computing or supercomputing. We present a first proof-of-principle of this using NavP (Navigational Programming) and fault-tolerant computing (FTC) in SDS, by employing program state migration facilitated by Checkpoint-Restart (C/R). NavP provides a new navigational view of computations in a distributed world for the application programmers. The tool of DHP (DMTCP Hop and Publish) we developed enables the application programmers to navigate the computation among instances or nodes by inserting hop(destination) statements in their app code, and choose when to publish partial results at stages of their algorithms that they think worthwhile for future continuation. The result of using DHP is that a parallel distributed SDS becomes easier to program and deploy, and this enables more efficient leveraging of the Amazon EC2 Spot market. This technical report describes a high-level design and an initial implementation.
Learning for Robust Combinatorial Optimization: Algorithm and Application
Venue: arXiv cs.LG 2021
Year: 2021

Learning to optimize (L2O) has recently emerged as a promising approach to solving optimization problems by exploiting the strong prediction power of neural networks and offering lower runtime complexity than conventional solvers. While L2O has been applied to various problems, a crucial yet challenging class of problems -- robust combinatorial optimization in the form of minimax optimization -- have largely remained under-explored. In addition to the exponentially large decision space, a key challenge for robust combinatorial optimization lies in the inner optimization problem, which is typically non-convex and entangled with outer optimization. In this paper, we study robust combinatorial optimization and propose a novel learning-based optimizer, called LRCO (Learning for Robust Combinatorial Optimization), which quickly outputs a robust solution in the presence of uncertain context. LRCO leverages a pair of learning-based optimizers -- one for the minimizer and the other for the maximizer -- that use their respective objective functions as losses and can be trained without the need of labels for training problem instances. To evaluate the performance of LRCO, we perform simulations for the task offloading problem in vehicular edge computing. Our results highlight that LRCO can greatly reduce the worst-case cost and improve robustness, while having a very low runtime complexity.
Automated Vision-Based Wellness Analysis for Elderly Care Centers
Venue: arXiv cs.MM 2021
Year: 2021

The growth in the aging population requires caregivers to improve both efficiency and quality of healthcare. In this study, we develop an automatic, vision-based system for monitoring and analyzing the physical and mental well-being of senior citizens. Through collaboration with Haven of Hope Christian Service, we collect video recording data in the care center with surveillance cameras. We then process and extract personalized facial, activity, and interaction features from the video data using deep neural networks. This integrated health information systems can assist caregivers to gain better insights into the seniors they are taking care of. These insights, including wellness metrics and long-term health patterns of senior citizens, can help caregivers update their caregiving strategies. We report the findings of our analysis and evaluate the system quantitatively. We also summarize technical challenges and additional functionalities and technologies needed for offering a comprehensive system.
Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction
Venue: arXiv cs.CL 2021
Year: 2021

Recent works have shown explainability and robustness are two crucial ingredients of trustworthy and reliable text classification. However, previous works usually address one of two aspects: i) how to extract accurate rationales for explainability while being beneficial to prediction; ii) how to make the predictive model robust to different types of adversarial attacks. Intuitively, a model that produces helpful explanations should be more robust against adversarial attacks, because we cannot trust the model that outputs explanations but changes its prediction under small perturbations. To this end, we propose a joint classification and rationale extraction model named AT-BMC. It includes two key mechanisms: mixed Adversarial Training (AT) is designed to use various perturbations in discrete and embedding space to improve the model's robustness, and Boundary Match Constraint (BMC) helps to locate rationales more precisely with the guidance of boundary information. Performances on benchmark datasets demonstrate that the proposed AT-BMC outperforms baselines on both classification and rationale extraction by a large margin. Robustness analysis shows that the proposed AT-BMC decreases the attack success rate effectively by up to 69%. The empirical results indicate that there are connections between robust models and better explanations.
Diaformer: Automatic Diagnosis via Symptoms Sequence Generation
Venue: arXiv cs.AI 2021
Year: 2021

Automatic diagnosis has attracted increasing attention but remains challenging due to multi-step reasoning. Recent works usually address it by reinforcement learning methods. However, these methods show low efficiency and require taskspecific reward functions. Considering the conversation between doctor and patient allows doctors to probe for symptoms and make diagnoses, the diagnosis process can be naturally seen as the generation of a sequence including symptoms and diagnoses. Inspired by this, we reformulate automatic diagnosis as a symptoms Sequence Generation (SG) task and propose a simple but effective automatic Diagnosis model based on Transformer (Diaformer). We firstly design the symptom attention framework to learn the generation of symptom inquiry and the disease diagnosis. To alleviate the discrepancy between sequential generation and disorder of implicit symptoms, we further design three orderless training mechanisms. Experiments on three public datasets show that our model outperforms baselines on disease diagnosis by 1%, 6% and 11.5% with the highest training efficiency. Detailed analysis on symptom inquiry prediction demonstrates that the potential of applying symptoms sequence generation for automatic diagnosis.
Online Unplugged and Block-Based Cryptography in Grade 10
Venue: arXiv cs.CR 2021
Year: 2021

We report our experience of an extracurricular online intervention on cryptography principles in 10th grade. This paper's first goal is to present the learning path we designed, influenced by cryptography core ideas rather than technical knowledge. We will detail how we used Snap! (a visual programming language) to realize hands-on activities: programming playgrounds to experiment with cryptosystems and their limits, and interactive support for an unplugged activity on the Diffie-Hellman key exchange. The second goal is to evaluate our intervention in terms of both student perceptions and learning of core cryptography ideas. The students appreciated the course and felt that, despite being remote, it was fun, interesting, and engaging. They said the course helped them understand the role of cryptography, CS, and Math in society and sparked their interest, especially in cryptography and CS. The third goal is to discuss what worked well and areas of improvement. Pedagogically, remote teaching caused high "instructor blindness" and prevented us from giving the optimal amount of guidance during the exploration activities with Snap! playgrounds, making them sometimes too challenging for total programming novices. On the other hand, the "remote-unplugged" Diffie-Hellman worked well: it embodies a coherent metaphor that engaged the students and made them grasp this groundbreaking protocol. The students praised the activities as engaging, even when challenging. The final assessment showed that the core cryptography ideas were well understood.
Prioritized Hierarchical Compliance Control for Dual-Arm Robot Stable Clamping
Venue: arXiv cs.RO 2021
Year: 2021

When a dual-arm robot clamps a rigid object in an environment for human beings, the environment or the collaborating human will impose incidental disturbance on the operated object or the robot arm, leading to clamping failure, damaging the robot even hurting the human. This research proposes a prioritized hierarchical compliance control to simultaneously deal with the two types of disturbances in the dual-arm robot clamping. First, we use hierarchical quadratic programming (HQP) to solve the robot inverse kinematics under the joint constraints and prioritize the compliance for the disturbance on the object over that on the robot arm. Second, we estimate the disturbance forces throughout the momentum observer with the F/T sensors and adopt admittance control to realize the compliances. Finally, we perform the verify experiments on a 14-DOF position-controlled dual-arm robot WalkerX, clamping a rigid object stably while realizing the compliance against the disturbances.
Learning with Label Noise for Image Retrieval by Selecting Interactions
Venue: arXiv cs.CV 2021
Year: 2021

Learning with noisy labels is an active research area for image classification. However, the effect of noisy labels on image retrieval has been less studied. In this work, we propose a noise-resistant method for image retrieval named Teacher-based Selection of Interactions, T-SINT, which identifies noisy interactions, ie. elements in the distance matrix, and selects correct positive and negative interactions to be considered in the retrieval loss by using a teacher-based training setup which contributes to the stability. As a result, it consistently outperforms state-of-the-art methods on high noise rates across benchmark datasets with synthetic noise and more realistic noise.
a novel attention-based network for fast salient object detection
Venue: arXiv cs.CV 2021
Year: 2021

In the current salient object detection network, the most popular method is using U-shape structure. However, the massive number of parameters leads to more consumption of computing and storage resources which are not feasible to deploy on the limited memory device. Some others shallow layer network will not maintain the same accuracy compared with U-shape structure and the deep network structure with more parameters will not converge to a global minimum loss with great speed. To overcome all of these disadvantages, we proposed a new deep convolution network architecture with three contributions: (1) using smaller convolution neural networks (CNNs) to compress the model in our improved salient object features compression and reinforcement extraction module (ISFCREM) to reduce parameters of the model. (2) introducing channel attention mechanism in ISFCREM to weigh different channels for improving the ability of feature representation. (3) applying a new optimizer to accumulate the long-term gradient information during training to adaptively tune the learning rate. The results demonstrate that the proposed method can compress the model to 1/3 of the original size nearly without losing the accuracy and converging faster and more smoothly on six widely used datasets of salient object detection compared with the others models. Our code is published in https://gitee.com/binzhangbinzhangbin/code-a-novel-attention-based-network-for-fast-salient-object-detection.git