OpenRaiser/Intern-Atlas huggingface dataset:
MIPE: A Metric Independent Pipeline for Effective Code-Mixed NLG Evaluation
Venue: arXiv cs.CL 2021
Year: 2021

Code-mixing is a phenomenon of mixing words and phrases from two or more languages in a single utterance of speech and text. Due to the high linguistic diversity, code-mixing presents several challenges in evaluating standard natural language generation (NLG) tasks. Various widely popular metrics perform poorly with the code-mixed NLG tasks. To address this challenge, we present a metric independent evaluation pipeline MIPE that significantly improves the correlation between evaluation metrics and human judgments on the generated code-mixed text. As a use case, we demonstrate the performance of MIPE on the machine-generated Hinglish (code-mixing of Hindi and English languages) sentences from the HinGE corpus. We can extend the proposed evaluation strategy to other code-mixed language pairs, NLG tasks, and evaluation metrics with minimal to no effort.
Extending Challenge Sets to Uncover Gender Bias in Machine Translation: Impact of Stereotypical Verbs and Adjectives
Venue: arXiv cs.CL 2021
Year: 2021

Human gender bias is reflected in language and text production. Because state-of-the-art machine translation (MT) systems are trained on large corpora of text, mostly generated by humans, gender bias can also be found in MT. For instance when occupations are translated from a language like English, which mostly uses gender neutral words, to a language like German, which mostly uses a feminine and a masculine version for an occupation, a decision must be made by the MT System. Recent research showed that MT systems are biased towards stereotypical translation of occupations. In 2019 the first, and so far only, challenge set, explicitly designed to measure the extent of gender bias in MT systems has been published. In this set measurement of gender bias is solely based on the translation of occupations. In this paper we present an extension of this challenge set, called WiBeMT, with gender-biased adjectives and adds sentences with gender-biased verbs. The resulting challenge set consists of over 70, 000 sentences and has been translated with three commercial MT systems: DeepL Translator, Microsoft Translator, and Google Translate. Results show a gender bias for all three MT systems. This gender bias is to a great extent significantly influenced by adjectives and to a lesser extent by verbs.
Deep Machine Learning Based Egyptian Vehicle License Plate Recognition Systems
Venue: arXiv cs.CV 2021
Year: 2021

Automated Vehicle License Plate (VLP) detection and recognition have ended up being a significant research issue as of late. VLP localization and recognition are some of the most essential techniques for managing traffic using digital techniques. In this paper, four smart systems are developed to recognize Egyptian vehicles license plates. Two systems are based on character recognition, which are (System1, Characters Recognition with Classical Machine Learning) and (System2, Characters Recognition with Deep Machine Learning). The other two systems are based on the whole plate recognition which are (System3, Whole License Plate Recognition with Classical Machine Learning) and (System4, Whole License Plate Recognition with Deep Machine Learning). We use object detection algorithms, and machine learning based object recognition algorithms. The performance of the developed systems has been tested on real images, and the experimental results demonstrate that the best detection accuracy rate for VLP is provided by using the deep learning method. Where the VLP detection accuracy rate is better than the classical system by 32%. However, the best detection accuracy rate for Vehicle License Plate Arabic Character (VLPAC) is provided by using the classical method. Where VLPAC detection accuracy rate is better than the deep learning-based system by 6%. Also, the results show that deep learning is better than the classical technique used in VLP recognition processes. Where the recognition accuracy rate is better than the classical system by 8%. Finally, the paper output recommends a robust VLP recognition system based on both statistical and deep machine learning.
Case Studies in Formal Reasoning About Lambda-Calculus: Semantics, Church-Rosser, Standardization and HOAS
Venue: arXiv cs.LO 2021
Year: 2021

We have previously published the Isabelle/HOL formalization of a general theory of syntax with bindings. In this companion paper, we instantiate the general theory to the syntax of lambda-calculus and formalize the development leading to several fundamental constructions and results: sound semantic interpretation, the Church-Rosser and standardization theorems, and higher-order abstract syntax (HOAS) encoding. For Church-Rosser and standardization, our work covers both the call-by-name and call-by-value versions of the calculus, following classic papers by Takahashi and Plotkin. During the formalization, we were able to stay focused on the high-level ideas of the development -- thanks to the arsenal provided by our general theory: a wealth of basic facts about the substitution, swapping and freshness operators, as well as recursive-definition and reasoning principles, including a specialization to semantic interpretation of syntax.
Breath to Pair (B2P): Respiration-Based Pairing Protocol for Wearable Devices
Venue: arXiv cs.CR 2021
Year: 2021

We propose Breath to Pair (B2P), a protocol for pairing and shared-key generation for wearable devices that leverages the wearer's respiration activity to ensure that the devices are part of the same body-area network. We assume that the devices exploit different types of sensors to extract and process the respiration signal. We illustrate B2P for the case of two devices that use respiratory inductance plethysmography (RIP) and accelerometer sensors, respectively. Allowing for different types of sensors in pairing allows us to include wearable devices that use a variety of different sensors. In practice, this form of sensor variety creates a number of challenges that limit the ability of the shared-key establishment algorithm to generate matching keys. The two main obstacles are the lack of synchronization across the devices and the need for correct noise-induced mismatches between the generated key bit-strings. B2P addresses the synchronization challenge by utilizing Change Point Detection (CPD) to detect abrupt changes in the respiration signal and consider their occurrences as synchronizing points. Any potential mismatches are handled by optimal quantization and encoding of the respiration signal in order to maximize the error correction rate and minimize the message overheads. Extensive evaluation on a dataset collected from 30 volunteers demonstrates that our protocol can generate a secure 256-bit key every 2.85 seconds (around one breathing cycle). Particular attention is given to secure B2P against device impersonation attacks.
DECIFE: Detecting Collusive Users Involved in Blackmarket Following Services on Twitter
Venue: arXiv cs.SI 2021
Year: 2021

The popularity of Twitter has fostered the emergence of various fraudulent user activities - one such activity is to artificially bolster the social reputation of Twitter profiles by gaining a large number of followers within a short time span. Many users want to gain followers to increase the visibility and reach of their profiles to wide audiences. This has provoked several blackmarket services to garner huge attention by providing artificial followers via the network of agreeable and compromised accounts in a collusive manner. Their activity is difficult to detect as the blackmarket services shape their behavior in such a way that users who are part of these services disguise themselves as genuine users. In this paper, we propose DECIFE, a framework to detect collusive users involved in producing 'following' activities through blackmarket services with the intention to gain collusive followers in return. We first construct a heterogeneous user-tweet-topic network to leverage the follower/followee relationships and linguistic properties of a user. The heterogeneous network is then decomposed to form four different subgraphs that capture the semantic relations between the users. An attention-based subgraph aggregation network is proposed to learn and combine the node representations from each subgraph. The combined representation is finally passed on to a hypersphere learning objective to detect collusive users. Comprehensive experiments on our curated dataset are conducted to validate the effectiveness of DECIFE by comparing it with other state-of-the-art approaches. To our knowledge, this is the first attempt to detect collusive users involved in blackmarket 'following services' on Twitter.
One-Leg Stance of Humanoid Robot using Active Balance Control
Venue: arXiv cs.RO 2021
Year: 2021

The task of self-balancing is one of the most important tasks when developing humanoid robots. This paper proposes a novel external balance mechanism for humanoid robot to maintain sideway balance. First, a dynamic model of the humanoid robot with balance mechanism and its simplified model are introduced. Secondly, a backstepping-based control method is utilized to split the system into two sub-systems. Then, a minimum observer-based controller is used to control the first sub-system. Since the second sub-system has unknown parameters, a model reference adaptive controller (MRAC) is used to control it. The proposed design divides the walking and balancing into two separated tasks, allowing the walking control can be executed independently of the balancing control. Furthermore, the use of the balance mechanism ensures the humanoid robot's hip movement does not exceed the threshold of a human when walking. Thus, making the overall pose of the humanoid robot looks more natural. An experiment is carried out on a commercial humanoid robot known as UXA-90 to evaluate the effectiveness of the proposed method.
PoseFace: Pose-Invariant Features and Pose-Adaptive Loss for Face Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Despite the great success achieved by deep learning methods in face recognition, severe performance drops are observed for large pose variations in unconstrained environments (e.g., in cases of surveillance and photo-tagging). To address it, current methods either deploy pose-specific models or frontalize faces by additional modules. Still, they ignore the fact that identity information should be consistent across poses and are not realizing the data imbalance between frontal and profile face images during training. In this paper, we propose an efficient PoseFace framework which utilizes the facial landmarks to disentangle the pose-invariant features and exploits a pose-adaptive loss to handle the imbalance issue adaptively. Extensive experimental results on the benchmarks of Multi-PIE, CFP, CPLFW and IJB have demonstrated the superiority of our method over the state-of-the-arts.
Improving Variational Autoencoder based Out-of-Distribution Detection for Embedded Real-time Applications
Venue: arXiv cs.LG 2021
Year: 2021

Uncertainties in machine learning are a significant roadblock for its application in safety-critical cyber-physical systems (CPS). One source of uncertainty arises from distribution shifts in the input data between training and test scenarios. Detecting such distribution shifts in real-time is an emerging approach to address the challenge. The high dimensional input space in CPS applications involving imaging adds extra difficulty to the task. Generative learning models are widely adopted for the task, namely out-of-distribution (OoD) detection. To improve the state-of-the-art, we studied existing proposals from both machine learning and CPS fields. In the latter, safety monitoring in real-time for autonomous driving agents has been a focus. Exploiting the spatiotemporal correlation of motion in videos, we can robustly detect hazardous motion around autonomous driving agents. Inspired by the latest advances in the Variational Autoencoder (VAE) theory and practice, we tapped into the prior knowledge in data to further boost OoD detection's robustness. Comparison studies over nuScenes and Synthia data sets show our methods significantly improve detection capabilities of OoD factors unique to driving scenarios, 42% better than state-of-the-art approaches. Our model also generalized near-perfectly, 97% better than the state-of-the-art across the real-world and simulation driving data sets experimented. Finally, we customized one proposed method into a twin-encoder model that can be deployed to resource limited embedded devices for real-time OoD detection. Its execution time was reduced over four times in low-precision 8-bit integer inference, while detection capability is comparable to its corresponding floating-point model.
Towards Controlled and Diverse Generation of Article Comments
Venue: arXiv cs.CL 2021
Year: 2021

Much research in recent years has focused on automatic article commenting. However, few of previous studies focus on the controllable generation of comments. Besides, they tend to generate dull and commonplace comments, which further limits their practical application. In this paper, we make the first step towards controllable generation of comments, by building a system that can explicitly control the emotion of the generated comments. To achieve this, we associate each kind of emotion category with an embedding and adopt a dynamic fusion mechanism to fuse this embedding into the decoder. A sentence-level emotion classifier is further employed to better guide the model to generate comments expressing the desired emotion. To increase the diversity of the generated comments, we propose a hierarchical copy mechanism that allows our model to directly copy words from the input articles. We also propose a restricted beam search (RBS) algorithm to increase intra-sentence diversity. Experimental results show that our model can generate informative and diverse comments that express the desired emotions with high accuracy.
Reinforced Imitation Learning by Free Energy Principle
Venue: arXiv cs.LG 2021
Year: 2021

Reinforcement Learning (RL) requires a large amount of exploration especially in sparse-reward settings. Imitation Learning (IL) can learn from expert demonstrations without exploration, but it never exceeds the expert's performance and is also vulnerable to distributional shift between demonstration and execution. In this paper, we radically unify RL and IL based on Free Energy Principle (FEP). FEP is a unified Bayesian theory of the brain that explains perception, action and model learning by a common fundamental principle. We present a theoretical extension of FEP and derive an algorithm in which an agent learns the world model that internalizes expert demonstrations and at the same time uses the model to infer the current and future states and actions that maximize rewards. The algorithm thus reduces exploration costs by partially imitating experts as well as maximizing its return in a seamless way, resulting in a higher performance than the suboptimal expert. Our experimental results show that this approach is promising in visual control tasks especially in sparse-reward environments.
Correlation, Linear Complexity, Maximum order Complexity on Families of binary Sequences
Venue: arXiv cs.IT 2021
Year: 2021

Correlation measure of order $k$ is an important measure of randomness in binary sequences. This measure tries to look for dependence between several shifted version of a sequence. We study the relation between the correlation measure of order $k$ and another two pseudorandom measures: the $N$th linear complexity and the $N$th maximum order complexity. We simplify and improve several state-of-the-art lower bounds for these two measures using the Hamming bound as well as weaker bounds derived from it.
Decision-forest voting scheme for classification of rare classes in network intrusion detection
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, Bayesian based aggregation of decision trees in an ensemble (decision forest) is investigated. The focus is laid on multi-class classification with number of samples significantly skewed toward one of the classes. The algorithm leverages out-of-bag datasets to estimate prediction errors of individual trees, which are then used in accordance with the Bayes rule to refine the decision of the ensemble. The algorithm takes prevalence of individual classes into account and does not require setting of any additional parameters related to class weights or decision-score thresholds. Evaluation is based on publicly available datasets as well as on an proprietary dataset comprising network traffic telemetry from hundreds of enterprise networks with over a million of users overall. The aim is to increase the detection capabilities of an operating malware detection system. While we were able to keep precision of the system higher than 94\%, that is only 6 out of 100 detections shown to the network administrator are false alarms, we were able to achieve increase of approximately 7\% in the number of detections. The algorithm effectively handles large amounts of data, and can be used in conjunction with most of the state-of-the-art algorithms used to train decision forests.
A brief note on understanding neural networks as Gaussian processes
Venue: arXiv cs.LG 2021
Year: 2021

As a generalization of the work in [Lee et al., 2017], this note briefly discusses when the prior of a neural network output follows a Gaussian process, and how a neural-network-induced Gaussian process is formulated. The posterior mean functions of such a Gaussian process regression lie in the reproducing kernel Hilbert space defined by the neural-network-induced kernel. In the case of two-layer neural networks, the induced Gaussian processes provide an interpretation of the reproducing kernel Hilbert spaces whose union forms a Barron space.
Boosting Entity-aware Image Captioning with Multi-modal Knowledge Graph
Venue: arXiv cs.CV 2021
Year: 2021

Entity-aware image captioning aims to describe named entities and events related to the image by utilizing the background knowledge in the associated article. This task remains challenging as it is difficult to learn the association between named entities and visual cues due to the long-tail distribution of named entities. Furthermore, the complexity of the article brings difficulty in extracting fine-grained relationships between entities to generate informative event descriptions about the image. To tackle these challenges, we propose a novel approach that constructs a multi-modal knowledge graph to associate the visual objects with named entities and capture the relationship between entities simultaneously with the help of external knowledge collected from the web. Specifically, we build a text sub-graph by extracting named entities and their relationships from the article, and build an image sub-graph by detecting the objects in the image. To connect these two sub-graphs, we propose a cross-modal entity matching module trained using a knowledge base that contains Wikipedia entries and the corresponding images. Finally, the multi-modal knowledge graph is integrated into the captioning model via a graph attention mechanism. Extensive experiments on both GoodNews and NYTimes800k datasets demonstrate the effectiveness of our method.
What Remains of Visual Semantic Embeddings
Venue: arXiv cs.CV 2021
Year: 2021

Zero shot learning (ZSL) has seen a surge in interest over the decade for its tight links with the mechanism making young children recognize novel objects. Although different paradigms of visual semantic embedding models are designed to align visual features and distributed word representations, it is unclear to what extent current ZSL models encode semantic information from distributed word representations. In this work, we introduce the split of tiered-ImageNet to the ZSL task, in order to avoid the structural flaws in the standard ImageNet benchmark. We build a unified framework for ZSL with contrastive learning as pre-training, which guarantees no semantic information leakage and encourages linearly separable visual features. Our work makes it fair for evaluating visual semantic embedding models on a ZSL setting in which semantic inference is decisive. With this framework, we show that current ZSL models struggle with encoding semantic relationships from word analogy and word hierarchy. Our analyses provide motivation for exploring the role of context language representations in ZSL tasks.
Synthetic Periocular Iris PAI from a Small Set of Near-Infrared-Images
Venue: arXiv cs.CV 2021
Year: 2021

Biometric has been increasing in relevance these days since it can be used for several applications such as access control for instance. Unfortunately, with the increased deployment of biometric applications, we observe an increase of attacks. Therefore, algorithms to detect such attacks (Presentation Attack Detection (PAD)) have been increasing in relevance. The LivDet-2020 competition which focuses on Presentation Attacks Detection (PAD) algorithms have shown still open problems, specially for unknown attacks scenarios. In order to improve the robustness of biometric systems, it is crucial to improve PAD methods. This can be achieved by augmenting the number of presentation attack instruments (PAI) and bona fide images that are used to train such algorithms. Unfortunately, the capture and creation of presentation attack instruments and even the capture of bona fide images is sometimes complex to achieve. This paper proposes a novel PAI synthetically created (SPI-PAI) using four state-of-the-art GAN algorithms (cGAN, WGAN, WGAN-GP, and StyleGAN2) and a small set of periocular NIR images. A benchmark between GAN algorithms is performed using the Frechet Inception Distance (FID) between the generated images and the original images used for training. The best PAD algorithm reported by the LivDet-2020 competition was tested for us using the synthetic PAI which was obtained with the StyleGAN2 algorithm. Surprisingly, The PAD algorithm was not able to detect the synthetic images as a Presentation Attack, categorizing all of them as bona fide. Such results demonstrated the feasibility of synthetic images to fool presentation attacks detection algorithms and the need for such algorithms to be constantly updated and trained with a larger number of images and PAI scenarios.
Double-RIS Communication with DF Relaying for Coverage Extension: Is One Relay Enough?
Venue: arXiv cs.IT 2021
Year: 2021

In this work, we investigate the decode-and-forward (DF) relay-aided double reconfigurable intelligent surface (RIS)-assisted networks, where the signal is subject to reflections from two RISs before reaching the destination. Different relay-aided network architectures are considered for maximum achievable rate under a total power constraint. Phase optimization for the double-RIS channels is tackled via the alternating optimization and majorization-minimization (MM) schemes. Moreover, closed-form solutions are obtained for each case. Numerical results indicate that the deployment of two relays, one near each RIS, achieves higher rates at low and medium signal-to noise ratios (SNRs) compared to placing a single relay between the two RISs; while at high SNRs, the latter approach achieves higher rates only if the inter-relay interference for the former case is considerably high.