OpenRaiser/Intern-Atlas huggingface dataset:
Vaccine Credential Technology Principles
Venue: arXiv cs.CY 2021
Year: 2021

The historically rapid development of effective COVID-19 vaccines has policymakers facing evergreen public health questions regarding vaccination records and verification. Governments and institutions around the world are already taking action on digital vaccine certificates, including guidance and recommendations from the European Commission, the WHO, and the Biden Administration. These could be encouraging efforts: an effective system for vaccine certificates could potentially be part of a safe return to work, travel, and daily life, and a secure technological implementation could improve on existing systems to prioritize privacy, streamline access, and build for necessary interoperability across countries and contexts. However, vaccine credentials are not without potential harms, and, particularly given major inequities in vaccine access and rollout, there are valid concerns that they may be used in ineffective or exclusionary ways that exacerbate inequality, allow for discrimination, violate privacy, and assume consent. While the present moment calls for urgency, we must also acknowledge that choices made in the vaccine credentialing rollout for COVID-19 are likely to have long-term implications, and must be made with care. In this paper, we outline potential implementation and ethical concerns that may arise from tech-enabled vaccine credentialing programs now and in the future, and discuss the technological tradeoffs implicated in these concerns. We suggest a set of principles that, if adopted, may mitigate these concerns, forestall preventable harms, and point the way forward; the paper is structured as a deep dive into each of these principles.
Inertial Sensor Data To Image Encoding For Human Action Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Convolutional Neural Networks (CNNs) are successful deep learning models in the field of computer vision. To get the maximum advantage of CNN model for Human Action Recognition (HAR) using inertial sensor data, in this paper, we use 4 types of spatial domain methods for transforming inertial sensor data to activity images, which are then utilized in a novel fusion framework. These four types of activity images are Signal Images (SI), Gramian Angular Field (GAF) Images, Markov Transition Field (MTF) Images and Recurrence Plot (RP) Images. Furthermore, for creating a multimodal fusion framework and to exploit activity image, we made each type of activity images multimodal by convolving with two spatial domain filters : Prewitt filter and High-boost filter. Resnet-18, a CNN model, is used to learn deep features from multi-modalities. Learned features are extracted from the last pooling layer of each ReNet and then fused by canonical correlation based fusion (CCF) for improving the accuracy of human action recognition. These highly informative features are served as input to a multiclass Support Vector Machine (SVM). Experimental results on three publicly available inertial datasets show the superiority of the proposed method over the current state-of-the-art.
Blending Advertising with Organic Content in E-Commerce: A Virtual Bids Optimization Approach
Venue: arXiv cs.LG 2021
Year: 2021

In e-commerce platforms, sponsored and non-sponsored content are jointly displayed to users and both may interactively influence their engagement behavior. The former content helps advertisers achieve their marketing goals and provides a stream of ad revenue to the platform. The latter content contributes to users' engagement with the platform, which is key to its long-term health. A burning issue for e-commerce platform design is how to blend advertising with content in a way that respects these interactions and balances these multiple business objectives. This paper describes a system developed for this purpose in the context of blending personalized sponsored content with non-sponsored content on the product detail pages of JD.COM, an e-commerce company. This system has three key features: (1) Optimization of multiple competing business objectives through a new virtual bids approach and the expressiveness of the latent, implicit valuation of the platform for the multiple objectives via these virtual bids. (2) Modeling of users' click behavior as a function of their characteristics, the individual characteristics of each sponsored content and the influence exerted by other sponsored and non-sponsored content displayed alongside through a deep learning approach; (3) Consideration of externalities in the allocation of ads, thereby making it directly compatible with a Vickrey-Clarke-Groves (VCG) auction scheme for the computation of payments in the presence of these externalities. The system is currently deployed and serving all traffic through JD.COM's mobile application. Experiments demonstrating the performance and advantages of the system are presented.
One-shot Learning with Absolute Generalization
Venue: arXiv cs.LG 2021
Year: 2021

One-shot learning is proposed to make a pretrained classifier workable on a new dataset based on one labeled samples from each pattern. However, few of researchers consider whether the dataset itself supports one-shot learning. In this paper, we propose a set of definitions to explain what kind of datasets can support one-shot learning and propose the concept "absolute generalization". Based on these definitions, we proposed a method to build an absolutely generalizable classifier. The proposed method concatenates two samples as a new single sample, and converts a classification problem to an identity identification problem or a similarity metric problem. Experiments demonstrate that the proposed method is superior to baseline on one-shot learning datasets and artificial datasets.
Investigating Code-Mixed Modern Standard Arabic-Egyptian to English Machine Translation
Venue: arXiv cs.LG 2021
Year: 2021

Recent progress in neural machine translation (NMT) has made it possible to translate successfully between monolingual language pairs where large parallel data exist, with pre-trained models improving performance even further. Although there exists work on translating in code-mixed settings (where one of the pairs includes text from two or more languages), it is still unclear what recent success in NMT and language modeling exactly means for translating code-mixed text. We investigate one such context, namely MT from code-mixed Modern Standard Arabic and Egyptian Arabic (MSAEA) into English. We develop models under different conditions, employing both (i) standard end-to-end sequence-to-sequence (S2S) Transformers trained from scratch and (ii) pre-trained S2S language models (LMs). We are able to acquire reasonable performance using only MSA-EN parallel data with S2S models trained from scratch. We also find LMs fine-tuned on data from various Arabic dialects to help the MSAEA-EN task. Our work is in the context of the Shared Task on Machine Translation in Code-Switching. Our best model achieves $\bf25.72$ BLEU, placing us first on the official shared task evaluation for MSAEA-EN.
Parallel Programming Applied Research Projects for Teaching Parallel Programming to Beginner Students
Venue: arXiv cs.DC 2021
Year: 2021

In this paper, we discuss the educational value of a few mid-size and one large applied research projects at the Computer Science Department of Okanagan College (OC) and at the Universities of Paris East Creteil (LACL) and Orleans (LIFO) in France. We found, that some freshmen students are very active and eager to be involved in applied research projects starting from the second semester. They are actively participating in programming competitions and want to be involved in applied research projects to compete with sophomore and older students. Our observation is based on five NSERC Engage College and Applied Research and Development (ARD) grants, and several small applied projects. Student involvement in applied research is a key motivation and success factor in our activities, but we are also involved in transferring some results of applied research, namely programming techniques, into the parallel programming courses for beginners at the senior- and first-year MSc levels. We illustrate this feedback process with programming notions for beginners, practical tools to acquire them and the overall success/failure of students as experienced for more than 10 years in our French University courses.
Placement Optimization and Power Control in Intelligent Reflecting Surface Aided Multiuser System
Venue: arXiv cs.IT 2021
Year: 2021

Intelligent reflecting surface (IRS) is a new and revolutionary technology capable of reconfiguring the wireless propagation environment by controlling its massive low-cost passive reflecting elements. Different from prior works that focus on optimizing IRS reflection coefficients or single-IRS placement, we aim to maximize the minimum throughput of a single-cell multiuser system aided by multiple IRSs, by joint multi-IRS placement and power control at the access point (AP), which is a mixed-integer non-convex problem with drastically increased complexity with the number of IRSs/users. To tackle this challenge, a ring-based IRS placement scheme is proposed along with a power control policy that equalizes the users' non-outage probability. An efficient searching algorithm is further proposed to obtain a close-to-optimal solution for arbitrary number of IRSs/rings. Numerical results validate our analysis and show that our proposed scheme significantly outperforms the benchmark schemes without IRS and/or with other power control policies. Moreover, it is shown that the IRSs are preferably deployed near AP for coverage range extension, while with more IRSs, they tend to spread out over the cell to cover more and get closer to target users.
Alleviating the Knowledge-Language Inconsistency: A Study for Deep Commonsense Knowledge
Venue: arXiv cs.CL 2021
Year: 2021

Knowledge facts are typically represented by relational triples, while we observe that some commonsense facts are represented by the triples whose forms are inconsistent with the expression of language. This inconsistency puts forward a challenge for pre-trained language models to deal with these commonsense knowledge facts. In this paper, we term such knowledge as deep commonsense knowledge and conduct extensive exploratory experiments on it. We show that deep commonsense knowledge occupies a significant part of commonsense knowledge while conventional methods fail to capture it effectively. We further propose a novel method to mine the deep commonsense knowledge distributed in sentences, alleviating the reliance of conventional methods on the triple representation form of knowledge. Experiments demonstrate that the proposal significantly improves the performance in mining deep commonsense knowledge.
FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learning
Venue: arXiv cs.CV 2021
Year: 2021

As GAN-based video and image manipulation technologies become more sophisticated and easily accessible, there is an urgent need for effective deepfake detection technologies. Moreover, various deepfake generation techniques have emerged over the past few years. While many deepfake detection methods have been proposed, their performance suffers from new types of deepfake methods on which they are not sufficiently trained. To detect new types of deepfakes, the model should learn from additional data without losing its prior knowledge about deepfakes (catastrophic forgetting), especially when new deepfakes are significantly different. In this work, we employ the Representation Learning (ReL) and Knowledge Distillation (KD) paradigms to introduce a transfer learning-based Feature Representation Transfer Adaptation Learning (FReTAL) method. We use FReTAL to perform domain adaptation tasks on new deepfake datasets while minimizing catastrophic forgetting. Our student model can quickly adapt to new types of deepfake by distilling knowledge from a pre-trained teacher model and applying transfer learning without using source domain data during domain adaptation. Through experiments on FaceForensics++ datasets, we demonstrate that FReTAL outperforms all baselines on the domain adaptation task with up to 86.97% accuracy on low-quality deepfakes.
SPFA: SFA on Multiple Persistent Faults
Venue: arXiv cs.CR 2021
Year: 2021

For classical fault analysis, a transient fault is required to be injected during runtime, e.g., only at a specific round. Instead, Persistent Fault Analysis (PFA) introduces a powerful class of fault attacks that allows for a fault to be present throughout the whole execution. One limitation of original PFA as introduced by Zhang et al. at CHES'18 is that the faulty values need to be known to the adversary. While this was addressed at a follow-up work at CHES'20, the solution is only applicable to a single faulty value. Instead, we use the potency of Statistical Fault Analysis (SFA) in the persistent fault setting, presenting Statistical Persistent Fault Analysis (SPFA) as a more general approach of PFA. As a result, any or even a multitude of unknown faults that cause an exploitable bias in the targeted round can be used to recover the cipher's secret key. Indeed, the undesired faults in the other rounds that occur due the persistent nature of the attack converge to a uniform distribution as required by SFA. We verify the effectiveness of our attack against LED and AES.
The Unpatchable Silicon: A Full Break of the Bitstream Encryption of Xilinx 7-Series FPGAs
Venue: arXiv cs.CR 2021
Year: 2021

The security of FPGAs is a crucial topic, as any vulnerability within the hardware can have severe consequences, if they are used in a secure design. Since FPGA designs are encoded in a bitstream, securing the bitstream is of the utmost importance. Adversaries have many motivations to recover and manipulate the bitstream, including design cloning, IP theft, manipulation of the design, or design subversions e.g., through hardware Trojans. Given that FPGAs are often part of cyber-physical systems e.g., in aviation, medical, or industrial devices, this can even lead to physical harm. Consequently, vendors have introduced bitstream encryption, offering authenticity and confidentiality. Even though attacks against bitstream encryption have been proposed in the past, e.g., side-channel analysis and probing, these attacks require sophisticated equipment and considerable technical expertise. In this paper, we introduce novel low-cost attacks against the Xilinx 7-Series (and Virtex-6) bitstream encryption, resulting in the total loss of authenticity and confidentiality. We exploit a design flaw which piecewise leaks the decrypted bitstream. In the attack, the FPGA is used as a decryption oracle, while only access to a configuration interface is needed. The attack does not require any sophisticated tools and, depending on the target system, can potentially be launched remotely. In addition to the attacks, we discuss several countermeasures.
ARMORY: Fully Automated and Exhaustive Fault Simulation on ARM-M Binaries
Venue: arXiv cs.CR 2021
Year: 2021

Embedded systems are ubiquitous. However, physical access of users and likewise attackers makes them often threatened by fault attacks: a single fault during the computation of a cryptographic primitive can lead to a total loss of system security. This can have serious consequences, e.g., in safetycritical systems, including bodily harm and catastrophic technical failures. However, countermeasures often focus on isolated fault models and high layers of abstraction. This leads to a dangerous sense of security, because exploitable faults that are only visible at machine code level might not be covered by countermeasures. In this work we present ARMORY, a fully automated open source framework for exhaustive fault simulation on binaries of the ubiquitous ARM-M class. It allows engineers and analysts to efficiently scan a binary for potential weaknesses against arbitrary combinations of multi-variate fault injections under a large variety of fault models. Using ARMORY, we demonstrate the power of fully automated fault analysis and the dangerous implications of applying countermeasures without knowledge of physical addresses and offsets. We exemplarily analyze two case studies, which are highly relevant for practice: a DFA on AES (cryptographic) and a secure bootloader (non-cryptographic). Our results show that indeed numerous exploitable faults found by ARMORY which occur in the actual implementations are easily missed in manual inspection. Crucially, most faults are only visible when taking machine code information, i.e., addresses and offsets, into account. Surprisingly, we show that a countermeasure that protects against one type of fault can actually largely increase the vulnerability to other fault models. Our work demonstrates the need for countermeasures that, at least in their evaluation, are not restricted to isolated fault models and consider low-level information [...].
Early Exiting with Ensemble Internal Classifiers
Venue: arXiv cs.CL 2021
Year: 2021

As a simple technique to accelerate inference of large-scale pre-trained models, early exiting has gained much attention in the NLP community. It allows samples to exit early at internal classifiers without passing through the entire model. Most existing work usually trains the internal classifiers independently and employs an exiting strategy to decide whether or not to exit based on the confidence of the current internal classifier. However, none of these works takes full advantage of the fact that the internal classifiers are trained to solve the same task therefore can be used to construct an ensemble. In this paper, we show that a novel objective function for the training of the ensemble internal classifiers can be naturally induced from the perspective of ensemble learning and information theory. The proposed training objective consists of two terms: one for accuracy and the other for the diversity of the internal classifiers. In contrast, the objective used in prior work is exactly the accuracy term of our training objective therefore only optimizes the accuracy but not diversity. Further, we propose a simple voting-based strategy that considers predictions of all the past internal classifiers to infer the correct label and decide whether to exit. Experimental results on various NLP tasks show that our proposed objective function and voting-based strategy can achieve better accuracy-speed trade-offs.
The Wits Intelligent Teaching System: Detecting Student Engagement During Lectures Using Convolutional Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

To perform contingent teaching and be responsive to students' needs during class, lecturers must be able to quickly assess the state of their audience. While effective teachers are able to gauge easily the affective state of the students, as class sizes grow this becomes increasingly difficult and less precise. The Wits Intelligent Teaching System (WITS) aims to assist lecturers with real-time feedback regarding student affect. The focus is primarily on recognising engagement or lack thereof. Student engagement is labelled based on behaviour and postures that are common to classroom settings. These proxies are then used in an observational checklist to construct a dataset of engagement upon which a CNN based on AlexNet is successfully trained and which significantly outperforms a Support Vector Machine approach. The deep learning approach provides satisfactory results on a challenging, real-world dataset with significant occlusion, lighting and resolution constraints.
A proxemics game between festival visitors and an industrial robot
Venue: arXiv cs.RO 2021
Year: 2021

With increased applications of collaborative robots (cobots) in industrial workplaces, behavioural effects of human-cobot interactions need to be further investigated. This is of particular importance as nonverbal behaviours of collaboration partners in human-robot teams significantly influence the experience of the human interaction partners and the success of the collaborative task. During the Ars Electronica 2020 Festival for Art, Technology and Society (Linz, Austria), we invited visitors to exploratively interact with an industrial robot, exhibiting restricted interaction capabilities: extending and retracting its arm, depending on the movements of the volunteer. The movements of the arm were pre-programmed and telecontrolled for safety reasons (which was not obvious to the participants). We recorded video data of these interactions and investigated general nonverbal behaviours of the humans interacting with the robot, as well as nonverbal behaviours of people in the audience. Our results showed that people were more interested in exploring the robot's action and perception capabilities than just reproducing the interaction game as introduced by the instructors. We also found that the majority of participants interacting with the robot approached it up to a distance which would be perceived as threatening or intimidating, if it were a human interaction partner. Regarding bystanders, we found examples where people made movements as if trying out variants of the current participant's behaviour.
Improving Facial Attribute Recognition by Group and Graph Learning
Venue: arXiv cs.CV 2021
Year: 2021

Exploiting the relationships between attributes is a key challenge for improving multiple facial attribute recognition. In this work, we are concerned with two types of correlations that are spatial and non-spatial relationships. For the spatial correlation, we aggregate attributes with spatial similarity into a part-based group and then introduce a Group Attention Learning to generate the group attention and the part-based group feature. On the other hand, to discover the non-spatial relationship, we model a group-based Graph Correlation Learning to explore affinities of predefined part-based groups. We utilize such affinity information to control the communication between all groups and then refine the learned group features. Overall, we propose a unified network called Multi-scale Group and Graph Network. It incorporates these two newly proposed learning strategies and produces coarse-to-fine graph-based group features for improving facial attribute recognition. Comprehensive experiments demonstrate that our approach outperforms the state-of-the-art methods.
4-Adic Complexity of Interleaved Quaternary Sequences
Venue: arXiv cs.IT 2021
Year: 2021

Tang and Ding \cite{X. Tang} present a series of quaternary sequences $w(a, b)$ interleaved by two binary sequences $a$ and $b$ with ideal autocorrelation and show that such interleaved quaternary sequences have optimal autocorrelation. In this paper we consider the 4-adic complexity $FC_{w}(4)$ of such quaternary sequence $w=w(a, b)$. We present a general formula on $FC_{w}(4)$, $w=w(a, b)$. As a direct consequence, we obtain a general lower bound $FC_{w}(4)\geq\log_{4}(4^{n}-1)$ where $2n$ is the period of the sequence $w$. By taking $a$ and $b$ to be several types of known binary sequences with ideal autocorrelation ($m$-sequences, twin-prime, Legendre, Hall sequences and their complement, shift or sample sequences), we compute the exact values of $FC_{w}(4)$, $w=w(a, b)$ and show that in most cases $FC_{w}(4)$ reaches or nearly reaches the maximum value $\log_{4}(4^{2n}-1)$. Our results show that the 4-adic complexity of the quaternary sequences defined in \cite{X. Tang} are large enough to resist the attack of the rational approximation algorithm.