OpenRaiser/Intern-Atlas huggingface dataset:
VR Hackathon with Goethe Institute: Lessons Learned from Organizing a Transdisciplinary VR Hackathon
Venue: arXiv cs.HC 2021
Year: 2021

In this article we report a case study of a Language Learning Bauhaus VR hackathon with Goethe Institute. It was organized as an educational and research project to tap into the dynamics of transdisciplinary teams challenged with a specific requirement. In our case, it was to build a Bauhaus-themed German Language Learning VR App. We constructed this experiment to simulate how representatives of different disciplines may work together towards a very specific purpose under time pressure. So, each participating team consisted of members of various expert-fields: software development (Unity or Unreal), design, psychology and linguistics. The results of this study cast light on the recommended cycle of design thinking and customer-centered design in VR. Especially in interdisciplinary rapid prototyping conditions, where stakeholders initially do not share competences. They also showcase educational benefits of working in transdisciplinary environments. This study, combined with our previous work on human factors in rapid software development and co-design, including hackathon dynamics, allowed us to formulate recommendations for organizing content creation VR hackathons for specific purposes. We also provide guidelines on how to prepare the participants to work in rapid prototyping VR environments and benefit from such experiences in the long term.
Automatic Micro-Expression Apex Frame Spotting using Local Binary Pattern from Six Intersection Planes
Venue: arXiv cs.CV 2021
Year: 2021

Facial expressions are one of the most effective ways for non-verbal communications, which can be expressed as the Micro-Expression (ME) in the high-stake situations. The MEs are involuntary, rapid, and, subtle, and they can reveal real human intentions. However, their feature extraction is very challenging due to their low intensity and very short duration. Although Local Binary Pattern from Three Orthogonal Plane (LBP-TOP) feature extractor is useful for the ME analysis, it does not consider essential information. To address this problem, we propose a new feature extractor called Local Binary Pattern from Six Intersection Planes (LBP-SIPl). This method extracts LBP code on six intersection planes, and then it combines them. Results show that the proposed method has superior performance in apex frame spotting automatically in comparison with the relevant methods on the CASME database. Simulation results show that, using the proposed method, the apex frame has been spotted in 43% of subjects in the CASME database, automatically. Also, the mean absolute error of 1.76 is achieved, using our novel proposed method.
Adaptive Clustering of Robust Semantic Representations for Adversarial Image Purification
Venue: arXiv cs.CV 2021
Year: 2021

Deep Learning models are highly susceptible to adversarial manipulations that can lead to catastrophic consequences. One of the most effective methods to defend against such disturbances is adversarial training but at the cost of generalization of unseen attacks and transferability across models. In this paper, we propose a robust defense against adversarial attacks, which is model agnostic and generalizable to unseen adversaries. Initially, with a baseline model, we extract the latent representations for each class and adaptively cluster the latent representations that share a semantic similarity. We obtain the distributions for the clustered latent representations and from their originating images, we learn semantic reconstruction dictionaries (SRD). We adversarially train a new model constraining the latent space representation to minimize the distance between the adversarial latent representation and the true cluster distribution. To purify the image, we decompose the input into low and high-frequency components. The high-frequency component is reconstructed based on the most adequate SRD from the clean dataset. In order to evaluate the most adequate SRD, we rely on the distance between robust latent representations and semantic cluster distributions. The output is a purified image with no perturbation. Image purification on CIFAR-10 and ImageNet-10 using our proposed method improved the accuracy by more than 10% compared to state-of-the-art results.
Near-Precise Parameter Approximation for Multiple Multiplications on A Single DSP Block
Venue: arXiv cs.AR 2021
Year: 2021

A multiply-accumulate (MAC) operation is the main computation unit for DSP applications. DSP blocks are one of the efficient solutions to implement MACs in FPGA's. However, since the DSP blocks have wide multiplier and adder blocks, MAC operations using low bit-length parameters lead to an underutilization problem. Hence, an efficient approximation technique is introduced. The technique includes manipulation and approximation of the low bit-length fixed-point parameters based upon a Single DSP - Multiple Multiplication (SDMM) execution. The SDMM changes the traditional MAC implementation in the DSP block by separating multiplication and accumulation operations. While the accumulator hardware available in the DSP block is used for multiple parameter multiplication, parallel LUTs are employed for the accumulation part of the MAC operation. The accuracy of the developed optimization technique was evaluated for different CNN weight bit precisions using the Alexnet and VGG-16 networks and the Tiny ImageNet dataset. The optimization can be implemented without loss of accuracy in almost all cases, while it causes slight accuracy losses in a few cases. Through these optimizations, the SDMM is performed at the cost of a small hardware overhead. For example, a single DSP block executes 3 8-bit fixed-point parameter multiplications. As a result of our optimizations, the parameters are represented in a different format on off-chip memory, providing up to 33% compression without any hardware cost. The compression rate can be further increased by up to 97% when used in conjunction with other compression methods for the VGG-16. Reaching this compression rate requires extra hardware cost. A prototype systolic array architecture was implemented employing our optimizations on a Xilinx Zynq FPGA. It reduced the number of DSP blocks by 66.6%, 75%, and 83.3% for 8, 6, and 4-bit input variables, respectively.
Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios
Venue: arXiv cs.SD 2021
Year: 2021

Often, the storage and computational constraints of embeddeddevices demand that a single on-device ASR model serve multiple use-cases / domains. In this paper, we propose aFlexibleTransducer(FlexiT) for on-device automatic speech recognition to flexibly deal with multiple use-cases / domains with different accuracy and latency requirements. Specifically, using a single compact model, FlexiT provides a fast response for voice commands, and accurate transcription but with more latency for dictation. In order to achieve flexible and better accuracy and latency trade-offs, the following techniques are used. Firstly, we propose using domain-specific altering of segment size for Emformer encoder that enables FlexiT to achieve flexible de-coding. Secondly, we use Alignment Restricted RNNT loss to achieve flexible fine-grained control on token emission latency for different domains. Finally, we add a domain indicator vector as an additional input to the FlexiT model. Using the combination of techniques, we show that a single model can be used to improve WERs and real time factor for dictation scenarios while maintaining optimal latency for voice commands use-cases
Multi-Scale Context Aggregation Network with Attention-Guided for Crowd Counting
Venue: arXiv cs.CV 2021
Year: 2021

Crowd counting aims to predict the number of people and generate the density map in the image. There are many challenges, including varying head scales, the diversity of crowd distribution across images and cluttered backgrounds. In this paper, we propose a multi-scale context aggregation network (MSCANet) based on single-column encoder-decoder architecture for crowd counting, which consists of an encoder based on a dense context-aware module (DCAM) and a hierarchical attention-guided decoder. To handle the issue of scale variation, we construct the DCAM to aggregate multi-scale contextual information by densely connecting the dilated convolution with varying receptive fields. The proposed DCAM can capture rich contextual information of crowd areas due to its long-range receptive fields and dense scale sampling. Moreover, to suppress the background noise and generate a high-quality density map, we adopt a hierarchical attention-guided mechanism in the decoder. This helps to integrate more useful spatial information from shallow feature maps of the encoder by introducing multiple supervision based on semantic attention module (SAM). Extensive experiments demonstrate that the proposed approach achieves better performance than other similar state-of-the-art methods on three challenging benchmark datasets for crowd counting. The code is available at https://github.com/KingMV/MSCANet
Two modifications for Loidreau's code-based cryptosystem
Venue: arXiv cs.IT 2021
Year: 2021

This paper presents two modifications for Loidreau's code-based cryptosystem. Loidreau's cryptosystem is a rank metric code-based cryptosystem constructed by using Gabidulin codes in the McEliece setting. Recently a polynomial-time key recovery attack was proposed to break Loidreau's cryptosystem in some cases. To prevent this attack, we propose the use of subcodes to disguise the secret codes in Modification \Rmnum{1}. In Modification \Rmnum{2}, we choose a random matrix of low column rank over $\mathbb{F}_q$ to mix with the secret matrix. According to our analysis, these two modifications can both resist the existing structural attacks. Additionally, we adopt the systematic generator matrix of the public code to make a reduction in the public-key size. In additon to stronger resistance against structural attacks and more compact representation of public keys, our modifications also have larger information transmission rates.
Enabling Inference Privacy with Adaptive Noise Injection
Venue: arXiv cs.CR 2021
Year: 2021

User-facing software services are becoming increasingly reliant on remote servers to host Deep Neural Network (DNN) models, which perform inference tasks for the clients. Such services require the client to send input data to the service provider, who processes it using a DNN and returns the output predictions to the client. Due to the rich nature of the inputs such as images and speech, the input often contains more information than what is necessary to perform the primary inference task. Consequently, in addition to the primary inference task, a malicious service provider could infer secondary (sensitive) attributes from the input, compromising the client's privacy. The goal of our work is to improve inference privacy by injecting noise to the input to hide the irrelevant features that are not conducive to the primary classification task. To this end, we propose Adaptive Noise Injection (ANI), which uses a light-weight DNN on the client-side to inject noise to each input, before transmitting it to the service provider to perform inference. Our key insight is that by customizing the noise to each input, we can achieve state-of-the-art trade-off between utility and privacy (up to 48.5% degradation in sensitive-task accuracy with <1% degradation in primary accuracy), significantly outperforming existing noise injection schemes. Our method does not require prior knowledge of the sensitive attributes and incurs minimal computational overheads.
Efficient and Privacy-Preserving Infection Control System for Covid-19-Like Pandemics using Blockchain
Venue: arXiv cs.CR 2021
Year: 2021

Contact tracing is a very effective way to control the COVID-19-like pandemics. It aims to identify individuals who closely contacted an infected person during the incubation period of the virus and notify them to quarantine. However, the existing systems suffer from privacy, security, and efficiency issues. To address these limitations, in this paper, we propose an efficient and privacy-preserving Blockchain-based infection control system. Instead of depending on a single authority to run the system, a group of health authorities, that form a consortium Blockchain, run our system. Using Blockchain technology not only secures our system against single point of failure and denial of service attacks, but also brings transparency because all transactions can be validated by different parties. Although contact tracing is important, it is not enough to effectively control an infection. Thus, unlike most of the existing systems that focus only on contact tracing, our system consists of three integrated subsystems, including contact tracing, public places access control, and safe-places recommendation. The access control subsystem prevents infected people from visiting public places to prevent spreading the virus, and the recommendation subsystem categorizes zones based on the infection level so that people can avoid visiting contaminated zones. Our analysis demonstrates that our system is secure and preserves the privacy of the users against identification, social graph disclosure, and tracking attacks, while thwarting false reporting (or panic) attacks. Moreover, our extensive performance evaluations demonstrate the scalability of our system (which is desirable in pandemics) due to its low communication, computation, and storage overheads.
Learnable Expansion-and-Compression Network for Few-shot Class-Incremental Learning
Venue: arXiv cs.CV 2021
Year: 2021

Few-shot class-incremental learning (FSCIL), which targets at continuously expanding model's representation capacity under few supervisions, is an important yet challenging problem. On the one hand, when fitting new tasks (novel classes), features trained on old tasks (old classes) could significantly drift, causing catastrophic forgetting. On the other hand, training the large amount of model parameters with few-shot novel-class examples leads to model over-fitting. In this paper, we propose a learnable expansion-and-compression network (LEC-Net), with the aim to simultaneously solve catastrophic forgetting and model over-fitting problems in a unified framework. By tentatively expanding network nodes, LEC-Net enlarges the representation capacity of features, alleviating feature drift of old network from the perspective of model regularization. By compressing the expanded network nodes, LEC-Net purses minimal increase of model parameters, alleviating over-fitting of the expanded network from a perspective of compact representation. Experiments on the CUB/CIFAR-100 datasets show that LEC-Net improves the baseline by 5~7% while outperforms the state-of-the-art by 5~6%. LEC-Net also demonstrates the potential to be a general incremental learning approach with dynamic model expansion capability.
Admissible heuristics for obstacle clearance optimization objectives
Venue: arXiv cs.RO 2021
Year: 2021

Obstacle clearance in state space is an important optimization objective in path planning because it can result in safe paths. This technical report presents admissible solution- and path-cost heuristics for this objective, which can be used to improve the performance of informed path planning algorithms.
Exploration of Hardware Acceleration Methods for an XNOR Traffic Signs Classifier
Venue: arXiv cs.CV 2021
Year: 2021

Deep learning algorithms are a key component of many state-of-the-art vision systems, especially as Convolutional Neural Networks (CNN) outperform most solutions in the sense of accuracy. To apply such algorithms in real-time applications, one has to address the challenges of memory and computational complexity. To deal with the first issue, we use networks with reduced precision, specifically a binary neural network (also known as XNOR). To satisfy the computational requirements, we propose to use highly parallel and low-power FPGA devices. In this work, we explore the possibility of accelerating XNOR networks for traffic sign classification. The trained binary networks are implemented on the ZCU 104 development board, equipped with a Zynq UltraScale+ MPSoC device using two different approaches. Firstly, we propose a custom HDL accelerator for XNOR networks, which enables the inference with almost 450 fps. Even better results are obtained with the second method - the Xilinx FINN accelerator - enabling to process input images with around 550 frame rate. Both approaches provide over 96% accuracy on the test set.
General Robot Dynamics Learning and Gen2Real
Venue: arXiv cs.RO 2021
Year: 2021

Acquiring dynamics is an essential topic in robot learning, but up-to-date methods, such as dynamics randomization, need to restart to check nominal parameters, generate simulation data, and train networks whenever they face different robots. To improve it, we novelly investigate general robot dynamics, its inverse models, and Gen2Real, which means transferring to reality. Our motivations are to build a model that learns the intrinsic dynamics of various robots and lower the threshold of dynamics learning by enabling an amateur to obtain robot models without being trapped in details. This paper achieves the "generality" by randomizing dynamics parameters, topology configurations, and model dimensions, which in sequence cover the property, the connection, and the number of robot links. A structure modified from GPT is applied to access the pre-training model of general dynamics. We also study various inverse models of dynamics to facilitate different applications. We step further to investigate a new concept, "Gen2Real", to transfer simulated, general models to physical, specific robots. Simulation and experiment results demonstrate the validity of the proposed models and method.\footnote{ These authors contribute equally.
mSHINE: A Multiple-meta-paths Simultaneous Learning Framework for Heterogeneous Information Network Embedding
Venue: arXiv cs.LG 2021
Year: 2021

Heterogeneous information networks(HINs) become popular in recent years for its strong capability of modelling objects with abundant information using explicit network structure. Network embedding has been proved as an effective method to convert information networks into lower-dimensional space, whereas the core information can be well preserved. However, traditional network embedding algorithms are sub-optimal in capturing rich while potentially incompatible semantics provided by HINs. To address this issue, a novel meta-path-based HIN representation learning framework named mSHINE is designed to simultaneously learn multiple node representations for different meta-paths. More specifically, one representation learning module inspired by the RNN structure is developed and multiple node representations can be learned simultaneously, where each representation is associated with one respective meta-path. By measuring the relevance between nodes with the designed objective function, the learned module can be applied in downstream link prediction tasks. A set of criteria for selecting initial meta-paths is proposed as the other module in mSHINE which is important to reduce the optimal meta-path selection cost when no prior knowledge of suitable meta-paths is available. To corroborate the effectiveness of mSHINE, extensive experimental studies including node classification and link prediction are conducted on five real-world datasets. The results demonstrate that mSHINE outperforms other state-of-the-art HIN embedding methods.
Computing the Characteristic Polynomial of Generic Toeplitz-like and Hankel-like Matrices
Venue: arXiv cs.SC 2021
Year: 2021

New algorithms are presented for computing annihilating polynomials of Toeplitz, Hankel, and more generally Toeplitz+ Hankel-like matrices over a field. Our approach follows works on Coppersmith's block Wiedemann method with structured projections, which have been recently successfully applied for computing the bivariate resultant. A first baby-step/giant step approach -- directly derived using known techniques on structured matrices -- gives a randomized Monte Carlo algorithm for the minimal polynomial of an $n\times n$ Toeplitz or Hankel-like matrix of displacement rank $α$ using $\tilde O(n^{ω- c(ω)} α^{c(ω)})$ arithmetic operations, where $ω$ is the exponent of matrix multiplication and $c(2.373)\approx 0.523$ for the best known value of $ω$. For generic Toeplitz+Hankel-like matrices a second algorithm computes the characteristic polynomial in $\tilde O(n^{2-1/ω})$ operations when the displacement rank is considered constant. Previous algorithms required $O(n^2)$ operations while the exponents presented here are respectively less than $1.86$ and $1.58$ with the best known estimate for $ω$.
Logging Practices with Mobile Analytics: An Empirical Study on Firebase
Venue: arXiv cs.SE 2021
Year: 2021

Software logs are of great value in both industrial and open-source projects. Mobile analytics logging enables developers to collect logs remotely from their apps running on end user devices at the cost of recording and transmitting logs across the Internet to a centralised infrastructure. This paper makes a first step in characterising logging practices with a widely adopted mobile analytics logging library, namely Firebase Analytics. We provide an empirical evaluation of the use of Firebase Analytics in 57 open-source Android applications by studying the evolution of code-bases to understand: a) the needs-in-common that push practitioners to adopt logging practices on mobile devices, and b) the differences in the ways developers use local and remote logging. Our results indicate mobile analytics logs are less pervasive and less maintained than traditional logging code. Based on our analysis, we believe logging using mobile analytics is more user centered compared to traditional logging, where the latter is mainly used to record information for debugging purposes.
AI4D -- African Language Program
Venue: arXiv cs.CL 2021
Year: 2021

Advances in speech and language technologies enable tools such as voice-search, text-to-speech, speech recognition and machine translation. These are however only available for high resource languages like English, French or Chinese. Without foundational digital resources for African languages, which are considered low-resource in the digital context, these advanced tools remain out of reach. This work details the AI4D - African Language Program, a 3-part project that 1) incentivised the crowd-sourcing, collection and curation of language datasets through an online quantitative and qualitative challenge, 2) supported research fellows for a period of 3-4 months to create datasets annotated for NLP tasks, and 3) hosted competitive Machine Learning challenges on the basis of these datasets. Key outcomes of the work so far include 1) the creation of 9+ open source, African language datasets annotated for a variety of ML tasks, and 2) the creation of baseline models for these datasets through hosting of competitive ML challenges.