OpenRaiser/Intern-Atlas huggingface dataset:
Vogtareuth Rehab Depth Datasets: Benchmark for Marker-less Posture Estimation in Rehabilitation
Venue: arXiv cs.CV 2021
Year: 2021

Posture estimation using a single depth camera has become a useful tool for analyzing movements in rehabilitation. Recent advances in posture estimation in computer vision research have been possible due to the availability of large-scale pose datasets. However, the complex postures involved in rehabilitation exercises are not represented in the existing benchmark depth datasets. To address this limitation, we propose two rehabilitation-specific pose datasets containing depth images and 2D pose information of patients, both adult and children, performing rehab exercises. We use a state-of-the-art marker-less posture estimation model which is trained on a non-rehab benchmark dataset. We evaluate it on our rehab datasets, and observe that the performance degrades significantly from non-rehab to rehab, highlighting the need for these datasets. We show that our dataset can be used to train pose models to detect rehab-specific complex postures. The datasets will be released for the benefit of the research community.
A Generalization of the ASR Search Algorithm to 2-Generator Quasi-Twisted Codes
Venue: arXiv cs.IT 2021
Year: 2021

One of the main goals of coding theory is to construct codes with best possible parameters and properties. A special class of codes called quasi-twisted (QT) codes is well-known to produce codes with good parameters. Most of the work on QT codes has been over the 1-generator case. In this work, we focus on 2-generator QT codes and generalize the ASR algorithm that has been very effective to produce new linear codes from 1-generator QT codes. Moreover, we also generalize a recent algorithm to test equivalence of cyclic codes to constacyclic codes. This algorithm makes the ASR search even more effective. As a result of implementing our algorithm, we have found 103 QT codes that are new among the class of QT codes. Additionally, most of these codes possess the following additional properties: a) they have the same parameters as best known linear codes, and b) many of the have additional desired properties such as being LCD and dual-containing. Further, we have also found a binary 2-generator QT code that is new (record breaking) among all binary linear codes and its extension yields another record breaking binary linear code.
Enhancing Security in VANETs with Efficient Sybil Attack Detection using Fog Computing
Venue: arXiv cs.CR 2021
Year: 2021

Vehicular ad hoc networks (VANETs) facilitate vehicles to broadcast beacon messages to ensure road safety. Rogue nodes in VANETs cause a Sybil attack to create an illusion of fake traffic congestion by broadcasting malicious information leading to catastrophic consequences, such as the collision of vehicles. Previous researchers used either cryptography, trust scores, or past vehicle data to detect rogue nodes, but they suffer from high processing delay, overhead, and false-positive rate (FPR). We propose a fog computing-based Sybil attack detection for VANETs (FSDV), which utilizes onboard units (OBUs) of all the vehicles in the region to create a dynamic fog for rogue nodes detection. We aim to reduce the data processing delays, overhead, and FPR in detecting rogue nodes causing Sybil attacks at high vehicle densities. The performance of our framework was carried out with simulations using OMNET++ and SUMO simulators. Results show that our framework ensures 43% lower processing delays, 13% lower overhead, and 35% lower FPR at high vehicle densities compared to existing Sybil attack detection schemes.
edge-SR: Super-Resolution For The Masses
Venue: arXiv cs.CV 2021
Year: 2021

Classic image scaling (e.g. bicubic) can be seen as one convolutional layer and a single upscaling filter. Its implementation is ubiquitous in all display devices and image processing software. In the last decade deep learning systems have been introduced for the task of image super-resolution (SR), using several convolutional layers and numerous filters. These methods have taken over the benchmarks of image quality for upscaling tasks. Would it be possible to replace classic upscalers with deep learning architectures on edge devices such as display panels, tablets, laptop computers, etc.? On one hand, the current trend in Edge-AI chips shows a promising future in this direction, with rapid development of hardware that can run deep-learning tasks efficiently. On the other hand, in image SR only few architectures have pushed the limit to extreme small sizes that can actually run on edge devices at real-time. We explore possible solutions to this problem with the aim to fill the gap between classic upscalers and small deep learning configurations. As a transition from classic to deep-learning upscaling we propose edge-SR (eSR), a set of one-layer architectures that use interpretable mechanisms to upscale images. Certainly, a one-layer architecture cannot reach the quality of deep learning systems. Nevertheless, we find that for high speed requirements, eSR becomes better at trading-off image quality and runtime performance. Filling the gap between classic and deep-learning architectures for image upscaling is critical for massive adoption of this technology. It is equally important to have an interpretable system that can reveal the inner strategies to solve this problem and guide us to future improvements and better understanding of larger networks.
Gender-based occupational segregation: a bit string approach
Venue: arXiv cs.SI 2021
Year: 2021

The systematic differences of gender representation across occupations, gender-based occupational segregation, has been suggested as one of the most important determinants of the still existing gender wage gap. Despite some signs of a decreasing trend, there is evidence that occupational gendered segregation is persistent even though gender differences in human capital variables have been disappearing. Using an agent-based model we provide a framework that introduces discriminatory behavior based on labour market theories of discrimination where workers and firms can exhibit gendered preferences. The introduction of discriminatory behavior transforms the otherwise random dynamics of occupational choice into a persistent gender-based occupational segregation consistent with empirical evidence.
Examining Covert Gender Bias: A Case Study in Turkish and English Machine Translation Models
Venue: arXiv cs.CL 2021
Year: 2021

As Machine Translation (MT) has become increasingly more powerful, accessible, and widespread, the potential for the perpetuation of bias has grown alongside its advances. While overt indicators of bias have been studied in machine translation, we argue that covert biases expose a problem that is further entrenched. Through the use of the gender-neutral language Turkish and the gendered language English, we examine cases of both overt and covert gender bias in MT models. Specifically, we introduce a method to investigate asymmetrical gender markings. We also assess bias in the attribution of personhood and examine occupational and personality stereotypes through overt bias indicators in MT models. Our work explores a deeper layer of bias in MT models and demonstrates the continued need for language-specific, interdisciplinary methodology in MT model development.
On the Foundation of Sparse Sensing (Part II): Diophantine Sampling and Array Configuration
Venue: arXiv cs.IT 2021
Year: 2021

In the second part of the series papers, we set out to study the algorithmic efficiency of sparse sensing. Stemmed from co-prime sensing, we propose a generalized framework, termed Diophantine sensing, which utilizes generic Diophantine equation theory and higher-order sparse ruler to strengthen the sampling time, the degree of freedom (DoF), and the sampling sparsity, simultaneously. Resorting to higher-moment statistics, the proposed Diophantine framework presents two fundamental improvements. First, on frequency estimation, we prove that given arbitrarily large down-sampling rates, there exist sampling schemes where the number of samples needed is only proportional to the sum of DoF and the number of snapshots required, which implies a linear sampling time. Second, on Direction-of-arrival (DoA) estimation, we propose two generic array constructions such that given N sensors, the minimal distance between sensors can be as large as a polynomial of N, O(N^q), which indicates that an arbitrarily sparse array (with arbitrarily small mutual coupling) exists given sufficiently many sensors. In addition, asymptotically, the proposed array configurations produce the best known DoF bound compared to existing sparse array designs.
Rewrite Rule Inference Using Equality Saturation
Venue: arXiv cs.PL 2021
Year: 2021

Many compilers, synthesizers, and theorem provers rely on rewrite rules to simplify expressions or prove equivalences. Developing rewrite rules can be difficult: rules may be subtly incorrect, profitable rules are easy to miss, and rulesets must be rechecked or extended whenever semantics are tweaked. Large rulesets can also be challenging to apply: redundant rules slow down rule-based search and frustrate debugging. This paper explores how equality saturation, a promising technique that uses e-graphs to apply rewrite rules, can also be used to infer rewrite rules. E-graphs can compactly represent the exponentially large sets of enumerated terms and potential rewrite rules. We show that equality saturation efficiently shrinks both sets, leading to faster synthesis of smaller, more general rulesets. We prototyped these strategies in a tool dubbed ruler. Compared to a similar tool built on CVC4, ruler synthesizes 5.8X smaller rulesets 25X faster without compromising on proving power. In an end-to-end case study, we show ruler-synthesized rules which perform as well as those crafted by domain experts, and addressed a longstanding issue in a popular open source tool.
Differential Music: Automated Music Generation Using LSTM Networks with Representation Based on Melodic and Harmonic Intervals
Venue: arXiv cs.SD 2021
Year: 2021

This paper presents a generative AI model for automated music composition with LSTM networks that takes a novel approach at encoding musical information which is based on movement in music rather than absolute pitch. Melodies are encoded as a series of intervals rather than a series of pitches, and chords are encoded as the set of intervals that each chord note makes with the melody at each timestep. Experimental results show promise as they sound musical and tonal. There are also weaknesses to this method, mainly excessive modulations in the compositions, but that is expected from the nature of the encoding. This issue is discussed later in the paper and is a potential topic for future work.
Creating Electronic Oscillator-based Ising Machines without External Injection Locking
Venue: arXiv cs.ET 2021
Year: 2021

Coupled electronic oscillators have recently been explored as a compact, integrated circuit- and room temperature operation- compatible hardware platform to design Ising machines. However, such implementations presently require the injection of an externally generated second-harmonic signal to impose the phase bipartition among the oscillators. In this work, we experimentally demonstrate a new electronic autaptic oscillator (EAO) that uses engineered feedback to eliminate the need for the generation and injection of the external second harmonic signal to minimize the Ising Hamiltonian. The feedback in the EAO is engineered to effectively generate the second harmonic signal internally. Using this oscillator design, we show experimentally, that a system of capacitively coupled EAOs exhibits the desired bipartition in the oscillator phases, and subsequently, demonstrate its application in solving the computationally hard Maximum Cut (MaxCut) problem. Our work not only establishes a new oscillator design aligned to the needs of the oscillator Ising machine but also advances the efforts to creating application specific analog computing platforms.
Modulating Intelligent Surfaces for Multi-User MIMO Systems: Beamforming and Modulation Design
Venue: arXiv cs.IT 2021
Year: 2021

This paper introduces a novel approach of utilizing the reconfigurable intelligent surface (RIS) for joint data modulation and signal beamforming in a multi-user downlink cellular network by leveraging the idea of backscatter communication. We present a general framework in which the RIS, referred to as modulating intelligent surface (MIS) in this paper, is used to: i) beamform the signals for a set of users whose data modulation is already performed by the base station (BS), and at the same time, ii) embed the data of a different set of users by passively modulating the deliberately sent carrier signals from the BS to the RIS. To maximize each user's spectral efficiency, a joint non-convex optimization problem is formulated under the sum minimum mean-square error (MMSE) criterion. Alternating optimization is used to divide the original joint problem into two tasks of: i) separately optimizing the MIS phase-shifts for passive beamforming along with data embedding for the BS- and MIS-served users, respectively, and ii) jointly optimizing the active precoder and the receive scaling factor for the BS- and MIS-served users, respectively. While the solution to the latter joint problem is found in closed-form using traditional optimization techniques, the optimal phase-shifts at the MIS are obtained by deriving the appropriate optimization-oriented vector approximate message passing (OOVAMP) algorithm. Moreover, the original joint problem is solved under both ideal and practical constraints on the MIS phase shifts, namely, the unimodular constraint and assuming each MIS element to be terminated by a variable reactive load. The proposed MIS-assisted scheme is compared against state-of-the-art RIS-assisted wireless communication schemes and simulation results reveal that it brings substantial improvements in terms of system throughput while supporting a much higher number of users.
CMML: Contextual Modulation Meta Learning for Cold-Start Recommendation
Venue: arXiv cs.IR 2021
Year: 2021

Practical recommender systems experience a cold-start problem when observed user-item interactions in the history are insufficient. Meta learning, especially gradient based one, can be adopted to tackle this problem by learning initial parameters of the model and thus allowing fast adaptation to a specific task from limited data examples. Though with significant performance improvement, it commonly suffers from two critical issues: the non-compatibility with mainstream industrial deployment and the heavy computational burdens, both due to the inner-loop gradient operation. These two issues make them hard to be applied in practical recommender systems. To enjoy the benefits of meta learning framework and mitigate these problems, we propose a recommendation framework called Contextual Modulation Meta Learning (CMML). CMML is composed of fully feed-forward operations so it is computationally efficient and completely compatible with the mainstream industrial deployment. CMML consists of three components, including a context encoder that can generate context embedding to represent a specific task, a hybrid context generator that aggregates specific user-item features with task-level context, and a contextual modulation network, which can modulate the recommendation model to adapt effectively. We validate our approach on both scenario-specific and user-specific cold-start setting on various real-world datasets, showing CMML can achieve comparable or even better performance with gradient based methods yet with much higher computational efficiency and better interpretability.
Improving Object Detection by Label Assignment Distillation
Venue: arXiv cs.CV 2021
Year: 2021

Label assignment in object detection aims to assign targets, foreground or background, to sampled regions in an image. Unlike labeling for image classification, this problem is not well defined due to the object's bounding box. In this paper, we investigate the problem from a perspective of distillation, hence we call Label Assignment Distillation (LAD). Our initial motivation is very simple, we use a teacher network to generate labels for the student. This can be achieved in two ways: either using the teacher's prediction as the direct targets (soft label), or through the hard labels dynamically assigned by the teacher (LAD). Our experiments reveal that: (i) LAD is more effective than soft-label, but they are complementary. (ii) Using LAD, a smaller teacher can also improve a larger student significantly, while soft-label can't. We then introduce Co-learning LAD, in which two networks simultaneously learn from scratch and the role of teacher and student are dynamically interchanged. Using PAA-ResNet50 as a teacher, our LAD techniques can improve detectors PAA-ResNet101 and PAA-ResNeXt101 to $46 \rm AP$ and $47.5\rm AP$ on the COCO test-dev set. With a stronger teacher PAA-SwinB, we improve the students PAA-ResNet50 to $43.7\rm AP$ by only 1x schedule training and standard setting, and PAA-ResNet101 to $47.9\rm AP$, significantly surpassing the current methods. Our source code and checkpoints are released at https://git.io/JrDZo.
Adaptation-Agnostic Meta-Training
Venue: arXiv cs.LG 2021
Year: 2021

Many meta-learning algorithms can be formulated into an interleaved process, in the sense that task-specific predictors are learned during inner-task adaptation and meta-parameters are updated during meta-update. The normal meta-training strategy needs to differentiate through the inner-task adaptation procedure to optimize the meta-parameters. This leads to a constraint that the inner-task algorithms should be solved analytically. Under this constraint, only simple algorithms with analytical solutions can be applied as the inner-task algorithms, limiting the model expressiveness. To lift the limitation, we propose an adaptation-agnostic meta-training strategy. Following our proposed strategy, we can apply stronger algorithms (e.g., an ensemble of different types of algorithms) as the inner-task algorithm to achieve superior performance comparing with popular baselines. The source code is available at https://github.com/jiaxinchen666/AdaptationAgnosticMetaLearning.
Image-based Insider Threat Detection via Geometric Transformation
Venue: arXiv cs.CR 2021
Year: 2021

Insider threat detection has been a challenging task over decades, existing approaches generally employ the traditional generative unsupervised learning methods to produce normal user behavior model and detect significant deviations as anomalies. However, such approaches are insufficient in precision and computational complexity. In this paper, we propose a novel insider threat detection method, Image-based Insider Threat Detector via Geometric Transformation (IGT), which converts the unsupervised anomaly detection into supervised image classification task, and therefore the performance can be boosted via computer vision techniques. To illustrate, our IGT uses a novel image-based feature representation of user behavior by transforming audit logs into grayscale images. By applying multiple geometric transformations on these behavior grayscale images, IGT constructs a self-labelled dataset and then train a behavior classifier to detect anomaly in self-supervised manner. The motivation behind our proposed method is that images converted from normal behavior data may contain unique latent features which keep unchanged after geometric transformation, while malicious ones cannot. Experimental results on CERT dataset show IGT outperforms the classical autoencoder-based unsupervised insider threat detection approaches, and improves the instance and user based Area under the Receiver Operating Characteristic Curve (AUROC) by 4% and 2%, respectively.
Optimal UAV Hitching on Ground Vehicles
Venue: arXiv cs.NI 2021
Year: 2021

Due to its mobility and agility, unmanned aerial vehicle (UAV) has emerged as a promising technology for various tasks, such as sensing, inspection and delivery. However, a typical UAV has limited energy storage and cannot fly a long distance without being recharged. This motivates several existing proposals to use trucks and other ground vehicles to offer riding to help UAVs save energy and expand the operation radius. We present the first theoretical study regarding how UAVs should optimally hitch on ground vehicles, considering vehicles' different travelling patterns and supporting capabilities. For a single UAV, we derive closed-form optimal vehicle selection and hitching strategy. When vehicles only support hitching, a UAV would prefer the vehicle that can carry it closest to its final destination. When vehicles can offer hitching plus charging, the UAV may hitch on a vehicle that carries it farther away from its destination and hitch a longer distance. The UAV may also prefer to hitch on a slower vehicle for the benefit of battery recharging. For multiple UAVs in need of hitching, we develop the max-saving algorithm (MSA) to optimally match UAV-vehicle collaboration. We prove that the MSA globally optimizes the total hitching benefits for the UAVs.
Detection of Criminal Texts for the Polish State Border Guard
Venue: arXiv cs.CL 2021
Year: 2021

This paper describes research on the detection of Polish criminal texts appearing on the Internet. We carried out experiments to find the best available setup for the efficient classification of unbalanced and noisy data. The best performance was achieved when our model was fine-tuned on a pre-trained Polish-based transformer language model. For the detection task, a large corpus of annotated Internet snippets was collected as training data. We share this dataset and create a new task for the detection of criminal texts using the Gonito platform as the benchmark.
Are the Multilingual Models Better? Improving Czech Sentiment with Transformers
Venue: arXiv cs.CL 2021
Year: 2021

In this paper, we aim at improving Czech sentiment with transformer-based models and their multilingual versions. More concretely, we study the task of polarity detection for the Czech language on three sentiment polarity datasets. We fine-tune and perform experiments with five multilingual and three monolingual models. We compare the monolingual and multilingual models' performance, including comparison with the older approach based on recurrent neural networks. Furthermore, we test the multilingual models and their ability to transfer knowledge from English to Czech (and vice versa) with zero-shot cross-lingual classification. Our experiments show that the huge multilingual models can overcome the performance of the monolingual models. They are also able to detect polarity in another language without any training data, with performance not worse than 4.4 % compared to state-of-the-art monolingual trained models. Moreover, we achieved new state-of-the-art results on all three datasets.