OpenRaiser/Intern-Atlas huggingface dataset:
HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition
Venue: arXiv cs.CL 2021
Year: 2021

This paper introduces HeBERT and HebEMO. HeBERT is a Transformer-based model for modern Hebrew text, which relies on a BERT (Bidirectional Encoder Representations for Transformers) architecture. BERT has been shown to outperform alternative architectures in sentiment analysis, and is suggested to be particularly appropriate for MRLs. Analyzing multiple BERT specifications, we find that while model complexity correlates with high performance on language tasks that aim to understand terms in a sentence, a more-parsimonious model better captures the sentiment of entire sentence. Either way, out BERT-based language model outperforms all existing Hebrew alternatives on all common language tasks. HebEMO is a tool that uses HeBERT to detect polarity and extract emotions from Hebrew UGC. HebEMO is trained on a unique Covid-19-related UGC dataset that we collected and annotated for this study. Data collection and annotation followed an active learning procedure that aimed to maximize predictability. We show that HebEMO yields a high F1-score of 0.96 for polarity classification. Emotion detection reaches F1-scores of 0.78-0.97 for various target emotions, with the exception of surprise, which the model failed to capture (F1 = 0.41). These results are better than the best-reported performance, even among English-language models of emotion detection.
Roughly Collected Dataset for Contact Force Sensing Catheter
Venue: arXiv cs.RO 2021
Year: 2021

With rise of interventional cardiology, Catheter Ablation Therapy (CAT) has established itself as a first-line solution to treat cardiac arrhythmia. Although CAT is a promising technique, cardiologist lacks vision inside the body during the procedure, which may cause serious clinical syndromes. To support accurate clinical procedure, Contact Force Sensing (CFS) system is developed to find a position of the catheter tip through the measure of contact force between catheter and heart tissue. However, the practical usability of commercialized CFS systems is not fully understood due to inaccuracy in the measurement. To support the development of more accurate system, we develop a full pipeline of CFS system with newly collected benchmark dataset through a contact force sensing catheter in simplest hardware form. Our dataset was roughly collected with human noise to increase data diversity. Through the analysis of the dataset, we identify a problem defined as Shift of Reference (SoR), which prevents accurate measurement of contact force. To overcome the problem, we conduct the contact force estimation via standard deep neural networks including for Recurrent Neural Network (RNN), Fully Convolutional Network (FCN) and Transformer. An average error in measurement for RNN, FCN and Transformer are, respectively, 2.46g, 3.03g and 3.01g. Through these studies, we try to lay a groundwork, serve a performance criteria for future CFS system research and open a publicly available dataset to public.
A Bayesian Neural Network based on Dropout Regulation
Venue: arXiv cs.LG 2021
Year: 2021

Bayesian Neural Networks (BNN) have recently emerged in the Deep Learning world for dealing with uncertainty estimation in classification tasks, and are used in many application domains such as astrophysics, autonomous driving...BNN assume a prior over the weights of a neural network instead of point estimates, enabling in this way the estimation of both aleatoric and epistemic uncertainty of the model prediction.Moreover, a particular type of BNN, namely MC Dropout, assumes a Bernoulli distribution on the weights by using Dropout.Several attempts to optimize the dropout rate exist, e.g. using a variational approach.In this paper, we present a new method called "Dropout Regulation" (DR), which consists of automatically adjusting the dropout rate during training using a controller as used in automation.DR allows for a precise estimation of the uncertainty which is comparable to the state-of-the-art while remaining simple to implement.
Efficient Byzantine Fault Tolerance using Trusted Execution Environment: Preventing Equivocation is only the Beginning
Venue: arXiv cs.DC 2021
Year: 2021

With the rapid development of blockchain, Byzantine fault-tolerant protocols have attracted revived interest recently. To overcome the theoretical bounds of Byzantine fault tolerance, many protocols attempt to use Trusted Execution Environment (TEE) to prevent equivocation and improve performance and scalability. However, due to the broken quorum intersection assumption caused by the reduction of the replica number, the improvement is mostly at the cost of increased communication complexity which prevents existing TEE-based protocols to be applied to large-scale blockchain systems. In this paper, we propose TBFT, an efficient Byzantine fault-tolerant protocol in the partial synchrony setting, which has O(n) message complexity in both normal-case and view-change. Compared to previous protocols, TBFT uses TEE-assisted primitives to limit more types of malicious behaviors of replicas rather than preventing equivocation only, thereby reducing the latency and communication complexity of clients and replicas. Besides, we also introduce lightweight cryptographic primitives including a novel leader election mechanism and an efficient voting message aggregation mechanism for better security and performance. We evaluate TBFT via systematic analysis and experiments, and the results show that TBFT has better performance and scalability compared to other protocols.
Extending Partial Representations of Rectangular Duals with Given Contact Orientations
Venue: arXiv cs.CG 2021
Year: 2021

A rectangular dual of a graph $G$ is a contact representation of $G$ by axis-aligned rectangles such that (i)~no four rectangles share a point and (ii)~the union of all rectangles is a rectangle. The partial representation extension problem for rectangular duals asks whether a given partial rectangular dual can be extended to a rectangular dual, that is, whether there exists a rectangular dual where some vertices are represented by prescribed rectangles. Combinatorially, a rectangular dual can be described by a regular edge labeling (REL), which determines the orientations of the rectangle contacts. We describe two approaches to solve the partial representation extension problem for rectangular duals with given REL. On the one hand, we characterise the RELs that admit an extension, which leads to a linear-time testing algorithm. In the affirmative, we can construct an extension in linear time. This partial representation extension problem can also be formulated as a linear program (LP). We use this LP to solve the simultaneous representation problem for the case of rectangular duals when each input graph is given together with a REL.
Optimal Non-Uniform Deployments of LoRa Networks
Venue: arXiv cs.NI 2021
Year: 2021

LoRa wireless technology is an increasingly prominent solution for massive connectivity and the Internet of Things. Stochastic geometry and numerical analysis of LoRa networks usually consider uniform end-device deployments. Real deployments however will often be non-uniform, for example due to mobility. This letter mathematically investigates how non-uniform deployments affect network coverage and suggest optimal deployment strategies and uplink random access transmission schemes. We find that concave deployments of LoRa end-devices with a sub-linear spread of random access inter-transmission times provide optimal network coverage performance.
Focusing Knowledge-based Graph Argument Mining via Topic Modeling
Venue: arXiv cs.IR 2021
Year: 2021

Decision-making usually takes five steps: identifying the problem, collecting data, extracting evidence, identifying pro and con arguments, and making decisions. Focusing on extracting evidence, this paper presents a hybrid model that combines latent Dirichlet allocation and word embeddings to obtain external knowledge from structured and unstructured data. We study the task of sentence-level argument mining, as arguments mostly require some degree of world knowledge to be identified and understood. Given a topic and a sentence, the goal is to classify whether a sentence represents an argument in regard to the topic. We use a topic model to extract topic- and sentence-specific evidence from the structured knowledge base Wikidata, building a graph based on the cosine similarity between the entity word vectors of Wikidata and the vector of the given sentence. Also, we build a second graph based on topic-specific articles found via Google to tackle the general incompleteness of structured knowledge bases. Combining these graphs, we obtain a graph-based model which, as our evaluation shows, successfully capitalizes on both structured and unstructured data.
PARAFAC2 AO-ADMM: Constraints in all modes
Venue: arXiv cs.LG 2021
Year: 2021

The PARAFAC2 model provides a flexible alternative to the popular CANDECOMP/PARAFAC (CP) model for tensor decompositions. Unlike CP, PARAFAC2 allows factor matrices in one mode (i.e., evolving mode) to change across tensor slices, which has proven useful for applications in different domains such as chemometrics, and neuroscience. However, the evolving mode of the PARAFAC2 model is traditionally modelled implicitly, which makes it challenging to regularise it. Currently, the only way to apply regularisation on that mode is with a flexible coupling approach, which finds the solution through regularised least-squares subproblems. In this work, we instead propose an alternating direction method of multipliers (ADMM)-based algorithm for fitting PARAFAC2 and widen the possible regularisation penalties to any proximable function. Our numerical experiments demonstrate that the proposed ADMM-based approach for PARAFAC2 can accurately recover the underlying components from simulated data while being both computationally efficient and flexible in terms of imposing constraints.
Design and Appropriation of Computer-supported Self-scheduling Practices in Healthcare Shift Work
Venue: arXiv cs.HC 2021
Year: 2021

Shift scheduling impacts healthcare workers' well-being because it sets the frame for their social life and recreational activities. Since it is complex and time-consuming, it has become a target for automation. However, existing systems mostly focus on improving efficiency. The workers' needs and their active participation do not play a pronounced role. Contrasting this trend, we designed a social practice-based, worker-centered, and well-being-oriented self-scheduling system which gives healthcare workers more control during shift planning. In a following nine month appropriation study, we found that workers who were cautious about their social standing in the group or who had a more spontaneous personal lifestyle used our system less often than others. Moreover, we revealed several conflict prevention practices and suggest to shift the focus away from a competitive shift distribution paradigm towards supporting these pro-social practices. We conclude with guidelines to support individual planning practices, self-leadership, and for dealing with conflicts.
Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platforms
Venue: arXiv cs.CV 2021
Year: 2021

Convolutional Neural Networks (CNNs) have proven to be a powerful state-of-the-art method for image classification tasks. One drawback however is the high computational complexity and high memory consumption of CNNs which makes them unfeasible for execution on embedded platforms which are constrained on physical resources needed to support CNNs. Quantization has often been used to efficiently optimize CNNs for memory and computational complexity at the cost of a loss of prediction accuracy. We therefore propose a method to optimally quantize the weights, biases and activations of each layer of a pre-trained CNN while controlling the loss in inference accuracy to enable quantized inference. We quantize the 32-bit floating-point precision parameters to low bitwidth fixed-point representations thereby finding optimal bitwidths and fractional offsets for parameters of each layer of a given CNN. We quantize parameters of a CNN post-training without re-training it. Our method is designed to quantize parameters of a CNN taking into account how other parameters are quantized because ignoring quantization errors due to other quantized parameters leads to a low precision CNN with accuracy losses of up to 50% which is far beyond what is acceptable. Our final method therefore gives a low precision CNN with accuracy losses of less than 1%. As compared to a method used by commercial tools that quantize all parameters to 8-bits, our approach provides quantized CNN with averages of 53% lower memory consumption and 77.5% lower cost of executing multiplications for the two CNNs trained on the four datasets that we tested our work on. We find that layer-wise quantization of parameters significantly helps in this process.
Polar Codes for Channels with Insertions, Deletions, and Substitutions
Venue: arXiv cs.IT 2021
Year: 2021

This paper presents a coding scheme for an insertion deletion substitution channel. We extend a previous scheme for the deletion channel where polar codes are modified by adding "guard bands" between segments. In the new scheme, each guard band is comprised of a middle segment of '1' symbols, and left and right segments of '0' symbols. Our coding scheme allows for a regular hidden-Markov input distribution, and achieves the information rate between the input and corresponding output of such a distribution. Thus, we prove that our scheme can be used to efficiently achieve the capacity of the channel. The probability of error of our scheme decays exponentially in the cube-root of the block length.
Real-Time Optimal Trajectory Planning for Autonomous Vehicles and Lap Time Simulation Using Machine Learning
Venue: arXiv cs.RO 2021
Year: 2021

Widespread development of driverless vehicles has led to the formation of autonomous racing, where technological development is accelerated by the high speeds and competitive environment of motorsport. A particular challenge for an autonomous vehicle is that of identifying a target trajectory - or, in the case of a competition vehicle, the racing line. Many existing approaches to finding the racing line are either not time-optimal solutions, or are computationally expensive - rendering them unsuitable for real-time application using on-board processing hardware. This study describes a machine learning approach to generating an accurate prediction of the racing line in real-time on desktop processing hardware. The proposed algorithm is a feed-forward neural network, trained using a dataset comprising racing lines for a large number of circuits calculated via traditional optimal control lap time simulation. The network predicts the racing line with a mean absolute error of +/-0.27m, and just +/-0.11m at corner apex - comparable to human drivers, and autonomous vehicle control subsystems. The approach generates predictions within 33ms, making it over 9,000 times faster than traditional methods of finding the optimal trajectory. Results suggest that for certain applications data-driven approaches to find near-optimal racing lines may be favourable to traditional computational methods.
Query Complexity of Least Absolute Deviation Regression via Robust Uniform Convergence
Venue: arXiv cs.LG 2021
Year: 2021

Consider a regression problem where the learner is given a large collection of $d$-dimensional data points, but can only query a small subset of the real-valued labels. How many queries are needed to obtain a $1+ε$ relative error approximation of the optimum? While this problem has been extensively studied for least squares regression, little is known for other losses. An important example is least absolute deviation regression ($\ell_1$ regression) which enjoys superior robustness to outliers compared to least squares. We develop a new framework for analyzing importance sampling methods in regression problems, which enables us to show that the query complexity of least absolute deviation regression is $Θ(d/ε^2)$ up to logarithmic factors. We further extend our techniques to show the first bounds on the query complexity for any $\ell_p$ loss with $p\in(1,2)$. As a key novelty in our analysis, we introduce the notion of robust uniform convergence, which is a new approximation guarantee for the empirical loss. While it is inspired by uniform convergence in statistical learning, our approach additionally incorporates a correction term to avoid unnecessary variance due to outliers. This can be viewed as a new connection between statistical learning theory and variance reduction techniques in stochastic optimization, which should be of independent interest.
Effects of Number of Filters of Convolutional Layers on Speech Recognition Model Accuracy
Venue: arXiv cs.LG 2021
Year: 2021

Inspired by the progress of the End-to-End approach [1], this paper systematically studies the effects of Number of Filters of convolutional layers on the model prediction accuracy of CNN+RNN (Convolutional Neural Networks adding to Recurrent Neural Networks) for ASR Models (Automatic Speech Recognition). Experimental results show that only when the CNN Number of Filters exceeds a certain threshold value is adding CNN to RNN able to improve the performance of the CNN+RNN speech recognition model, otherwise some parameter ranges of CNN can render it useless to add the CNN to the RNN model. Our results show a strong dependency of word accuracy on the Number of Filters of convolutional layers. Based on the experimental results, the paper suggests a possible hypothesis of Sound-2-Vector Embedding (Convolutional Embedding) to explain the above observations. Based on this Embedding hypothesis and the optimization of parameters, the paper develops an End-to-End speech recognition system which has a high word accuracy but also has a light model-weight. The developed LVCSR (Large Vocabulary Continuous Speech Recognition) model has achieved quite a high word accuracy of 90.2% only by its Acoustic Model alone, without any assistance from intermediate phonetic representation and any Language Model. Its acoustic model contains only 4.4 million weight parameters, compared to the 35~68 million acoustic-model weight parameters in DeepSpeech2 [2] (one of the top state-of-the-art LVCSR models) which can achieve a word accuracy of 91.5%. The light-weighted model is good for improving the transcribing computing efficiency and also useful for mobile devices, Driverless Vehicles, etc. Our model weight is reduced to ~10% the size of DeepSpeech2, but our model accuracy remains close to that of DeepSpeech2. If combined with a Language Model, our LVCSR system is able to achieve 91.5% word accuracy.
A Practical Coding Scheme for the BSC with Feedback
Venue: arXiv cs.IT 2021
Year: 2021

We provide a practical implementation of the rubber method of Ahlswede et al. for binary alphabets. The idea is to create the "skeleton" sequence therein via an arithmetic decoder designed for a particular $k$-th order Markov chain. For the stochastic binary symmetric channel, we show that the scheme is nearly optimal in a strong sense for certain parameters.
On Stochastic Rewriting and Combinatorics via Rule-Algebraic Methods
Venue: arXiv cs.LO 2021
Year: 2021

Building upon the rule-algebraic stochastic mechanics framework, we present new results on the relationship of stochastic rewriting systems described in terms of continuous-time Markov chains, their embedded discrete-time Markov chains and certain types of generating function expressions in combinatorics. We introduce a number of generating function techniques that permit a novel form of static analysis for rewriting systems based upon marginalizing distributions over the states of the rewriting systems via pattern-counting observables.
Parallel Independence in Attributed Graph Rewriting
Venue: arXiv cs.LO 2021
Year: 2021

In order to define graph transformations by the simultaneous application of concurrent rules, we have adopted in previous work a structure of attributed graphs stable by unions. We analyze the consequences on parallel independence, a property that characterizes the possibility to resort to sequential rewriting. This property turns out to depend not only on the left-hand side of rules, as in algebraic approaches to graph rewriting, but also on their right-hand side. It is then shown that, of three possible definitions of parallel rewriting, only one is convenient in the light of parallel independence.
A Possible Artificial Intelligence Ecosystem Avatar: the Moorea case (IDEA)
Venue: arXiv cs.AI 2021
Year: 2021

High-throughput data collection techniques and largescale (cloud) computing are transforming our understanding of ecosystems at all scales by allowing the integration of multimodal data such as physics, chemistry, biology, ecology, fishing, economics and other social sciences in a common computational framework. We focus in this paper on a large scale data assimilation and prediction backbone based on Deep Stacking Networks (DSN) in the frame of the IDEA (Island Digital Ecosystem Avatars) project (Moorea Island), based on the subdivision of the island in watersheds and lagoon units. We also describe several kinds of raw data that can train and constrain such an ecosystem avatar model, as well as second level data such as ecological or physical indexes / indicators.
On Single-User Interactive Beam Alignment in Millimeter Wave Systems: Impact of Feedback Delay
Venue: arXiv cs.IT 2021
Year: 2021

Narrow beams are key to wireless communications in millimeter wave frequency bands. Beam alignment (BA) allows the base station (BS) to adjust the direction and width of the beam used for communication. During BA, the BS transmits a number of scanning beams covering different angular regions. The goal is to minimize the expected width of the uncertainty region (UR) that includes the angle of departure of the user. Conventionally, in interactive BA, it is assumed that the feedback corresponding to each scanning packet is received prior to transmission of the next one. However, in practice, the feedback delay could be larger because of propagation or system constraints. This paper investigates BA strategies that operate under arbitrary fixed feedback delays. This problem is analyzed through a source coding prospective where the feedback sequences are viewed as source codewords. It is shown that these codewords form a codebook with a particular characteristic which is used to define a new class of codes called d-unimodal codes. By analyzing the properties of these codes, a lower bound on the minimum achievable expected beamwidth is provided. The results reveal potential performance improvements in terms of the BA duration it takes to achieve a fixed expected width of the UR over the state-of-the-art BA methods which do not consider the effect of delay.
Graph Coding for Model Selection and Anomaly Detection in Gaussian Graphical Models
Venue: arXiv cs.LG 2021
Year: 2021

A classic application of description length is for model selection with the minimum description length (MDL) principle. The focus of this paper is to extend description length for data analysis beyond simple model selection and sequences of scalars. More specifically, we extend the description length for data analysis in Gaussian graphical models. These are powerful tools to model interactions among variables in a sequence of i.i.d Gaussian data in the form of a graph. Our method uses universal graph coding methods to accurately account for model complexity, and therefore provide a more rigorous approach for graph model selection. The developed method is tested with synthetic and electrocardiogram (ECG) data to find the graph model and anomaly in Gaussian graphical models. The experiments show that our method gives better performance compared to commonly used methods.