OpenRaiser/Intern-Atlas huggingface dataset:
Distributed Adaptive Learning Under Communication Constraints
Venue: arXiv cs.LG 2021
Year: 2021

This work examines adaptive distributed learning strategies designed to operate under communication constraints. We consider a network of agents that must solve an online optimization problem from continual observation of streaming data. The agents implement a distributed cooperative strategy where each agent is allowed to perform local exchange of information with its neighbors. In order to cope with communication constraints, the exchanged information must be unavoidably compressed. We propose a diffusion strategy nicknamed as ACTC (Adapt-Compress-Then-Combine), which relies on the following steps: i) an adaptation step where each agent performs an individual stochastic-gradient update with constant step-size; ii) a compression step that leverages a recently introduced class of stochastic compression operators; and iii) a combination step where each agent combines the compressed updates received from its neighbors. The distinguishing elements of this work are as follows. First, we focus on adaptive strategies, where constant (as opposed to diminishing) step-sizes are critical to respond in real time to nonstationary variations. Second, we consider the general class of directed graphs and left-stochastic combination policies, which allow us to enhance the interplay between topology and learning. Third, in contrast with related works that assume strong convexity for all individual agents' cost functions, we require strong convexity only at a network level, a condition satisfied even if a single agent has a strongly-convex cost and the remaining agents have non-convex costs. Fourth, we focus on a diffusion (as opposed to consensus) strategy. Under the demanding setting of compressed information, we establish that the ACTC iterates fluctuate around the desired optimizer, achieving remarkable savings in terms of bits exchanged between neighboring agents.
A reliable Turing machine
Venue: arXiv cs.CC 2021
Year: 2021

We consider computations of a Turing machine subjected to noise. In every step, the action (the new state and the new content of the observed cell, the direction of the head movement) can differ from that prescribed by the transition function with a small probability (independently of previous such events). We construct a universal 1-tape Turing machine that for a low enough (constant) noise probability performs arbitrarily large computations. For this unavoidably, the input needs to be encoded -- by a simple code depending on its size. The work uses a technique familiar from reliable cellular automata, complemented by some new ones.
The h-index
Venue: arXiv cs.DL 2021
Year: 2021

The h-index is a mainstream bibliometric indicator, since it is widely used in academia, research management and research policy. While its advantages have been highlighted, such as its simple calculation, it has also received widespread criticism. The criticism is mainly based on the negative effects it may have on scholars, when the index is used to describe the quality of a scholar. The "h" means "highly-cited" and "high achievement", and should not be confused with the last name of its inventor, Hirsch. Put simply, the h-index combines a measure of quantity and impact in a single indicator. Several initiatives try to provide alternatives to the h-index to counter some of its shortcomings.
V2X Misbehavior and Collective Perception Service: Considerations for Standardization
Venue: arXiv cs.CR 2021
Year: 2021

Connected and Automated Vehicles use sensors and wireless communication to improve road safety and efficiency. However, attackers may target Vehicle-to-Everything communication. Indeed, an attacker may send authenticated but wrong data to send false location information, alert incorrect events or report a bogus object endangering the safety of other CAVs. Currently, Standardization Development Organizations are working on developing security standards against such attacks. Unfortunately, current standardization efforts do not include misbehavior specifications for advanced V2X services such as Collective Perception yet. This work assesses the security of Collective Perception Messages and proposes inputs for consideration in existing standards.
Analysis of Communication Channels Related to Physical Unclonable Functions
Venue: arXiv cs.IT 2021
Year: 2021

Cryptographic algorithms rely on the secrecy of their corresponding keys. On embedded systems with standard CMOS chips, where secure permanent memory such as flash is not available as a key storage, the secret key can be derived from Physical Unclonable Functions (PUFs) that make use of minuscule manufacturing variations of, for instance, SRAM cells. Since PUFs are affected by environmental changes, the reliable reproduction of the PUF key requires error correction. For silicon PUFs with binary output, errors occur in the form of bitflips within the PUFs response. Modelling the channel as a Binary Symmetric Channel (BSC) with fixed crossover probability $p$ is only a first-order approximation of the real behavior of the PUF response. We propose a more realistic channel model, refered to as the Varying Binary Symmetric Channel (VBSC), which takes into account that the reliability of different PUF response bits may not be equal. We investigate its channel capacity for various scenarios which differ in the channel state information (CSI) present at encoder and decoder. We compare the capacity results for the VBSC for the different CSI cases with reference to the distribution of the bitflip probability according a work by Maes et al.
A Parallel Scalable Domain Decomposition Preconditioner for Elastic Crack Simulation Using XFEM
Venue: arXiv cs.CE 2021
Year: 2021

In this paper, a parallel overlapping domain decomposition preconditioner is proposed to solve the linear system of equations arising from the extended finite element discretization of elastic crack problems. The algorithm partitions the computational mesh into two types of subdomains: the regular subdomains and the crack tip subdomains based on the observation that the crack tips have a significant impact on the convergence of the iterative method while the impact of the crack lines is not that different from those of regular mesh points. The tip subdomains consist of mesh points at crack tips and all neighboring points where the branch enrichment functions are applied. The regular subdomains consist of all other mesh points, including those on the crack lines. To overcome the mismatch between the number of subdomains and the number of processor cores, the proposed method is divided into two steps: solve the crack tip problem and then the regular subdomain problem during each iteration. The proposed method was used to develop a parallel XFEM package which is able to test different types of iterative methods. To achieve good parallel efficiency, additional methods were introduced to reduce communication and to maintain the load balance between processors. Numerical experiments indicate that the proposed method significantly reduces the number of iterations and the total computation time compared to the classical methods. In addition, the method scales up to 8192 processor cores with over 70\% parallel efficiency to solve problems with more than $2\times10^8$ degrees of freedom.
Efficient FPGA-based ECDSA Verification Engine for Permissioned Blockchains
Venue: arXiv cs.CR 2021
Year: 2021

As enterprises embrace blockchain technology, many real-world applications have been developed and deployed using permissioned blockchain platforms (access to network is controlled and given to only nodes with known identities). Such blockchain platforms heavily depend on cryptography to provide a layer of trust within the network, thus verification of cryptographic signatures often becomes the bottleneck. The Elliptic Curve Digital Signature Algorithm (ECDSA) is the most commonly used cryptographic scheme in permissioned blockchains. In this paper, we propose an efficient implementation of ECDSA signature verification on an FPGA, in order to improve the performance of permissioned blockchains that aim to use FPGA-based hardware accelerators. We propose several optimizations for modular arithmetic (e.g., custom multipliers and fast modular reduction) and point arithmetic (e.g., reduced number of point double and addition operations, and optimal width NAF representation). Based on these optimized modular and point arithmetic modules, we propose an ECDSA verification engine that can be used by any application for fast verification of ECDSA signatures. We further optimize our ECDSA verification engine for Hyperledger Fabric (one of the most widely used permissioned blockchain platforms) by moving carefully selected operations to a precomputation block, thus simplifying the critical path of ECDSA signature verification. From our implementation on Xilinx Alveo U250 accelerator board with target frequency of 250MHz, our ECDSA verification engine can perform a single verification in $760μs$ resulting in a throughput of 1,315 verifications per second, which is ~2.5x faster than state-of-the-art FPGA-based implementations. Our Hyperledger Fabric-specific ECDSA engine can perform a single verification in $368μs$ with a throughput of 2,717 verifications per second.
A Triple-Double Convolutional Neural Network for Panchromatic Sharpening
Venue: arXiv cs.CV 2021
Year: 2021

Pansharpening refers to the fusion of a panchromatic image with a high spatial resolution and a multispectral image with a low spatial resolution, aiming to obtain a high spatial resolution multispectral image. In this paper, we propose a novel deep neural network architecture with level-domain based loss function for pansharpening by taking into account the following double-type structures, \emph{i.e.,} double-level, double-branch, and double-direction, called as triple-double network (TDNet). By using the structure of TDNet, the spatial details of the panchromatic image can be fully exploited and utilized to progressively inject into the low spatial resolution multispectral image, thus yielding the high spatial resolution output. The specific network design is motivated by the physical formula of the traditional multi-resolution analysis (MRA) methods. Hence, an effective MRA fusion module is also integrated into the TDNet. Besides, we adopt a few ResNet blocks and some multi-scale convolution kernels to deepen and widen the network to effectively enhance the feature extraction and the robustness of the proposed TDNet. Extensive experiments on reduced- and full-resolution datasets acquired by WorldView-3, QuickBird, and GaoFen-2 sensors demonstrate the superiority of the proposed TDNet compared with some recent state-of-the-art pansharpening approaches. An ablation study has also corroborated the effectiveness of the proposed approach.
Recommender systems: when memory matters
Venue: arXiv cs.IR 2021
Year: 2021

In this paper, we study the effect of long memory in the learnability of a sequential recommender system including users' implicit feedback. We propose an online algorithm, where model parameters are updated user per user over blocks of items constituted by a sequence of unclicked items followed by a clicked one. We illustrate through thorough empirical evaluations that filtering users with respect to the degree of long memory contained in their interactions with the system allows to substantially gain in performance with respect to MAP and NDCG, especially in the context of training large-scale Recommender Systems.
Controllable Response Generation for Assistive Use-cases
Venue: arXiv cs.CL 2021
Year: 2021

Conversational agents have become an integral part of the general population for simple task enabling situations. However, these systems are yet to have any social impact on the diverse and minority population, for example, helping people with neurological disorders, for example ALS, and people with speech, language and social communication disorders. Language model technology can play a huge role to help these users carry out daily communication and social interactions. To enable this population, we build a dialog system that can be controlled by users using cues or keywords. We build models that can suggest relevant cues in the dialog response context which is used to control response generation and can speed up communication. We also introduce a keyword loss to lexically constrain the model output. We show both qualitatively and quantitatively that our models can effectively induce the keyword into the model response without degrading the quality of response. In the context of usage of such systems for people with degenerative disorders, we present human evaluation of our cue or keyword predictor and the controllable dialog system and show that our models perform significantly better than models without control. Our study shows that keyword control on end to end response generation models is powerful and can enable and empower users with degenerative disorders to carry out their day to day communication.
STJLA: A Multi-Context Aware Spatio-Temporal Joint Linear Attention Network for Traffic Forecasting
Venue: arXiv cs.LG 2021
Year: 2021

Traffic prediction has gradually attracted the attention of researchers because of the increase in traffic big data. Therefore, how to mine the complex spatio-temporal correlations in traffic data to predict traffic conditions more accurately become a difficult problem. Previous works combined graph convolution networks (GCNs) and self-attention mechanism with deep time series models (e.g. recurrent neural networks) to capture the spatio-temporal correlations separately, ignoring the relationships across time and space. Besides, GCNs are limited by over-smoothing issue and self-attention is limited by quadratic problem, result in GCNs lack global representation capabilities, and self-attention inefficiently capture the global spatial dependence. In this paper, we propose a novel deep learning model for traffic forecasting, named Multi-Context Aware Spatio-Temporal Joint Linear Attention (STJLA), which applies linear attention to the spatio-temporal joint graph to capture global dependence between all spatio-temporal nodes efficiently. More specifically, STJLA utilizes static structural context and dynamic semantic context to improve model performance. The static structure context based on node2vec and one-hot encoding enriches the spatio-temporal position information. Furthermore, the multi-head diffusion convolution network based dynamic spatial context enhances the local spatial perception ability, and the GRU based dynamic temporal context stabilizes sequence position information of the linear attention, respectively. Experiments on two real-world traffic datasets, England and PEMSD7, demonstrate that our STJLA can achieve up to 9.83% and 3.08% accuracy improvement in MAE measure over state-of-the-art baselines.
On the Implementation of Fixed-point Exponential Function for Machine Learning and Signal Processing Accelerators
Venue: arXiv cs.AR 2021
Year: 2021

The natural exponential function is widely used in modeling many engineering and scientific systems. It is also an integral part of many neural network activation function such as sigmoid, tanh, ELU, RBF etc. Dedicated hardware accelerator and processors are designed for faster execution of such applications. Such accelerators can immensely benefit from an optimal implementation of exponential function. This can be achieved for most applications with the knowledge that the exponential function for a negative domain is more widely used than the positive domain. This paper presents an optimized implementation of exponential function for variable precision fixed point negative input. The implementation presented here significantly reduces the number of multipliers and adders. This is further optimized using mixed world-length implementation for the series expansion. The reduction in area and power consumption is more than 30% and 50% respectively over previous equivalent method.
A refined consumer behavior model for energy systems: Application to the pricing and energy-efficiency problems
Venue: arXiv cs.GT 2021
Year: 2021

The sum-utility maximization problem is known to be important in the energy systems literature. The conventional assumption to address this problem is that the utility is concave. But for some key applications, such an assumption is not reasonable and does not reflect well the actual behavior of the consumer. To address this issue, the authors pose and address a more general optimization problem, namely by assuming the consumer's utility to be sigmoidal and in a given class of functions. The considered class of functions is very attractive for at least two reasons. First, the classical NP-hardness issue associated with sum-utility maximization is circumvented. Second, the considered class of functions encompasses well-known performance metrics used to analyze the problems of pricing and energy-efficiency. This allows one to design a new and optimal inclining block rates (IBR) pricing policy which also has the virtue of flattening the power consumption and reducing the peak power. We also show how to maximize the energy-efficiency by a low-complexity algorithm. When compared to existing policies, simulations fully support the benefit from using the proposed approach.
Overcome Anterograde Forgetting with Cycled Memory Networks
Venue: arXiv cs.LG 2021
Year: 2021

Learning from a sequence of tasks for a lifetime is essential for an agent towards artificial general intelligence. This requires the agent to continuously learn and memorize new knowledge without interference. This paper first demonstrates a fundamental issue of lifelong learning using neural networks, named anterograde forgetting, i.e., preserving and transferring memory may inhibit the learning of new knowledge. This is attributed to the fact that the learning capacity of a neural network will be reduced as it keeps memorizing historical knowledge, and the fact that conceptual confusion may occur as it transfers irrelevant old knowledge to the current task. This work proposes a general framework named Cycled Memory Networks (CMN) to address the anterograde forgetting in neural networks for lifelong learning. The CMN consists of two individual memory networks to store short-term and long-term memories to avoid capacity shrinkage. A transfer cell is designed to connect these two memory networks, enabling knowledge transfer from the long-term memory network to the short-term memory network to mitigate the conceptual confusion, and a memory consolidation mechanism is developed to integrate short-term knowledge into the long-term memory network for knowledge accumulation. Experimental results demonstrate that the CMN can effectively address the anterograde forgetting on several task-related, task-conflict, class-incremental and cross-domain benchmarks.
A Bio-inspired Modular System for Humanoid Posture Control
Venue: arXiv cs.RO 2021
Year: 2021

Bio-inspired sensorimotor control systems may be appealing to roboticists who try to solve problems of multiDOF humanoids and human-robot interactions. This paper presents a simple posture control concept from neuroscience, called disturbance estimation and compensation, DEC concept [1]. It provides human-like mechanical compliance due to low loop gain, tolerance of time delays, and automatic adjustment to changes in external disturbance scenarios. Its outstanding feature is that it uses feedback of multisensory disturbance estimates rather than 'raw' sensory signals for disturbance compensation. After proof-of-principle tests in 1 and 2 DOF posture control robots, we present here a generalized DEC control module for multi-DOF robots. In the control layout, one DEC module controls one DOF (modular control architecture). Modules of neighboring joints are synergistically interconnected using vestibular information in combination with joint angle and torque signals. These sensory interconnections allow each module to control the kinematics of the more distal links as if they were a single link. This modular design makes the complexity of the robot control scale linearly with the DOFs and error robustness high compared to monolithic control architectures. The presented concept uses Matlab/Simulink (The MathWorks, Natick, USA) for both, model simulation and robot control and will be available as open library
TransBoost: A Boosting-Tree Kernel Transfer Learning Algorithm for Improving Financial Inclusion
Venue: arXiv cs.LG 2021
Year: 2021

The prosperity of mobile and financial technologies has bred and expanded various kinds of financial products to a broader scope of people, which contributes to advocating financial inclusion. It has non-trivial social benefits of diminishing financial inequality. However, the technical challenges in individual financial risk evaluation caused by the distinct characteristic distribution and limited credit history of new users, as well as the inexperience of newly-entered companies in handling complex data and obtaining accurate labels, impede further promoting financial inclusion. To tackle these challenges, this paper develops a novel transfer learning algorithm (i.e., TransBoost) that combines the merits of tree-based models and kernel methods. The TransBoost is designed with a parallel tree structure and efficient weights updating mechanism with theoretical guarantee, which enables it to excel in tackling real-world data with high dimensional features and sparsity in $O(n)$ time complexity. We conduct extensive experiments on two public datasets and a unique large-scale dataset from Tencent Mobile Payment. The results show that the TransBoost outperforms other state-of-the-art benchmark transfer learning algorithms in terms of prediction accuracy with superior efficiency, shows stronger robustness to data sparsity, and provides meaningful model interpretation. Besides, given a financial risk level, the TransBoost enables financial service providers to serve the largest number of users including those who would otherwise be excluded by other algorithms. That is, the TransBoost improves financial inclusion.
Towards automated verification of multi-party consensus protocols
Venue: arXiv cs.CR 2021
Year: 2021

Blockchain technology and related frameworks have recently received extensive attention. Blockchain systems use multi-party consensus protocols to reach agreements on transactions. Hyperledger Fabric framework exposes a multi-party consensus, based on endorsement policy protocol, to reach a consensus on a transaction. In this paper, we define a problem of verification of a blockchain multi-party consensus with probabilistic properties. Further, we propose a verification technique of endorsement policies using statistical model checking and hypothesis testing. We analyze several aspects of the policies, including the ability to assign weights to organizations and the refusal probabilities of organizations. We demonstrate on experiments the work of our verification technique and how one can use experimental results to make the model satisfiable the specification. One can use our technique to design enterprise applications with the Hyperledger Fabric framework.