Reviewerly/exHarmony huggingface dataset:
A Taxonomy of Recurrent Learning Rules

Backpropagation through time (BPTT) is the de facto standard for training recurrent neural networks (RNNs), but it is non-causal and non-local. Real-time recurrent learning is a causal alternative, but it is highly inefficient. Recently, e-prop was proposed as a causal, local, and efficient practical alternative to these algorithms, providing an approximation of the exact gradient by radically pruning the recurrent dependencies carried over time. Here, we derive RTRL from BPTT using a detailed notation bringing intuition and clarification to how they are connected. Furthermore, we frame e-prop within in the picture, formalising what it approximates. Finally, we derive a family of algorithms of which e-prop is a special case.
Neural Criticality Metric for Object Detection Deep Neural Networks

The complexity of state-of-the-art Deep Neural Network (DNN) architectures exacerbates the search for safety relevant metrics and methods that could be used for functional safety assessments. In this article, we investigate Neurons’ Criticality (the ability to affect the decision process) for several object detection DNN architectures. As a first step, we introduce the Neural Criticality metric for object detection DNNs and set a theoretical background. Subsequently, by conducting experiments, we verify that removing one neuron from the computational graph of a DNN can have a significant (positive, as well as negative) influence on the prediction’s precision (object classification and localization). Finally, we build statistics for each neuron from pre-trained networks on the COCO object detection validation dataset and examine the network stability for the most critical neurons in order to prove our metric’s validity.
Increasing IoT Security by Supply Power Measurement

IoT devices are pervasive and provide helpful data in many aspects of life. People very often rely on the information provided by these devices. In contrast to this, IoT devices often are not designed with the right degree of security. Sometimes the assumptions made during development regarding their environment become invalid soon after roll-out. In addition, new threats arise, and new vulnerabilities are discovered frequently. Therefore, IoT devices are of great interest to attackers and often easily fall victim to them. This paper will describe a method to detect whether a medical IoT device is under attack and whether it runs any unexpected tasks. The method uses supply current monitoring and is both easy to implement and only requires very few resources (both in CPU performance and in memory). It can be used as an inexpensive early warning system.
Self-organization of a Dynamical Orthogonal Basis Acquiring Large Memory Capacity in Modular Reservoir Computing

The ability of the brain to generate complex spatiotemporal patterns with a specific timing is essential for motor learning and time series prediction. An approach that tries to replicate this ability using the self-sustained neural activity of a randomly connected recurrent neural network (reservoir) meets the difficulty of orbital instability. We propose a novel system that learns an arbitrary time series as the linear sum (readout) of stable trajectories produced by numerous small network modules. Our experimental results show that the trajectories of the module outputs are orthogonal to each other, that is, the reservoir self-organizes an orthogonal basis. Furthermore, the system can learn the timing of extremely long intervals, say tens of seconds for a millisecond computation unit and the complex time series of the Lorenz system.
Towards Robust Uncertainty Estimation in the Presence of Noisy Labels

In security-critical applications, it is essential to know how confident the model is in its predictions. Many uncertainty estimation methods have been proposed recently, and these methods are reliable when the training data do not contain labeling errors. However, we find that the quality of these uncertainty estimation methods decreases dramatically when noisy labels are present in the training data. In some datasets, the uncertainty estimates would become completely absurd, even though these labeling noises barely affect the test accuracy. We further analyze the impact of existing label noise handling methods on the reliability of uncertainty estimates, although most of these methods focus only on improving the accuracy of the models. We identify that the data cleaning-based approach can alleviate the influence of label noise on uncertainty estimates to some extent, but there are still some drawbacks. Finally, we propose a robust uncertainty estimation method under label noise. Compared with other algorithms, our approach achieves a more reliable uncertainty estimates in the presence of noisy labels, especially when there are large-scale labeling errors in the training data.
Neural-Gas VAE

Most deep learning models are known to be black-box models due to their overwhelming complexity. One approach to make models more interpretable is to reduce the representations to a finite number of objects. This can be achieved by clustering latent spaces or training models which include quantization by design such as the Vector Quantised-Variational AutoEncoder (VQ-VAE). However, if the architecture is not chosen carefully, a phenomenon called index collapse can be observed. Here, a large part of the codebook containing the prototypes is not used decreasing the possible performance. Approaches to circumvent this either rely on data-depending initialization or decreasing the dimensionality of the codebook vectors. In this paper, we present a novel variant of the VQ-VAE, the Neural-Gas VAE, which adapts the codebook loss inspired by neural-gas to avoid index collapse. We show that the Neural-Gas VAE achieves competitive performance on CIFAR and Speech Commands for different codebook sizes and dimensions. Moreover, we show that the resulting architecture learns a meaningful latent space and topology for both features or objects.
Testing and Verification of the Deep Neural Networks Against Sparse Pixel Defects

Deep neural networks can produce outstanding results when applied to image recognition tasks but are susceptible to image defects and modifications. Substantial degradation of the image can be detected by automatic or interactive prevention techniques. However, sparse pixel defects may have a significant impact on the dependability of safety-critical systems, especially autonomous driving vehicles. Such perturbations can limit the perception capabilities of the system while remaining undetected by human observer. The effective generation of such cases facilitates the simulation of real-life challenges caused by sparse pixel defects, like occluded or stained objects. This work introduces a novel sparse adversarial attack generation method based on differential evolution strategy. Additionally, we introduce a novel framework for sparse adversarial attack generation, which can be integrated into the safety-critical systems development process. An empirical evaluation demonstrates that the proposed method outperforms and complements state-of-the-art techniques allowing for complete evaluation of an image recognition system.
Boosting Both Robustness and Hardware Efficiency via Random Pruning Mask Selection

Deep neural networks (DNNs) are notorious for two key drawbacks: the vulnerability against adversarial attacks and the prohibitive cost of storage and computation, which greatly hinders DNNs’ deployment on safety-critical yet resource-limited platforms. Although researchers have proposed adversary-aware pruning methods where adversarial training and network pruning are studied jointly to improve the robustness of pruned networks, they failed to attain a double-win, i.e., the achieved robustness is still limited and cannot surpass that of dense networks. In this work, pursuing a win-win in robustness and efficiency, we demonstrate that the robustness of pruned networks can be easily boosted by leveraging the stochastic policy. More specifically, we propose a Random Mask Selection (RMS) strategy where pruning masks are randomly sampled during inference to confuse attackers. Furthermore, a necessary hardware-aware algorithm optimization is introduced to eliminate the potential hardware overhead of RMS, and thus ensures a convenient implementation of RMS on existing hardware accelerators without sacrificing processing speed or power efficiency. Extensive experiments show that our approach achieves a double-win in robustness and compactness compared to dense models and outperforms the SOTA adversary-aware pruning method in terms of robustness.
Multi-Knowledge Attention Transfer Framework for Action Recognition

Action recognition is a crucial task in computer vision. The Two-Stream Network and 3D ConvNets have achieved outstanding performance. However, due to the enormous amount of computation of optical flow and 3D convolution, they can not be effectively applied to some real-time applications. Therefore, some researchers have used motion vectors and residuals in the compressed video to replace optical flow, but their lack of fine structure leads to decreased model performance, such as noise and inaccurate motion blocks. In this paper, we propose a Multi-Knowledge Attention Transfer (MKAT) framework based on the three-stream network structure, including Multi-Knowledge Enhancement (MKE) module and Feature Loss Enhancement (FLE) module. The MKE module adopts the distillation methods of self-learning and multi-level information fusion, which allow the student network to learn from the decision-making and thinking aspects of the teacher. We use the attention enhancement (AE) module to process the output of each feature layer, so that the FLE module can highlight the differences between the key information of the feature layer. Experimental results on two public benchmarks (i.e., UCF-101, HMDB-51) significantly outperform the current state of the art on the compressed domain.
Relation-Aware Global-Augmented Transformer for TextCaps

Text-based image captioning (TextCaps) task aims to describe the given image reasonably based on scene text and visual objects simultaneously. Although previous works have shown great success, they pay too much attention to the text modality while ignoring other important visual information, and the correlations between objects and text are not fully exploited. Moreover, traditional transformer-based architectures ignore global information reflecting the entire image, which may cause object missing and erroneous reasoning problems. In this paper, we propose a Relation-aware Global-augmented Transformer (RGT) framework to tackle these problems. Specifically, we utilize a scene graph extracted from the image to explicitly model the relative semantic and spatial relationships of objects via a graph convolutional network, which not only enhances the visual representations but also encodes explicit semantic features of objects. Besides, we add a multi-modal alignment (MMA) module as a supplement for the multi-modal transformer to strengthen the association between scene text and objects. Finally, a global-augmented transformer (GAT) is designed to get a more comprehensive representation of the image, which could alleviate object missing and erroneous reasoning problems. Our method outperforms state-of-the-art models on the TextCaps dataset, improving from 105.0 to 107.2 in CIDEr.
Associative Memory Networks with Multidimensional Neurons

Neural networks normally used to model associative memory can be regarded as consisting of dissipative units (the neurons) that interact in such a way that the network itself admits a global energy or Liapunov function. The network’s global dynamics is such that the system always evolves “downhill” in the energy landscape. In most models for associative memory, the individual neurons are described as one-dimensional, dynamical systems. In the present contribution, we explore the possibility of extending the structural scheme of associative memory neural networks to more general scenarios, where the units (that is, the neurons) are modeled as multi-dimensional, dissipative systems. With that aim in mind, we advance a coupling scheme for dissipative, multi-dimensional units, that generates dynamical features akin to those required when modeling associative memory.
MLPPose: Human Keypoint Localization via MLP-Mixer

Although existing methods have made great progress in human pose estimation, there are still a lot of challenging situations not well-handled, such as occluded limbs, invisible body parts or complex scenarios. In this work, we propose a novel approach called MLPPose, which combining the MLP-Mixer layers with the convolutional token embedding for human pose estimation. The MLP-Mixer layers are consisted of two types of MLP blocks, one concerns the global receptive field and the other mixes the channel feature at each location. This composition can not only obtain the association between different keypoints, but also efficiently capture the global dependency relationships between keypoints and scenes. Thus, it allows our model to efficiently locate the keypoints, despite that some of them are occluded, invisible or in complex scenarios. Meanwhile, it is able to simplify the progress of extracting the global dependency relationships compared to the attentional mechanism which is widely used in transformer models. Experiments show that our model achieves competitive results with state-of-the-art methods on the MS-COCO and MPII human pose estimation benchmarks. Moreover, our model is more lightweight and faster than other best performance methods.
A Coarse-to-Fine Training Paradigm for Dialogue Summarization

Pre-trained language models (PLMs) have achieved promising results on dialogue summarization. Previous works mainly encode semantic features from wordy dialogues to help PLMs model dialogues, but extracting those features from the original dialogue text is costly. Besides, the resulting semantic features may be also redundant, which is harmful for PLMs to catch the dialogue’s main idea. Without searching for dispensable features, this paper proposes a coarse-to-fine training paradigm for dialogue summarization. Instead of directly fine-tuning PLMs to obtain complete summaries, this paradigm constructs a coarse-grained summarizer which automatically infers the key information to annotate each dialogue. Further, a fine-grained summarizer would generate detailed summaries based on the annotated dialogues. Moreover, to utilize the knowledge from out-of-domain pre-training, a meta learning mechanism is adopted, which could cooperate with our training paradigm and help the model pre-trained on other domains adapt to the dialogue summarization. Experimental results demonstrate that our method could outperform competitive baselines.
PlAA: Pixel-level Adversarial Attack on Attention for Deep Neural Network

Deep Neural Networks (DNNs) have demonstrated excellent performance in many fields. However, existing studies have shown that deep neural networks are very susceptible to well-designed adversarial samples. Adversarial samples cause the system to make incorrect classifications or predictions and may lead to security risks in the real world. Many adversarial attacks methods for making adversarial samples have been proposed. However, the excessive perturbation of most attack methods causes adversarial perturbations to be visible in human vision. In this paper, we proposed pixel-level adversarial attack on attention, named PlAA, which aims to attack a very few pixels in the attention area of DNNs to generate adversarial samples to achieve a high attack success rate and better hide adversarial disturbance. The experimental results show that in the single-pixel attack scenario, our PlAA attack method has an improvement up to 34.16% in the attack success rate compared with the existing one-pixel attack methods. In the multi-pixel attack scenario, compared with the existing attack on attention (AoA) methods, PlAA can better hide the adversarial disturbance while ensuring the same high attack success rate.
High-Performance and Robust Binarized Neural Network Accelerator Based on Modified Content-Addressable Memory

The binarized neural network (BNN) is one of the most promising candidates for low-cost convolutional neural networks (CNNs). This is because of its significant reduction in memory and computational costs, and reasonable classification accuracy. Content-addressable memory (CAM) can perform binarized convolution operations efficiently since the bitwise comparison in CAM matches well with the binarized multiply operation in a BNN. However, a significant design issue in CAM-based BNN accelerators is that the operational reliability is severely degraded by process variations during match-line (ML) sensing operations. In this paper, we proposed a novel ML sensing scheme to reduce the hardware error probability. Most errors occur when the difference between the number of matches in the evaluation ML and the reference ML is small; thus, the proposed hardware identified cases that are vulnerable to process variations using dual references. The proposed dual-reference sensing structure has &gt;49% less ML sensing errors than that of the conventional design, leading to a &gt;1.0% accuracy improvement for Fashion MNIST image classification. In addition, owing to the parallel convolution operation of the CAM-based BNN accelerator, the proposed hardware achieved &gt;34% processing-time improvement compared with that of the digital logic implementation.
Don’t Shoot the Message: Regulating Disinformation Beyond Content

This paper approaches regulatory strategies against disinformation with two main goals: (i) exploring the policies recently implemented in different legal contexts to provide insight into both the risks they pose to free speech and their potential to address the rationales that motivated them, and (ii) to do so by bridging policy debates and recent social and communications studies findings on disinformation. An interdisciplinary theoretical framework informs both the paper’s scope (anchored on understandings of regulatory strategies and disinformation) and the analysis of the legitimate motivations for states to establish statutory regulation aiming at disinformation. Departing from this analysis, I suggest an organisation of recently implemented and proposed policies into three groups according to their regulatory target: content, data, and structure. Combining the analysis of these three types of policies with the theoretical framework, I will argue that, in the realm of statutory regulation, state action is better off targeted at data or structure, as aiming at content represents disproportional risks to freedom of expression. Furthermore, content-targeted regulation shows little potential to address the structural transformations on the public sphere of communications that, among other factors, influence current production practices and disinformation spread.
A Formal Model of Aspect Shifting: The Case of Dot Diagrams

Many diagrams can be read in different ways, allowing readers to extract different kinds of information about the represented situation from the same representation. Such diagrams are often used to solve problems that require the combination of information from different views. This has been widely remarked upon in the literature and is sometimes referred to as aspect shifting. However, we know of no formal account of the phenomenon, or what an “aspect” of a diagram might be. In this paper, we give such an account. In an application of our previous work, we describe a theory of representation systems in which the same representation token can be interpreted within multiple distinct representation systems. Each gives a different view of the underlying domain, accounting for multiple readability. We also describe how these different representation systems can be combined into a single overarching system which allows inference across the different aspects given by the component systems. This accounts for why just apprehending a diagram can be a substantial inferential step.
Fake News and the 2018 Brazilian Presidential Election

This paper analyzes the phenomenon of the spread of false information in cyberspace (fake news or disinformation). Starting from the contextualization and conceptualization of fake news, an examination of the legal models currently existing in the European Union, Germany, and the United States is undertaken to deal with the phenomenon. Then, a critical assessment of the Brazilian Superior Electoral Court cases is made when judging the alleged disclosure of fake news in the 2018 Brazilian presidential campaign. Finally, the work concludes with assessing of the compatibility of the different models of combating disinformation with the exercise of freedom of expression by users of cyberspace.
Online Hate Speech and the Role of Digital Platforms: What Are the Prospects for Freedom of Expression?

In recent years, the subject of freedom of expression has been extended to two new phenomena: the web and hate speech. The former has to do with the extent (potentially infinite and uncontrolled) of freedom; the latter has to do with its limits and brings into play the fundamental principles of protection of the individual and respect for human dignity, as well as the principle of non-discrimination. The present essay addresses the controversial issue of the repression of hate speech by online platforms and the new role assigned to them, namely regulating users’ fundamental rights.
“Digital Democracy”: A Threat to the Democratic System or Oxygenation of Representative Democracy and Free Speech?

The so-called “digital democracy” has little to do with representative democracy or even with deliberative democracy, functioning rather, in liberal or libertarian logic, as a multipolar channel of free speech in pluralist societies, which develops without dependence on the filter media, predicting in the future, a growing impact on political action. The political use of social networks in cyberspace in democratic states can both oxygenate representative democracy, stimulating direct communication between politicians and citizens and improving public debate, as well as degrading its quality through hate speech, false news, and vexing messages. However, in the second case, network communications do not, by themselves, constitute a threat to the same democracy but rather a strain on its quality. Since cyberspace cannot be a “land without law”, surveillance of fake or extreme online discourse must be based on balanced rules that allow public and private entities to prevent and repress the incitement or practice of serious crimes but avoiding the use of direct or collateral censorship aimed at shaping a single thought pattern or an index of “cursed” political thoughts. In Europe, there is sometimes an exaggerated perception of virtual risks linked to communication in cyberspace, which means that in some countries, like Germany, the curbing of political debate operates through a “bureaucratic” private and collateral soft form of censorship imposed by public authorities on digital platforms, subsidizing the action of the courts, which constitutes a constitutionally problematic restriction on freedom of expression. In the United States, the platforms generated some sort of digital cartel banning or suspending, sometimes concertedly, citizens and social networks based on their deontological codes of behaviour without further control. In any case, in electoral periods, courts with electoral functions must have effective power to intervene, especially in the removal of messages that involve manifestly fake news with major social impact and also relevant manipulative communications in the elective process, namely those created artificially from abroad or that involve gross violation of citizen’s data protection rules.