OpenRaiser/Intern-Atlas huggingface dataset:
Detecting soccer balls with reduced neural networks: a comparison of multiple architectures under constrained hardware scenarios
Venue: arXiv cs.CV 2021
Year: 2021

Object detection techniques that achieve state-of-the-art detection accuracy employ convolutional neural networks, implemented to have optimal performance in graphics processing units. Some hardware systems, such as mobile robots, operate under constrained hardware situations, but still benefit from object detection capabilities. Multiple network models have been proposed, achieving comparable accuracy with reduced architectures and leaner operations. Motivated by the need to create an object detection system for a soccer team of mobile robots, this work provides a comparative study of recent proposals of neural networks targeted towards constrained hardware environments, in the specific task of soccer ball detection. We train multiple open implementations of MobileNetV2 and MobileNetV3 models with different underlying architectures, as well as YOLOv3, TinyYOLOv3, YOLOv4 and TinyYOLOv4 in an annotated image data set captured using a mobile robot. We then report their mean average precision on a test data set and their inference times in videos of different resolutions, under constrained and unconstrained hardware configurations. Results show that MobileNetV3 models have a good trade-off between mAP and inference time in constrained scenarios only, while MobileNetV2 with high width multipliers are appropriate for server-side inference. YOLO models in their official implementations are not suitable for inference in CPUs.
Grow-Push-Prune: aligning deep discriminants for effective structural network compression
Venue: arXiv cs.CV 2021
Year: 2021

Most of today's popular deep architectures are hand-engineered to be generalists. However, this design procedure usually leads to massive redundant, useless, or even harmful features for specific tasks. Unnecessarily high complexities render deep nets impractical for many real-world applications, especially those without powerful GPU support. In this paper, we attempt to derive task-dependent compact models from a deep discriminant analysis perspective. We propose an iterative and proactive approach for classification tasks which alternates between (1) a pushing step, with an objective to simultaneously maximize class separation, penalize co-variances, and push deep discriminants into alignment with a compact set of neurons, and (2) a pruning step, which discards less useful or even interfering neurons. Deconvolution is adopted to reverse 'unimportant' filters' effects and recover useful contributing sources. A simple network growing strategy based on the basic Inception module is proposed for challenging tasks requiring larger capacity than what the base net can offer. Experiments on the MNIST, CIFAR10, and ImageNet datasets demonstrate our approach's efficacy. On ImageNet, by pushing and pruning our grown Inception-88 model, we achieve more accurate models than Inception nets generated during growing, residual nets, and popular compact nets at similar sizes. We also show that our grown Inception nets (without hard-coded dimension alignment) clearly outperform residual nets of similar complexities.
Graph-based methods for analyzing orchard tree structure using noisy point cloud data
Venue: arXiv cs.CV 2021
Year: 2021

Digitisation of fruit trees using LiDAR enables analysis which can be used to better growing practices to improve yield. Sophisticated analysis requires geometric and semantic understanding of the data, including the ability to discern individual trees as well as identifying leafy and structural matter. Extraction of this information should be rapid, as should data capture, so that entire orchards can be processed, but existing methods for classification and segmentation rely on high-quality data or additional data sources like cameras. We present a method for analysis of LiDAR data specifically for individual tree location, segmentation and matter classification, which can operate on low-quality data captured by handheld or mobile LiDAR. Our methods for tree location and segmentation improved on existing methods with an F1 score of 0.774 and a v-measure of 0.915 respectively, while trunk matter classification performed poorly in absolute terms with an average F1 score of 0.490 on real data, though consistently outperformed existing methods and displayed a significantly shorter runtime.
Geometric Disentanglement by Random Convex Polytopes
Venue: arXiv cs.LG 2021
Year: 2021

We propose a new geometric method for measuring the quality of representations obtained from deep learning. Our approach, called Random Polytope Descriptor, provides an efficient description of data points based on the construction of random convex polytopes. We demonstrate the use of our technique by qualitatively comparing the behavior of classic and regularized autoencoders. This reveals that applying regularization to autoencoder networks may decrease the out-of-distribution detection performance in latent space. While our technique is similar in spirit to $k$-means clustering, we achieve significantly better false positive/negative balance in clustering tasks on autoencoded datasets.
User-item matching for recommendation fairness
Venue: arXiv cs.IR 2021
Year: 2021

As we all know, users and item-providers are two main parties of participants in recommender systems. However, most existing research efforts on recommendation were focused on better serving users and overlooked the purpose of item-providers. This paper is devoted to improve the item exposure fairness for item-providers' objective, and keep the recommendation accuracy not decreased or even improved for users' objective. We propose to set stock volume constraints on items, to be specific, limit the maximally allowable recommended times of an item to be proportional to the frequency of its being interacted in the past, which is validated to achieve superior item exposure fairness to common recommenders and thus mitigates the Matthew Effect on item popularity. With the two constraints of pre-existing recommendation length of users and our stock volumes of items, a heuristic strategy based on normalized scores and a Minimum Cost Maximum Flow (MCMF) based model are proposed to solve the optimal user-item matching problem, whose accuracy performances are even better than that of baseline algorithm in regular recommendation context, and in line with state-of-the-art enhancement of the baseline. What's more, our MCMF based strategy is parameter-free, while those counterpart algorithms have to resort to parameter traversal process to achieve their best performance.
Learning Hard Retrieval Decoder Attention for Transformers
Venue: arXiv cs.CL 2021
Year: 2021

The Transformer translation model is based on the multi-head attention mechanism, which can be parallelized easily. The multi-head attention network performs the scaled dot-product attention function in parallel, empowering the model by jointly attending to information from different representation subspaces at different positions. In this paper, we present an approach to learning a hard retrieval attention where an attention head only attends to one token in the sentence rather than all tokens. The matrix multiplication between attention probabilities and the value sequence in the standard scaled dot-product attention can thus be replaced by a simple and efficient retrieval operation. We show that our hard retrieval attention mechanism is 1.43 times faster in decoding, while preserving translation quality on a wide range of machine translation tasks when used in the decoder self- and cross-attention networks.
Timing Cache Accesses to Eliminate Side Channels in Shared Software
Venue: arXiv cs.CR 2021
Year: 2021

Timing side channels have been used to extract cryptographic keys and sensitive documents, even from trusted enclaves. In this paper, we focus on cache side channels created by access to shared code or data in the memory hierarchy. This vulnerability is exploited by several known attacks, e.g, evict+reload for recovering an RSA key and Spectre variants for data leaked due to speculative accesses. The key insight in this paper is the importance of the first access to the shared data after a victim brings the data into the cache. To eliminate the timing side channel, we ensure that the first access by a process to any cache line loaded by another process results in a miss. We accomplish this goal by using a combination of timestamps and a novel hardware design to allow efficient parallel comparisons of the timestamps. The solution works at all the cache levels and defends against an attacker process running on another core, same core, or another hyperthread. Our design retains the benefits of a shared cache: allowing processes to utilize the entire cache for their execution and retaining a single copy of shared code and data (data deduplication). Our implementation in the GEM5 simulator demonstrates that the system is able to defend against RSA key extraction. We evaluate performance using SPECCPU2006 and observe overhead due to first access delay to be 2.17%. The overhead due to the security context bookkeeping is of the order of 0.3%.
Computing Graph Neural Networks: A Survey from Algorithms to Accelerators
Venue: arXiv cs.LG 2021
Year: 2021

Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent years owing to their capability to model and learn from graph-structured data. Such an ability has strong implications in a wide variety of fields whose data is inherently relational, for which conventional neural networks do not perform well. Indeed, as recent reviews can attest, research in the area of GNNs has grown rapidly and has lead to the development of a variety of GNN algorithm variants as well as to the exploration of groundbreaking applications in chemistry, neurology, electronics, or communication networks, among others. At the current stage of research, however, the efficient processing of GNNs is still an open challenge for several reasons. Besides of their novelty, GNNs are hard to compute due to their dependence on the input graph, their combination of dense and very sparse operations, or the need to scale to huge graphs in some applications. In this context, this paper aims to make two main contributions. On the one hand, a review of the field of GNNs is presented from the perspective of computing. This includes a brief tutorial on the GNN fundamentals, an overview of the evolution of the field in the last decade, and a summary of operations carried out in the multiple phases of different GNN algorithm variants. On the other hand, an in-depth analysis of current software and hardware acceleration schemes is provided, from which a hardware-software, graph-aware, and communication-centric vision for GNN accelerators is distilled.
Who Are the `Silent Spreaders'?: Contact Tracing in Spatio-Temporal Memory Models
Venue: arXiv cs.SI 2021
Year: 2021

The COVID-19 epidemic has swept the world for over a year. However, a large number of infectious asymptomatic COVID-19 cases (\textit{ACC}s) are still making the breaking up of the transmission chains very difficult. Efforts by epidemiological researchers in many countries have thrown light on the clinical features of ACCs, but there is still a lack of practical approaches to detect ACCs so as to help contain the pandemic. To address the issue of ACCs, this paper presents a neural network model called Spatio-Temporal Episodic Memory for COVID-19 (\textit{STEM-COVID}) to identify ACCs from contact tracing data. Based on the fusion Adaptive Resonance Theory (\textit{ART}), the model encodes a collective spatio-temporal episodic memory of individuals and incorporates an effective mechanism of parallel searches for ACCs. Specifically, the episodic traces of the identified positive cases are used to map out the episodic traces of suspected ACCs using a weighted evidence pooling method. To evaluate the efficacy of STEM-COVID, a realistic agent based simulation model for COVID-19 spreading is implemented based on the recent epidemiological findings on ACCs. The experiments based on rigorous simulation scenarios, manifesting the current situation of COVID-19 spread, show that the STEM-COVID model with weighted evidence pooling has a higher level of accuracy and efficiency for identifying ACCs when compared with several baselines. Moreover, the model displays strong robustness against noisy data and different ACC proportions, which partially reflects the effect of breakthrough infections after vaccination on the virus transmission.
Phase retrieval with Bregman divergences and application to audio signal recovery
Venue: arXiv cs.SD 2021
Year: 2021

Phase retrieval (PR) aims to recover a signal from the magnitudes of a set of inner products. This problem arises in many audio signal processing applications which operate on a short-time Fourier transform magnitude or power spectrogram, and discard the phase information. Recovering the missing phase from the resulting modified spectrogram is indeed necessary in order to synthesize time-domain signals. PR is commonly addressed by considering a minimization problem involving a quadratic loss function. In this paper, we adopt a different standpoint. Indeed, the quadratic loss does not properly account for some perceptual properties of audio, and alternative discrepancy measures such as beta-divergences have been preferred in many settings. Therefore, we formulate PR as a new minimization problem involving Bregman divergences. Since these divergences are not symmetric with respect to their two input arguments in general, they lead to two different formulations of the problem. To optimize the resulting objective, we derive two algorithms based on accelerated gradient descent and alternating direction method of multipliers. Experiments conducted on audio signal recovery from spectrograms that are either exact or estimated from noisy observations highlight the potential of our proposed methods for audio restoration. In particular, leveraging some of these Bregman divergences induce better performance than the quadratic loss when performing PR from spectrograms under very noisy conditions.
Understanding Self-supervised Learning with Dual Deep Networks
Venue: arXiv cs.LG 2021
Year: 2021

We propose a novel theoretical framework to understand contrastive self-supervised learning (SSL) methods that employ dual pairs of deep ReLU networks (e.g., SimCLR). First, we prove that in each SGD update of SimCLR with various loss functions, including simple contrastive loss, soft Triplet loss and InfoNCE loss, the weights at each layer are updated by a \emph{covariance operator} that specifically amplifies initial random selectivities that vary across data samples but survive averages over data augmentations. To further study what role the covariance operator plays and which features are learned in such a process, we model data generation and augmentation processes through a \emph{hierarchical latent tree model} (HLTM) and prove that the hidden neurons of deep ReLU networks can learn the latent variables in HLTM, despite the fact that the network receives \emph{no direct supervision} from these unobserved latent variables. This leads to a provable emergence of hierarchical features through the amplification of initially random selectivities through contrastive SSL. Extensive numerical studies justify our theoretical findings. Code is released in https://github.com/facebookresearch/luckmatters/tree/master/ssl.
Enhancing Fine-grained Sentiment Classification Exploiting Local Context Embedding
Venue: arXiv cs.CL 2021
Year: 2021

Target-oriented sentiment classification is a fine-grained task of natural language processing to analyze the sentiment polarity of the targets. To improve the performance of sentiment classification, many approaches proposed various attention mechanisms to capture the important context words of a target. However, previous approaches ignored the significant relatedness of a target's sentiment and its local context. This paper proposes a local context-aware network (LCA-Net), equipped with the local context embedding and local context prediction loss, to strengthen the model by emphasizing the sentiment information of the local context. The experimental results on three common datasets show that local context-aware network performs superior to existing approaches in extracting local context features. Besides, the local context-aware framework is easy to adapt to many models, with the potential to improve other target-level tasks.
Proof Repair across Type Equivalences
Venue: arXiv cs.PL 2021
Year: 2021

We describe a new approach to automatically repairing broken proofs in the Coq proof assistant in response to changes in types. Our approach combines a configurable proof term transformation with a decompiler from proof terms to tactic scripts. The proof term transformation implements transport across equivalences in a way that removes references to the old version of the changed type and does not rely on axioms beyond those Coq assumes. We have implemented this approach in PUMPKIN Pi, an extension to the PUMPKIN PATCH Coq plugin suite for proof repair. We demonstrate PUMPKIN Pi's flexibility on eight case studies, including supporting a benchmark from a user study, easing development with dependent types, porting functions and proofs between unary and binary numbers, and supporting an industrial proof engineer to interoperate between Coq and other verification tools more easily.
Tight Approximation Guarantees for Concave Coverage Problems
Venue: arXiv cs.DS 2021
Year: 2021

In the maximum coverage problem, we are given subsets $T_1, \ldots, T_m$ of a universe $[n]$ along with an integer $k$ and the objective is to find a subset $S \subseteq [m]$ of size $k$ that maximizes $C(S) := \Big|\bigcup_{i \in S} T_i\Big|$. It is a classic result that the greedy algorithm for this problem achieves an optimal approximation ratio of $1-e^{-1}$. In this work we consider a generalization of this problem wherein an element $a$ can contribute by an amount that depends on the number of times it is covered. Given a concave, nondecreasing function $φ$, we define $C^φ(S) := \sum_{a \in [n]}w_aφ(|S|_a)$, where $|S|_a = |\{i \in S : a \in T_i\}|$. The standard maximum coverage problem corresponds to taking $φ(j) = \min\{j,1\}$. For any such $φ$, we provide an efficient algorithm that achieves an approximation ratio equal to the Poisson concavity ratio of $φ$, defined by $α_φ := \min_{x \in \mathbb{N}^*} \frac{\mathbb{E}[φ(\text{Poi}(x))]}{φ(\mathbb{E}[\text{Poi}(x)])}$. Complementing this approximation guarantee, we establish a matching NP-hardness result when $φ$ grows in a sublinear way. As special cases, we improve the result of [Barman et al., IPCO, 2020] about maximum multi-coverage, that was based on the unique games conjecture, and we recover the result of [Dudycz et al., IJCAI, 2020] on multi-winner approval-based voting for geometrically dominant rules. Our result goes beyond these special cases and we illustrate it with applications to distributed resource allocation problems, welfare maximization problems and approval-based voting for general rules.
$X$-Secure $T$-Private Federated Submodel Learning with Elastic Dropout Resilience
Venue: arXiv cs.IT 2021
Year: 2021

Motivated by recent interest in federated submodel learning, this work explores the fundamental problem of privately reading from and writing to a database comprised of $K$ files (submodels) that are stored across $N$ distributed servers according to an $X$-secure threshold secret sharing scheme. One after another, various users wish to retrieve their desired file, locally process the information and then update the file in the distributed database while keeping the identity of their desired file private from any set of up to $T$ colluding servers. The availability of servers changes over time, so elastic dropout resilience is required. The main contribution of this work is an adaptive scheme, called ACSA-RW, that takes advantage of all currently available servers to reduce its communication costs, fully updates the database after each write operation even though the database is only partially accessible due to server dropouts, and ensures a memoryless operation of the network in the sense that the storage structure is preserved and future users may remain oblivious of the past history of server dropouts. The ACSA-RW construction builds upon CSA codes that were originally introduced for XSTPIR and have been shown to be natural solutions for secure distributed matrix multiplication problems. ACSA-RW achieves the desired private read and write functionality with elastic dropout resilience, matches the best results for private-read from PIR literature, improves significantly upon available baselines for private-write, reveals a striking symmetry between upload and download costs, and exploits redundant storage dimensions to accommodate arbitrary read and write dropout servers up to certain threshold values. It also answers in the affirmative an open question by Kairouz et al. by exploiting synergistic gains from the joint design of private read and write operations.
Hierarchical Domain-Adapted Feature Learning for Video Saliency Prediction
Venue: arXiv cs.CV 2021
Year: 2021

In this work, we propose a 3D fully convolutional architecture for video saliency prediction that employs hierarchical supervision on intermediate maps (referred to as conspicuity maps) generated using features extracted at different abstraction levels. We provide the base hierarchical learning mechanism with two techniques for domain adaptation and domain-specific learning. For the former, we encourage the model to unsupervisedly learn hierarchical general features using gradient reversal at multiple scales, to enhance generalization capabilities on datasets for which no annotations are provided during training. As for domain specialization, we employ domain-specific operations (namely, priors, smoothing and batch normalization) by specializing the learned features on individual datasets in order to maximize performance. The results of our experiments show that the proposed model yields state-of-the-art accuracy on supervised saliency prediction. When the base hierarchical model is empowered with domain-specific modules, performance improves, outperforming state-of-the-art models on three out of five metrics on the DHF1K benchmark and reaching the second-best results on the other two. When, instead, we test it in an unsupervised domain adaptation setting, by enabling hierarchical gradient reversal layers, we obtain performance comparable to supervised state-of-the-art.