OpenRaiser/Intern-Atlas huggingface dataset:
Data Leakage in Federated Averaging
Venue: arXiv cs.LG 2022
Year: 2022

Recent attacks have shown that user data can be recovered from FedSGD updates, thus breaking privacy. However, these attacks are of limited practical relevance as federated learning typically uses the FedAvg algorithm. Compared to FedSGD, recovering data from FedAvg updates is much harder as: (i) the updates are computed at unobserved intermediate network weights, (ii) a large number of batches are used, and (iii) labels and network weights vary simultaneously across client steps. In this work, we propose a new optimization-based attack which successfully attacks FedAvg by addressing the above challenges. First, we solve the optimization problem using automatic differentiation that forces a simulation of the client's update that generates the unobserved parameters for the recovered labels and inputs to match the received client update. Second, we address the large number of batches by relating images from different epochs with a permutation invariant prior. Third, we recover the labels by estimating the parameters of existing FedSGD attacks at every FedAvg step. On the popular FEMNIST dataset, we demonstrate that on average we successfully recover >45% of the client's images from realistic FedAvg updates computed on 10 local epochs of 10 batches each with 5 images, compared to only <10% using the baseline. Our findings show many real-world federated learning implementations based on FedAvg are vulnerable.
Joint Representation Training in Sequential Tasks with Shared Structure
Venue: arXiv cs.LG 2022
Year: 2022

Classical theory in reinforcement learning (RL) predominantly focuses on the single task setting, where an agent learns to solve a task through trial-and-error experience, given access to data only from that task. However, many recent empirical works have demonstrated the significant practical benefits of leveraging a joint representation trained across multiple, related tasks. In this work we theoretically analyze such a setting, formalizing the concept of task relatedness as a shared state-action representation that admits linear dynamics in all the tasks. We introduce the Shared-MatrixRL algorithm for the setting of Multitask MatrixRL. In the presence of $P$ episodic tasks of dimension $d$ sharing a joint $r \ll d$ low-dimensional representation, we show the regret on the the $P$ tasks can be improved from $O(PHd\sqrt{NH})$ to $O((Hd\sqrt{rP} + HP\sqrt{rd})\sqrt{NH})$ over $N$ episodes of horizon $H$. These gains coincide with those observed in other linear models in contextual bandits and RL. In contrast with previous work that have studied multi task RL in other function approximation models, we show that in the presence of bilinear optimization oracle and finite state action spaces there exists a computationally efficient algorithm for multitask MatrixRL via a reduction to quadratic programming. We also develop a simple technique to shave off a $\sqrt{H}$ factor from the regret upper bounds of some episodic linear problems.
Vibration fault detection in wind turbines based on normal behaviour models without feature engineering
Venue: arXiv cs.LG 2022
Year: 2022

Most wind turbines are remotely monitored 24/7 to allow for an early detection of operation problems and developing damage. We present a new fault detection method for vibration-monitored drivetrains that does not require any feature engineering. Our method relies on a simple model architecture to enable a straightforward implementation in practice. We propose to apply convolutional autoencoders for identifying and extracting the most relevant features from the half spectrum in an automated manner, saving time and effort. Thereby, a spectral model of the normal vibration response is learnt for the monitored component from past measurements. We demonstrate that the model can successfully distinguish damaged from healthy components and detect a damaged generator bearing and damaged gearbox parts from their vibration responses. Using measurements from commercial wind turbines and a test rig, we show that vibration-based fault detection in wind turbine drivetrains can be performed without the usual upfront definition of spectral features. Another advantage of the presented method is that the entire half spectrum is monitored instead of the usual focus on monitoring individual frequencies and harmonics.
Arcadia: A Fast and Reliable Persistent Memory Replicated Log
Venue: arXiv cs.DC 2022
Year: 2022

The performance properties of byte-addressable persistent memory (PMEM) have the potential to significantly improve system performance over a wide spectrum of applications. But persistent memory brings considerable new challenges to the programmer: only 8-byte write atomicity, out of order flush and availability limited by node failure. It's possible to work with the atomicity and ordering constraints of PMEM directly by carefully sequencing the order of store operations and inserting explicit flush and fence operations at each ordering point. But this is tedious and error-prone: too many flush operations defeat the performance benefits of PMEM, and even with generous use, it is difficult to prove that a given program is crash-consistent. Logging is a great abstraction to deal with these issues but prior work on PMEM logging has not successfully hidden the idiosyncrasies of PMEM. Moreover, shortcomings in the log interface and design have prevented attainment of full PMEM performance. We believe that a log design that hides the idiosyncrasies from programmers while delivering full performance is key to success. In this paper, we present the design and implementation of Arcadia, a generic replicated log on PMEM to address these problems. Arcadia handles atomicity, integrity, and replication of log records to reduce programmer burden. Our design has several novel aspects including concurrent log writes with in-order commit, atomicity and integrity primitives for local and remote PMEM writes, and a frequency-based log force policy for providing low overhead persistence with guaranteed bounded loss of uncommitted records. Our evaluation shows that Arcadia outperforms state-of-the-art PMEM logs, such as PMDK's libpmemlog, FLEX, and Query Fresh by several times while providing stronger log record durability guarantees. We expect Arcadia to become the leading off-the-shelf PMEM log design.
Performance Analysis of Joint Active User Detection and Channel Estimation for Massive Connectivity
Venue: arXiv cs.IT 2022
Year: 2022

This paper considers joint active user detection (AUD) and channel estimation (CE) for massive connectivity scenarios with sporadic traffic. The state-of-art method under a Bayesian framework to perform joint AUD and CE in such scenarios is approximate message passing (AMP). However, the existing theoretical analysis of AMP-based joint AUD and CE can only be performed with a given fixed point of the AMP state evolution function, lacking the analysis of AMP phase transition and Bayes-optimality. In this paper, we propose a novel theoretical framework to analyze the performance of the joint AUD and CE problem by adopting the replica method in the Bayes-optimal condition. Specifically, our analysis is based on a general channel model, which reduces to particular channel models in multiple typical MIMO communication scenarios. Our theoretical framework allows ones to measure the optimality and phase transition of AMP-based joint AUD and CE as well as to predict the corresponding performance metrics under our model. To reify our proposed theoretical framework, we analyze two typical scenarios from the massive random access literature, i.e., the isotropic channel scenario and the spatially correlated channel scenario. Accordingly, our performance analysis produces some novel results for both the isotropic Raleigh channel and spatially correlated channel case.
Single Threshold Packet Scheduling Policy for AoI Minimization in Resource-Constrained Network
Venue: arXiv cs.IT 2022
Year: 2022

This paper investigates the tradeoff between the average age of information (AoI) and the transmission cost for networks with stochastic packet arrival and random erasure channel. Specifically, we model the resource-constrained AoI minimization problem as a constrained Markov decision process (CMDP) and propose a low-complexity single threshold packet scheduling policy for it. The key advantage of the proposed policy is its tractability and convenience for implementation. The AoI distribution and long-term average transmission cost of the proposed policy are derived as closed-form functions of the selected threshold. Furthermore, we show that the proposed policy reduces to the optimal policies under special settings and achieves close-to-optimal performance under general settings.
Generating Diverse Vocal Bursts with StyleGAN2 and MEL-Spectrograms
Venue: arXiv cs.SD 2022
Year: 2022

We describe our approach for the generative emotional vocal burst task (ExVo Generate) of the ICML Expressive Vocalizations Competition. We train a conditional StyleGAN2 architecture on mel-spectrograms of preprocessed versions of the audio samples. The mel-spectrograms generated by the model are then inverted back to the audio domain. As a result, our generated samples substantially improve upon the baseline provided by the competition from a qualitative and quantitative perspective for all emotions. More precisely, even for our worst-performing emotion (awe), we obtain an FAD of 1.76 compared to the baseline of 4.81 (as a reference, the FAD between the train/validation sets for awe is 0.776).
Construct a Sentence with Multiple Specified Words
Venue: arXiv cs.CL 2022
Year: 2022

This paper demonstrates a task to finetune a BART model so it can construct a sentence from an arbitrary set of words, which used to be a difficult NLP task. The training task is making sentences with four words, but the trained model can generate sentences when fewer or more words are provided. The output sentences have high quality in general. The model can have some real-world applications, and this task can be used as an evaluation mechanism for any language model as well.
CV 3315 Is All You Need : Semantic Segmentation Competition
Venue: arXiv cs.CV 2022
Year: 2022

This competition focus on Urban-Sense Segmentation based on the vehicle camera view. Class highly unbalanced Urban-Sense images dataset challenge the existing solutions and further studies. Deep Conventional neural network-based semantic segmentation methods such as encoder-decoder architecture and multi-scale and pyramid-based approaches become flexible solutions applicable to real-world applications. In this competition, we mainly review the literature and conduct experiments on transformer-driven methods especially SegFormer, to achieve an optimal trade-off between performance and efficiency. For example, SegFormer-B0 achieved 74.6% mIoU with the smallest FLOPS, 15.6G, and the largest model, SegFormer- B5 archived 80.2% mIoU. According to multiple factors, including individual case failure analysis, individual class performance, training pressure and efficiency estimation, the final candidate model for the competition is SegFormer- B2 with 50.6 GFLOPS and 78.5% mIoU evaluated on the testing set. Checkout our code implementation at https://vmv.re/cv3315.
Inverted Semantic-Index for Image Retrieval
Venue: arXiv cs.CV 2022
Year: 2022

This paper addresses the construction of inverted index for large-scale image retrieval. The inverted index proposed by J. Sivic brings a significant acceleration by reducing distance computations with only a small fraction of the database. The state-of-the-art inverted indices aim to build finer partitions that produce a concise and accurate candidate list. However, partitioning in these frameworks is generally achieved by unsupervised clustering methods which ignore the semantic information of images. In this paper, we replace the clustering method with image classification, during the construction of codebook. We then propose a merging and splitting method to solve the problem that the number of partitions is unchangeable in the inverted semantic-index. Next, we combine our semantic-index with the product quantization (PQ) so as to alleviate the accuracy loss caused by PQ compression. Finally, we evaluate our model on large-scale image retrieval benchmarks. Experiment results demonstrate that our model can significantly improve the retrieval accuracy by generating high-quality candidate lists.
FreSCo: Frequency-Domain Scan Context for LiDAR-based Place Recognition with Translation and Rotation Invariance
Venue: arXiv cs.RO 2022
Year: 2022

Place recognition plays a crucial role in re-localization and loop closure detection tasks for robots and vehicles. This paper seeks a well-defined global descriptor for LiDAR-based place recognition. Compared to local descriptors, global descriptors show remarkable performance in urban road scenes but are usually viewpoint-dependent. To this end, we propose a simple yet robust global descriptor dubbed FreSCo that decomposes the viewpoint difference during revisit and achieves both translation and rotation invariance by leveraging Fourier Transform and circular shift technique. Besides, a fast two-stage pose estimation method is proposed to estimate the relative pose after place retrieval by utilizing the compact 2D point clouds extracted from the original data. Experiments show that FreSCo exhibited superior performance than contemporaneous methods on sequences of different scenes from multiple datasets. Code will be publicly available at https://github.com/soytony/FreSCo.
Sentiment Analysis with R: Natural Language Processing for Semi-Automated Assessments of Qualitative Data
Venue: arXiv cs.CL 2022
Year: 2022

Sentiment analysis is a sub-discipline in the field of natural language processing and computational linguistics and can be used for automated or semi-automated analyses of text documents. One of the aims of these analyses is to recognize an expressed attitude as positive or negative as it can be contained in comments on social media platforms or political documents and speeches as well as fictional and nonfictional texts. Regarding analyses of comments on social media platforms, this is an extension of the previous tutorial on semi-automated screenings of social media network data. A longitudinal perspective regarding social media comments as well as cross-sectional perspectives regarding fictional and nonfictional texts, e.g. entire books and libraries, can lead to extensive text documents. Their analyses can be simplified and accelerated by using sentiment analysis with acceptable inter-rater reliability. Therefore, this tutorial introduces the basic functions for performing a sentiment analysis with R and explains how text documents can be analysed step by step - regardless of their underlying formatting. All prerequisites and steps are described in detail and associated codes are available on GitHub. A comparison of two political speeches illustrates a possible use case.
BRL/Pisa/IIT SoftHand: A Low-cost, 3D-Printed, Underactuated, Tendon-Driven Hand with Soft and Adaptive Synergies
Venue: arXiv cs.RO 2022
Year: 2022

This paper introduces the BRL/Pisa/IIT (BPI) SoftHand: a single actuator-driven, low-cost, 3D-printed, tendon-driven, underactuated robot hand that can be used to perform a range of grasping tasks. Based on the adaptive synergies of the Pisa/IIT SoftHand, we design a new joint system and tendon routing to facilitate the inclusion of both soft and adaptive synergies, which helps us balance durability, affordability and grasping performance of the hand. The focus of this work is on the design, simulation, synergies and grasping tests of this SoftHand. The novel phalanges are designed and printed based on linkages, gear pairs and geometric restraint mechanisms, and can be applied to most tendon-driven robotic hands. We show that the robot hand can successfully grasp and lift various target objects and adapt to hold complex geometric shapes, reflecting the successful adoption of the soft and adaptive synergies. We intend to open-source the design of the hand so that it can be built cheaply on a home 3D-printer. For more detail: https://sites.google.com/view/bpi-softhandtactile-group-bri/brlpisaiit-softhand-design
TEVR: Improving Speech Recognition by Token Entropy Variance Reduction
Venue: arXiv cs.CL 2022
Year: 2022

This paper presents TEVR, a speech recognition model designed to minimize the variation in token entropy w.r.t. to the language model. This takes advantage of the fact that if the language model will reliably and accurately predict a token anyway, then the acoustic model doesn't need to be accurate in recognizing it. We train German ASR models with 900 million parameters and show that on CommonVoice German, TEVR scores a very competitive 3.64% word error rate, which outperforms the best reported results by a relative 16.89% reduction in word error rate. We hope that releasing our fully trained speech recognition pipeline to the community will lead to privacy-preserving offline virtual assistants in the future.
The Bandwagon Effect: Not Just Another Bias
Venue: arXiv cs.IR 2022
Year: 2022

Optimizing recommender systems based on user interaction data is mainly seen as a problem of dealing with selection bias, where most existing work assumes that interactions from different users are independent. However, it has been shown that in reality user feedback is often influenced by earlier interactions of other users, e.g. via average ratings, number of views or sales per item, etc. This phenomenon is known as the bandwagon effect. In contrast with previous literature, we argue that the bandwagon effect should not be seen as a problem of statistical bias. In fact, we prove that this effect leaves both individual interactions and their sample mean unbiased. Nevertheless, we show that it can make estimators inconsistent, introducing a distinct set of problems for convergence in relevance estimation. Our theoretical analysis investigates the conditions under which the bandwagon effect poses a consistency problem and explores several approaches for mitigating these issues. This work aims to show that the bandwagon effect poses an underinvestigated open problem that is fundamentally distinct from the well-studied selection bias in recommendation.
Deploying Robots in Everyday Environments: Towards Dependable and Practical Robotic Systems
Venue: arXiv cs.RO 2022
Year: 2022

Robot deployment in realistic dynamic environments is a challenging problem despite the fact that robots can be quite skilled at a large number of isolated tasks. One reason for this is that robots are rarely equipped with powerful introspection capabilities, which means that they cannot always deal with failures in a reasonable manner; in addition, manual diagnosis is often a tedious task that requires technicians to have a considerable set of robotics skills. In this paper, we discuss our ongoing efforts - in the context of the ROPOD project - to address some of these problems. In particular, we (i) present our early efforts at developing a robotic black box and consider some factors that complicate its design, (ii) explain our component and system monitoring concept, and (iii) describe the necessity for remote monitoring and experimentation as well as our initial attempts at performing those. Our preliminary work opens a range of promising directions for making robots more usable and reliable in practice - not only in the context of ROPOD, but in a more general sense as well.
On how to avoid exacerbating spurious correlations when models are overparameterized
Venue: arXiv cs.LG 2022
Year: 2022

Overparameterized models fail to generalize well in the presence of data imbalance even when combined with traditional techniques for mitigating imbalances. This paper focuses on imbalanced classification datasets, in which a small subset of the population -- a minority -- may contain features that correlate spuriously with the class label. For a parametric family of cross-entropy loss modifications and a representative Gaussian mixture model, we derive non-asymptotic generalization bounds on the worst-group error that shed light on the role of different hyper-parameters. Specifically, we prove that, when appropriately tuned, the recently proposed VS-loss learns a model that is fair towards minorities even when spurious features are strong. On the other hand, alternative heuristics, such as the weighted CE and the LA-loss, can fail dramatically. Compared to previous works, our bounds hold for more general models, they are non-asymptotic, and, they apply even at scenarios of extreme imbalance.
Implementation of SquashFS Support in U-Boot
Venue: arXiv cs.OS 2022
Year: 2022

U-Boot is a notorious bootloader and Open Source project. This work had as objective adding support for the SquashFS filesystem to U-Boot and the support developed was submitted as a contribution to the project. The bootloader is responsible, in this context, for loading the kernel and the device tree blob into RAM. It needs to be capable of reading a storage device's partition formatted with a specific filesystem type. Adding this support allows U-Boot to read from SquashFS partitions. The source code was submitted to U-Boot's mailing list through a series of patches to be reviewed by one of the project's maintainer. Once it gets merged, the support will be used and modified by U-Boot's international community.
Exploiting Transformation Invariance and Equivariance for Self-supervised Sound Localisation
Venue: arXiv cs.CV 2022
Year: 2022

We present a simple yet effective self-supervised framework for audio-visual representation learning, to localize the sound source in videos. To understand what enables to learn useful representations, we systematically investigate the effects of data augmentations, and reveal that (1) composition of data augmentations plays a critical role, i.e. explicitly encouraging the audio-visual representations to be invariant to various transformations~({\em transformation invariance}); (2) enforcing geometric consistency substantially improves the quality of learned representations, i.e. the detected sound source should follow the same transformation applied on input video frames~({\em transformation equivariance}). Extensive experiments demonstrate that our model significantly outperforms previous methods on two sound localization benchmarks, namely, Flickr-SoundNet and VGG-Sound. Additionally, we also evaluate audio retrieval and cross-modal retrieval tasks. In both cases, our self-supervised models demonstrate superior retrieval performances, even competitive with the supervised approach in audio retrieval. This reveals the proposed framework learns strong multi-modal representations that are beneficial to sound localisation and generalization to further applications. \textit{All codes will be available}.