OpenRaiser/Intern-Atlas huggingface dataset:
EEG-based Emotion Recognition with Spatial and Functional Brain Mapping of CNS and PNS Signals
Venue: arXiv cs.MM 2022
Year: 2022

Emotion plays a significant role in our daily life. Recognition of emotion is wide-spread in the field of health care and human-computer interaction. Emotion is the result of the coordinated activities of cortical and subcortical neural processes, which correlate to specific physiological responses. However, the existing emotion recognition techniques failed to combine various physiological signals as one integrated feature representation. Meanwhile, many researchers ignored the problem of over-fitting model with high accuracy, which was actually false high accuracy caused by improper pre-processing. In this paper, sigmoid baseline filtering is conducted to solve the over-fitting problem from source. To construct a physiological-based algorithm, a 3D spatial and functional brain mapping is proposed based on human physiological mechanism and international electrode system, which combines the signals of the central and peripheral nervous system together. By combining the baseline filtering, 3D brain mapping, and simple 4D-CNN, a novel emotion recognition model is finally proposed. Experiment results demonstrate that the performance of the proposed model is comparable to the state of art algorithms.
Improving the Diagnosis of Psychiatric Disorders with Self-Supervised Graph State Space Models
Venue: arXiv cs.LG 2022
Year: 2022

Single subject prediction of brain disorders from neuroimaging data has gained increasing attention in recent years. Yet, for some heterogeneous disorders such as major depression disorder (MDD) and autism spectrum disorder (ASD), the performance of prediction models on large-scale multi-site datasets remains poor. We present a two-stage framework to improve the diagnosis of heterogeneous psychiatric disorders from resting-state functional magnetic resonance imaging (rs-fMRI). First, we propose a self-supervised mask prediction task on data from healthy individuals that can exploit differences between healthy controls and patients in clinical datasets. Next, we train a supervised classifier on the learned discriminative representations. To model rs-fMRI data, we develop Graph-S4; an extension to the recently proposed state-space model S4 to graph settings where the underlying graph structure is not known in advance. We show that combining the framework and Graph-S4 can significantly improve the diagnostic performance of neuroimaging-based single subject prediction models of MDD and ASD on three open-source multi-center rs-fMRI clinical datasets.
Assessing Project-Level Fine-Tuning of ML4SE Models
Venue: arXiv cs.SE 2022
Year: 2022

Machine Learning for Software Engineering (ML4SE) is an actively growing research area that focuses on methods that help programmers in their work. In order to apply the developed methods in practice, they need to achieve reasonable quality in order to help rather than distract developers. While the development of new approaches to code representation and data collection improves the overall quality of the models, it does not take into account the information that we can get from the project at hand. In this work, we investigate how the model's quality can be improved if we target a specific project. We develop a framework to assess quality improvements that models can get after fine-tuning for the method name prediction task on a particular project. We evaluate three models of different complexity and compare their quality in three settings: trained on a large dataset of Java projects, further fine-tuned on the data from a particular project, and trained from scratch on this data. We show that per-project fine-tuning can greatly improve the models' quality as they capture the project's domain and naming conventions. We open-source the tool we used for data collection, as well as the code to run the experiments: https://zenodo.org/record/6040745.
Specification-Guided Learning of Nash Equilibria with High Social Welfare
Venue: arXiv cs.GT 2022
Year: 2022

Reinforcement learning has been shown to be an effective strategy for automatically training policies for challenging control problems. Focusing on non-cooperative multi-agent systems, we propose a novel reinforcement learning framework for training joint policies that form a Nash equilibrium. In our approach, rather than providing low-level reward functions, the user provides high-level specifications that encode the objective of each agent. Then, guided by the structure of the specifications, our algorithm searches over policies to identify one that provably forms an $ε$-Nash equilibrium (with high probability). Importantly, it prioritizes policies in a way that maximizes social welfare across all agents. Our empirical evaluation demonstrates that our algorithm computes equilibrium policies with high social welfare, whereas state-of-the-art baselines either fail to compute Nash equilibria or compute ones with comparatively lower social welfare.
Revealing Single Frame Bias for Video-and-Language Learning
Venue: arXiv cs.CV 2022
Year: 2022

Training an effective video-and-language model intuitively requires multiple frames as model inputs. However, it is unclear whether using multiple frames is beneficial to downstream tasks, and if yes, whether the performance gain is worth the drastically-increased computation and memory costs resulting from using more frames. In this work, we explore single-frame models for video-and-language learning. On a diverse set of video-and-language tasks (including text-to-video retrieval and video question answering), we show the surprising result that, with large-scale pre-training and a proper frame ensemble strategy at inference time, a single-frame trained model that does not consider temporal information can achieve better performance than existing methods that use multiple frames for training. This result reveals the existence of a strong "static appearance bias" in popular video-and-language datasets. Therefore, to allow for a more comprehensive evaluation of video-and-language models, we propose two new retrieval tasks based on existing fine-grained action recognition datasets that encourage temporal modeling. Our code is available at https://github.com/jayleicn/singularity
Discrete State-Action Abstraction via the Successor Representation
Venue: arXiv cs.AI 2022
Year: 2022

While the difficulty of reinforcement learning problems is typically related to the complexity of their state spaces, Abstraction proposes that solutions often lie in simpler underlying latent spaces. Prior works have focused on learning either a continuous or dense abstraction, or require a human to provide one. Information-dense representations capture features irrelevant for solving tasks, and continuous spaces can struggle to represent discrete objects. In this work we automatically learn a sparse discrete abstraction of the underlying environment. We do so using a simple end-to-end trainable model based on the successor representation and max-entropy regularization. We describe an algorithm to apply our model, named Discrete State-Action Abstraction (DSAA), which computes an action abstraction in the form of temporally extended actions, i.e., Options, to transition between discrete abstract states. Empirically, we demonstrate the effects of different exploration schemes on our resulting abstraction, and show that it is efficient for solving downstream tasks.
SHRED: 3D Shape Region Decomposition with Learned Local Operations
Venue: arXiv cs.CV 2022
Year: 2022

We present SHRED, a method for 3D SHape REgion Decomposition. SHRED takes a 3D point cloud as input and uses learned local operations to produce a segmentation that approximates fine-grained part instances. We endow SHRED with three decomposition operations: splitting regions, fixing the boundaries between regions, and merging regions together. Modules are trained independently and locally, allowing SHRED to generate high-quality segmentations for categories not seen during training. We train and evaluate SHRED with fine-grained segmentations from PartNet; using its merge-threshold hyperparameter, we show that SHRED produces segmentations that better respect ground-truth annotations compared with baseline methods, at any desired decomposition granularity. Finally, we demonstrate that SHRED is useful for downstream applications, out-performing all baselines on zero-shot fine-grained part instance segmentation and few-shot fine-grained semantic segmentation when combined with methods that learn to label shape regions.
How does overparametrization affect performance on minority groups?
Venue: arXiv cs.LG 2022
Year: 2022

The benefits of overparameterization for the overall performance of modern machine learning (ML) models are well known. However, the effect of overparameterization at a more granular level of data subgroups is less understood. Recent empirical studies demonstrate encouraging results: (i) when groups are not known, overparameterized models trained with empirical risk minimization (ERM) perform better on minority groups; (ii) when groups are known, ERM on data subsampled to equalize group sizes yields state-of-the-art worst-group-accuracy in the overparameterized regime. In this paper, we complement these empirical studies with a theoretical investigation of the risk of overparameterized random feature models on minority groups. In a setting in which the regression functions for the majority and minority groups are different, we show that overparameterization always improves minority group performance.
A Penny for Your (visual) Thoughts: Self-Supervised Reconstruction of Natural Movies from Brain Activity
Venue: arXiv cs.CV 2022
Year: 2022

Reconstructing natural videos from fMRI brain recordings is very challenging, for two main reasons: (i) As fMRI data acquisition is difficult, we only have a limited amount of supervised samples, which is not enough to cover the huge space of natural videos; and (ii) The temporal resolution of fMRI recordings is much lower than the frame rate of natural videos. In this paper, we propose a self-supervised approach for natural-movie reconstruction. By employing cycle-consistency over Encoding-Decoding natural videos, we can: (i) exploit the full framerate of the training videos, and not be limited only to clips that correspond to fMRI recordings; (ii) exploit massive amounts of external natural videos which the subjects never saw inside the fMRI machine. These enable increasing the applicable training data by several orders of magnitude, introducing natural video priors to the decoding network, as well as temporal coherence. Our approach significantly outperforms competing methods, since those train only on the limited supervised data. We further introduce a new and simple temporal prior of natural videos, which - when folded into our fMRI decoder further - allows us to reconstruct videos at a higher frame-rate (HFR) of up to x8 of the original fMRI sample rate.
IoT based SMEs shop management system
Venue: arXiv cs.HC 2022
Year: 2022

The Internet of Things (IoT) is an idea that intends to interface arranged data frameworks to actual items. The Internet of Things (IoT) has applications in pretty much every part of life in this day and age, and stock administration is no special case. IoT gives an answer for this issue by making it simpler to interface every one of the various organizations in a strategic framework utilizing Wireless Sensor Networks. An Interactive Shopping Model and an Automated Inventory Intelligent Management System that uses the Internet of Things to give constant item following, the board, and observing. A study and examination of the commonness of IoT among assembling SMEs is introduced, just as the current impediments and possibilities for permitting prescient investigation. The four examination capacities are depicted alongside an outline of the IoT empowering agents. Future patterns and difficulties in arising innovative work subjects are featured, for example, making IoT advances available to SMEs. The motivation behind this paper is to look at how the Internet of Things is changing our lives and work spaces, just as to feature probably the best strategic approaches, insights, and patterns. Considering the developing significance of big business IoT and the exploration hole in this field, an IoT design and the IoT administration industry will be examined. A model is needed to choose and send IoT administrations in different authoritative settings.
Towards Scalable Hyperbolic Neural Networks using Taylor Series Approximations
Venue: arXiv cs.LG 2022
Year: 2022

Hyperbolic networks have shown prominent improvements over their Euclidean counterparts in several areas involving hierarchical datasets in various domains such as computer vision, graph analysis, and natural language processing. However, their adoption in practice remains restricted due to (i) non-scalability on accelerated deep learning hardware, (ii) vanishing gradients due to the closure of hyperbolic space, and (iii) information loss due to frequent mapping between local tangent space and fully hyperbolic space. To tackle these issues, we propose the approximation of hyperbolic operators using Taylor series expansions, which allows us to reformulate the computationally expensive tangent and cosine hyperbolic functions into their polynomial equivariants which are more efficient. This allows us to retain the benefits of preserving the hierarchical anatomy of the hyperbolic space, while maintaining the scalability over current accelerated deep learning infrastructure. The polynomial formulation also enables us to utilize the advancements in Euclidean networks such as gradient clipping and ReLU activation to avoid vanishing gradients and remove errors due to frequent switching between tangent space and hyperbolic space. Our empirical evaluation on standard benchmarks in the domain of graph analysis and computer vision shows that our polynomial formulation is as scalable as Euclidean architectures, both in terms of memory and time complexity, while providing results as effective as hyperbolic models. Moreover, our formulation also shows a considerable improvement over its baselines due to our solution to vanishing gradients and information loss.
Network Report: A Structured Description for Network Datasets
Venue: arXiv cs.SI 2022
Year: 2022

The rapid development of network science and technologies depends on shareable datasets. Currently, there is no standard practice for reporting and sharing network datasets. Some network dataset providers only share links, while others provide some contexts or basic statistics. As a result, critical information may be unintentionally dropped, and network dataset consumers may misunderstand or overlook critical aspects. Inappropriately using a network dataset can lead to severe consequences (e.g., discrimination) especially when machine learning models on networks are deployed in high-stake domains. Challenges arise as networks are often used across different domains (e.g., network science, physics, etc) and have complex structures. To facilitate the communication between network dataset providers and consumers, we propose network report. A network report is a structured description that summarizes and contextualizes a network dataset. Network report extends the idea of dataset reports (e.g., Datasheets for Datasets) from prior work with network-specific descriptions of the non-i.i.d. nature, demographic information, network characteristics, etc. We hope network reports encourage transparency and accountability in network research and development across different fields.
Minimal Joint Entropy and Order-Preserving Couplings
Venue: arXiv cs.IT 2022
Year: 2022

This paper focuses on the extreme-value problem for Shannon entropy of the joint distribution with given marginals. It is proved that the minimum-entropy coupling must be of order-preserving, while the maximum-entropy coupling coincides with the independent one. Note that in this sense, we interpret entropy as a measure of system disorder.
1Cademy at Semeval-2022 Task 1: Investigating the Effectiveness of Multilingual, Multitask, and Language-Agnostic Tricks for the Reverse Dictionary Task
Venue: arXiv cs.CL 2022
Year: 2022

This paper describes our system for the SemEval2022 task of matching dictionary glosses to word embeddings. We focus on the Reverse Dictionary Track of the competition, which maps multilingual glosses to reconstructed vector representations. More specifically, models convert the input of sentences to three types of embeddings: SGNS, Char, and Electra. We propose several experiments for applying neural network cells, general multilingual and multitask structures, and language-agnostic tricks to the task. We also provide comparisons over different types of word embeddings and ablation studies to suggest helpful strategies. Our initial transformer-based model achieves relatively low performance. However, trials on different retokenization methodologies indicate improved performance. Our proposed Elmobased monolingual model achieves the highest outcome, and its multitask, and multilingual varieties show competitive results as well.
LADDER: Latent Boundary-guided Adversarial Training
Venue: arXiv cs.LG 2022
Year: 2022

Deep Neural Networks (DNNs) have recently achieved great success in many classification tasks. Unfortunately, they are vulnerable to adversarial attacks that generate adversarial examples with a small perturbation to fool DNN models, especially in model sharing scenarios. Adversarial training is proved to be the most effective strategy that injects adversarial examples into model training to improve the robustness of DNN models against adversarial attacks. However, adversarial training based on the existing adversarial examples fails to generalize well to standard, unperturbed test data. To achieve a better trade-off between standard accuracy and adversarial robustness, we propose a novel adversarial training framework called LAtent bounDary-guided aDvErsarial tRaining (LADDER) that adversarially trains DNN models on latent boundary-guided adversarial examples. As opposed to most of the existing methods that generate adversarial examples in the input space, LADDER generates a myriad of high-quality adversarial examples through adding perturbations to latent features. The perturbations are made along the normal of the decision boundary constructed by an SVM with an attention mechanism. We analyze the merits of our generated boundary-guided adversarial examples from a boundary field perspective and visualization view. Extensive experiments and detailed analysis on MNIST, SVHN, CelebA, and CIFAR-10 validate the effectiveness of LADDER in achieving a better trade-off between standard accuracy and adversarial robustness as compared with vanilla DNNs and competitive baselines.
Decentralized, not Dehumanized in the Metaverse: Bringing Utility to NFTs through Multimodal Interaction
Venue: arXiv cs.MM 2022
Year: 2022

User Interaction for NFTs (Non-fungible Tokens) is gaining increasing attention. Although NFTs have been traditionally single-use and monolithic, recent applications aim to connect multimodal interaction with human behaviour. This paper reviews the related technological approaches and business practices in NFT art. We highlight that multimodal interaction is a currently under-studied issue in mainstream NFT art, and conjecture that multimodal interaction is a crucial enabler for decentralization in the NFT community. We propose a framework that combines a bottom-up approach with AI multimodal process. Through this framework, we put forward integrating human behaviour data into generative NFT units, as "multimodal interactive NFT". Our work displays the possibilities of NFTs in the art world, beyond the traditional 2D and 3D static content.
On the quick search for the shortest paths in an unweighted dynamic graph by its projections in brief
Venue: arXiv cs.DC 2022
Year: 2022

For the first time proposed: a method for representing the projections of a graph in computer memory and a description based on it of a quick search for shortest paths in unweighted dynamic graphs. The spatial complexity of the projection description does not exceed $(d + 1)\times n$ words, where $d$ is the diameter and $n$ is the number of vertices of the graph. The temporal difficulty of finding one shortest path between two vertices does not exceed d steps with the duration of elementary time of sampling a machine word. The solution can be applied in time delay-critical routing protocols of computer networks and supercomputers.
Robust PCA for Subspace Estimation in User-Centric Cell-Free Wireless Networks
Venue: arXiv cs.IT 2022
Year: 2022

We consider a scalable user-centric cell-free massive MIMO network with distributed remote radio units (RUs), enabling macrodiversity and joint processing. Due to the limited uplink (UL) pilot dimension, multiuser interference in the UL pilot transmission phase makes channel estimation a non-trivial problem. We make use of two types of UL pilot signals, sounding reference signal (SRS) and demodulation reference signal (DMRS) pilots, for the estimation of the channel subspace and its instantaneous realization, respectively. The SRS pilots are transmitted over multiple time slots and resource blocks according to a Latin squares based hopping scheme, which aims at averaging out the interference of different SRS co-pilot users. We propose a robust principle component analysis approach for channel subspace estimation from the SRS signal samples, employed at the RUs for each associated user. The estimated subspace is further used at the RUs for DMRS pilot decontamination and instantaneous channel estimation. We provide numerical simulations to compare the system performance using our subspace and channel estimation scheme with the cases of ideal partial subspace/channel knowledge and pilot matching channel estimation. The results show that a system with a properly designed SRS pilot hopping scheme can closely approximate the performance of a genie-aided system.
Authenticated Byzantine Gossip Protocol
Venue: arXiv cs.DC 2022
Year: 2022

ABGP refers to Authenticated Byzantine Gossip Protocol. The ABGP is a partial-synchronous, weak consistent, BFT based consensus algorithm. The algorithm implements the gossip protocol, but with BFT features inside (like multisig record approval). The algorithm has been developed as an alternative to classic private ledger solutions, like Hyperledger.
Opening up echo chambers via optimal content recommendation
Venue: arXiv cs.SI 2022
Year: 2022

Online social platforms have become central in the political debate. In this context, the existence of echo chambers is a problem of primary relevance. These clusters of like-minded individuals tend to reinforce prior beliefs, elicit animosity towards others and aggravate the spread of misinformation. We study this phenomenon on a Twitter dataset related to the 2017 French presidential elections and propose a method to tackle it with content recommendations. We use a quadratic program to find optimal recommendations that maximise the diversity of content users are exposed to, while still accounting for their preferences. Our method relies on a theoretical model that can sufficiently describe how content flows through the platform. We show that the model provides good approximations of empirical measures and demonstrate the effectiveness of the optimisation algorithm at mitigating the echo chamber effect on this dataset, even with limited budget for recommendations.