OpenRaiser/Intern-Atlas huggingface dataset:
Nuclear Norm Maximization Based Curiosity-Driven Learning
Venue: arXiv cs.LG 2022
Year: 2022

To handle the sparsity of the extrinsic rewards in reinforcement learning, researchers have proposed intrinsic reward which enables the agent to learn the skills that might come in handy for pursuing the rewards in the future, such as encouraging the agent to visit novel states. However, the intrinsic reward can be noisy due to the undesirable environment's stochasticity and directly applying the noisy value predictions to supervise the policy is detrimental to improve the learning performance and efficiency. Moreover, many previous studies employ $\ell^2$ norm or variance to measure the exploration novelty, which will amplify the noise due to the square operation. In this paper, we address aforementioned challenges by proposing a novel curiosity leveraging the nuclear norm maximization (NNM), which can quantify the novelty of exploring the environment more accurately while providing high-tolerance to the noise and outliers. We conduct extensive experiments across a variety of benchmark environments and the results suggest that NNM can provide state-of-the-art performance compared with previous curiosity methods. On 26 Atari games subset, when trained with only intrinsic reward, NNM achieves a human-normalized score of 1.09, which doubles that of competitive intrinsic rewards-based approaches. Our code will be released publicly to enhance the reproducibility.
Scaling Laws and Interpretability of Learning from Repeated Data
Venue: arXiv cs.LG 2022
Year: 2022

Recent large language models have been trained on vast datasets, but also often on repeated data, either intentionally for the purpose of upweighting higher quality data, or unintentionally because data deduplication is not perfect and the model is exposed to repeated data at the sentence, paragraph, or document level. Some works have reported substantial negative performance effects of this repeated data. In this paper we attempt to study repeated data systematically and to understand its effects mechanistically. To do this, we train a family of models where most of the data is unique but a small fraction of it is repeated many times. We find a strong double descent phenomenon, in which repeated data can lead test loss to increase midway through training. A predictable range of repetition frequency leads to surprisingly severe degradation in performance. For instance, performance of an 800M parameter model can be degraded to that of a 2x smaller model (400M params) by repeating 0.1% of the data 100 times, despite the other 90% of the training tokens remaining unique. We suspect there is a range in the middle where the data can be memorized and doing so consumes a large fraction of the model's capacity, and this may be where the peak of degradation occurs. Finally, we connect these observations to recent mechanistic interpretability work - attempting to reverse engineer the detailed computations performed by the model - by showing that data repetition disproportionately damages copying and internal structures associated with generalization, such as induction heads, providing a possible mechanism for the shift from generalization to memorization. Taken together, these results provide a hypothesis for why repeating a relatively small fraction of data in large language models could lead to disproportionately large harms to performance.
Improving Long Tailed Document-Level Relation Extraction via Easy Relation Augmentation and Contrastive Learning
Venue: arXiv cs.CL 2022
Year: 2022

Towards real-world information extraction scenario, research of relation extraction is advancing to document-level relation extraction(DocRE). Existing approaches for DocRE aim to extract relation by encoding various information sources in the long context by novel model architectures. However, the inherent long-tailed distribution problem of DocRE is overlooked by prior work. We argue that mitigating the long-tailed distribution problem is crucial for DocRE in the real-world scenario. Motivated by the long-tailed distribution problem, we propose an Easy Relation Augmentation(ERA) method for improving DocRE by enhancing the performance of tailed relations. In addition, we further propose a novel contrastive learning framework based on our ERA, i.e., ERACL, which can further improve the model performance on tailed relations and achieve competitive overall DocRE performance compared to the state-of-arts.
Robot Person Following in Uniform Crowd Environment
Venue: arXiv cs.CV 2022
Year: 2022

Person-tracking robots have many applications, such as in security, elderly care, and socializing robots. Such a task is particularly challenging when the person is moving in a Uniform crowd. Also, despite significant progress of trackers reported in the literature, state-of-the-art trackers have hardly addressed person following in such scenarios. In this work, we focus on improving the perceptivity of a robot for a person following task by developing a robust and real-time applicable object tracker. We present a new robot person tracking system with a new RGB-D tracker, Deep Tracking with RGB-D (DTRD) that is resilient to tricky challenges introduced by the uniform crowd environment. Our tracker utilizes transformer encoder-decoder architecture with RGB and depth information to discriminate the target person from similar distractors. A substantial amount of comprehensive experiments and results demonstrate that our tracker has higher performance in two quantitative evaluation metrics and confirms its superiority over other SOTA trackers.
Shared-Control Robotic Manipulation in Virtual Reality
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, we present the implementation details of a Virtual Reality (VR)-based teleoperation interface for moving a robotic manipulator. We propose an iterative human-in-the-loop design where the user sets the next task-space waypoint for the robot's end effector and executes the action on the physical robot before setting the next waypoints. Information from the robot's surroundings is provided to the user in two forms: as a point cloud in 3D space and a video stream projected on a virtual wall. The feasibility of the selected end effector pose is communicated to the user by the color of the virtual end effector. The interface is demonstrated to successfully work for a pick and place scenario, however, our trials showed that the fluency of the interaction and the autonomy level of the system can be increased.
HLATR: Enhance Multi-stage Text Retrieval with Hybrid List Aware Transformer Reranking
Venue: arXiv cs.IR 2022
Year: 2022

Deep pre-trained language models (e,g. BERT) are effective at large-scale text retrieval task. Existing text retrieval systems with state-of-the-art performance usually adopt a retrieve-then-reranking architecture due to the high computational cost of pre-trained language models and the large corpus size. Under such a multi-stage architecture, previous studies mainly focused on optimizing single stage of the framework thus improving the overall retrieval performance. However, how to directly couple multi-stage features for optimization has not been well studied. In this paper, we design Hybrid List Aware Transformer Reranking (HLATR) as a subsequent reranking module to incorporate both retrieval and reranking stage features. HLATR is lightweight and can be easily parallelized with existing text retrieval systems so that the reranking process can be performed in a single yet efficient processing. Empirical experiments on two large-scale text retrieval datasets show that HLATR can efficiently improve the ranking performance of existing multi-stage text retrieval methods.
A graphical representation of binary linear codes
Venue: arXiv cs.IT 2022
Year: 2022

A binary $[n,k]$-linear code $\mathcal{C}$ is a $k$-dimensional subspace of $\mathbb{F}_2^n$. For $\boldsymbol{x}\in \mathbb{F}_2^n$, the set $\boldsymbol{x}+\mathcal{C}$ is a coset of $\mathcal{C}$. In this work we study a partial ordering on the set of cosets of a binary linear code $\mathcal{C}$ of length $n$ and we construct a graph using the orphan structure of this code.
Evaluating Performance of Machine Learning Models for Diabetic Sensorimotor Polyneuropathy Severity Classification using Biomechanical Signals during Gait
Venue: arXiv cs.LG 2022
Year: 2022

Diabetic sensorimotor polyneuropathy (DSPN) is one of the prevalent forms of neuropathy affected by diabetic patients that involves alterations in biomechanical changes in human gait. In literature, for the last 50 years, researchers are trying to observe the biomechanical changes due to DSPN by studying muscle electromyography (EMG), and ground reaction forces (GRF). However, the literature is contradictory. In such a scenario, we are proposing to use Machine learning techniques to identify DSPN patients by using EMG, and GRF data. We have collected a dataset consists of three lower limb muscles EMG (tibialis anterior (TA), vastus lateralis (VL), gastrocnemius medialis (GM) and 3-dimensional GRF components (GRFx, GRFy, and GRFz). Raw EMG and GRF signals were preprocessed, and a newly proposed feature extraction technique scheme from literature was applied to extract the best features from the signals. The extracted feature list was ranked using Relief feature ranking techniques, and highly correlated features were removed. We have trained different ML models to find out the best-performing model and optimized that model. We trained the optimized ML models for different combinations of muscles and GRF components features, and the performance matrix was evaluated. This study has found ensemble classifier model was performing in identifying DSPN Severity, and we optimized it before training. For EMG analysis, we have found the best accuracy of 92.89% using the Top 14 features for features from GL, VL and TA muscles combined. In the GRF analysis, the model showed 94.78% accuracy by using the Top 15 features for the feature combinations extracted from GRFx, GRFy and GRFz signals. The performance of ML-based DSPN severity classification models, improved significantly, indicating their reliability in DSPN severity classification, for biomechanical data.
Micro-video recommendation model based on graph neural network and attention mechanism
Venue: arXiv cs.IR 2022
Year: 2022

With the rapid development of Internet technology and the comprehensive popularity of Internet applications, online activities have gradually become an indispensable part of people's daily life. The original recommendation learning algorithm is mainly based on user-microvideo interaction for learning, modeling the user-micro-video connection relationship, which is difficult to capture the more complex relationships between nodes. To address the above problems, we propose a personalized recommendation model based on graph neural network, which utilizes the feature that graph neural network can tap deep information of graph data more effectively, and transforms the input user rating information and item side information into graph structure, for effective feature extraction, based on the importance sampling strategy. The importance-based sampling strategy measures the importance of neighbor nodes to the central node by calculating the relationship tightness between the neighbor nodes and the central node, and selects the neighbor nodes for recommendation tasks based on the importance level, which can be more targeted to select the sampling neighbors with more influence on the target micro-video nodes. The pooling aggregation strategy, on the other hand, trains the aggregation weights by inputting the neighborhood node features into the fully connected layer before aggregating the neighborhood features, and then introduces the pooling layer for feature aggregation, and finally aggregates the obtained neighborhood aggregation features with the target node itself, which directly introduces a symmetric trainable function to fuse the neighborhood weight training into the model to better capture the different neighborhood nodes' differential features in a learnable manner to allow for a more accurate representation of the current node features.
DProQ: A Gated-Graph Transformer for Protein Complex Structure Assessment
Venue: arXiv cs.LG 2022
Year: 2022

Proteins interact to form complexes to carry out essential biological functions. Computational methods have been developed to predict the structures of protein complexes. However, an important challenge in protein complex structure prediction is to estimate the quality of predicted protein complex structures without any knowledge of the corresponding native structures. Such estimations can then be used to select high-quality predicted complex structures to facilitate biomedical research such as protein function analysis and drug discovery. We challenge this significant task with DProQ, which introduces a gated neighborhood-modulating Graph Transformer (GGT) designed to predict the quality of 3D protein complex structures. Notably, we incorporate node and edge gates within a novel Graph Transformer framework to control information flow during graph message passing. We train and evaluate DProQ on four newly-developed datasets that we make publicly available in this work. Our rigorous experiments demonstrate that DProQ achieves state-of-the-art performance in ranking protein complex structures.
SplitPlace: AI Augmented Splitting and Placement of Large-Scale Neural Networks in Mobile Edge Environments
Venue: arXiv cs.DC 2022
Year: 2022

In recent years, deep learning models have become ubiquitous in industry and academia alike. Deep neural networks can solve some of the most complex pattern-recognition problems today, but come with the price of massive compute and memory requirements. This makes the problem of deploying such large-scale neural networks challenging in resource-constrained mobile edge computing platforms, specifically in mission-critical domains like surveillance and healthcare. To solve this, a promising solution is to split resource-hungry neural networks into lightweight disjoint smaller components for pipelined distributed processing. At present, there are two main approaches to do this: semantic and layer-wise splitting. The former partitions a neural network into parallel disjoint models that produce a part of the result, whereas the latter partitions into sequential models that produce intermediate results. However, there is no intelligent algorithm that decides which splitting strategy to use and places such modular splits to edge nodes for optimal performance. To combat this, this work proposes a novel AI-driven online policy, SplitPlace, that uses Multi-Armed-Bandits to intelligently decide between layer and semantic splitting strategies based on the input task's service deadline demands. SplitPlace places such neural network split fragments on mobile edge devices using decision-aware reinforcement learning for efficient and scalable computing. Moreover, SplitPlace fine-tunes its placement engine to adapt to volatile environments. Our experiments on physical mobile-edge environments with real-world workloads show that SplitPlace can significantly improve the state-of-the-art in terms of average response time, deadline violation rate, inference accuracy, and total reward by up to 46, 69, 3 and 12 percent respectively.
Identifying and Mitigating Instability in Embeddings of the Degenerate Core
Venue: arXiv cs.SI 2022
Year: 2022

Are the embeddings of a graph's degenerate core stable? What happens to the embeddings of nodes in the degenerate core as we systematically remove periphery nodes (by repeated peeling off $k$-cores)? We discover three patterns w.r.t. instability in degenerate-core embeddings across a variety of popular graph embedding algorithms and datasets. We use regression to quantify the change point in graph embedding stability. Furthermore, we present the STABLE algorithm, which takes an existing graph embedding algorithm and makes it stable. We show the effectiveness of STABLE in terms of making the degenerate-core embedding stable and still producing state-of-the-art link prediction performance.
Towards the Effects of Alignment Edits on the Quality of Experience of 360 Videos
Venue: arXiv cs.MM 2022
Year: 2022

The optimization of viewers' quality of experience (QoE) in 360 videos faces two major roadblocks: inaccurate adaptive streaming and viewers missing the plot of a story. Alignment edit emerged as a promising mechanism to avoid both issues at once. Alignment edits act on the content, matching the users' viewport with a region of interest in the video content. As a consequence, viewers' attention is focused, reducing exploratory behavior and enabling the optimization of network resources; in addition, it allows for a precise selection of events to be shown to viewers, supporting viewers to follow the storyline. In this work, we investigate the effects of alignment edits on QoE by conducting two user studies. Specifically, we measured three QoE factors: presence, comfort, and overall QoE. We introduce a new alignment edit, named \textit{Fade-rotation}, based on a mechanism to reduce cybersickness in VR games. In the user studies, we tested four versions of fade-rotation and compared them with instant alignment. We observed from the results that gradual alignment achieves good levels of comfort for all contents and rotational speed tested, showing its validity. We observed a decrease in head motion after both alignment edits, with the gradual edit reaching a reduction in head speed of 8\% greater than that of instant alignment, confirming the usefulness of these edits for streaming video on-demand. Finally, parameters to implement \textit{Fade-rotation} are described.
MultiBiSage: A Web-Scale Recommendation System Using Multiple Bipartite Graphs at Pinterest
Venue: arXiv cs.LG 2022
Year: 2022

Graph Convolutional Networks (GCN) can efficiently integrate graph structure and node features to learn high-quality node embeddings. These embeddings can then be used for several tasks such as recommendation and search. At Pinterest, we have developed and deployed PinSage, a data-efficient GCN that learns pin embeddings from the Pin-Board graph. The Pin-Board graph contains pin and board entities and the graph captures the pin belongs to a board interaction. However, there exist several entities at Pinterest such as users, idea pins, creators, and there exist heterogeneous interactions among these entities such as add-to-cart, follow, long-click. In this work, we show that training deep learning models on graphs that captures these diverse interactions would result in learning higher-quality pin embeddings than training PinSage on only the Pin-Board graph. To that end, we model the diverse entities and their diverse interactions through multiple bipartite graphs and propose a novel data-efficient MultiBiSage model. MultiBiSage can capture the graph structure of multiple bipartite graphs to learn high-quality pin embeddings. We take this pragmatic approach as it allows us to utilize the existing infrastructure developed at Pinterest -- such as Pixie system that can perform optimized random-walks on billion node graphs, along with existing training and deployment workflows. We train MultiBiSage on six bipartite graphs including our Pin-Board graph. Our offline metrics show that MultiBiSage significantly outperforms the deployed latest version of PinSage on multiple user engagement metrics.
Online Coreference Resolution for Dialogue Processing: Improving Mention-Linking on Real-Time Conversations
Venue: arXiv cs.CL 2022
Year: 2022

This paper suggests a direction of coreference resolution for online decoding on actively generated input such as dialogue, where the model accepts an utterance and its past context, then finds mentions in the current utterance as well as their referents, upon each dialogue turn. A baseline and four incremental-updated models adapted from the mention-linking paradigm are proposed for this new setting, which address different aspects including the singletons, speaker-grounded encoding and cross-turn mention contextualization. Our approach is assessed on three datasets: Friends, OntoNotes, and BOLT. Results show that each aspect brings out steady improvement, and our best models outperform the baseline by over 10%, presenting an effective system for this setting. Further analysis highlights the task characteristics, such as the significance of addressing the mention recall.
Diversity Preference-Aware Link Recommendation for Online Social Networks
Venue: arXiv cs.LG 2022
Year: 2022

Link recommendation, which recommends links to connect unlinked online social network users, is a fundamental social network analytics problem with ample business implications. Existing link recommendation methods tend to recommend similar friends to a user but overlook the user's diversity preference, although social psychology theories suggest the criticality of diversity preference to link recommendation performance. In recommender systems, a field related to link recommendation, a number of diversification methods have been proposed to improve the diversity of recommended items. Nevertheless, diversity preference is distinct from diversity studied by diversification methods. To address these research gaps, we define and operationalize the concept of diversity preference for link recommendation and propose a new link recommendation problem: the diversity preference-aware link recommendation problem. We then analyze key properties of the new link recommendation problem and develop a novel link recommendation method to solve the problem. Using two large-scale online social network data sets, we conduct extensive empirical evaluations to demonstrate the superior performance of our method over representative diversification methods adapted for link recommendation as well as state-of-the-art link recommendation methods.
All You Need Is Logs: Improving Code Completion by Learning from Anonymous IDE Usage Logs
Venue: arXiv cs.SE 2022
Year: 2022

In this work, we propose an approach for collecting completion usage logs from the users in an IDE and using them to train a machine learning based model for ranking completion candidates. We developed a set of features that describe completion candidates and their context, and deployed their anonymized collection in the Early Access Program of IntelliJ-based IDEs. We used the logs to collect a dataset of code completions from users, and employed it to train a ranking CatBoost model. Then, we evaluated it in two settings: on a held-out set of the collected completions and in a separate A/B test on two different groups of users in the IDE. Our evaluation shows that using a simple ranking model trained on the past user behavior logs significantly improved code completion experience. Compared to the default heuristics-based ranking, our model demonstrated a decrease in the number of typing actions necessary to perform the completion in the IDE from 2.073 to 1.832. The approach adheres to privacy requirements and legal constraints, since it does not require collecting personal information, performing all the necessary anonymization on the client's side. Importantly, it can be improved continuously: implementing new features, collecting new data, and evaluating new models - this way, we have been using it in production since the end of 2020.