OpenRaiser/Intern-Atlas huggingface dataset:
Being at Home in the Metaverse? Prospectus for a Social Imaginary
Venue: arXiv cs.CY 2022
Year: 2022

The metaverse has seen growing corporate and popular interest over the past few years. While visions vary, the metaverse is generally seen as an extension of the internet that may be developed through advances in a number of digital technologies, such as augmented and virtual reality, as well as new technical infrastructure and standards. The metaverse constitutes an emerging social imaginary, a way of both understanding and directing our shared existence. This paper examines this emerging social imaginary through the phenomenological concept of dwelling, or being at home in the world, as developed by Martin Heidegger. To examine in depth one influential articulation of this social imaginary, this paper focuses on the metaverse as envisioned by Mark Zuckerberg, CEO of Meta (formerly Facebook). The paper presents a thematic analysis of Zuckerberg's public statements regarding the metaverse to provide a close reading of this particular vision. Then, through the lens of Heidegger's philosophy of dwelling, this paper identifies numerous threats to dwelling posed by the metaverse social imaginary. This paper explains these threats and their prognoses, and it closes with some considerations for how the metaverse could be designed to better facilitate human dwelling.
Optimising 2D Pose Representation: Improve Accuracy, Stability and Generalisability Within Unsupervised 2D-3D Human Pose Estimation
Venue: arXiv cs.CV 2022
Year: 2022

This paper addresses the problem of 2D pose representation during unsupervised 2D to 3D pose lifting to improve the accuracy, stability and generalisability of 3D human pose estimation (HPE) models. All unsupervised 2D-3D HPE approaches provide the entire 2D kinematic skeleton to a model during training. We argue that this is sub-optimal and disruptive as long-range correlations are induced between independent 2D key points and predicted 3D ordinates during training. To this end, we conduct the following study. With a maximum architecture capacity of 6 residual blocks, we evaluate the performance of 5 models which each represent a 2D pose differently during the adversarial unsupervised 2D-3D HPE process. Additionally, we show the correlations between 2D key points which are learned during the training process, highlighting the unintuitive correlations induced when an entire 2D pose is provided to a lifting model. Our results show that the most optimal representation of a 2D pose is that of two independent segments, the torso and legs, with no shared features between each lifting network. This approach decreased the average error by 20\% on the Human3.6M dataset when compared to a model with a near identical parameter count trained on the entire 2D kinematic skeleton. Furthermore, due to the complex nature of adversarial learning, we show how this representation can also improve convergence during training allowing for an optimum result to be obtained more often.
Monotonic Gaussian process for physics-constrained machine learning with materials science applications
Venue: arXiv cs.LG 2022
Year: 2022

Physics-constrained machine learning is emerging as an important topic in the field of machine learning for physics. One of the most significant advantages of incorporating physics constraints into machine learning methods is that the resulting model requires significantly less data to train. By incorporating physical rules into the machine learning formulation itself, the predictions are expected to be physically plausible. Gaussian process (GP) is perhaps one of the most common methods in machine learning for small datasets. In this paper, we investigate the possibility of constraining a GP formulation with monotonicity on three different material datasets, where one experimental and two computational datasets are used. The monotonic GP is compared against the regular GP, where a significant reduction in the posterior variance is observed. The monotonic GP is strictly monotonic in the interpolation regime, but in the extrapolation regime, the monotonic effect starts fading away as one goes beyond the training dataset. Imposing monotonicity on the GP comes at a small accuracy cost, compared to the regular GP. The monotonic GP is perhaps most useful in applications where data is scarce and noisy, and monotonicity is supported by strong physical evidence.
DecVi: Adaptive Video Conferencing on Open Peer-to-Peer Networks
Venue: arXiv cs.NI 2022
Year: 2022

Video conferencing has become the preferred way of interacting virtually. Current video conferencing applications, like Zoom, Teams or WebEx, are centralized, cloud-based platforms whose performance crucially depends on the proximity of clients to their data centers. Clients from low-income countries are particularly affected as most data centers from major cloud providers are located in economically advanced nations. Centralized conferencing applications also suffer from occasional outages and are embattled by serious privacy violation allegations. In recent years, decentralized video conferencing applications built over p2p networks and incentivized through blockchain are becoming popular. A key characteristic of these networks is their openness: anyone can host a media server on the network and gain reward for providing service. Strong economic incentives combined with lower entry barrier to join the network, makes increasing server coverage to even remote regions of the world. These reasons, however, also lead to a security problem: a server may obfuscate its true location in order to gain an unfair business advantage. In this paper, we consider the problem of multicast tree construction for video conferencing sessions in open p2p conferencing applications. We propose DecVi, a decentralized multicast tree construction protocol that adaptively discovers efficient tree structures based on an exploration-exploitation framework. DecVi is motivated by the combinatorial multi-armed bandit problem and uses a succinct learning model to compute effective actions. Despite operating in a multi-agent setting with each server having only limited knowledge of the global network and without cooperation among servers, experimentally we show DecVi achieves similar quality-of-experience compared to a centralized globally optimal algorithm while achieving higher reliability and flexibility.
Cooperative coevolutionary Modified Differential Evolution with Distance-based Selection for Large-Scale Optimization Problems in noisy environments through an automatic Random Grouping
Venue: arXiv cs.NE 2022
Year: 2022

Many optimization problems suffer from noise, and nonlinearity check-based decomposition methods (e.g. Differential Grouping) will completely fail to detect the interactions between variables in multiplicative noisy environments, thus, it is difficult to decompose the large-scale optimization problems (LSOPs) in noisy environments. In this paper, we propose an automatic Random Grouping (aRG), which does not need any explicit hyperparameter specified by users. Simulation experiments and mathematical analysis show that aRG can detect the interactions between variables without the fitness landscape knowledge, and the sub-problems decomposed by aRG have smaller scales, which is easier for EAs to optimize. Based on the cooperative coevolution (CC) framework, we introduce an advanced optimizer named Modified Differential Evolution with Distance-based Selection (MDE-DS) to enhance the search ability in noisy environments. Compared with canonical DE, the parameter self-adaptation, the balance between diversification and intensification, and the distance-based probability selection endow MDE-DS with stronger ability in exploration and exploitation. To evaluate the performance of our proposal, we design $500$-D and $1000$-D problems with various separability in noisy environments based on the CEC2013 LSGO Suite. Numerical experiments show that our proposal has broad prospects to solve LSOPs in noisy environments and can be easily extended to higher-dimensional problems.
Semantic Communication-Empowered Physical-layer Network Coding
Venue: arXiv cs.NI 2022
Year: 2022

In a two-way relay channel (TWRC), physical-layer network coding (PNC) doubles the system throughput by turning superimposed signals transmitted simultaneously by different end nodes into useful network-coded information (known as PNC decoding). Prior works indicated that the PNC decoding performance is affected by the relative phase offset between the received signals from different nodes. In particular, some "bad" relative phase offsets could lead to huge performance degradation. Previous solutions to mitigate the relative phase offset effect were limited to the conventional bit-oriented communication paradigm, aiming at delivering a given information stream as quickly and reliably as possible. In contrast, this paper puts forth the first semantic communication-empowered PNC-enabled TWRC to address the relative phase offset issue, referred to as SC-PNC. Despite the bad relative phase offsets, SC-PNC directly extracts the semantic meaning of transmitted messages rather than ensuring accurate bit stream transmission. We jointly design deep neural network (DNN)-based transceivers at the end nodes and propose a semantic PNC decoder at the relay. Taking image delivery as an example, experimental results show that the SC-PNC TWRC achieves high and stable reconstruction quality for images under different channel conditions and relative phase offsets, compared with the conventional bit-oriented counterparts.
Structure-Preserving Graph Representation Learning
Venue: arXiv cs.LG 2022
Year: 2022

Though graph representation learning (GRL) has made significant progress, it is still a challenge to extract and embed the rich topological structure and feature information in an adequate way. Most existing methods focus on local structure and fail to fully incorporate the global topological structure. To this end, we propose a novel Structure-Preserving Graph Representation Learning (SPGRL) method, to fully capture the structure information of graphs. Specifically, to reduce the uncertainty and misinformation of the original graph, we construct a feature graph as a complementary view via k-Nearest Neighbor method. The feature graph can be used to contrast at node-level to capture the local relation. Besides, we retain the global topological structure information by maximizing the mutual information (MI) of the whole graph and feature embeddings, which is theoretically reduced to exchanging the feature embeddings of the feature and the original graphs to reconstruct themselves. Extensive experiments show that our method has quite superior performance on semi-supervised node classification task and excellent robustness under noise perturbation on graph structure or node features.
Domain Adaptation from Scratch
Venue: arXiv cs.CL 2022
Year: 2022

Natural language processing (NLP) algorithms are rapidly improving but often struggle when applied to out-of-distribution examples. A prominent approach to mitigate the domain gap is domain adaptation, where a model trained on a source domain is adapted to a new target domain. We present a new learning setup, ``domain adaptation from scratch'', which we believe to be crucial for extending the reach of NLP to sensitive domains in a privacy-preserving manner. In this setup, we aim to efficiently annotate data from a set of source domains such that the trained model performs well on a sensitive target domain from which data is unavailable for annotation. Our study compares several approaches for this challenging setup, ranging from data selection and domain adaptation algorithms to active learning paradigms, on two NLP tasks: sentiment analysis and Named Entity Recognition. Our results suggest that using the abovementioned approaches eases the domain gap, and combining them further improves the results.
Real-time 3D Single Object Tracking with Transformer
Venue: arXiv cs.CV 2022
Year: 2022

LiDAR-based 3D single object tracking is a challenging issue in robotics and autonomous driving. Currently, existing approaches usually suffer from the problem that objects at long distance often have very sparse or partially-occluded point clouds, which makes the features extracted by the model ambiguous. Ambiguous features will make it hard to locate the target object and finally lead to bad tracking results. To solve this problem, we utilize the powerful Transformer architecture and propose a Point-Track-Transformer (PTT) module for point cloud-based 3D single object tracking task. Specifically, PTT module generates fine-tuned attention features by computing attention weights, which guides the tracker focusing on the important features of the target and improves the tracking ability in complex scenarios. To evaluate our PTT module, we embed PTT into the dominant method and construct a novel 3D SOT tracker named PTT-Net. In PTT-Net, we embed PTT into the voting stage and proposal generation stage, respectively. PTT module in the voting stage could model the interactions among point patches, which learns context-dependent features. Meanwhile, PTT module in the proposal generation stage could capture the contextual information between object and background. We evaluate our PTT-Net on KITTI and NuScenes datasets. Experimental results demonstrate the effectiveness of PTT module and the superiority of PTT-Net, which surpasses the baseline by a noticeable margin, ~10% in the Car category. Meanwhile, our method also has a significant performance improvement in sparse scenarios. In general, the combination of transformer and tracking pipeline enables our PTT-Net to achieve state-of-the-art performance on both two datasets. Additionally, PTT-Net could run in real-time at 40FPS on NVIDIA 1080Ti GPU. Our code is open-sourced for the research community at https://github.com/shanjiayao/PTT.
Dialogue Evaluation with Offline Reinforcement Learning
Venue: arXiv cs.CL 2022
Year: 2022

Task-oriented dialogue systems aim to fulfill user goals through natural language interactions. They are ideally evaluated with human users, which however is unattainable to do at every iteration of the development phase. Simulated users could be an alternative, however their development is nontrivial. Therefore, researchers resort to offline metrics on existing human-human corpora, which are more practical and easily reproducible. They are unfortunately limited in reflecting real performance of dialogue systems. BLEU for instance is poorly correlated with human judgment, and existing corpus-based metrics such as success rate overlook dialogue context mismatches. There is still a need for a reliable metric for task-oriented systems with good generalization and strong correlation with human judgements. In this paper, we propose the use of offline reinforcement learning for dialogue evaluation based on a static corpus. Such an evaluator is typically called a critic and utilized for policy optimization. We go one step further and show that offline RL critics can be trained on a static corpus for any dialogue system as external evaluators, allowing dialogue performance comparisons across various types of systems. This approach has the benefit of being corpus- and model-independent, while attaining strong correlation with human judgements, which we confirm via an interactive user trial.
Instance-Dependent Noisy Label Learning via Graphical Modelling
Venue: arXiv cs.CV 2022
Year: 2022

Noisy labels are unavoidable yet troublesome in the ecosystem of deep learning because models can easily overfit them. There are many types of label noise, such as symmetric, asymmetric and instance-dependent noise (IDN), with IDN being the only type that depends on image information. Such dependence on image information makes IDN a critical type of label noise to study, given that labelling mistakes are caused in large part by insufficient or ambiguous information about the visual classes present in images. Aiming to provide an effective technique to address IDN, we present a new graphical modelling approach called InstanceGM, that combines discriminative and generative models. The main contributions of InstanceGM are: i) the use of the continuous Bernoulli distribution to train the generative model, offering significant training advantages, and ii) the exploration of a state-of-the-art noisy-label discriminative classifier to generate clean labels from instance-dependent noisy-label samples. InstanceGM is competitive with current noisy-label learning approaches, particularly in IDN benchmarks using synthetic and real-world datasets, where our method shows better accuracy than the competitors in most experiments.
Exploiting Pretrained Biochemical Language Models for Targeted Drug Design
Venue: arXiv cs.LG 2022
Year: 2022

Motivation: The development of novel compounds targeting proteins of interest is one of the most important tasks in the pharmaceutical industry. Deep generative models have been applied to targeted molecular design and have shown promising results. Recently, target-specific molecule generation has been viewed as a translation between the protein language and the chemical language. However, such a model is limited by the availability of interacting protein-ligand pairs. On the other hand, large amounts of unlabeled protein sequences and chemical compounds are available and have been used to train language models that learn useful representations. In this study, we propose exploiting pretrained biochemical language models to initialize (i.e. warm start) targeted molecule generation models. We investigate two warm start strategies: (i) a one-stage strategy where the initialized model is trained on targeted molecule generation (ii) a two-stage strategy containing a pre-finetuning on molecular generation followed by target specific training. We also compare two decoding strategies to generate compounds: beam search and sampling. Results: The results show that the warm-started models perform better than a baseline model trained from scratch. The two proposed warm-start strategies achieve similar results to each other with respect to widely used metrics from benchmarks. However, docking evaluation of the generated compounds for a number of novel proteins suggests that the one-stage strategy generalizes better than the two-stage strategy. Additionally, we observe that beam search outperforms sampling in both docking evaluation and benchmark metrics for assessing compound quality. Availability and implementation: The source code is available at https://github.com/boun-tabi/biochemical-lms-for-drug-design and the materials are archived in Zenodo at https://doi.org/10.5281/zenodo.6832145
Programming with Context-Sensitive Holes using Dependency-Aware Tuning
Venue: arXiv cs.PL 2022
Year: 2022

Developing efficient and maintainable software systems is both hard and time consuming. In particular, non-functional performance requirements involve many design and implementation decisions that can be difficult to take early during system development. Choices -- such as selection of data structures or where and how to parallelize code -- typically require extensive manual tuning that is both time consuming and error-prone. Although various auto-tuning approaches exist, they are either specialized for certain domains or require extensive code rewriting to work for different contexts in the code. In this paper, we introduce a new methodology for writing programs with holes, that is, decision variables explicitly stated in the program code that enable developers to postpone decisions during development. We introduce and evaluate two novel ideas: (i) context-sensitive holes that are expanded by the compiler into sets of decision variables for automatic tuning, and (ii) dependency-aware tuning, where static analysis reduces the search space by finding the set of decision variables that can be tuned independently of each other. We evaluate the two new concepts in a system called Miking, where we show how the general methodology can be used for automatic algorithm selection, data structure decisions, and parallelization choices.
Mutual Witness Gabriel Drawings of Complete Bipartite Graphs
Venue: arXiv cs.CG 2022
Year: 2022

Let $Γ$ be a straight-line drawing of a graph and let $u$ and $v$ be two vertices of $Γ$. The Gabriel disk of $u,v$ is the disk having $u$ and $v$ as antipodal points. A pair $\langle Γ_0,Γ_1 \rangle$ of vertex-disjoint straight-line drawings form a mutual witness Gabriel drawing when, for $i=0,1$, any two vertices $u$ and $v$ of $Γ_i$ are adjacent if and only if their Gabriel disk does not contain any vertex of $Γ_{1-i}$. We characterize the pairs $\langle G_0,G_1 \rangle $ of complete bipartite graphs that admit a mutual witness Gabriel drawing. The characterization leads to a linear time testing algorithm. We also show that when at least one of the graphs in the pair $\langle G_0, G_1 \rangle $ is complete $k$-partite with $k>2$ and all partition sets in the two graphs have size greater than one, the pair does not admit a mutual witness Gabriel drawing.
In-Place Gestures Classification via Long-term Memory Augmented Network
Venue: arXiv cs.HC 2022
Year: 2022

In-place gesture-based virtual locomotion techniques enable users to control their viewpoint and intuitively move in the 3D virtual environment. A key research problem is to accurately and quickly recognize in-place gestures, since they can trigger specific movements of virtual viewpoints and enhance user experience. However, to achieve real-time experience, only short-term sensor sequence data (up to about 300ms, 6 to 10 frames) can be taken as input, which actually affects the classification performance due to limited spatio-temporal information. In this paper, we propose a novel long-term memory augmented network for in-place gestures classification. It takes as input both short-term gesture sequence samples and their corresponding long-term sequence samples that provide extra relevant spatio-temporal information in the training phase. We store long-term sequence features with an external memory queue. In addition, we design a memory augmented loss to help cluster features of the same class and push apart features from different classes, thus enabling our memory queue to memorize more relevant long-term sequence features. In the inference phase, we input only short-term sequence samples to recall the stored features accordingly, and fuse them together to predict the gesture class. We create a large-scale in-place gestures dataset from 25 participants with 11 gestures. Our method achieves a promising accuracy of 95.1% with a latency of 192ms, and an accuracy of 97.3% with a latency of 312ms, and is demonstrated to be superior to recent in-place gesture classification techniques. User study also validates our approach. Our source code and dataset will be made available to the community.