OpenRaiser/Intern-Atlas huggingface dataset:
Violet: Architecturally Exposed Orchestration, Movement, and Placement for Generalized Deep Learning
Venue: arXiv cs.AR 2023
Year: 2023

Deep learning and hardware for it has garnered immense academic and industry interest in the past 5 years, with many novel proposals. However, the state-of-art remains NVIDIA's TensorCore-based systems that provide top-of-line performance and coverage across a wide-spectrum of deep learning applications. In this paper, we first identify four key problems any new DL solution must solve: 1) Data orchestration, 2) Data movement, 3) Work placement and blending these to achieve 4) Coverage across different types of DL applications. With this as a guide, we propose Violet, a novel architecture with roots in multicore SIMD which balances the responsibilities for these four problems between the architecture, microarchitecture and software stack. Compared to the NVIDIA A100 GPU, we find Violet achieves geo-mean 2.4X/10.6X and 2.1X/9.5X performance/efficiency for inference and training across the MLPerf benchmark suite. We present detailed operator-level analysis of the MLPerf benchmark suite, extracting out key behaviors - with implications for architecture research beyond this paper, that underpin the speedup and efficiency. Overall, this paper motivates the importance of balance, that the break down of responsibilities must be thought through carefully in order to compete with incumbent architecture designs.
Dense Extreme Inception Network for Edge Detection
Venue: arXiv cs.CV 2023
Year: 2023

<<<This is a pre-acceptance version, please, go through Pattern Recognition Journal on Sciencedirect to read the final version>>>. Edge detection is the basis of many computer vision applications. State of the art predominantly relies on deep learning with two decisive factors: dataset content and network's architecture. Most of the publicly available datasets are not curated for edge detection tasks. Here, we offer a solution to this constraint. First, we argue that edges, contours and boundaries, despite their overlaps, are three distinct visual features requiring separate benchmark datasets. To this end, we present a new dataset of edges. Second, we propose a novel architecture, termed Dense Extreme Inception Network for Edge Detection (DexiNed), that can be trained from scratch without any pre-trained weights. DexiNed outperforms other algorithms in the presented dataset. It also generalizes well to other datasets without any fine-tuning. The higher quality of DexiNed is also perceptually evident thanks to the sharper and finer edges it outputs.
Bounded Distance-control for Multi-UAV Formation Safety and Preservation in Target-tracking Applications
Venue: arXiv cs.RO 2023
Year: 2023

The notion of safety in multi-agent systems assumes great significance in many emerging collaborative multi-robot applications. In this paper, we present a multi-UAV collaborative target-tracking application by defining bounded inter-UAV distances in the formation in order to ensure safe operation. In doing so, we address the problem of prioritizing specific objectives over others in a multi-objective control framework. We propose a barrier Lyapunov function-based distributed control law to enforce the bounds on the distances and assess its Lyapunov stability using a kinematic model. The theoretical analysis is supported by numerical results, which account for measurement noise and moving targets. Straight-line and circular motion of the target are considered, and results for quadratic Lyapunov function-based control, often used in multi-agent multi-objective problems, are also presented. A comparison of the two control approaches elucidates the advantages of our proposed safe-control in bounding the inter-agent distances in a formation. A concluding evaluation using ROS simulations illustrates the practical applicability of the proposed control to a pair of multi-rotors visually estimating and maintaining their mutual separation within specified bounds, as they track a moving target.
GaTector: A Unified Framework for Gaze Object Prediction
Venue: arXiv cs.CV 2023
Year: 2023

Gaze object prediction is a newly proposed task that aims to discover the objects being stared at by humans. It is of great application significance but still lacks a unified solution framework. An intuitive solution is to incorporate an object detection branch into an existing gaze prediction method. However, previous gaze prediction methods usually use two different networks to extract features from scene image and head image, which would lead to heavy network architecture and prevent each branch from joint optimization. In this paper, we build a novel framework named GaTector to tackle the gaze object prediction problem in a unified way. Particularly, a specific-general-specific (SGS) feature extractor is firstly proposed to utilize a shared backbone to extract general features for both scene and head images. To better consider the specificity of inputs and tasks, SGS introduces two input-specific blocks before the shared backbone and three task-specific blocks after the shared backbone. Specifically, a novel Defocus layer is designed to generate object-specific features for the object detection task without losing information or requiring extra computations. Moreover, the energy aggregation loss is introduced to guide the gaze heatmap to concentrate on the stared box. In the end, we propose a novel wUoC metric that can reveal the difference between boxes even when they share no overlapping area. Extensive experiments on the GOO dataset verify the superiority of our method in all three tracks, i.e. object detection, gaze estimation, and gaze object prediction.
PLACE dropout: A Progressive Layer-wise and Channel-wise Dropout for Domain Generalization
Venue: arXiv cs.LG 2023
Year: 2023

Domain generalization (DG) aims to learn a generic model from multiple observed source domains that generalizes well to arbitrary unseen target domains without further training. The major challenge in DG is that the model inevitably faces a severe overfitting issue due to the domain gap between source and target domains. To mitigate this problem, some dropout-based methods have been proposed to resist overfitting by discarding part of the representation of the intermediate layers. However, we observe that most of these methods only conduct the dropout operation in some specific layers, leading to an insufficient regularization effect on the model. We argue that applying dropout at multiple layers can produce stronger regularization effects, which could alleviate the overfitting problem on source domains more adequately than previous layer-specific dropout methods. In this paper, we develop a novel layer-wise and channel-wise dropout for DG, which randomly selects one layer and then randomly selects its channels to conduct dropout. Particularly, the proposed method can generate a variety of data variants to better deal with the overfitting issue. We also provide theoretical analysis for our dropout method and prove that it can effectively reduce the generalization error bound. Besides, we leverage the progressive scheme to increase the dropout ratio with the training progress, which can gradually boost the difficulty of training the model to enhance its robustness. Extensive experiments on three standard benchmark datasets have demonstrated that our method outperforms several state-of-the-art DG methods. Our code is available at https://github.com/lingeringlight/PLACEdropout.
Cross-modal Manifold Cutmix for Self-supervised Video Representation Learning
Venue: arXiv cs.CV 2023
Year: 2023

Contrastive representation learning of videos highly relies on the availability of millions of unlabelled videos. This is practical for videos available on web but acquiring such large scale of videos for real-world applications is very expensive and laborious. Therefore, in this paper we focus on designing video augmentation for self-supervised learning, we first analyze the best strategy to mix videos to create a new augmented video sample. Then, the question remains, can we make use of the other modalities in videos for data mixing? To this end, we propose Cross-Modal Manifold Cutmix (CMMC) that inserts a video tesseract into another video tesseract in the feature space across two different modalities. We find that our video mixing strategy STC-mix, i.e. preliminary mixing of videos followed by CMMC across different modalities in a video, improves the quality of learned video representations. We conduct thorough experiments for two downstream tasks: action recognition and video retrieval on two small scale video datasets UCF101, and HMDB51. We also demonstrate the effectiveness of our STC-mix on NTU dataset where domain knowledge is limited. We show that the performance of our STC-mix on both the downstream tasks is on par with the other self-supervised approaches while requiring less training data.
SoK: Anti-Facial Recognition Technology
Venue: arXiv cs.CR 2023
Year: 2023

The rapid adoption of facial recognition (FR) technology by both government and commercial entities in recent years has raised concerns about civil liberties and privacy. In response, a broad suite of so-called "anti-facial recognition" (AFR) tools has been developed to help users avoid unwanted facial recognition. The set of AFR tools proposed in the last few years is wide-ranging and rapidly evolving, necessitating a step back to consider the broader design space of AFR systems and long-term challenges. This paper aims to fill that gap and provides the first comprehensive analysis of the AFR research landscape. Using the operational stages of FR systems as a starting point, we create a systematic framework for analyzing the benefits and tradeoffs of different AFR approaches. We then consider both technical and social challenges facing AFR tools and propose directions for future research in this field.
Reinforcement Learning with Almost Sure Constraints
Venue: arXiv cs.LG 2023
Year: 2023

In this work we address the problem of finding feasible policies for Constrained Markov Decision Processes under probability one constraints. We argue that stationary policies are not sufficient for solving this problem, and that a rich class of policies can be found by endowing the controller with a scalar quantity, so called budget, that tracks how close the agent is to violating the constraint. We show that the minimal budget required to act safely can be obtained as the smallest fixed point of a Bellman-like operator, for which we analyze its convergence properties. We also show how to learn this quantity when the true kernel of the Markov decision process is not known, while providing sample-complexity bounds. The utility of knowing this minimal budget relies in that it can aid in the search of optimal or near-optimal policies by shrinking down the region of the state space the agent must navigate. Simulations illustrate the different nature of probability one constraints against the typically used constraints in expectation.
Meterstick: Benchmarking Performance Variability in Cloud and Self-hosted Minecraft-like Games Extended Technical Report
Venue: arXiv cs.PF 2023
Year: 2023

Due to increasing popularity and strict performance requirements, online games have become a workload of interest for the performance engineering community. One of the most popular types of online games is the Minecraft-like Game (MLG), in which players can terraform the environment. The most popular MLG, Minecraft, provides not only entertainment, but also educational support and social interaction, to over 130 million people world-wide. MLGs currently support their many players by replicating isolated instances that support each only up to a few hundred players under favorable conditions. In practice, as we show here, the real upper limit of supported players can be much lower. In this work, we posit that performance variability is a key cause for the lack of scalability in MLGs. We propose a novel operational model for MLGs and use it to design the first benchmark that focuses on MLG performance variability, defining specialized workloads, metrics, and processes. We conduct real-world benchmarking of MLGs and find environment-based workloads and cloud deployment to be significant sources of performance variability: peak-latency degrades sharply to 20.7 times the arithmetic mean, and exceeds by a factor of 7.4 the performance requirements. We derive actionable insights for game-developers, game-operators, and other stakeholders to tame performance variability.
Hop-Spanners for Geometric Intersection Graphs
Venue: arXiv cs.CG 2023
Year: 2023

A $t$-spanner of a graph $G=(V,E)$ is a subgraph $H=(V,E')$ that contains a $uv$-path of length at most $t$ for every $uv\in E$. It is known that every $n$-vertex graph admits a $(2k-1)$-spanner with $O(n^{1+1/k})$ edges for $k\geq 1$. This bound is the best possible for $1\leq k\leq 9$ and is conjectured to be optimal due to Erdős' girth conjecture. We study $t$-spanners for $t\in \{2,3\}$ for geometric intersection graphs in the plane. These spanners are also known as \emph{$t$-hop spanners} to emphasize the use of graph-theoretic distances (as opposed to Euclidean distances between the geometric objects or their centers). We obtain the following results: (1) Every $n$-vertex unit disk graph (UDG) admits a 2-hop spanner with $O(n)$ edges; improving upon the previous bound of $O(n\log n)$. (2) The intersection graph of $n$ axis-aligned fat rectangles admits a 2-hop spanner with $O(n\log n)$ edges, and this bound is tight up to a factor of $\log \log n$. (3) The intersection graph of $n$ fat convex bodies in the plane admits a 3-hop spanner with $O(n\log n)$ edges. (4) The intersection graph of $n$ axis-aligned rectangles admits a 3-hop spanner with $O(n\log^2 n)$ edges.
Reconfigurable Intelligent Surface-Empowered Self-Interference Cancellation for 6G Full-Duplex MIMO Communication Systems
Venue: arXiv cs.IT 2023
Year: 2023

Substantially increasing wireless traffic and extending serving coverage is required with the advent of sixth-generation (6G) wireless communication networks. Reconfigurable intelligent surface (RIS) is widely considered as a promising technique which is capable of improving the system sum rate and energy efficiency. Moreover, full-duplex (FD) multi-input-multi-output (MIMO) transmission provides simultaneous transmit and received signals, which theoretically provides twice of spectrum efficiency. However, the self-interference (SI) in FD system is a challenging task requiring high-overhead cancellation, which can be resolved by configuring appropriate phase shifts of RIS. This paper has proposed an RIS-empowered full-duplex interference cancellation (RFIC) scheme in order to alleviate the severe interference in an RIS-FD system. We consider the interference minimization of RIS-FD MIMO while guaranteeing quality-of-service (QoS) of whole system. The closed-form solution of RIS phase shifts is theoretically derived with the discussion of different numbers of RIS elements and receiving antennas. Simulation results reveal that the proposed RFIC scheme outperforms existing benchmarks with more than 50% of performance gain of sum rate.
Central-Smoothing Hypergraph Neural Networks for Predicting Drug-Drug Interactions
Venue: arXiv cs.LG 2023
Year: 2023

Predicting drug-drug interactions (DDI) is the problem of predicting side effects (unwanted outcomes) of a pair of drugs using drug information and known side effects of many pairs. This problem can be formulated as predicting labels (i.e. side effects) for each pair of nodes in a DDI graph, of which nodes are drugs and edges are interacting drugs with known labels. State-of-the-art methods for this problem are graph neural networks (GNNs), which leverage neighborhood information in the graph to learn node representations. For DDI, however, there are many labels with complicated relationships due to the nature of side effects. Usual GNNs often fix labels as one-hot vectors that do not reflect label relationships and potentially do not obtain the highest performance in the difficult cases of infrequent labels. In this paper, we formulate DDI as a hypergraph where each hyperedge is a triple: two nodes for drugs and one node for a label. We then present CentSmoothie, a hypergraph neural network that learns representations of nodes and labels altogether with a novel central-smoothing formulation. We empirically demonstrate the performance advantages of CentSmoothie in simulations as well as real datasets.
Event Linking: Grounding Event Mentions to Wikipedia
Venue: arXiv cs.CL 2023
Year: 2023

Comprehending an article requires understanding its constituent events. However, the context where an event is mentioned often lacks the details of this event. A question arises: how can the reader obtain more knowledge about this particular event in addition to what is provided by the local context in the article? This work defines Event Linking, a new natural language understanding task at the event level. Event linking tries to link an event mention appearing in an article to the most appropriate Wikipedia page. This page is expected to provide rich knowledge about what the event mention refers to. To standardize the research in this new direction, we contribute in four-fold. First, this is the first work in the community that formally defines Event Linking task. Second, we collect a dataset for this new task. Specifically, we automatically gather training set from Wikipedia, and then create two evaluation sets: one from the Wikipedia domain, reporting the in-domain performance, and a second from the real-world news domain, to evaluate out-of-domain performance. Third, we retrain and evaluate two state-of-the-art (SOTA) entity linking models, showing the challenges of event linking, and we propose an event-specific linking system EVELINK to set a competitive result for the new task. Fourth, we conduct a detailed and insightful analysis to help understand the task and the limitation of the current model. Overall, as our analysis shows, Event Linking is a considerably challenging and essential task requiring more effort from the community. Data and code are available here: https://github.com/CogComp/event-linking.
LookinGood^π: Real-time Person-independent Neural Re-rendering for High-quality Human Performance Capture
Venue: arXiv cs.CV 2023
Year: 2023

We propose LookinGood^π, a novel neural re-rendering approach that is aimed to (1) improve the rendering quality of the low-quality reconstructed results from human performance capture system in real-time; (2) improve the generalization ability of the neural rendering network on unseen people. Our key idea is to utilize the rendered image of reconstructed geometry as the guidance to assist the prediction of person-specific details from few reference images, thus enhancing the re-rendered result. In light of this, we design a two-branch network. A coarse branch is designed to fix some artifacts (i.e. holes, noise) and obtain a coarse version of the rendered input, while a detail branch is designed to predict "correct" details from the warped references. The guidance of the rendered image is realized by blending features from two branches effectively in the training of the detail branch, which improves both the warping accuracy and the details' fidelity. We demonstrate that our method outperforms state-of-the-art methods at producing high-fidelity images on unseen people.
Nirikshak: A Clustering Based Autonomous API Testing Framework
Venue: arXiv cs.SE 2023
Year: 2023

Quality Assurance (QA) is a critical component in product development, particularly in software testing. Despite the evolution of automated methods, testing for REST APIs often involves repetitive tasks. A significant portion of resources is dedicated more to scripting tests than to detecting and resolving actual software bugs. Additionally, conventional testing methods frequently struggle to adapt to software updates. However, with advancements in data science, a new paradigm is emerging: a self-reliant testing framework. This innovative approach minimizes the need for user intervention, achieving level 2 of autonomy in executing REST API testing procedures. It does so by employing a clustering method and analysis on logs categorizing test cases efficiently and thereby streamlining the testing process as well as ensuring more dynamic adaptability to software changes. Nirikshak is publicly available as an open-source software for the community at https://github.com/yashmahalwal/nirikshak.
Curriculum learning for data-driven modeling of dynamical systems
Venue: arXiv cs.LG 2023
Year: 2023

The reliable prediction of the temporal behavior of complex systems is key in numerous scientific fields. This strong interest is however hindered by modeling issues: often, the governing equations describing the physics of the system under consideration are not accessible or, if known, their solution might require a computational time incompatible with the prediction time constraints. Not surprisingly, approximating complex systems in a generic functional format and informing it ex-nihilo from available observations has become common practice in the age of machine learning, as illustrated by the numerous successful examples based on deep neural networks. However, generalizability of the models, margins of guarantee and the impact of data are often overlooked or examined mainly by relying on prior knowledge of the physics. We tackle these issues from a different viewpoint, by adopting a curriculum learning strategy. In curriculum learning, the dataset is structured such that the training process starts from simple samples towards more complex ones in order to favor convergence and generalization. The concept has been developed and successfully applied in robotics and control of systems. Here, we apply this concept for the learning of complex dynamical systems in a systematic way. First, leveraging insights from the ergodic theory, we assess the amount of data sufficient for a-priori guaranteeing a faithful model of the physical system and thoroughly investigate the impact of the training set and its structure on the quality of long-term predictions. Based on that, we consider entropy as a metric of complexity of the dataset; we show how an informed design of the training set based on the analysis of the entropy significantly improves the resulting models in terms of generalizability, and provide insights on the amount and the choice of data required for an effective data-driven modeling.