OpenRaiser/Intern-Atlas huggingface dataset:
Prompting for Multi-Modal Tracking
Venue: arXiv cs.CV 2022
Year: 2022

Multi-modal tracking gains attention due to its ability to be more accurate and robust in complex scenarios compared to traditional RGB-based tracking. Its key lies in how to fuse multi-modal data and reduce the gap between modalities. However, multi-modal tracking still severely suffers from data deficiency, thus resulting in the insufficient learning of fusion modules. Instead of building such a fusion module, in this paper, we provide a new perspective on multi-modal tracking by attaching importance to the multi-modal visual prompts. We design a novel multi-modal prompt tracker (ProTrack), which can transfer the multi-modal inputs to a single modality by the prompt paradigm. By best employing the tracking ability of pre-trained RGB trackers learning at scale, our ProTrack can achieve high-performance multi-modal tracking by only altering the inputs, even without any extra training on multi-modal data. Extensive experiments on 5 benchmark datasets demonstrate the effectiveness of the proposed ProTrack.
Deep Deformable 3D Caricatures with Learned Shape Control
Venue: arXiv cs.CV 2022
Year: 2022

A 3D caricature is an exaggerated 3D depiction of a human face. The goal of this paper is to model the variations of 3D caricatures in a compact parameter space so that we can provide a useful data-driven toolkit for handling 3D caricature deformations. To achieve the goal, we propose an MLP-based framework for building a deformable surface model, which takes a latent code and produces a 3D surface. In the framework, a SIREN MLP models a function that takes a 3D position on a fixed template surface and returns a 3D displacement vector for the input position. We create variations of 3D surfaces by learning a hypernetwork that takes a latent code and produces the parameters of the MLP. Once learned, our deformable model provides a nice editing space for 3D caricatures, supporting label-based semantic editing and point-handle-based deformation, both of which produce highly exaggerated and natural 3D caricature shapes. We also demonstrate other applications of our deformable model, such as automatic 3D caricature creation.
Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates
Venue: arXiv cs.LG 2022
Year: 2022

Vertical federated learning (VFL) is an emerging paradigm that allows different parties (e.g., organizations or enterprises) to collaboratively build machine learning models with privacy protection. In the training phase, VFL only exchanges the intermediate statistics, i.e., forward activations and backward derivatives, across parties to compute model gradients. Nevertheless, due to its geo-distributed nature, VFL training usually suffers from the low WAN bandwidth. In this paper, we introduce CELU-VFL, a novel and efficient VFL training framework that exploits the local update technique to reduce the cross-party communication rounds. CELU-VFL caches the stale statistics and reuses them to estimate model gradients without exchanging the ad hoc statistics. Significant techniques are proposed to improve the convergence performance. First, to handle the stochastic variance problem, we propose a uniform sampling strategy to fairly choose the stale statistics for local updates. Second, to harness the errors brought by the staleness, we devise an instance weighting mechanism that measures the reliability of the estimated gradients. Theoretical analysis proves that CELU-VFL achieves a similar sub-linear convergence rate as vanilla VFL training but requires much fewer communication rounds. Empirical results on both public and real-world workloads validate that CELU-VFL can be up to six times faster than the existing works.
Joint Beam Placement and Load Balancing Optimization for Non-Geostationary Satellite Systems
Venue: arXiv cs.IT 2022
Year: 2022

Non-geostationary (Non-GSO) satellite constellations have emerged as a promising solution to enable ubiquitous high-speed low-latency broadband services by generating multiple spot-beams placed on the ground according to the user locations. However, there is an inherent trade-off between the number of active beams and the complexity of generating a large number of beams. This paper formulates and solves a joint beam placement and load balancing problem to carefully optimize the satellite beam and enhance the link budgets with a minimal number of active beams. We propose a two-stage algorithm design to overcome the combinatorial structure of the considered optimization problem providing a solution in polynomial time. The first stage minimizes the number of active beams, while the second stage performs a load balancing to distribute users in the coverage area of the active beams. Numerical results confirm the benefits of the proposed methodology both in carrier-to-noise ratio and multiplexed users per beam over other benchmarks.
Using Graph Neural Networks for Program Termination
Venue: arXiv cs.SE 2022
Year: 2022

Termination analyses investigate the termination behavior of programs, intending to detect nontermination, which is known to cause a variety of program bugs (e.g. hanging programs, denial-of-service vulnerabilities). Beyond formal approaches, various attempts have been made to estimate the termination behavior of programs using neural networks. However, the majority of these approaches continue to rely on formal methods to provide strong soundness guarantees and consequently suffer from similar limitations. In this paper, we move away from formal methods and embrace the stochastic nature of machine learning models. Instead of aiming for rigorous guarantees that can be interpreted by solvers, our objective is to provide an estimation of a program's termination behavior and of the likely reason for nontermination (when applicable) that a programmer can use for debugging purposes. Compared to previous approaches using neural networks for program termination, we also take advantage of the graph representation of programs by employing Graph Neural Networks. To further assist programmers in understanding and debugging nontermination bugs, we adapt the notions of attention and semantic segmentation, previously used for other application domains, to programs. Overall, we designed and implemented classifiers for program termination based on Graph Convolutional Networks and Graph Attention Networks, as well as a semantic segmentation Graph Neural Network that localizes AST nodes likely to cause nontermination. We also illustrated how the information provided by semantic segmentation can be combined with program slicing to further aid debugging.
Global-Local Self-Distillation for Visual Representation Learning
Venue: arXiv cs.CV 2022
Year: 2022

The downstream accuracy of self-supervised methods is tightly linked to the proxy task solved during training and the quality of the gradients extracted from it. Richer and more meaningful gradients updates are key to allow self-supervised methods to learn better and in a more efficient manner. In a typical self-distillation framework, the representation of two augmented images are enforced to be coherent at the global level. Nonetheless, incorporating local cues in the proxy task can be beneficial and improve the model accuracy on downstream tasks. This leads to a dual objective in which, on the one hand, coherence between global-representations is enforced and on the other, coherence between local-representations is enforced. Unfortunately, an exact correspondence mapping between two sets of local-representations does not exist making the task of matching local-representations from one augmentation to another non-trivial. We propose to leverage the spatial information in the input images to obtain geometric matchings and compare this geometric approach against previous methods based on similarity matchings. Our study shows that not only 1) geometric matchings perform better than similarity based matchings in low-data regimes but also 2) that similarity based matchings are highly hurtful in low-data regimes compared to the vanilla baseline without local self-distillation. The code is available at https://github.com/tileb1/global-local-self-distillation.
$\mathbb{Z}_p\mathbb{Z}_{p^2}\dots\mathbb{Z}_{p^s}$-Additive Generalized Hadamard Codes
Venue: arXiv cs.IT 2022
Year: 2022

The $\mathbb{Z}_p\mathbb{Z}_{p^2}\dots\mathbb{Z}_{p^s}$-additive codes are subgroups of $\mathbb{Z}_p^{α_1} \times \mathbb{Z}_{p^2}^{α_2} \times \cdots \times \mathbb{Z}_{p^s}^{α_s}$, and can be seen as linear codes over $\mathbb{Z}_p$ when $α_i=0$ for all $i \in \{2,\dots, s\}$, a $\mathbb{Z}_{p^s}$-additive code when $α_i=0$ for all $i \in \{1,\dots, s-1\}$ , or a $\mathbb{Z}_p\mathbb{Z}_{p^2}$-additive code when $s=2$, or $\mathbb{Z}_2\mathbb{Z}_4$-additive codes when $p=2$ and $s=2$. A $\mathbb{Z}_p\mathbb{Z}_{p^2}\dots\mathbb{Z}_{p^s}$-linear generalized Hadamard (GH) code is a GH code over $\mathbb{Z}_p$ which is the Gray map image of a $\mathbb{Z}_p\mathbb{Z}_{p^2}\dots\mathbb{Z}_{p^s}$-additive code. In this paper, we generalize some known results for $\mathbb{Z}_p\mathbb{Z}_{p^2}\dots\mathbb{Z}_{p^s}$-linear GH codes with $p$ prime and $s\geq 2$. First, we give a recursive construction of $\mathbb{Z}_p\mathbb{Z}_{p^2}\dots \mathbb{Z}_{p^s}$-additive GH codes of type $(α_1,\dots,α_s;t_1,\dots,t_s)$ with $t_1\geq 1, t_2,\dots,t_{s-1}\geq 0$, and $t_s\geq1$. Then, we show for which types the corresponding $\mathbb{Z}_p\mathbb{Z}_{p^2}\dots\mathbb{Z}_{p^s}$-linear GH codes are nonlinear over $\mathbb{Z}_p$. We also compute the kernel and its dimension whenever they are nonlinear.
Graph Neural Networks for Channel Decoding
Venue: arXiv cs.IT 2022
Year: 2022

In this work, we propose a fully differentiable graph neural network (GNN)-based architecture for channel decoding and showcase a competitive decoding performance for various coding schemes, such as low-density parity-check (LDPC) and BCH codes. The idea is to let a neural network (NN) learn a generalized message passing algorithm over a given graph that represents the forward error correction (FEC) code structure by replacing node and edge message updates with trainable functions. Contrary to many other deep learning-based decoding approaches, the proposed solution enjoys scalability to arbitrary block lengths and the training is not limited by the curse of dimensionality. We benchmark our proposed decoder against state-of-the-art in conventional channel decoding as well as against recent deep learning-based results. For the (63,45) BCH code, our solution outperforms weighted belief propagation (BP) decoding by approximately 0.4 dB with significantly less decoding iterations and even for 5G NR LDPC codes, we observe a competitive performance when compared to conventional BP decoding. For the BCH codes, the resulting GNN decoder can be fully parametrized with only 9640 weights.
Personalized Promotion Decision Making Based on Direct and Enduring Effect Predictions
Venue: arXiv cs.IR 2022
Year: 2022

Promotions have been trending in the e-commerce marketplace to build up customer relationships and guide customers towards the desired actions. Since incentives are effective to engage customers and customers have different preferences for different types of incentives, the demand for personalized promotion decision making is increasing over time. However, research on promotion decision making has focused specifically on purchase conversion during the promotion period (the direct effect), while generally disregarding the enduring effect in the post promotion period. To achieve a better lift return on investment (lift ROI) on the enduring effect of the promotion and improve customer retention and loyalty, we propose a framework of multiple treatment promotion decision making by modeling each customer's direct and enduring response. First, we propose a customer direct and enduring effect (CDEE) model which predicts the customer direct and enduring response. With the help of the predictions of the CDEE, we personalize incentive allocation to optimize the enduring effect while keeping the cost under the budget. To estimate the effect of decision making, we apply an unbiased evaluation approach of business metrics with randomized control trial (RCT) data. We compare our method with benchmarks using two promotions in Mercari and achieve significantly better results.
On infrastructure for facilitation of inner source in small development teams
Venue: arXiv cs.SE 2022
Year: 2022

The phenomenon of adopting open source software development practices in a corporate environment is known by many names, one being inner source. The objective of this study is to investigate how an organization consisting of small development teams can benefit from adopting inner source and assess the level of applicability. The research has been conducted as a case study at a software development company. Data collection was carried out through interviews and a series of focus group meetings, and then analyzed by mapping it to an available framework. The analysis shows that the organization possesses potential, and also identified a number of challenges and benefits of special importance to the case company. To address these challenges, the case study synthesized the organizational and infrastructural needs of the organization in a requirements specification describing a technical infrastructure, also known as a software forge, with an adapted organizational context and work process.
Some Practice for Improving the Search Results of E-commerce
Venue: arXiv cs.IR 2022
Year: 2022

In the Amazon KDD Cup 2022, we aim to apply natural language processing methods to improve the quality of search results that can significantly enhance user experience and engagement with search engines for e-commerce. We discuss our practical solution for this competition, ranking 6th in task one, 2nd in task two, and 2nd in task 3. The code is available at https://github.com/wufanyou/KDD-Cup-2022-Amazon.
Traveler: Navigating Task Parallel Traces for Performance Analysis
Venue: arXiv cs.HC 2022
Year: 2022

Understanding the behavior of software in execution is a key step in identifying and fixing performance issues. This is especially important in high performance computing contexts where even minor performance tweaks can translate into large savings in terms of computational resource use. To aid performance analysis, developers may collect an execution trace - a chronological log of program activity during execution. As traces represent the full history, developers can discover a wide array of possibly previously unknown performance issues, making them an important artifact for exploratory performance analysis. However, interactive trace visualization is difficult due to issues of data size and complexity of meaning. Traces represent nanosecond-level events across many parallel processes, meaning the collected data is often large and difficult to explore. The rise of asynchronous task parallel programming paradigms complicates the relation between events and their probable cause. To address these challenges, we conduct a continuing design study in collaboration with high performance computing researchers. We develop diverse and hierarchical ways to navigate and represent execution trace data in support of their trace analysis tasks. Through an iterative design process, we developed Traveler, an integrated visualization platform for task parallel traces. Traveler provides multiple linked interfaces to help navigate trace data from multiple contexts. We evaluate the utility of Traveler through feedback from users and a case study, finding that integrating multiple modes of navigation in our design supported performance analysis tasks and led to the discovery of previously unknown behavior in a distributed array library.
Reinforcement learning with experience replay and adaptation of action dispersion
Venue: arXiv cs.LG 2022
Year: 2022

Effective reinforcement learning requires a proper balance of exploration and exploitation defined by the dispersion of action distribution. However, this balance depends on the task, the current stage of the learning process, and the current environment state. Existing methods that designate the action distribution dispersion require problem-dependent hyperparameters. In this paper, we propose to automatically designate the action distribution dispersion using the following principle: This distribution should have sufficient dispersion to enable the evaluation of future policies. To that end, the dispersion should be tuned to assure a sufficiently high probability (densities) of the actions in the replay buffer and the modes of the distributions that generated them, yet this dispersion should not be higher. This way, a policy can be effectively evaluated based on the actions in the buffer, but exploratory randomness in actions decreases when this policy converges. The above principle is verified here on challenging benchmarks Ant, HalfCheetah, Hopper, and Walker2D, with good results. Our method makes the action standard deviations converge to values similar to those resulting from trial-and-error optimization.
Masked Autoencoders As The Unified Learners For Pre-Trained Sentence Representation
Venue: arXiv cs.CL 2022
Year: 2022

Despite the progresses on pre-trained language models, there is a lack of unified frameworks for pre-trained sentence representation. As such, it calls for different pre-training methods for specific scenarios, and the pre-trained models are likely to be limited by their universality and representation quality. In this work, we extend the recently proposed MAE style pre-training strategy, RetroMAE, such that it may effectively support a wide variety of sentence representation tasks. The extended framework consists of two stages, with RetroMAE conducted throughout the process. The first stage performs RetroMAE over generic corpora, like Wikipedia, BookCorpus, etc., from which the base model is learned. The second stage takes place on domain-specific data, e.g., MS MARCO and NLI, where the base model is continuingly trained based on RetroMAE and contrastive learning. The pre-training outputs at the two stages may serve different applications, whose effectiveness are verified with comprehensive experiments. Concretely, the base model are proved to be effective for zero-shot retrieval, with remarkable performances achieved on BEIR benchmark. The continuingly pre-trained models further benefit more downstream tasks, including the domain-specific dense retrieval on MS MARCO, Natural Questions, and the sentence embeddings' quality for standard STS and transfer tasks in SentEval. The empirical insights of this work may inspire the future design of sentence representation pre-training. Our pre-trained models and source code will be released to the public communities.
'PeriHack': Designing a Serious Game for Cybersecurity Awareness
Venue: arXiv cs.CR 2022
Year: 2022

This paper describes the design process for the cybersecurity serious game 'PeriHack'. Publicly released under a CC (BY-NC-SA) license, PeriHack is a board and card game for two players or teams that simulates the struggle between a red team (attackers) and a blue team (defenders). The game requires players to explore a sample network looking for vulnerabilities and then chain different attacks to exploit possible weaknesses of different nature, which may include both technical and social engineering exploits. At the same time, it also simulates budget level constraints for the blue team by providing limited resources to evaluate and prioritize different critical vulnerabilities. The game is discussed via the lenses of the AGE and 6-11 Frameworks and was primarily designed as a learning tool for students in the cybersecurity and technology related fields.
Optimizing Differentially-Maintained Recursive Queries on Dynamic Graphs
Venue: arXiv cs.DB 2022
Year: 2022

Differential computation (DC) is a highly general incremental computation/view maintenance technique that can maintain the output of an arbitrary and possibly recursive dataflow computation upon changes to its base inputs. As such, it is a promising technique for graph database management systems (GDBMS) that support continuous recursive queries over dynamic graphs. Although differential computation can be highly efficient for maintaining these queries, it can require a prohibitively large amount of memory. This paper studies how to reduce the memory overhead of DC with the goal of increasing the scalability of systems that adopt it. We propose a suite of optimizations that are based on dropping the differences of operators, both completely or partially, and recomputing these differences when necessary. We propose deterministic and probabilistic data structures to keep track of the dropped differences. Extensive experiments demonstrate that the optimizations can improve the scalability of a DC-based continuous query processor.
Towards Visualization of Time-Series Ecological Momentary Assessment (EMA) Data on Standalone Voice-First Virtual Assistants
Venue: arXiv cs.HC 2022
Year: 2022

Population aging is an increasingly important consideration for health care in the 21th century, and continuing to have access and interact with digital health information is a key challenge for aging populations. Voice-based Intelligent Virtual Assistants (IVAs) are promising to improve the Quality of Life (QoL) of older adults, and coupled with Ecological Momentary Assessments (EMA) they can be effective to collect important health information from older adults, especially when it comes to repeated time-based events. However, this same EMA data is hard to access for the older adult: although the newest IVAs are equipped with a display, the effectiveness of visualizing time-series based EMA data on standalone IVAs has not been explored. To investigate the potential opportunities for visualizing time-series based EMA data on standalone IVAs, we designed a prototype system, where older adults are able to query and examine the time-series EMA data on Amazon Echo Show - a widely used commercially available standalone screen-based IVA. We conducted a preliminary semi-structured interview with a geriatrician and an older adult, and identified three findings that should be carefully considered when designing such visualizations.
Delving into Effective Gradient Matching for Dataset Condensation
Venue: arXiv cs.LG 2022
Year: 2022

As deep learning models and datasets rapidly scale up, network training is extremely time-consuming and resource-costly. Instead of training on the entire dataset, learning with a small synthetic dataset becomes an efficient solution. Extensive research has been explored in the direction of dataset condensation, among which gradient matching achieves state-of-the-art performance. The gradient matching method directly targets the training dynamics by matching the gradient when training on the original and synthetic datasets. However, there are limited deep investigations into the principle and effectiveness of this method. In this work, we delve into the gradient matching method from a comprehensive perspective and answer the critical questions of what, how, and where to match. We propose to match the multi-level gradients to involve both intra-class and inter-class gradient information. We demonstrate that the distance function should focus on the angle, considering the magnitude simultaneously to delay the overfitting. An overfitting-aware adaptive learning step strategy is also proposed to trim unnecessary optimization steps for algorithmic efficiency improvement. Ablation and comparison experiments demonstrate that our proposed methodology shows superior accuracy, efficiency, and generalization compared to prior work.