Miracle-dz/UniFAR-FTUs huggingface dataset:
TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors

Simulation has the potential to massively scale evaluation of self-driving systems, enabling rapid development as well as safe deployment. Bridging the gap between simulation and the real world requires realistic multi-agent behaviors. Existing simulation environments rely on heuristic-based models that directly encode traffic rules, which cannot capture irregular maneuvers (e.g., nudging, U-turns) and complex interactions (e.g., yielding, merging). In contrast, we leverage real-world data to learn directly from human demonstration, and thus capture more naturalistic driving behaviors. To this end, we propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we parameterize the policy with an implicit la-tent variable model that generates socially-consistent plans for all actors in the scene jointly. To learn a robust policy amenable for long horizon simulation, we unroll the policy in training and optimize through the fully differentiable simulation across time. Our learning objective incorporates both human demonstrations as well as common sense. We show TrafficSim generates significantly more realistic traffic scenarios as compared to a diverse set of baselines. Notably, we can exploit trajectories generated by TrafficSim as effective data augmentation for training better motion planner.
Advances in Video Compression System Using Deep Neural Network: A Review and Case Studies

Significant advances in video compression systems have been made in the past several decades to satisfy the near-exponential growth of Internet-scale video traffic. From the application perspective, we have identified three major functional blocks, including preprocessing, coding, and postprocessing, which have been continuously investigated to maximize the end-user quality of experience (QoE) under a limited bit rate budget. Recently, artificial intelligence (AI)-powered techniques have shown great potential to further increase the efficiency of the aforementioned functional blocks, both individually and jointly. In this article, we review recent technical advances in video compression systems extensively, with an emphasis on deep neural network (DNN)-based approaches, and then present three comprehensive case studies. On preprocessing, we show a switchable texture-based video coding example that leverages DNN-based scene understanding to extract semantic areas for the improvement of a subsequent video coder. On coding, we present an end-to-end neural video coding framework that takes advantage of the stacked DNNs to efficiently and compactly code input raw videos via fully data-driven learning. On postprocessing, we demonstrate two neural adaptive filters to, respectively, facilitate the in-loop and postfiltering for the enhancement of compressed frames. Finally, a companion website hosting the contents developed in this work can be accessed publicly at https://purdueviper.github. io/dnn-coding/.
ConE: A Concurrent Edit Detection Tool for Large-scale Software Development

Modern, complex software systems are being continuously extended and adjusted. The developers responsible for this may come from different teams or organizations, and may be distributed over the world. This may make it difficult to keep track of what other developers are doing, which may result in multiple developers concurrently editing the same code areas. This, in turn, may lead to hard-to-merge changes or even merge conflicts, logical bugs that are difficult to detect, duplication of work, and wasted developer productivity. To address this, we explore the extent of this problem in the pull-request-based software development model. We study half a year of changes made to six large repositories in Microsoft in which at least 1,000 pull requests are created each month. We find that files concurrently edited in different pull requests are more likely to introduce bugs. Motivated by these findings, we design, implement, and deploy a service named Concurrent Edit Detector (ConE) that proactively detects pull requests containing concurrent edits, to help mitigate the problems caused by them. ConE has been designed to scale, and to minimize false alarms while still flagging relevant concurrently edited files. Key concepts of ConE include the detection of the Extent of Overlap between pull requests, and the identification of Rarely Concurrently Edited Files. To evaluate ConE, we report on its operational deployment on 234 repositories inside Microsoft. ConE assessed 26,000 pull requests and made 775 recommendations about conflicting changes, which were rated as useful in over 70% (554) of the cases. From interviews with 48 users, we learned that they believed ConE would save time in conflict resolution and avoiding duplicate work, and that over 90% intend to keep using the service on a daily basis.
JITuNE: Just-In-Time Hyperparameter Tuning for Network Embedding Algorithms

Network embedding (NE) can generate succinct node representations for massive-scale networks and enable direct applications of common machine learning methods to the network structure. Various NE algorithms have been proposed and used in a number of applications, such as node classification and link prediction. NE algorithms typically contain hyperparameters that are key to performance, but the hyperparameter tuning process can be time consuming. It is desirable to have the hyperparameters tuned within a specified length of time. Although AutoML methods have been applied to the hyperparameter tuning of NE algorithms, the problem of how to tune hyperparameters in a given period of time is not studied for NE algorithms before. In this paper, we propose JITuNE, a just-in-time hyperparameter tuning framework for NE algorithms. Our JITuNE framework enables the time-constrained hyperparameter tuning for NE algorithms by employing the tuning over hierarchical network synopses and transferring the knowledge obtained on synopses to the whole network. The hierarchical generation of synopsis and a time-constrained tuning method enable the constraining of overall tuning time. Extensive experiments demonstrate that JITuNE can significantly improve performances of NE algorithms, outperforming state-of-the-art methods within the same number of algorithm runs.
HinFlair: pre-trained contextual string embeddings for pos tagging and text classification in the Hindi language

Recent advancements in language models based on recurrent neural networks and transformers architecture have achieved state-of-the-art results on a wide range of natural language processing tasks such as pos tagging, named entity recognition, and text classification. However, most of these language models are pre-trained in high resource languages like English, German, Spanish. Multi-lingual language models include Indian languages like Hindi, Telugu, Bengali in their training corpus, but they often fail to represent the linguistic features of these languages as they are not the primary language of the study. We introduce HinFlair, which is a language representation model (contextual string embeddings) pre-trained on a large monolingual Hindi corpus. Experiments were conducted on 6 text classification datasets and a Hindi dependency treebank to analyze the performance of these contextualized string embeddings for the Hindi language. Results show that HinFlair outperforms previous state-of-the-art publicly available pre-trained embeddings for downstream tasks like text classification and pos tagging. Also, HinFlair when combined with FastText embeddings outperforms many transformers based language model trained particularly for Hindi language. The datasets and other resources used for this study are publicly available at https://github.com/harshpatel1014/ HinFlair
Informative core identification in complex networks

In network analysis, the core structure of modeling interest is usually hidden in a larger network in which most structures are not informative. The noise and bias introduced by the non-informative component in networks can obscure the salient structure and limit many network modeling procedures’ effectiveness. This paper introduces a novel core-periphery model for the non-informative periphery structure of networks without imposing a specific form for the informative core structure. We propose spectral algorithms for core identification as a data preprocessing step for general downstream network analysis tasks based on the model. The algorithm enjoys a strong theoretical guarantee of accuracy and is scalable for large networks. We evaluate the proposed method by extensive simulation studies demonstrating various advantages over many traditional core-periphery methods. The method is applied to extract the informative core structure from a citation network and give more informative results in the downstream hierarchical community detection. 1 ar X iv :2 10 1. 06 38 8v 1 [ st at . M L ] 1 6 Ja n 20 21
MP3: A Unified Model to Map, Perceive, Predict and Plan

High-definition maps (HD maps) are a key component of most modern self-driving systems due to their valuable semantic and geometric information. Unfortunately, building HD maps has proven hard to scale due to their cost as well as the requirements they impose in the localization system that has to work everywhere with centimeter-level accuracy. Being able to drive without an HD map would be very beneficial to scale self-driving solutions as well as to increase the failure tolerance of existing ones (e.g., if localization fails or the map is not up-to-date). Towards this goal, we propose MP3, an end-to-end approach to mapless1 driving where the input is raw sensor data and a high-level command (e.g., turn left at the intersection). MP3 predicts intermediate representations in the form of an online map and the current and future state of dynamic agents, and exploits them in a novel neural motion planner to make interpretable decisions taking into account uncertainty. We show that our approach is significantly safer, more comfortable, and can follow commands better than the baselines in challenging long-term closed-loop simulations, as well as when compared to an expert driver in a large-scale real-world dataset.
$(\epsilon, n)$ Fixed-Length Strong Coordination Capacity

This paper investigates the problem of synthesizing joint distributions in the finite-length regime. For a fixed blocklength $n$ and an upper bound on the distribution approximation $\epsilon$, we prove a capacity result for fixed-length strong coordination. It is shown analytically that the rate conditions for the fixed-length regime are lower-bounded by the mutual information that appears in the asymptotical condition plus $Q^{-1} \left(\epsilon \right) \sqrt{ V/n}$, where $V$ is the channel dispersion, and $Q^{-1}$ is the inverse of the Gaussian cumulative distribution function.
In situ AC-hysteresis measurements of SPD-processed Cu20(Fe15Co85)80

The changes of magnetic properties upon heat treatment of a metastable supersaturated solid solution processed by severe plastic deformation are investigated by in-situ AC-hysteresis measurements. Data are analyzed in the framework of dynamic loss theory, with correlative investigations of the microstructural properties. The evolution of hysteresis upon annealing points out that the single-phase supersaturated solid solution remains stable up to 400◦C, then hindering of domain wall motion sets in at this temperature. At 600◦C, a multi phase microstructure is present, causing a significant increase in coercivity.
G ENIE A Leaderboard for Human-in-the-Loop Evaluation of Text Generation

Leaderboards have eased model development for many NLP datasets by standardizing their evaluation and delegating it to an independent external repository. Their adoption, however, is so far limited to tasks which can be reliably evaluated in an automatic manner. This work introduces G ENIE , an extensible human evaluation leaderboard, which brings the ease of leaderboards to text generation tasks. G E - NIE automatically posts leaderboard submissions to crowdsourcing platforms asking human annotators to evaluate them on various axes (e.g., correctness, conciseness, ﬂuency), and compares their answers to various automatic metrics. We introduce several datasets in English to G ENIE , representing four core challenges in text generation: machine translation, summarization, commonsense reasoning, and machine comprehension. We provide formal granular evaluation metrics and identify areas for future research. We make G ENIE publicly available, 1 and hope that it will spur progress in language generation models as well as their automatic and manual evaluation.
Online Caching with Optimal Switching Regret

We consider the classical uncoded caching problem from an online learning point-of-view. A cache of limited storage capacity can hold $C$ files at a time from a large catalog. A user requests an arbitrary file from the catalog at each time slot. The request sequence could be adversarial. Before the file request from the user arrives, a caching policy populates the cache with any $C$ files of its choice. In the case of a cache-hit, the policy receives a unit reward and zero rewards otherwise. In addition to that, there is a cost associated with fetching files to the cache, which we refer to as the switching cost. The objective is to design a caching policy that incurs minimal regret while considering both the rewards due to cache-hits and the switching cost due to the file fetches. The main contribution of this paper is the switching regret analysis of a Follow The Perturbed LEADER-based anytime caching policy, which is shown to have an order optimal switching regret. In this pursuit, we improve the best-known switching regret bound for this problem by a factor of Θ(√C). We conclude the paper by comparing the performance of different popular caching policies using a publicly available trace from a commercial CDN server.
Unsupervised Link and Unlink Prediction on Dynamic Networks

Understanding and characterizing the process deriving the creation and dissolution of social interactions is a fundamental challenge for social network analysis. In the dynamic setting, it is essential to be able to, given the collection of link states of the network in a certain period, accurately predict the link and unlink states of the network in a future time. Addressing this task is more complicated compared to its static counterpart especially for increasingly large networks due to the prohibitive expensiveness of computational complexity. Consequently, mainstreams of current researches in unsupervised settings ignore the temporal information. Additionally, only a few approaches study on unlink prediction, which is also important to understand the evolution of social networks. In this work, we address such mining tasks by unsupervised learning, and propose a model for l ink and u nlink p rediction with t emporal information (LUPT). Given a sequence of snapshots of network over time, LUPT utilizes the spectral diﬀusion by variants of local random walks to calculate the probability vector started from each node at each snapshot. Then, it calculates the similarity score for each of the nodes by the probability vectors of all the previous snapshots. Finally, LUPT predicts the link and unlink states by ranking the similarity scores according to the link and unlink tasks, respectively. Experiments on real-world networks demonstrate that LUPT provides superior
GeoSim: Photorealistic Image Simulation with Geometry-Aware Composition

Scalable sensor simulation is an important yet challenging open problem for safety-critical domains such as self-driving. Current work in image simulation either fail to be photorealistic or do not model the 3D environment and the dynamic objects within, losing high-level control and physical realism. In this paper we present GeoSim, a geometry-aware image composition process which synthesizes novel urban driving scenes by augmenting existing images with dynamic objects extracted from other scenes and rendered at novel poses. Towards this goal, we ﬁrst build a diverse bank of 3D objects with both realistic geometry and appearance from sensor data. During simulation, we perform a novel geometry-aware simulation-by-composition procedure which 1) proposes plausible and realistic object placements into a given scene, 2) renders novel views of dynamic objects from the asset bank, and 3) composes and blends the rendered image segments. The resulting synthetic images are photorealistic, trafﬁc aware, and geometrically consistent, allowing image simulation to scale to complex use cases. We demonstrate two such important applications: long-range realistic video simulation across multiple camera sensors, and synthetic data generation for data augmentation on downstream segmentation tasks.
AGChain: A Blockchain-based Gateway for Permanent, Distributed, and Secure App Delegation from Existing Mobile App Markets

Mobile app markets are emerging with the popularity of smartphones. However, they fall short in several aspects, including no transparent app listing, no world-wide app access, and even insecure app downloading. To address these problems, we propose a novel blockchain-based gateway, AGChain, to bridge end users and app markets so that existing app markets could still provide services while users enjoy permanent, distributed, and secure app delegation from AGChain. To this end, we identify two previously underestimated challenges and propose mechanisms to significantly reduce gas costs in our smart contract and make IPFS (Inter-planetary File System) based file storage really distributed. We also address three AGChain-specific system challenges to make it secure and sustainable. We have implemented an AGChain prototype ( https://www.agchain.ltd/ ) on Ethereum. The evaluation shows that it achieves security and decentralization with minimal gas costs and reasonable performance.
Fusing Wav2vec2.0 and BERT into End-to-end Model for Low-resource Speech Recognition

—Self-supervised acoustic pre-training has achieved impressive results on low-resource speech recognition tasks. It indicates that the pretrain-and-ﬁnetune paradigm is a promising direction. In this work, we propose an end-to-end model for the low-resource speech recognition, which fuses a pre-trained audio encoder (wav2vec2.0) and a pre-trained text decoder (BERT). The two modules are connected by a linear attention mechanism without parameters. A fully connected layer is introduced for hidden mapping between speech and language modalities. Besides, we design an effective ﬁne-tuning strategy to preserve and utilize the text context modeling ability of the pre-trained decoder. Armed with this strategy, our model exhibits distinct faster convergence and better performance. Our model achieves approaching recognition performance in CALLHOME corpus (15h) as the SOTA pipeline modeling.
Novel Bayesian Procrustes Variance-based Inferences in Geometric Morphometrics&Novel R package: BPviGM1

Compared to abundant classical statistics-based literature, to date, very little Bayesian literature exists on Procrustes shape analysis in Geometric Morphometrics, probably because of being a relatively new branch of statistical research and because of inherent computational difficulty associated with Bayesian analysis. Moreover, we may obtain a plethora of novel inferences from Bayesian Procrustes analysis of shape parameter distributions. In this paper, we propose to regard the posterior of Procrustes shape variance as morphological variability indicators. Here we propose novel Bayesian methodologies for Procrustes shape analysis based on landmark data's isotropic variance assumption and propose a Bayesian statistical test for model validation of new species discovery using morphological variation reflected in the posterior distribution of landmark-variance of objects studied under Geometric Morphometrics. We will consider Gaussian distribution-based and heavy-tailed t distribution-based models for Procrustes analysis. To date, we are not aware of any direct R package for Bayesian Procrustes analysis for landmark-based Geometric Morphometrics. Hence, we introduce a novel, simple R package \textbf{BPviGM1} ("Bayesian Procrustes Variance-based inferences in Geometric Morphometrics 1"), which essentially contains the R code implementations of the computations for proposed models and methodologies, such as R function for Markov Chain Monte Carlo (MCMC) run for drawing samples from posterior of parameters of concern and R function for the proposed Bayesian test of model validation based on significance morphological variation. As an application, we can quantitatively show that primate male-face may be genetically viable to more shape-variation than the same for females.
Martingale solution of stochastic hybrid Korteweg - de Vries - Burgers equation

In the paper, we consider a stochastic hybrid Korteweg - de Vries - Burgers type equation with multiplicative noise in the form of cylindrical Wiener process. We prove the existence of a martingale solution to the equation studied. The proof of the existence of the solution is based on two approximations of the considered problem and the compactness method. First, we introduce an auxiliary problem corresponding to the equation studied. Then, we prove the existence of a martingale solution to this problem. Finally, we show that the solution of the auxiliary problem converges, in some sense, to the solution of the equation under consideration.
The stellar distribution function and local vertical potential from Gaia DR2

We develop a novel method to simultaneously determine the vertical potential, force and stellar $z-v_z$ phase space distribution function (DF) in our local patch of the Galaxy. We assume that the Solar Neighborhood can be treated as a one-dimensional system in dynamical equilibrium and directly fit the number density in the $z-v_z$ plane to what we call the Rational Linear DF (RLDF) model. This model can be regarded as a continuous sum of isothermal DFs though it has only one more parameter than the isothermal model. We apply our method to a sample of giant stars from Gaia Data Release 2 and show that the RLDF provides an excellent fit to the data. The well-known phase space spiral emerges in the residual map of the $z-v_z$ plane. We use the best-fit potential to plot the residuals in terms of the frequency and angle of vertical oscillations and show that the spiral maps into a straight line. From its slope, we estimate that the phase spirals were generated by a perturbation $\sim540$ Myr years ago. We also determine the differential surface density as a function of vertical velocity dispersion, a.k. a. the vertical temperature distribution. The result is qualitatively similar to what was previously found for SDSS/SEGUE G dwarfs. Finally, we address parameter degeneracies and the validity of the 1D approximation. Particularly, the mid-plane density derived from a cold subsample, where the 1D approximation is more secure, is closer to literature values than that derived from the sample as a whole.