OpenRaiser/Intern-Atlas huggingface dataset:
Data Augmentation for Neural NLP
Venue: arXiv cs.CL 2023
Year: 2023

Data scarcity is a problem that occurs in languages and tasks where we do not have large amounts of labeled data but want to use state-of-the-art models. Such models are often deep learning models that require a significant amount of data to train. Acquiring data for various machine learning problems is accompanied by high labeling costs. Data augmentation is a low-cost approach for tackling data scarcity. This paper gives an overview of current state-of-the-art data augmentation methods used for natural language processing, with an emphasis on methods for neural and transformer-based models. Furthermore, it discusses the practical challenges of data augmentation, possible mitigations, and directions for future research.
Saliency Guided Contrastive Learning on Scene Images
Venue: arXiv cs.CV 2023
Year: 2023

Self-supervised learning holds promise in leveraging large numbers of unlabeled data. However, its success heavily relies on the highly-curated dataset, e.g., ImageNet, which still needs human cleaning. Directly learning representations from less-curated scene images is essential for pushing self-supervised learning to a higher level. Different from curated images which include simple and clear semantic information, scene images are more complex and mosaic because they often include complex scenes and multiple objects. Despite being feasible, recent works largely overlooked discovering the most discriminative regions for contrastive learning to object representations in scene images. In this work, we leverage the saliency map derived from the model's output during learning to highlight these discriminative regions and guide the whole contrastive learning. Specifically, the saliency map first guides the method to crop its discriminative regions as positive pairs and then reweighs the contrastive losses among different crops by its saliency scores. Our method significantly improves the performance of self-supervised learning on scene images by +1.1, +4.3, +2.2 Top1 accuracy in ImageNet linear evaluation, Semi-supervised learning with 1% and 10% ImageNet labels, respectively. We hope our insights on saliency maps can motivate future research on more general-purpose unsupervised representation learning from scene data.
Distilling Calibrated Student from an Uncalibrated Teacher
Venue: arXiv cs.CV 2023
Year: 2023

Knowledge distillation is a common technique for improving the performance of a shallow student network by transferring information from a teacher network, which in general, is comparatively large and deep. These teacher networks are pre-trained and often uncalibrated, as no calibration technique is applied to the teacher model while training. Calibration of a network measures the probability of correctness for any of its predictions, which is critical in high-risk domains. In this paper, we study how to obtain a calibrated student from an uncalibrated teacher. Our approach relies on the fusion of the data-augmentation techniques, including but not limited to cutout, mixup, and CutMix, with knowledge distillation. We extend our approach beyond traditional knowledge distillation and find it suitable for Relational Knowledge Distillation and Contrastive Representation Distillation as well. The novelty of the work is that it provides a framework to distill a calibrated student from an uncalibrated teacher model without compromising the accuracy of the distilled student. We perform extensive experiments to validate our approach on various datasets, including CIFAR-10, CIFAR-100, CINIC-10 and TinyImageNet, and obtained calibrated student models. We also observe robust performance of our approach while evaluating it on corrupted CIFAR-100C data.
Transformer-Based Sensor Fusion for Autonomous Driving: A Survey
Venue: arXiv cs.CV 2023
Year: 2023

Sensor fusion is an essential topic in many perception systems, such as autonomous driving and robotics. Transformers-based detection head and CNN-based feature encoder to extract features from raw sensor-data has emerged as one of the best performing sensor-fusion 3D-detection-framework, according to the dataset leaderboards. In this work we provide an in-depth literature survey of transformer based 3D-object detection task in the recent past, primarily focusing on the sensor fusion. We also briefly go through the Vision transformers (ViT) basics, so that readers can easily follow through the paper. Moreover, we also briefly go through few of the non-transformer based less-dominant methods for sensor fusion for autonomous driving. In conclusion we summarize with sensor-fusion trends to follow and provoke future research. More updated summary can be found at: https://github.com/ApoorvRoboticist/Transformers-Sensor-Fusion
Greedy Discovery of Ordinal Factors
Venue: arXiv cs.LG 2023
Year: 2023

In large datasets, it is hard to discover and analyze structure. It is thus common to introduce tags or keywords for the items. In applications, such datasets are then filtered based on these tags. Still, even medium-sized datasets with a few tags result in complex and for humans hard-to-navigate systems. In this work, we adopt the method of ordinal factor analysis to address this problem. An ordinal factor arranges a subset of the tags in a linear order based on their underlying structure. A complete ordinal factorization, which consists of such ordinal factors, precisely represents the original dataset. Based on such an ordinal factorization, we provide a way to discover and explain relationships between different items and attributes in the dataset. However, computing even just one ordinal factor of high cardinality is computationally complex. We thus propose the greedy algorithm in this work. This algorithm extracts ordinal factors using already existing fast algorithms developed in formal concept analysis. Then, we leverage to propose a comprehensive way to discover relationships in the dataset. We furthermore introduce a distance measure based on the representation emerging from the ordinal factorization to discover similar items. To evaluate the method, we conduct a case study on different datasets.
CryptoScratch: Developing and evaluating a block-based programming tool for teaching K-12 cryptography education using Scratch
Venue: arXiv cs.CR 2023
Year: 2023

This paper presents the design, implementation, and evaluation of a new framework called CryptoScratch, which extends the Scratch programming environment with modern cryptographic algorithms (e.g., AES, RSA, SHA-256) implemented as visual blocks. Using the simple interface of CryptoScratch, K-12 students can study how to use cryptographic algorithms for services like confidentiality, authentication, and integrity protection; and then use these blocks to build complex modern cryptographic schemes (e.g., Pretty Good Privacy, Digital Signatures). In addition, we present the design and implementation of a Task Block that provides students instruction on various cryptography problems and verifies that they have successfully completed the problem. The task block also generates feedback, nudging learners to implement more secure solutions for cryptographic problems. An initial usability study was performed with 16 middle-school students where students were taught basic cryptographic concepts and then asked to complete tasks using those concepts. Once students had knowledge of a variety of basic cryptographic algorithms, they were asked to use those algorithms to implement complex cryptographic schemes such as Pretty Good Privacy and Digital Signatures. Using the successful implementation of the cryptographic and task blocks in Scratch, the initial testing indicated that $\approx 60\%$ of the students could quickly grasp and implement complex cryptography concepts using CryptoScratch, while $\approx 90\%$ showed comfort with cryptography concepts and use-cases. Based on the positive results from the initial testing, a larger study of students is being developed to investigate the effectiveness across the socioeconomic spectrum.
Pattern detection in ordered graphs
Venue: arXiv cs.DS 2023
Year: 2023

A popular way to define or characterize graph classes is via forbidden subgraphs or forbidden minors. These characterizations play a key role in graph theory, but they rarely lead to efficient algorithms to recognize these classes. In contrast, many essential graph classes can be recognized efficiently thanks to characterizations of the following form: there must exist an ordering of the vertices such that some ordered pattern does not appear, where a pattern is basically an ordered subgraph. These pattern characterizations have been studied for decades, but there have been recent efforts to better understand them systematically. In this paper, we focus on a simple problem at the core of this topic: given an ordered graph of size $n$, how fast can we detect whether a fixed pattern of size $k$ is present? Following the literature on graph classes recognition, we first look for patterns that can be detected in linear time. We prove, among other results, that almost all patterns on three vertices (which capture many interesting classes, such as interval, chordal, split, bipartite, and comparability graphs) fall in this category. Then, in a finer-grained complexity perspective, we prove conditional lower bounds for this problem. In particular we show that for a large family of patterns on four vertices it is unlikely that subquadratic algorithm exist. Finally, we define a parameter for patterns, the merge-width, and prove that for patterns of merge-width $t$, one can solve the problem in $O(n^{ct})$ for some constant~$c$. As a corollary, we get that detecting outerplanar patterns and other classes of patterns can be done in time independent of the size of the pattern.
Unsupervised 3D Object Learning through Neuron Activity aware Plasticity
Venue: arXiv cs.AI 2023
Year: 2023

We present an unsupervised deep learning model for 3D object classification. Conventional Hebbian learning, a well-known unsupervised model, suffers from loss of local features leading to reduced performance for tasks with complex geometric objects. We present a deep network with a novel Neuron Activity Aware (NeAW) Hebbian learning rule that dynamically switches the neurons to be governed by Hebbian learning or anti-Hebbian learning, depending on its activity. We analytically show that NeAW Hebbian learning relieves the bias in neuron activity, allowing more neurons to attend to the representation of the 3D objects. Empirical results show that the NeAW Hebbian learning outperforms other variants of Hebbian learning and shows higher accuracy over fully supervised models when training data is limited.
Finding a Small Vertex Cut on Distributed Networks
Venue: arXiv cs.DS 2023
Year: 2023

We present an algorithm for distributed networks to efficiently find a small vertex cut in the CONGEST model. Given a positive integer $κ$, our algorithm can, with high probability, either find $κ$ vertices whose removal disconnects the network or return that such $κ$ vertices do not exist. Our algorithm takes $κ^3\cdot \tilde{O}(D+\sqrt{n})$ rounds, where $n$ is the number of vertices in the network and $D$ denotes the network's diameter. This implies $\tilde{O}(D+\sqrt{n})$ round complexity whenever $κ=\text{polylog}(n)$. Prior to our result, a bound of $\tilde{O}(D)$ is known only when $κ=1,2$ [Parter, Petruschka DISC'22]. For $κ\geq 3$, this bound can be obtained only by an $O(\log n)$-approximation algorithm [Censor-Hillel, Ghaffari, Kuhn PODC'14], and the only known exact algorithm takes $O\left((κΔD)^{O(κ)}\right)$ rounds, where $Δ$ is the maximum degree [Parter DISC'19]. Our result answers an open problem by Nanongkai, Saranurak, and Yingchareonthawornchai [STOC'19].
A Deep Neural Network Based Approach to Building Budget-Constrained Models for Big Data Analysis
Venue: arXiv cs.LG 2023
Year: 2023

Deep learning approaches require collection of data on many different input features or variables for accurate model training and prediction. Since data collection on input features could be costly, it is crucial to reduce the cost by selecting a subset of features and developing a budget-constrained model (BCM). In this paper, we introduce an approach to eliminating less important features for big data analysis using Deep Neural Networks (DNNs). Once a DNN model has been developed, we identify the weak links and weak neurons, and remove some input features to bring the model cost within a given budget. The experimental results show our approach is feasible and supports user selection of a suitable BCM within a given budget.
Shield Model Predictive Path Integral: A Computationally Efficient Robust MPC Approach Using Control Barrier Functions
Venue: arXiv cs.RO 2023
Year: 2023

Model Predictive Path Integral (MPPI) control is a type of sampling-based model predictive control that simulates thousands of trajectories and uses these trajectories to synthesize optimal controls on-the-fly. In practice, however, MPPI encounters problems limiting its application. For instance, it has been observed that MPPI tends to make poor decisions if unmodeled dynamics or environmental disturbances exist, preventing its use in safety-critical applications. Moreover, the multi-threaded simulations used by MPPI require significant onboard computational resources, making the algorithm inaccessible to robots without modern GPUs. To alleviate these issues, we propose a novel (Shield-MPPI) algorithm that provides robustness against unpredicted disturbances and achieves real-time planning using a much smaller number of parallel simulations on regular CPUs. The novel Shield-MPPI algorithm is tested on an aggressive autonomous racing platform both in simulation and using experiments. The results show that the proposed controller greatly reduces the number of constraint violations compared to state-of-the-art robust MPPI variants and stochastic MPC methods.
On Code Reuse from StackOverflow: An Exploratory Study on Jupyter Notebook
Venue: arXiv cs.SE 2023
Year: 2023

Jupyter Notebook is a popular tool among data analysts and scientists for working with data. It provides a way to combine code, documentation, and visualizations in a single, interactive environment, facilitating code reuse. While code reuse can improve programming efficiency, it can also decrease readability, security, and overall performance. We conduct a large-scale exploratory study of code reuse practices in the Jupyter Notebook development community on the Stack Overflow platform to understand the potential negative impacts of code reuse. Our findings identified 1,097,470 Jupyter Notebook clone pairs that reuse Stack Overflow code snippets, and the average code snippet has 7.91 code quality violations. Through our research, we gain insight into the reasons behind Jupyter Notebook developers' decision to reuse code and the potential drawbacks of this practice.
Hera: A Heterogeneity-Aware Multi-Tenant Inference Server for Personalized Recommendations
Venue: arXiv cs.DC 2023
Year: 2023

While providing low latency is a fundamental requirement in deploying recommendation services, achieving high resource utility is also crucial in cost-effectively maintaining the datacenter. Co-locating multiple workers of a model is an effective way to maximize query-level parallelism and server throughput, but the interference caused by concurrent workers at shared resources can prevent server queries from meeting its SLA. Hera utilizes the heterogeneous memory requirement of multi-tenant recommendation models to intelligently determine a productive set of co-located models and its resource allocation, providing fast response time while achieving high throughput. We show that Hera achieves an average 37.3% improvement in effective machine utilization, enabling 26% reduction in required servers, significantly improving upon the baseline recommedation inference server.
Detecting software vulnerabilities using Language Models
Venue: arXiv cs.CR 2023
Year: 2023

Recently, deep learning techniques have garnered substantial attention for their ability to identify vulnerable code patterns accurately. However, current state-of-the-art deep learning models, such as Convolutional Neural Networks (CNN), and Long Short-Term Memories (LSTMs) require substantial computational resources. This results in a level of overhead that makes their implementation unfeasible for deployment in realtime settings. This study presents a novel transformer-based vulnerability detection framework, referred to as VulDetect, which is achieved through the fine-tuning of a pre-trained large language model, (GPT) on various benchmark datasets of vulnerable code. Our empirical findings indicate that our framework is capable of identifying vulnerable software code with an accuracy of up to 92.65%. Our proposed technique outperforms SyseVR and VulDeBERT, two state-of-the-art vulnerability detection techniques
Efficient Context Integration through Factorized Pyramidal Learning for Ultra-Lightweight Semantic Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Semantic segmentation is a pixel-level prediction task to classify each pixel of the input image. Deep learning models, such as convolutional neural networks (CNNs), have been extremely successful in achieving excellent performances in this domain. However, mobile application, such as autonomous driving, demand real-time processing of incoming stream of images. Hence, achieving efficient architectures along with enhanced accuracy is of paramount importance. Since, accuracy and model size of CNNs are intrinsically contentious in nature, the challenge is to achieve a decent trade-off between accuracy and model size. To address this, we propose a novel Factorized Pyramidal Learning (FPL) module to aggregate rich contextual information in an efficient manner. On one hand, it uses a bank of convolutional filters with multiple dilation rates which leads to multi-scale context aggregation; crucial in achieving better accuracy. On the other hand, parameters are reduced by a careful factorization of the employed filters; crucial in achieving lightweight models. Moreover, we decompose the spatial pyramid into two stages which enables a simple and efficient feature fusion within the module to solve the notorious checkerboard effect. We also design a dedicated Feature-Image Reinforcement (FIR) unit to carry out the fusion operation of shallow and deep features with the downsampled versions of the input image. This gives an accuracy enhancement without increasing model parameters. Based on the FPL module and FIR unit, we propose an ultra-lightweight real-time network, called FPLNet, which achieves state-of-the-art accuracy-efficiency trade-off. More specifically, with only less than 0.5 million parameters, the proposed network achieves 66.93\% and 66.28\% mIoU on Cityscapes validation and test set, respectively. Moreover, FPLNet has a processing speed of 95.5 frames per second (FPS).
Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification
Venue: arXiv cs.CV 2023
Year: 2023

Motion-based association for Multi-Object Tracking (MOT) has recently re-achieved prominence with the rise of powerful object detectors. Despite this, little work has been done to incorporate appearance cues beyond simple heuristic models that lack robustness to feature degradation. In this paper, we propose a novel way to leverage objects' appearances to adaptively integrate appearance matching into existing high-performance motion-based methods. Building upon the pure motion-based method OC-SORT, we achieve 1st place on MOT20 and 2nd place on MOT17 with 63.9 and 64.9 HOTA, respectively. We also achieve 61.3 HOTA on the challenging DanceTrack benchmark as a new state-of-the-art even compared to more heavily-designed methods. The code and models are available at \url{https://github.com/GerardMaggiolino/Deep-OC-SORT}.
Generalization of Auto-Regressive Hidden Markov Models to Non-Linear Dynamics and Unit Quaternion Observation Space
Venue: arXiv cs.RO 2023
Year: 2023

Latent variable models are widely used to perform unsupervised segmentation of time series in different context such as robotics, speech recognition, and economics. One of the most widely used latent variable model is the Auto-Regressive Hidden Markov Model (ARHMM), which combines a latent mode governed by a Markov chain dynamics with a linear Auto-Regressive dynamics of the observed state. In this work, we propose two generalizations of the ARHMM. First, we propose a more general AR dynamics in Cartesian space, described as a linear combination of non-linear basis functions. Second, we propose a linear dynamics in unit quaternion space, in order to properly describe orientations. These extensions allow to describe more complex dynamics of the observed state. Although this extension is proposed for the ARHMM, it can be easily extended to other latent variable models with AR dynamics in the observed space, such as Auto-Regressive Hidden semi-Markov Models.
Minimum-Entropy Coupling Approximation Guarantees Beyond the Majorization Barrier
Venue: arXiv cs.IT 2023
Year: 2023

Given a set of discrete probability distributions, the minimum entropy coupling is the minimum entropy joint distribution that has the input distributions as its marginals. This has immediate relevance to tasks such as entropic causal inference for causal graph discovery and bounding mutual information between variables that we observe separately. Since finding the minimum entropy coupling is NP-Hard, various works have studied approximation algorithms. The work of [Compton, ISIT 2022] shows that the greedy coupling algorithm of [Kocaoglu et al., AAAI 2017] is always within $log_2(e) \approx 1.44$ bits of the optimal coupling. Moreover, they show that it is impossible to obtain a better approximation guarantee using the majorization lower-bound that all prior works have used: thus establishing a majorization barrier. In this work, we break the majorization barrier by designing a stronger lower-bound that we call the profile method. Using this profile method, we are able to show that the greedy algorithm is always within $log_2(e)/e \approx 0.53$ bits of optimal for coupling two distributions (previous best-known bound is within 1 bit), and within $(1 + log_2(e))/2 \approx 1.22$ bits for coupling any number of distributions (previous best-known bound is within 1.44 bits). We also examine a generalization of the minimum entropy coupling problem: Concave Minimum-Cost Couplings. We are able to obtain similar guarantees for this generalization in terms of the concave cost function. Additionally, we make progress on the open problem of [Kovačević et al., Inf. Comput. 2015] regarding NP membership of the minimum entropy coupling problem by showing that any hardness of minimum entropy coupling beyond NP comes from the difficulty of computing arithmetic in the complexity class NP. Finally, we present exponential-time algorithms for computing the exactly optimal solution.
Understanding Collaborative Practices and Tools of Professional UX Practitioners in Software Organizations
Venue: arXiv cs.HC 2023
Year: 2023

User experience (UX) has undergone a revolution in collaborative practices, due to tools that enable quick feedback and continuous collaboration with a varied team across a design's lifecycle. However, it is unclear how this shift in collaboration has been received in professional UX practice, and whether new pain points have arisen. To this end, we conducted a survey (N=114) with UX practitioners at software organizations based in the U.S. to better understand their collaborative practices and tools used throughout the design process. We found that while an increase in collaborative activity enhanced many aspects of UX work, some long-standing challenges -- such as handing off designs to developers -- still persist. Moreover, we observed new challenges emerging from activities enabled by collaborative tools such as design system management. Based on our findings, we discuss how UX practices can improve collaboration moving forward and provide concrete design implications for collaborative UX tools.