OpenRaiser/Intern-Atlas huggingface dataset:
Using simulation to quantify the performance of automotive perception systems
Venue: arXiv cs.CV 2023
Year: 2023

The design and evaluation of complex systems can benefit from a software simulation - sometimes called a digital twin. The simulation can be used to characterize system performance or to test its performance under conditions that are difficult to measure (e.g., nighttime for automotive perception systems). We describe the image system simulation software tools that we use to evaluate the performance of image systems for object (automobile) detection. We describe experiments with 13 different cameras with a variety of optics and pixel sizes. To measure the impact of camera spatial resolution, we designed a collection of driving scenes that had cars at many different distances. We quantified system performance by measuring average precision and we report a trend relating system resolution and object detection performance. We also quantified the large performance degradation under nighttime conditions, compared to daytime, for all cameras and a COCO pre-trained network.
Physics-informed neural networks for solving forward and inverse problems in complex beam systems
Venue: arXiv cs.LG 2023
Year: 2023

This paper proposes a new framework using physics-informed neural networks (PINNs) to simulate complex structural systems that consist of single and double beams based on Euler-Bernoulli and Timoshenko theory, where the double beams are connected with a Winkler foundation. In particular, forward and inverse problems for the Euler-Bernoulli and Timoshenko partial differential equations (PDEs) are solved using nondimensional equations with the physics-informed loss function. Higher-order complex beam PDEs are efficiently solved for forward problems to compute the transverse displacements and cross-sectional rotations with less than 1e-3 percent error. Furthermore, inverse problems are robustly solved to determine the unknown dimensionless model parameters and applied force in the entire space-time domain, even in the case of noisy data. The results suggest that PINNs are a promising strategy for solving problems in engineering structures and machines involving beam systems.
Distilling Multi-Level X-vector Knowledge for Small-footprint Speaker Verification
Venue: arXiv cs.SD 2023
Year: 2023

Even though deep speaker models have demonstrated impressive accuracy in speaker verification tasks, this often comes at the expense of increased model size and computation time, presenting challenges for deployment in resource-constrained environments. Our research focuses on addressing this limitation through the development of small footprint deep speaker embedding extraction using knowledge distillation. While previous work in this domain has concentrated on speaker embedding extraction at the utterance level, our approach involves amalgamating embeddings from different levels of the x-vector model (teacher network) to train a compact student network. The results highlight the significance of frame-level information, with the student models exhibiting a remarkable size reduction of 85%-91% compared to their teacher counterparts, depending on the size of the teacher embeddings. Notably, by concatenating teacher embeddings, we achieve student networks that maintain comparable performance to the teacher while enjoying a substantial 75% reduction in model size. These findings and insights extend to other x-vector variants, underscoring the broad applicability of our approach.
Effective Visualization and Analysis of Recommender Systems
Venue: arXiv cs.IR 2023
Year: 2023

Recommender system exists everywhere in the business world. From Goodreads to TikTok, customers of internet products become more addicted to the products thanks to the technology. Industrial practitioners focus on increasing the technical accuracy of recommender systems while at same time balancing other factors such as diversity and serendipity. In spite of the length of the research and development history of recommender systems, there has been little discussion on how to take advantage of visualization techniques to facilitate the algorithmic design of the technology. In this paper, we use a series of data analysis and visualization techniques such as Takens Embedding, Determinantal Point Process and Social Network Analysis to help people develop effective recommender systems by predicting intermediate computational cost and output performance. Our work is pioneering in the field, as to our limited knowledge, there have been few publications (if any) on visualization of recommender systems.
Algebraic Monograph Transformations
Venue: arXiv cs.LO 2023
Year: 2023

Monographs are graph-like structures with directed edges of unlimited length that are freely adjacent to each other. The standard nodes are represented as edges of length zero. They can be drawn in a way consistent with standard graphs and many others, like E-graphs or $\infty$-graphs. The category of monographs share many properties with the categories of graph structures (algebras of monadic many-sorted signatures), except that there is no terminal monograph. It is universal in the sense that its slice categories (or categories of typed monographs) are equivalent to the categories of graph structures. Type monographs thus emerge as a natural way of specifying graph structures. A detailed analysis of single and double pushout transformations of monographs is provided, and a notion of attributed typed monographs generalizing typed attributed E-graphs is analyzed w.r.t. attribute-preserving transformations.
Interpretable System Identification and Long-term Prediction on Time-Series Data
Venue: arXiv cs.LG 2023
Year: 2023

Time-series prediction has drawn considerable attention during the past decades fueled by the emerging advances of deep learning methods. However, most neural network based methods lack interpretability and fail in extracting the hidden mechanism of the targeted physical system. To overcome these shortcomings, an interpretable sparse system identification method without any prior knowledge is proposed in this study. This method adopts the Fourier transform to reduces the irrelevant items in the dictionary matrix, instead of indiscriminate usage of polynomial functions in most system identification methods. It shows an interpretable system representation and greatly reduces computing cost. With the adoption of $l_1$ norm in regularizing the parameter matrix, a sparse description of the system model can be achieved. Moreover, Three data sets including the water conservancy data, global temperature data and financial data are used to test the performance of the proposed method. Although no prior knowledge was known about the physical background, experimental results show that our method can achieve long-term prediction regardless of the noise and incompleteness in the original data more accurately than the widely-used baseline data-driven methods. This study may provide some insight into time-series prediction investigations, and suggests that an white-box system identification method may extract the easily overlooked yet inherent periodical features and may beat neural-network based black-box methods on long-term prediction tasks.
UZH_CLyp at SemEval-2023 Task 9: Head-First Fine-Tuning and ChatGPT Data Generation for Cross-Lingual Learning in Tweet Intimacy Prediction
Venue: arXiv cs.CL 2023
Year: 2023

This paper describes the submission of UZH_CLyp for the SemEval 2023 Task 9 "Multilingual Tweet Intimacy Analysis". We achieved second-best results in all 10 languages according to the official Pearson's correlation regression evaluation measure. Our cross-lingual transfer learning approach explores the benefits of using a Head-First Fine-Tuning method (HeFiT) that first updates only the regression head parameters and then also updates the pre-trained transformer encoder parameters at a reduced learning rate. Additionally, we study the impact of using a small set of automatically generated examples (in our case, from ChatGPT) for low-resource settings where no human-labeled data is available. Our study shows that HeFiT stabilizes training and consistently improves results for pre-trained models that lack domain adaptation to tweets. Our study also shows a noticeable performance increase in cross-lingual learning when synthetic data is used, confirming the usefulness of current text generation systems to improve zero-shot baseline results. Finally, we examine how possible inconsistencies in the annotated data contribute to cross-lingual interference issues.
Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection
Venue: arXiv cs.LG 2023
Year: 2023

Detecting Out-of-distribution (OOD) inputs have been a critical issue for neural networks in the open world. However, the unstable behavior of OOD detection along the optimization trajectory during training has not been explored clearly. In this paper, we first find the performance of OOD detection suffers from overfitting and instability during training: 1) the performance could decrease when the training error is near zero, and 2) the performance would vary sharply in the final stage of training. Based on our findings, we propose Average of Pruning (AoP), consisting of model averaging and pruning, to mitigate the unstable behaviors. Specifically, model averaging can help achieve a stable performance by smoothing the landscape, and pruning is certified to eliminate the overfitting by eliminating redundant features. Comprehensive experiments on various datasets and architectures are conducted to verify the effectiveness of our method.
Domain-aware Triplet loss in Domain Generalization
Venue: arXiv cs.CV 2023
Year: 2023

Despite much progress being made in the field of object recognition with the advances of deep learning, there are still several factors negatively affecting the performance of deep learning models. Domain shift is one of these factors and is caused by discrepancies in the distributions of the testing and training data. In this paper, we focus on the problem of compact feature clustering in domain generalization to help optimize the embedding space from multi-domain data. We design a domainaware triplet loss for domain generalization to help the model to not only cluster similar semantic features, but also to disperse features arising from the domain. Unlike previous methods focusing on distribution alignment, our algorithm is designed to disperse domain information in the embedding space. The basic idea is motivated based on the assumption that embedding features can be clustered based on domain information, which is mathematically and empirically supported in this paper. In addition, during our exploration of feature clustering in domain generalization, we note that factors affecting the convergence of metric learning loss in domain generalization are more important than the pre-defined domains. To solve this issue, we utilize two methods to normalize the embedding space, reducing the internal covariate shift of the embedding features. The ablation study demonstrates the effectiveness of our algorithm. Moreover, the experiments on the benchmark datasets, including PACS, VLCS and Office-Home, show that our method outperforms related methods focusing on domain discrepancy. In particular, our results on RegnetY-16 are significantly better than state-of-the-art methods on the benchmark datasets. Our code will be released at https://github.com/workerbcd/DCT
Privacy-Preserving Tree-Based Inference with TFHE
Venue: arXiv cs.CR 2023
Year: 2023

Privacy enhancing technologies (PETs) have been proposed as a way to protect the privacy of data while still allowing for data analysis. In this work, we focus on Fully Homomorphic Encryption (FHE), a powerful tool that allows for arbitrary computations to be performed on encrypted data. FHE has received lots of attention in the past few years and has reached realistic execution times and correctness. More precisely, we explain in this paper how we apply FHE to tree-based models and get state-of-the-art solutions over encrypted tabular data. We show that our method is applicable to a wide range of tree-based models, including decision trees, random forests, and gradient boosted trees, and has been implemented within the Concrete-ML library, which is open-source at https://github.com/zama-ai/concrete-ml. With a selected set of use-cases, we demonstrate that our FHE version is very close to the unprotected version in terms of accuracy.
Combining Generative Artificial Intelligence (AI) and the Internet: Heading towards Evolution or Degradation?
Venue: arXiv cs.CV 2023
Year: 2023

In the span of a few months, generative Artificial Intelligence (AI) tools that can generate realistic images or text have taken the Internet by storm, making them one of the technologies with fastest adoption ever. Some of these generative AI tools such as DALL-E, MidJourney, or ChatGPT have gained wide public notoriety. Interestingly, these tools are possible because of the massive amount of data (text and images) available on the Internet. The tools are trained on massive data sets that are scraped from Internet sites. And now, these generative AI tools are creating massive amounts of new data that are being fed into the Internet. Therefore, future versions of generative AI tools will be trained with Internet data that is a mix of original and AI-generated data. As time goes on, a mixture of original data and data generated by different versions of AI tools will populate the Internet. This raises a few intriguing questions: how will future versions of generative AI tools behave when trained on a mixture of real and AI generated data? Will they evolve with the new data sets or degenerate? Will evolution introduce biases in subsequent generations of generative AI tools? In this document, we explore these questions and report some very initial simulation results using a simple image-generation AI tool. These results suggest that the quality of the generated images degrades as more AI-generated data is used for training thus suggesting that generative AI may degenerate. Although these results are preliminary and cannot be generalised without further study, they serve to illustrate the potential issues of the interaction between generative AI and the Internet.
Reasoning-Based Software Testing
Venue: arXiv cs.SE 2023
Year: 2023

With software systems becoming increasingly pervasive and autonomous, our ability to test for their quality is severely challenged. Many systems are called to operate in uncertain and highly-changing environment, not rarely required to make intelligent decisions by themselves. This easily results in an intractable state space to explore at testing time. The state-of-the-art techniques try to keep the pace, e.g., by augmenting the tester's intuition with some form of (explicit or implicit) learning from observations to search this space efficiently. For instance, they exploit historical data to drive the search (e.g., ML-driven testing) or the tests execution data itself (e.g., adaptive or search-based testing). Despite the indubitable advances, the need for smartening the search in such a huge space keeps to be pressing. We introduce Reasoning-Based Software Testing (RBST), a new way of thinking at the testing problem as a causal reasoning task. Compared to mere intuition-based or state-of-the-art learning-based strategies, we claim that causal reasoning more naturally emulates the process that a human would do to ''smartly" search the space. RBST aims to mimic and amplify, with the power of computation, this ability. The conceptual leap can pave the ground to a new trend of techniques, which can be variously instantiated from the proposed framework, by exploiting the numerous tools for causal discovery and inference. Preliminary results reported in this paper are promising.
Active Learning Enhances Classification of Histopathology Whole Slide Images with Attention-based Multiple Instance Learning
Venue: arXiv cs.CV 2023
Year: 2023

In many histopathology tasks, sample classification depends on morphological details in tissue or single cells that are only visible at the highest magnification. For a pathologist, this implies tedious zooming in and out, while for a computational decision support algorithm, it leads to the analysis of a huge number of small image patches per whole slide image (WSI). Attention-based multiple instance learning (MIL), where attention estimation is learned in a weakly supervised manner, has been successfully applied in computational histopathology, but it is challenged by large numbers of irrelevant patches, reducing its accuracy. Here, we present an active learning approach to the problem. Querying the expert to annotate regions of interest in a WSI guides the formation of high-attention regions for MIL. We train an attention-based MIL and calculate a confidence metric for every image in the dataset to select the most uncertain WSIs for expert annotation. We test our approach on the CAMELYON17 dataset classifying metastatic lymph node sections in breast cancer. With a novel attention guiding loss, this leads to an accuracy boost of the trained models with few regions annotated for each class. Active learning thus improves WSIs classification accuracy, leads to faster and more robust convergence, and speeds up the annotation process. It may in the future serve as an important contribution to train MIL models in the clinically relevant context of cancer classification in histopathology.
Pricing in Ride-sharing Markets : Effects of network competition and autonomous vehicles
Venue: arXiv cs.GT 2023
Year: 2023

Autonomous vehicles will be an integral part of ride-sharing services in the future. This setting differs from traditional ride-sharing marketplaces because of the absence of the supply side (drivers). However, it has far-reaching consequences because in addition to pricing, players now have to make decisions on how to distribute fleets across network locations and re-balance vehicles in order to serve future demand. In this paper, we explore a duopoly setting in the ride-sharing marketplace where the players have fully autonomous fleets. Each ride-service provider (RSP)'s prices depend on the prices and the supply of the other player. We formulate their decision-making problems using a game-theoretic setup where each player seeks to find the optimal prices and supplies at each node while considering the decisions of the other player. This leads to a scenario where the players' optimization problems are coupled and it is challenging to find the equilibrium. We characterize the types of demand functions (e.g.: linear) for which this game admits an exact potential function and can be solved efficiently. For other types of demand functions, we propose an iterative algorithm to compute the equilibrium. We conclude by providing numerical insights into how different kinds of equilibria would play out in the market when the players are asymmetric. Our numerical evaluations also provide insights into how the regulator needs to consider network effects while deciding regulation in order to avoid unfavorable outcomes.
iART: Learning from Demonstration for Assisted Robotic Therapy Using LSTM
Venue: arXiv cs.RO 2023
Year: 2023

In this paper, we present an intelligent Assistant for Robotic Therapy (iART), that provides robotic assistance during 3D trajectory tracking tasks. We propose a novel LSTM-based robot learning from demonstration (LfD) paradigm to mimic a therapist's assistance behavior. iART presents a trajectory agnostic LfD routine that can generalize learned behavior from a single trajectory to any 3D shape. Once the therapist's behavior has been learned, iART enables the patient to modify this behavior as per their preference. The system requires only a single demonstration of 2 minutes and exhibits a mean accuracy of 91.41% in predicting, and hence mimicking a therapist's assistance behavior. The system delivers stable assistance in realtime and successfully reproduces different types of assistance behaviors.
Dynamic discretization discovery under hard node storage constraints
Venue: arXiv cs.DM 2023
Year: 2023

The recently developed dynamic discretization discovery (DDD) is a powerful method that allows many time-dependent problems to become more tractable. While DDD has been applied to a variety of problems, one particular challenge has been to deal with storage constraints without leading to a weak relaxation in each iteration. Specifically, the current approach to deal with certain hard storage constraints in continuous settings is to remove a subset of the storage constraints completely in each iteration of DDD. In this work, we show that for discrete problems, such weak relaxations are not necessary. Specifically, we find bounds on the additional storage that must be permitted in each iteration. We demonstrate our techniques in the case of the classical universal packet routing problem in the presence of bounded node storage, which can currently only be solved via integer programming. We present computational results demonstrating the effectiveness of DDD when solving universal packet routing.
Optimal transfer protocol by incremental layer defrosting
Venue: arXiv cs.LG 2023
Year: 2023

Transfer learning is a powerful tool enabling model training with limited amounts of data. This technique is particularly useful in real-world problems where data availability is often a serious limitation. The simplest transfer learning protocol is based on ``freezing" the feature-extractor layers of a network pre-trained on a data-rich source task, and then adapting only the last layers to a data-poor target task. This workflow is based on the assumption that the feature maps of the pre-trained model are qualitatively similar to the ones that would have been learned with enough data on the target task. In this work, we show that this protocol is often sub-optimal, and the largest performance gain may be achieved when smaller portions of the pre-trained network are kept frozen. In particular, we make use of a controlled framework to identify the optimal transfer depth, which turns out to depend non-trivially on the amount of available training data and on the degree of source-target task correlation. We then characterize transfer optimality by analyzing the internal representations of two networks trained from scratch on the source and the target task through multiple established similarity measures.