OpenRaiser/Intern-Atlas huggingface dataset:
Homomorphic Hashing Based on Elliptic Curve Cryptography
Venue: arXiv cs.CR 2023
Year: 2023

For avoiding the exposure of plaintexts in cloud environments, some homomorphic hashing algorithms have been proposed to generate the hash value of each plaintext, and cloud environments only store the hash values and calculate the hash values for future needs. However, longer hash value generation time and longer hash value summary time may be required by these homomorphic hashing algorithms with higher security strengths. Therefore, this study proposes a homomorphic hashing based on elliptic curve cryptography (ECC) to provide a homomorphic hashing function in accordance with the characteristics of ECC. Furthermore, mathematical models and practical cases have been given to prove the proposed method. In experiments, the results show that the proposed method have higher efficiency with different security strengths.
Landscape of High-performance Python to Develop Data Science and Machine Learning Applications
Venue: arXiv cs.DC 2023
Year: 2023

Python has become the prime language for application development in the Data Science and Machine Learning domains. However, data scientists are not necessarily experienced programmers. While Python lets them quickly implement their algorithms, when moving at scale, computation efficiency becomes inevitable. Thus, harnessing high-performance devices such as multicore processors and Graphical Processing Units (GPUs) to their potential is generally not trivial. The present narrative survey was thought as a reference document for such practitioners to help them make their way in the wealth of tools and techniques available for the Python language. Our document revolves around user scenarios, which are meant to cover most situations they may face. We believe that this document may also be of practical use to tool developers, who may use our work to identify potential lacks in existing tools and help them motivate their contributions.
Towards Understanding the Effects of Evolving the MCTS UCT Selection Policy
Venue: arXiv cs.AI 2023
Year: 2023

Monte Carlo Tree Search (MCTS) is a sampling best-first method to search for optimal decisions. The success of MCTS depends heavily on how the MCTS statistical tree is built and the selection policy plays a fundamental role in this. A particular selection policy that works particularly well, widely adopted in MCTS, is the Upper Confidence Bounds for Trees, referred to as UCT. Other more sophisticated bounds have been proposed by the community with the goal to improve MCTS performance on particular problems. Thus, it is evident that while the MCTS UCT behaves generally well, some variants might behave better. As a result of this, multiple works have been proposed to evolve a selection policy to be used in MCTS. Although all these works are inspiring, none of them have carried out an in-depth analysis shedding light under what circumstances an evolved alternative of MCTS UCT might be beneficial in MCTS due to focusing on a single type of problem. In sharp contrast to this, in this work we use five functions of different nature, going from a unimodal function, covering multimodal functions to deceptive functions. We demonstrate how the evolution of the MCTS UCT might be beneficial in multimodal and deceptive scenarios, whereas the MCTS UCT is robust in unimodal scenarios and competitive in the rest of the scenarios used in this study.
Deep-OSG: Deep Learning of Operators in Semigroup
Venue: arXiv cs.LG 2023
Year: 2023

This paper proposes a novel deep learning approach for learning operators in semigroup, with applications to modeling unknown autonomous dynamical systems using time series data collected at varied time lags. It is a sequel to the previous flow map learning (FML) works [T. Qin, K. Wu, and D. Xiu, J. Comput. Phys., 395:620--635, 2019], [K. Wu and D. Xiu, J. Comput. Phys., 408:109307, 2020], and [Z. Chen, V. Churchill, K. Wu, and D. Xiu, J. Comput. Phys., 449:110782, 2022], which focused on learning single evolution operator with a fixed time step. This paper aims to learn a family of evolution operators with variable time steps, which constitute a semigroup for an autonomous system. The semigroup property is very crucial and links the system's evolutionary behaviors across varying time scales, but it was not considered in the previous works. We propose for the first time a framework of embedding the semigroup property into the data-driven learning process, through a novel neural network architecture and new loss functions. The framework is very feasible, can be combined with any suitable neural networks, and is applicable to learning general autonomous ODEs and PDEs. We present the rigorous error estimates and variance analysis to understand the prediction accuracy and robustness of our approach, showing the remarkable advantages of semigroup awareness in our model. Moreover, our approach allows one to arbitrarily choose the time steps for prediction and ensures that the predicted results are well self-matched and consistent. Extensive numerical experiments demonstrate that embedding the semigroup property notably reduces the data dependency of deep learning models and greatly improves the accuracy, robustness, and stability for long-time prediction.
Using t-distributed stochastic neighbor embedding for visualization and segmentation of 3D point clouds of plants
Venue: arXiv cs.CV 2023
Year: 2023

In this work, the use of t-SNE is proposed to embed 3D point clouds of plants into 2D space for plant characterization. It is demonstrated that t-SNE operates as a practical tool to flatten and visualize a complete 3D plant model in 2D space. The perplexity parameter of t-SNE allows 2D rendering of plant structures at various organizational levels. Aside from the promise of serving as a visualization tool for plant scientists, t-SNE also provides a gateway for processing 3D point clouds of plants using their embedded counterparts in 2D. In this paper, simple methods were proposed to perform semantic segmentation and instance segmentation via grouping the embedded 2D points. The evaluation of these methods on a public 3D plant data set conveys the potential of t-SNE for enabling of 2D implementation of various steps involved in automatic 3D phenotyping pipelines.
Diverse Probabilistic Trajectory Forecasting with Admissibility Constraints
Venue: arXiv cs.LG 2023
Year: 2023

Predicting multiple trajectories for road users is important for automated driving systems: ego-vehicle motion planning indeed requires a clear view of the possible motions of the surrounding agents. However, the generative models used for multiple-trajectory forecasting suffer from a lack of diversity in their proposals. To avoid this form of collapse, we propose a novel method for structured prediction of diverse trajectories. To this end, we complement an underlying pretrained generative model with a diversity component, based on a determinantal point process (DPP). We balance and structure this diversity with the inclusion of knowledge-based quality constraints, independent from the underlying generative model. We combine these two novel components with a gating operation, ensuring that the predictions are both diverse and within the drivable area. We demonstrate on the nuScenes driving dataset the relevance of our compound approach, which yields significant improvements in the diversity and the quality of the generated trajectories.
MPC-based Motion Planning for Autonomous Truck-Trailer Maneuvering
Venue: arXiv cs.RO 2023
Year: 2023

Time-optimal motion planning of autonomous vehicles in complex environments is a highly researched topic. This paper describes a novel approach to optimize and execute locally feasible trajectories for the maneuvering of a truck-trailer Autonomous Mobile Robot (AMR), by dividing the environment in a sequence or route of freely accessible overlapping corridors. Multi-stage optimal control generates local trajectories through advancing subsets of this route. To cope with the advancing subsets and changing environments, the optimal control problem is solved online with a receding horizon in a Model Predictive Control (MPC) fashion with an improved update strategy. This strategy seamlessly integrates the computationally expensive MPC updates with a low-cost feedback controller for trajectory tracking, for disturbance rejection, and for stabilization of the unstable kinematics of the reversing truck-trailer AMR. This methodology is implemented in a flexible software framework for an effortless transition from offline simulations to deployment of experiments. An experimental setup showcasing the truck-trailer AMR performing two reverse parking maneuvers validates the presented method.
"I am the follower, also the boss": Exploring Different Levels of Autonomy and Machine Forms of Guiding Robots for the Visually Impaired
Venue: arXiv cs.HC 2023
Year: 2023

Guiding robots, in the form of canes or cars, have recently been explored to assist blind and low vision (BLV) people. Such robots can provide full or partial autonomy when guiding. However, the pros and cons of different forms and autonomy for guiding robots remain unknown. We sought to fill this gap. We designed autonomy-switchable guiding robotic cane and car. We conducted a controlled lab-study (N=12) and a field study (N=9) on BLV. Results showed that full autonomy received better walking performance and subjective ratings in the controlled study, whereas participants used more partial autonomy in the natural environment as demanding more control. Besides, the car robot has demonstrated abilities to provide a higher sense of safety and navigation efficiency compared with the cane robot. Our findings offered empirical evidence about how the BLV community perceived different machine forms and autonomy, which can inform the design of assistive robots.
Keeping Pace with Ever-Increasing Data: Towards Continual Learning of Code Intelligence Models
Venue: arXiv cs.SE 2023
Year: 2023

Previous research on code intelligence usually trains a deep learning model on a fixed dataset in an offline manner. However, in real-world scenarios, new code repositories emerge incessantly, and the carried new knowledge is beneficial for providing up-to-date code intelligence services to developers. In this paper, we aim at the following problem: How to enable code intelligence models to continually learn from ever-increasing data? One major challenge here is catastrophic forgetting, meaning that the model can easily forget knowledge learned from previous datasets when learning from the new dataset. To tackle this challenge, we propose REPEAT, a novel method for continual learning of code intelligence models. Specifically, REPEAT addresses the catastrophic forgetting problem with representative exemplars replay and adaptive parameter regularization. The representative exemplars replay component selects informative and diverse exemplars in each dataset and uses them to retrain model periodically. The adaptive parameter regularization component recognizes important parameters in the model and adaptively penalizes their changes to preserve the knowledge learned before. We evaluate the proposed approach on three code intelligence tasks including code summarization, software vulnerability detection, and code clone detection. Extensive experiments demonstrate that REPEAT consistently outperforms baseline methods on all tasks. For example, REPEAT improves the conventional fine-tuning method by 1.22, 5.61, and 1.72 on code summarization, vulnerability detection and clone detection, respectively.
Efficiently Upgrading Multilingual Machine Translation Models to Support More Languages
Venue: arXiv cs.CL 2023
Year: 2023

With multilingual machine translation (MMT) models continuing to grow in size and number of supported languages, it is natural to reuse and upgrade existing models to save computation as data becomes available in more languages. However, adding new languages requires updating the vocabulary, which complicates the reuse of embeddings. The question of how to reuse existing models while also making architectural changes to provide capacity for both old and new languages has also not been closely studied. In this work, we introduce three techniques that help speed up effective learning of the new languages and alleviate catastrophic forgetting despite vocabulary and architecture mismatches. Our results show that by (1) carefully initializing the network, (2) applying learning rate scaling, and (3) performing data up-sampling, it is possible to exceed the performance of a same-sized baseline model with 30% computation and recover the performance of a larger model trained from scratch with over 50% reduction in computation. Furthermore, our analysis reveals that the introduced techniques help learn the new directions more effectively and alleviate catastrophic forgetting at the same time. We hope our work will guide research into more efficient approaches to growing languages for these MMT models and ultimately maximize the reuse of existing models.
Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision
Venue: arXiv cs.SD 2023
Year: 2023

We introduce SPEAR-TTS, a multi-speaker text-to-speech (TTS) system that can be trained with minimal supervision. By combining two types of discrete speech representations, we cast TTS as a composition of two sequence-to-sequence tasks: from text to high-level semantic tokens (akin to "reading") and from semantic tokens to low-level acoustic tokens ("speaking"). Decoupling these two tasks enables training of the "speaking" module using abundant audio-only data, and unlocks the highly efficient combination of pretraining and backtranslation to reduce the need for parallel data when training the "reading" component. To control the speaker identity, we adopt example prompting, which allows SPEAR-TTS to generalize to unseen speakers using only a short sample of 3 seconds, without any explicit speaker representation or speaker-id labels. Our experiments demonstrate that SPEAR-TTS achieves a character error rate that is competitive with state-of-the-art methods using only 15 minutes of parallel data, while matching ground-truth speech in terms of naturalness and acoustic quality, as measured in subjective tests.
Towards a Deeper Understanding of Concept Bottleneck Models Through End-to-End Explanation
Venue: arXiv cs.AI 2023
Year: 2023

Concept Bottleneck Models (CBMs) first map raw input(s) to a vector of human-defined concepts, before using this vector to predict a final classification. We might therefore expect CBMs capable of predicting concepts based on distinct regions of an input. In doing so, this would support human interpretation when generating explanations of the model's outputs to visualise input features corresponding to concepts. The contribution of this paper is threefold: Firstly, we expand on existing literature by looking at relevance both from the input to the concept vector, confirming that relevance is distributed among the input features, and from the concept vector to the final classification where, for the most part, the final classification is made using concepts predicted as present. Secondly, we report a quantitative evaluation to measure the distance between the maximum input feature relevance and the ground truth location; we perform this with the techniques, Layer-wise Relevance Propagation (LRP), Integrated Gradients (IG) and a baseline gradient approach, finding LRP has a lower average distance than IG. Thirdly, we propose using the proportion of relevance as a measurement for explaining concept importance.
Multi-Scale Message Passing Neural PDE Solvers
Venue: arXiv cs.LG 2023
Year: 2023

We propose a novel multi-scale message passing neural network algorithm for learning the solutions of time-dependent PDEs. Our algorithm possesses both temporal and spatial multi-scale resolution features by incorporating multi-scale sequence models and graph gating modules in the encoder and processor, respectively. Benchmark numerical experiments are presented to demonstrate that the proposed algorithm outperforms baselines, particularly on a PDE with a range of spatial and temporal scales.
A Review of existing GDPR Solutions for Citizens and SMEs
Venue: arXiv cs.CR 2023
Year: 2023

The GDPR grants data subjects certain rights, like the right to access their data from companies, but in practice multiple problems exist with exercising these rights such as unknown data holders or interpreting the received data. Small and medium enterprises on the other hand need to facilitate the obligations given by the GDPR, but often lack proper systems, staff and other resources to do so effectively. For the GDPR to be effective in practice, these problems need to be addressed. With the work at hand we provide an overview of existing software solutions for these problems (from an internet research), discuss to which degree they solve the various problems and what issues remain.
CALaMo: a Constructionist Assessment of Language Models
Venue: arXiv cs.CL 2023
Year: 2023

This paper presents a novel framework for evaluating Neural Language Models' linguistic abilities using a constructionist approach. Not only is the usage-based model in line with the underlying stochastic philosophy of neural architectures, but it also allows the linguist to keep meaning as a determinant factor in the analysis. We outline the framework and present two possible scenarios for its application.
A Random Dictator Is All You Need
Venue: arXiv cs.GT 2023
Year: 2023

We study information aggregation with a decision maker aggregating binary recommendations from symmetric agents. Each agent's recommendation depends on her private information about a hidden state. While the decision maker knows the prior distribution over states and the marginal distribution of each agent's recommendation, the recommendations are adversarially-correlated. The decision maker's goal is choosing a robustly-optimal aggregation rule. We prove that for a large number of agents, for the three standard robustness paradigms - minimax, regret and approximation ratio - the unique optimal aggregation rule is random dictator. We further characterize the minimal regret for any agents' number through concavification.
Storing a Trie with Compact and Predictable Space
Venue: arXiv cs.DS 2023
Year: 2023

This paper proposed a storing approach for trie structures, called coordinate hash trie. The basic idea is using a global hash table with a special hash function to store all edges of a trie. For a trie with $n$ nodes and an alphabet with size $m$, the execution time of finding, inserting and deleting a child node, is $O(1)$ for the average case, $O(m)$ for the worst case. The space used by this approach is $O(n)$, unrelated to $m$. The constant of space consumption is predictable, with no need for reallocation or resizing. In addition, this approach is very easy to implement.
"STILL AROUND": Experiences and Survival Strategies of Veteran Women Software Developers
Venue: arXiv cs.SE 2023
Year: 2023

The intersection of ageism and sexism can create a hostile environment for veteran software developers belonging to marginalized genders. In this study, we conducted 14 interviews to examine the experiences of people at this intersection, primarily women, in order to discover the strategies they employed in order to successfully remain in the field. We identified 283 codes, which fell into three main categories: Strategies, Experiences, and Perception. Several strategies we identified, such as (Deliberately) Not Trying to Look Younger, were not previously described in the software engineering literature. We found that, in some companies, older women developers are recognized as having particular value, further strengthening the known benefits of diversity in the workforce. Based on the experiences and strategies, we suggest organizations employing software developers to consider the benefits of hiring veteran women software developers. For example, companies can draw upon the life experiences of older women developers in order to better understand the needs of customers from a similar demographic. While we recognize that many of the strategies employed by our study participants are a response to systemic issues, we still consider that, in the short-term, there is benefit in describing these strategies for developers who are experiencing such issues today.
Understanding Why ViT Trains Badly on Small Datasets: An Intuitive Perspective
Venue: arXiv cs.CV 2023
Year: 2023

Vision transformer (ViT) is an attention neural network architecture that is shown to be effective for computer vision tasks. However, compared to ResNet-18 with a similar number of parameters, ViT has a significantly lower evaluation accuracy when trained on small datasets. To facilitate studies in related fields, we provide a visual intuition to help understand why it is the case. We first compare the performance of the two models and confirm that ViT has less accuracy than ResNet-18 when trained on small datasets. We then interpret the results by showing attention map visualization for ViT and feature map visualization for ResNet-18. The difference is further analyzed through a representation similarity perspective. We conclude that the representation of ViT trained on small datasets is hugely different from ViT trained on large datasets, which may be the reason why the performance drops a lot on small datasets.
Transformer-based Models for Long-Form Document Matching: Challenges and Empirical Analysis
Venue: arXiv cs.CL 2023
Year: 2023

Recent advances in the area of long document matching have primarily focused on using transformer-based models for long document encoding and matching. There are two primary challenges associated with these models. Firstly, the performance gain provided by transformer-based models comes at a steep cost - both in terms of the required training time and the resource (memory and energy) consumption. The second major limitation is their inability to handle more than a pre-defined input token length at a time. In this work, we empirically demonstrate the effectiveness of simple neural models (such as feed-forward networks, and CNNs) and simple embeddings (like GloVe, and Paragraph Vector) over transformer-based models on the task of document matching. We show that simple models outperform the more complex BERT-based models while taking significantly less training time, energy, and memory. The simple models are also more robust to variations in document length and text perturbations.
A Multimodal Sensing Ring for Quantification of Scratch Intensity
Venue: arXiv cs.LG 2023
Year: 2023

An objective measurement of chronic itch is necessary for improvements in patient care for numerous medical conditions. While wearables have shown promise for scratch detection, they are currently unable to estimate scratch intensity, preventing a comprehensive understanding of the effect of itch on an individual. In this work, we present a framework for the estimation of scratch intensity in addition to the detection of scratch. This is accomplished with a multimodal ring device, consisting of an accelerometer and a contact microphone, a pressure-sensitive tablet for capturing ground truth intensity values, and machine learning algorithms for regression of scratch intensity on a 0-600 milliwatts (mW) power scale that can be mapped to a 0-10 continuous scale. We evaluate the performance of our algorithms on 20 individuals using leave one subject out cross-validation and using data from 14 additional participants, we show that our algorithms achieve clinically-relevant discrimination of scratching intensity levels. By doing so, our device enables the quantification of the substantial variations in the interpretation of the 0-10 scale frequently utilized in patient self-reported clinical assessments. This work demonstrates that a finger-worn device can provide multidimensional, objective, real-time measures for the action of scratching.