OpenRaiser/Intern-Atlas huggingface dataset:
Dynamic Euclidean Bottleneck Matching
Venue: arXiv cs.CG 2023
Year: 2023

A fundamental question in computational geometry is for a set of input points in the Euclidean space, that is subject to discrete changes (insertion/deletion of points at each time step), whether it is possible to maintain an approximate bottleneck matching in sublinear update time. In this work, we answer this question in the affirmative for points on a real line and for points in the plane with a bounded geometric spread. For a set $P$ of $n$ points on a line, we show that there exists a dynamic algorithm that maintains a bottleneck matching of $P$ and supports insertion and deletion in $O(\log n)$ time. Moreover, we show that a modified version of this algorithm maintains a minimum-weight matching with $O(\log n)$ update (insertion and deletion) time. Next, for a set $P$ of $n$ points in the plane, we show that a ($6\sqrt{2}$)-factor approximate bottleneck matching of $P_k$, at each time step $k$, can be maintained in $O(\logΔ)$ amortized time per insertion and $O(\logΔ + |P_k|)$ amortized time per deletion, where $Δ$ is the geometric spread of $P$.
Tree-Based Machine Learning Methods For Vehicle Insurance Claims Size Prediction
Venue: arXiv cs.LG 2023
Year: 2023

Vehicle insurance claims size prediction needs methods to efficiently handle these claims. Machine learning (ML) is one of the methods that solve this problem. Tree-based ensemble learning algorithms are highly effective and widely used ML methods. This study considers how vehicle insurance providers incorporate ML methods in their companies and explores how the models can be applied to insurance big data. We utilize various tree-based ML methods, such as bagging, random forest, and gradient boosting, to determine the relative importance of predictors in predicting claims size and to explore the relationships between claims size and predictors. Furthermore, we evaluate and compare these models' performances. The results show that tree-based ensemble methods are better than the classical least square method. Keywords: claims size prediction; machine learning; tree-based ensemble methods; vehicle insurance.
Lightweight-Yet-Efficient: Revitalizing Ball-Tree for Point-to-Hyperplane Nearest Neighbor Search
Venue: arXiv cs.DB 2023
Year: 2023

Finding the nearest neighbor to a hyperplane (or Point-to-Hyperplane Nearest Neighbor Search, simply P2HNNS) is a new and challenging problem with applications in many research domains. While existing state-of-the-art hashing schemes (e.g., NH and FH) are able to achieve sublinear time complexity without the assumption of the data being in a unit hypersphere, they require an asymmetric transformation, which increases the data dimension from $d$ to $Ω(d^2)$. This leads to considerable overhead for indexing and incurs significant distortion errors. In this paper, we investigate a tree-based approach for solving P2HNNS using the classical Ball-Tree index. Compared to hashing-based methods, tree-based methods usually require roughly linear costs for construction, and they provide different kinds of approximations with excellent flexibility. A simple branch-and-bound algorithm with a novel lower bound is first developed on Ball-Tree for performing P2HNNS. Then, a new tree structure named BC-Tree, which maintains the Ball and Cone structures in the leaf nodes of Ball-Tree, is described together with two effective strategies, i.e., point-level pruning and collaborative inner product computing. BC-Tree inherits both the low construction cost and lightweight property of Ball-Tree while providing a similar or more efficient search. Experimental results over 16 real-world data sets show that Ball-Tree and BC-Tree are around 1.1$\sim$10$\times$ faster than NH and FH, and they can reduce the index size and indexing time by about 1$\sim$3 orders of magnitudes on average. The code is available at \url{https://github.com/HuangQiang/BC-Tree}.
An Elementary Formal Proof of the Group Law on Weierstrass Elliptic Curves in any Characteristic
Venue: arXiv cs.LO 2023
Year: 2023

Elliptic curves are fundamental objects in number theory and algebraic geometry, whose points over a field form an abelian group under a geometric addition law. Any elliptic curve over a field admits a Weierstrass model, but prior formal proofs that the addition law is associative in this model involve either advanced algebraic geometry or tedious computation, especially in characteristic two. We formalise in the Lean theorem prover, the type of nonsingular points of a Weierstrass curve over a field of any characteristic and a purely algebraic proof that it forms an abelian group.
Multi-Target Tobit Models for Completing Water Quality Data
Venue: arXiv cs.AI 2023
Year: 2023

Monitoring microbiological behaviors in water is crucial to manage public health risk from waterborne pathogens, although quantifying the concentrations of microbiological organisms in water is still challenging because concentrations of many pathogens in water samples may often be below the quantification limit, producing censoring data. To enable statistical analysis based on quantitative values, the true values of non-detected measurements are required to be estimated with high precision. Tobit model is a well-known linear regression model for analyzing censored data. One drawback of the Tobit model is that only the target variable is allowed to be censored. In this study, we devised a novel extension of the classical Tobit model, called the \emph{multi-target Tobit model}, to handle multiple censored variables simultaneously by introducing multiple target variables. For fitting the new model, a numerical stable optimization algorithm was developed based on elaborate theories. Experiments conducted using several real-world water quality datasets provided an evidence that estimating multiple columns jointly gains a great advantage over estimating them separately.
Predicting Privacy Preferences for Smart Devices as Norms
Venue: arXiv cs.AI 2023
Year: 2023

Smart devices, such as smart speakers, are becoming ubiquitous, and users expect these devices to act in accordance with their preferences. In particular, since these devices gather and manage personal data, users expect them to adhere to their privacy preferences. However, the current approach of gathering these preferences consists in asking the users directly, which usually triggers automatic responses failing to capture their true preferences. In response, in this paper we present a collaborative filtering approach to predict user preferences as norms. These preference predictions can be readily adopted or can serve to assist users in determining their own preferences. Using a dataset of privacy preferences of smart assistant users, we test the accuracy of our predictions.
Supervisory Control of Modular Discrete-Event Systems under Partial Observation: Normality
Venue: arXiv cs.FL 2023
Year: 2023

Complex systems are often composed of many small communicating components called modules. We investigate the synthesis of supervisory controllers for modular systems under partial observation that, as the closed-loop system, realize the supremal normal sublanguage of the specification. We call such controllers maximally permissive normal supervisors. The challenge in modular systems is to find conditions under which the global nonblocking and maximally permissive normal supervisor can be achieved locally as the parallel composition of local normal supervisors. We show that a structural concept of hierarchical supervisory control called modified observation consistency (MOC) is such a condition. However, the algorithmic verification of MOC is an open problem, and therefore it is necessary to find easily-verifiable conditions that ensure MOC. We show that the condition that all shared events are observable is such a condition. Considering specifications, we examine both local specifications, where each module has its own specification, and global specifications. We combine our results for normality with the existing results for controllability to locally synthesize the nonblocking and maximally permissive controllable and normal supervisor. Finally, we illustrate the results on an industrial case study of the patient table of an MRI scanner.
Evaluating the effect of data augmentation and BALD heuristics on distillation of Semantic-KITTI dataset
Venue: arXiv cs.CV 2023
Year: 2023

Active Learning (AL) has remained relatively unexplored for LiDAR perception tasks in autonomous driving datasets. In this study we evaluate Bayesian active learning methods applied to the task of dataset distillation or core subset selection (subset with near equivalent performance as full dataset). We also study the effect of application of data augmentation (DA) within Bayesian AL based dataset distillation. We perform these experiments on the full Semantic-KITTI dataset. We extend our study over our existing work only on 1/4th of the same dataset. Addition of DA and BALD have a negative impact over the labeling efficiency and thus the capacity to distill datasets. We demonstrate key issues in designing a functional AL framework and finally conclude with a review of challenges in real world active learning.
Effects of Architectures on Continual Semantic Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Research in the field of Continual Semantic Segmentation is mainly investigating novel learning algorithms to overcome catastrophic forgetting of neural networks. Most recent publications have focused on improving learning algorithms without distinguishing effects caused by the choice of neural architecture.Therefore, we study how the choice of neural network architecture affects catastrophic forgetting in class- and domain-incremental semantic segmentation. Specifically, we compare the well-researched CNNs to recently proposed Transformers and Hybrid architectures, as well as the impact of the choice of novel normalization layers and different decoder heads. We find that traditional CNNs like ResNet have high plasticity but low stability, while transformer architectures are much more stable. When the inductive biases of CNN architectures are combined with transformers in hybrid architectures, it leads to higher plasticity and stability. The stability of these models can be explained by their ability to learn general features that are robust against distribution shifts. Experiments with different normalization layers show that Continual Normalization achieves the best trade-off in terms of adaptability and stability of the model. In the class-incremental setting, the choice of the normalization layer has much less impact. Our experiments suggest that the right choice of architecture can significantly reduce forgetting even with naive fine-tuning and confirm that for real-world applications, the architecture is an important factor in designing a continual learning model.
Memory-augmented Online Video Anomaly Detection
Venue: arXiv cs.CV 2023
Year: 2023

The ability to understand the surrounding scene is of paramount importance for Autonomous Vehicles (AVs). This paper presents a system capable to work in an online fashion, giving an immediate response to the arise of anomalies surrounding the AV, exploiting only the videos captured by a dash-mounted camera. Our architecture, called MOVAD, relies on two main modules: a Short-Term Memory Module to extract information related to the ongoing action, implemented by a Video Swin Transformer (VST), and a Long-Term Memory Module injected inside the classifier that considers also remote past information and action context thanks to the use of a Long-Short Term Memory (LSTM) network. The strengths of MOVAD are not only linked to its excellent performance, but also to its straightforward and modular architecture, trained in a end-to-end fashion with only RGB frames with as less assumptions as possible, which makes it easy to implement and play with. We evaluated the performance of our method on Detection of Traffic Anomaly (DoTA) dataset, a challenging collection of dash-mounted camera videos of accidents. After an extensive ablation study, MOVAD is able to reach an AUC score of 82.17\%, surpassing the current state-of-the-art by +2.87 AUC. Our code will be available on https://github.com/IMPLabUniPr/movad/tree/movad_vad
ChatGPT: Jack of all trades, master of none
Venue: arXiv cs.CL 2023
Year: 2023

OpenAI has released the Chat Generative Pre-trained Transformer (ChatGPT) and revolutionized the approach in artificial intelligence to human-model interaction. Several publications on ChatGPT evaluation test its effectiveness on well-known natural language processing (NLP) tasks. However, the existing studies are mostly non-automated and tested on a very limited scale. In this work, we examined ChatGPT's capabilities on 25 diverse analytical NLP tasks, most of them subjective even to humans, such as sentiment analysis, emotion recognition, offensiveness, and stance detection. In contrast, the other tasks require more objective reasoning like word sense disambiguation, linguistic acceptability, and question answering. We also evaluated GPT-4 model on five selected subsets of NLP tasks. We automated ChatGPT and GPT-4 prompting process and analyzed more than 49k responses. Our comparison of its results with available State-of-the-Art (SOTA) solutions showed that the average loss in quality of the ChatGPT model was about 25% for zero-shot and few-shot evaluation. For GPT-4 model, a loss for semantic tasks is significantly lower than for ChatGPT. We showed that the more difficult the task (lower SOTA performance), the higher the ChatGPT loss. It especially refers to pragmatic NLP problems like emotion recognition. We also tested the ability to personalize ChatGPT responses for selected subjective tasks via Random Contextual Few-Shot Personalization, and we obtained significantly better user-based predictions. Additional qualitative analysis revealed a ChatGPT bias, most likely due to the rules imposed on human trainers by OpenAI. Our results provide the basis for a fundamental discussion of whether the high quality of recent predictive NLP models can indicate a tool's usefulness to society and how the learning and validation procedures for such systems should be established.
MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-based Malware Detection
Venue: arXiv cs.LG 2023
Year: 2023

ML models are known to be vulnerable to adversarial query attacks. In these attacks, queries are iteratively perturbed towards a particular class without any knowledge of the target model besides its output. The prevalence of remotely-hosted ML classification models and Machine-Learning-as-a-Service platforms means that query attacks pose a real threat to the security of these systems. To deal with this, stateful defenses have been proposed to detect query attacks and prevent the generation of adversarial examples by monitoring and analyzing the sequence of queries received by the system. Several stateful defenses have been proposed in recent years. However, these defenses rely solely on similarity or out-of-distribution detection methods that may be effective in other domains. In the malware detection domain, the methods to generate adversarial examples are inherently different, and therefore we find that such detection mechanisms are significantly less effective. Hence, in this paper, we present MalProtect, which is a stateful defense against query attacks in the malware detection domain. MalProtect uses several threat indicators to detect attacks. Our results show that it reduces the evasion rate of adversarial query attacks by 80+\% in Android and Windows malware, across a range of attacker scenarios. In the first evaluation of its kind, we show that MalProtect outperforms prior stateful defenses, especially under the peak adversarial threat.
Hybridization of K-means with improved firefly algorithm for automatic clustering in high dimension
Venue: arXiv cs.LG 2023
Year: 2023

K-means Clustering is the most well-known partitioning algorithm among all clustering, by which we can partition the data objects very easily in to more than one clusters. However, for K-means to choose an appropriate number of clusters without any prior domain knowledge about the dataset is challenging, especially in high-dimensional data objects. Hence, we have implemented the Silhouette and Elbow methods with PCA to find an optimal number of clusters. Also, previously, so many meta-heuristic swarm intelligence algorithms inspired by nature have been employed to handle the automatic data clustering problem. Firefly is efficient and robust for automatic clustering. However, in the Firefly algorithm, the entire population is automatically subdivided into sub-populations that decrease the convergence rate speed and trapping to local minima in high-dimensional optimization problems. Thus, our study proposed an enhanced firefly, i.e., a hybridized K-means with an ODFA model for automatic clustering. The experimental part shows output and graphs of the Silhouette and Elbow methods as well as the Firefly algorithm
Bridging the Transparency Gap: What Can Explainable AI Learn From the AI Act?
Venue: arXiv cs.AI 2023
Year: 2023

The European Union has proposed the Artificial Intelligence Act which introduces detailed requirements of transparency for AI systems. Many of these requirements can be addressed by the field of explainable AI (XAI), however, there is a fundamental difference between XAI and the Act regarding what transparency is. The Act views transparency as a means that supports wider values, such as accountability, human rights, and sustainable innovation. In contrast, XAI views transparency narrowly as an end in itself, focusing on explaining complex algorithmic properties without considering the socio-technical context. We call this difference the ``transparency gap''. Failing to address the transparency gap, XAI risks leaving a range of transparency issues unaddressed. To begin to bridge this gap, we overview and clarify the terminology of how XAI and European regulation -- the Act and the related General Data Protection Regulation (GDPR) -- view basic definitions of transparency. By comparing the disparate views of XAI and regulation, we arrive at four axes where practical work could bridge the transparency gap: defining the scope of transparency, clarifying the legal status of XAI, addressing issues with conformity assessment, and building explainability for datasets.
Utilizing Domain Knowledge: Robust Machine Learning for Building Energy Prediction with Small, Inconsistent Datasets
Venue: arXiv cs.LG 2023
Year: 2023

The demand for a huge amount of data for machine learning (ML) applications is currently a bottleneck in an empirically dominated field. We propose a method to combine prior knowledge with data-driven methods to significantly reduce their data dependency. In this study, component-based machine learning (CBML) as the knowledge-encoded data-driven method is examined in the context of energy-efficient building engineering. It encodes the abstraction of building structural knowledge as semantic information in the model organization. We design a case experiment to understand the efficacy of knowledge-encoded ML in sparse data input (1% - 0.0125% sampling rate). The result reveals its three advanced features compared with pure ML methods: 1. Significant improvement in the robustness of ML to extremely small-size and inconsistent datasets; 2. Efficient data utilization from different entities' record collections; 3. Characteristics of accepting incomplete data with high interpretability and reduced training time. All these features provide a promising path to alleviating the deployment bottleneck of data-intensive methods and contribute to efficient real-world data usage. Moreover, four necessary prerequisites are summarized in this study that ensures the target scenario benefits by combining prior knowledge and ML generalization.
Online estimation methods for irregular autoregressive models
Venue: arXiv cs.LG 2023
Year: 2023

In the last decades, due to the huge technological growth observed, it has become increasingly common that a collection of temporal data rapidly accumulates in vast amounts. This provides an opportunity for extracting valuable information through the estimation of increasingly precise models. But at the same time it imposes the challenge of continuously updating the models as new data become available. Currently available methods for addressing this problem, the so-called online learning methods, use current parameter estimations and novel data to update the estimators. These approaches avoid using the full raw data and speeding up the computations. In this work we consider three online learning algorithms for parameters estimation in the context of time series models. In particular, the methods implemented are: gradient descent, Newton-step and Kalman filter recursions. These algorithms are applied to the recently developed irregularly observed autoregressive (iAR) model. The estimation accuracy of the proposed methods is assessed by means of Monte Carlo experiments. The results obtained show that the proposed online estimation methods allow for a precise estimation of the parameters that generate the data both for the regularly and irregularly observed time series. These online approaches are numerically efficient, allowing substantial computational time savings. Moreover, we show that the proposed methods are able to adapt the parameter estimates quickly when the time series behavior changes, unlike batch estimation methods.
Exact Performance Analysis of THz Link Under Transceiver Hardware Impairments
Venue: arXiv cs.IT 2023
Year: 2023

Transceiver hardware impairment (THI) is inevitable for high-date rate terahertz (THz) communication. Existing statistical analysis either neglects THI's effect or provides approximate results when analyzing the performance of the THz system combined with channel fading and antenna misalignment. In this paper, we develop exact analytical expressions for the average signal-to-noise ratio (SNR), ergodic capacity, and average bit-error-rate (BER) performance of a THz wireless link under the combined effect of $α$-$μ$ fading channel, zero-boresight pointing errors, and the Gaussian distributed THI. We also derive asymptotic expressions for the outage probability and average BER, which shows that the diversity order of the THz link is independent of THI's parameters. Simulations validate the derived analytical results and demonstrate the impact of the THI parameters on the THz performance.
Estimating Bibliometric Links using Google Scholar: A Semi-Systematic Literature Mapping of Migration and Housing
Venue: arXiv cs.DL 2023
Year: 2023

As the number of empirical studies increases unprecedentedly in line with expansions in higher education, theoretical developments in population studies suffer due to a discoverability crisis of related work. The systematic use of previous research is common in the medical sciences through various types of structured reviews. However, these are less common in the social sciences, despite their potential, especially in cross-disciplinary fields such as migration. We use Google Scholar to examine the niche of housing research within migration studies through a broad range of documents. The contribution of this meta-analysis is threefold. Firstly, we illustrate the association of keywords across the corpus of literature related to migration and housing and map the growth of migration literature since the 1960s. Secondly, we highlight key bridging documents using network measures. Finally, we estimate the distance in reading time between documents. Our findings suggest that the corpus of previous work at the intersection of migration and housing consists of many additive case studies, indicating a gap in integrative approaches replicable across a growing knowledge landscape.