OpenRaiser/Intern-Atlas huggingface dataset:
Assisted Learning for Organizations with Limited Imbalanced Data
Venue: arXiv cs.LG 2024
Year: 2024

In the era of big data, many big organizations are integrating machine learning into their work pipelines to facilitate data analysis. However, the performance of their trained models is often restricted by limited and imbalanced data available to them. In this work, we develop an assisted learning framework for assisting organizations to improve their learning performance. The organizations have sufficient computation resources but are subject to stringent data-sharing and collaboration policies. Their limited imbalanced data often cause biased inference and sub-optimal decision-making. In assisted learning, an organizational learner purchases assistance service from an external service provider and aims to enhance its model performance within only a few assistance rounds. We develop effective stochastic training algorithms for both assisted deep learning and assisted reinforcement learning. Different from existing distributed algorithms that need to frequently transmit gradients or models, our framework allows the learner to only occasionally share information with the service provider, but still obtain a model that achieves near-oracle performance as if all the data were centralized.
A Distance-based Anomaly Detection Framework for Deep Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

In deep reinforcement learning (RL) systems, abnormal states pose significant risks by potentially triggering unpredictable behaviors and unsafe actions, thus impeding the deployment of RL systems in real-world scenarios. It is crucial for reliable decision-making systems to have the capability to cast an alert whenever they encounter unfamiliar observations that they are not equipped to handle. In this paper, we propose a novel Mahalanobis distance-based (MD) anomaly detection framework, called \textit{MDX}, for deep RL algorithms. MDX simultaneously addresses random, adversarial, and out-of-distribution (OOD) state outliers in both offline and online settings. It utilizes Mahalanobis distance within class-conditional distributions for each action and operates within a statistical hypothesis testing framework under the Gaussian assumption. We further extend it to robust and distribution-free versions by incorporating Robust MD and conformal inference techniques. Through extensive experiments on classical control environments, Atari games, and autonomous driving scenarios, we demonstrate the effectiveness of our MD-based detection framework. MDX offers a simple, unified, and practical anomaly detection tool for enhancing the safety and reliability of RL systems in real-world applications.
A Few-Shot Learning Approach for Sound Source Distance Estimation Using Relation Networks
Venue: arXiv cs.SD 2024
Year: 2024

In this paper, we study the performance of few-shot learning, specifically meta learning empowered few-shot relation networks, over supervised deep learning and conventional machine learning approaches in the problem of Sound Source Distance Estimation (SSDE). In previous research on deep supervised SSDE, low accuracies have often resulted from the mismatch between the training data (from known environments) and the test data (from unknown environments). By performing comparative experiments on a sufficient amount of data, we show that the few-shot relation network outperforms other competitors including eXtreme Gradient Boosting (XGBoost), Support Vector Machine (SVM), Convolutional Neural Network (CNN), and MultiLayer Perceptron (MLP). Hence it is possible to calibrate a microphone-equipped system, with a few labeled samples of audio recorded in a particular unknown environment to adjust and generalize our classifier to the possible input data and gain higher accuracies.
Recent Advances of Continual Learning in Computer Vision: An Overview
Venue: arXiv cs.CV 2024
Year: 2024

In contrast to batch learning where all training data is available at once, continual learning represents a family of methods that accumulate knowledge and learn continuously with data available in sequential order. Similar to the human learning process with the ability of learning, fusing, and accumulating new knowledge coming at different time steps, continual learning is considered to have high practical significance. Hence, continual learning has been studied in various artificial intelligence tasks. In this paper, we present a comprehensive review of the recent progress of continual learning in computer vision. In particular, the works are grouped by their representative techniques, including regularization, knowledge distillation, memory, generative replay, parameter isolation, and a combination of the above techniques. For each category of these techniques, both its characteristics and applications in computer vision are presented. At the end of this overview, several subareas, where continuous knowledge accumulation is potentially helpful while continual learning has not been well studied, are discussed.
Apple Tasting Revisited: Bayesian Approaches to Partially Monitored Online Binary Classification
Venue: arXiv cs.LG 2024
Year: 2024

We consider a variant of online binary classification where a learner sequentially assigns labels ($0$ or $1$) to items with unknown true class. If, but only if, the learner chooses label $1$ they immediately observe the true label of the item. The learner faces a trade-off between short-term classification accuracy and long-term information gain. This problem has previously been studied under the name of the `apple tasting' problem. We revisit this problem as a partial monitoring problem with side information, and focus on the case where item features are linked to true classes via a logistic regression model. Our principal contribution is a study of the performance of Thompson Sampling (TS) for this problem. Using recently developed information-theoretic tools, we show that TS achieves a Bayesian regret bound of an improved order to previous approaches. Further, we experimentally verify that efficient approximations to TS and Information Directed Sampling via Pólya-Gamma augmentation have superior empirical performance to existing methods.
A Group Key Establishment Scheme
Venue: arXiv cs.CR 2024
Year: 2024

Group authentication is a method of confirmation that a set of users belong to a group and of distributing a common key among them. Unlike the standard authentication schemes where one central authority authenticates users one by one, group authentication can handle the authentication process at once for all members of the group. The recently presented group authentication algorithms mainly exploit Lagrange's polynomial interpolation along with elliptic curve groups over finite fields. As a fresh approach, this work suggests use of linear spaces for group authentication and key establishment for a group of any size. The approach with linear spaces introduces a reduced computation and communication load to establish a common shared key among the group members. The advantages of using vector spaces make the proposed method applicable to energy and resource constrained devices. In addition to providing lightweight authentication and key agreement, this proposal allows any user in a group to make a non-member to be a member, which is expected to be useful for autonomous systems in the future. The scheme is designed in a way that the sponsors of such members can easily be recognized by anyone in the group. Unlike the other group authentication schemes based on Lagrange's polynomial interpolation, the proposed scheme doesn't provide a tool for adversaries to compromise the whole group secrets by using only a few members' shares as well as it allows to recognize a non-member easily, which prevents service interruption attacks.
Skew cyclic codes over $\mathbb{Z}_4+v\mathbb{Z}_4$ with derivation: structural properties and computational results
Venue: arXiv cs.IT 2024
Year: 2024

In this work, we study a class of skew cyclic codes over the ring $R:=\mathbb{Z}_4+v\mathbb{Z}_4,$ where $v^2=v,$ with an automorphism $θ$ and a derivation $Δ_θ,$ namely codes as modules over a skew polynomial ring $R[x;θ,Δ_θ],$ whose multiplication is defined using an automorphism $θ$ and a derivation $Δ_θ.$ We investigate the structures of a skew polynomial ring $R[x;θ,Δ_θ].$ We define $Δ_θ$-cyclic codes as a generalization of the notion of cyclic codes. The properties of $Δ_θ$-cyclic codes as well as dual $Δ_θ$-cyclic codes are derived. As an application, some new linear codes over $\mathbb{Z}_4$ with good parameters are obtained by Plotkin sum construction, also via a Gray map as well as residue and torsion codes of these codes.
Coresets for Kernel Clustering
Venue: arXiv cs.DS 2024
Year: 2024

We devise coresets for kernel $k$-Means with a general kernel, and use them to obtain new, more efficient, algorithms. Kernel $k$-Means has superior clustering capability compared to classical $k$-Means, particularly when clusters are non-linearly separable, but it also introduces significant computational challenges. We address this computational issue by constructing a coreset, which is a reduced dataset that accurately preserves the clustering costs. Our main result is a coreset for kernel $k$-Means that works for a general kernel and has size $\mathrm{poly}(kε^{-1})$. Our new coreset both generalizes and greatly improves all previous results; moreover, it can be constructed in time near-linear in $n$. This result immediately implies new algorithms for kernel $k$-Means, such as a $(1+ε)$-approximation in time near-linear in $n$, and a streaming algorithm using space and update time $\mathrm{poly}(k ε^{-1} \log n)$. We validate our coreset on various datasets with different kernels. Our coreset performs consistently well, achieving small errors while using very few points. We show that our coresets can speed up kernel $k$-Means++ (the kernelized version of the widely used $k$-Means++ algorithm), and we further use this faster kernel $k$-Means++ for spectral clustering. In both applications, we achieve significant speedup and a better asymptotic growth while the error is comparable to baselines that do not use coresets.
A New Simple Vision Algorithm for Detecting the Enzymic Browning Defects in Golden Delicious Apples
Venue: arXiv cs.CV 2024
Year: 2024

In this work, a simple vision algorithm is designed and implemented to extract and identify the surface defects on the Golden Delicious apples caused by the enzymic browning process. 34 Golden Delicious apples were selected for the experiments, of which 17 had enzymic browning defects and the other 17 were sound. The image processing part of the proposed vision algorithm extracted the defective surface area of the apples with high accuracy of 97.15%. The area and mean of the segmented images were selected as the 2x1 feature vectors to feed into a designed artificial neural network. The analysis based on the above features indicated that the images with a mean less than 0.0065 did not belong to the defective apples; rather, they were extracted as part of the calyx and stem of the healthy apples. The classification accuracy of the neural network applied in this study was 99.19%
A Model Selection Approach for Corruption Robust Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

We develop a model selection approach to tackle reinforcement learning with adversarial corruption in both transition and reward. For finite-horizon tabular MDPs, without prior knowledge on the total amount of corruption, our algorithm achieves a regret bound of $\widetilde{\mathcal{O}}(\min\{\frac{1}Δ, \sqrt{T}\}+C)$ where $T$ is the number of episodes, $C$ is the total amount of corruption, and $Δ$ is the reward gap between the best and the second-best policy. This is the first worst-case optimal bound achieved without knowledge of $C$, improving previous results of Lykouris et al. (2021); Chen et al. (2021); Wu et al. (2021). For finite-horizon linear MDPs, we develop a computationally efficient algorithm with a regret bound of $\widetilde{\mathcal{O}}(\sqrt{(1+C)T})$, and another computationally inefficient one with $\widetilde{\mathcal{O}}(\sqrt{T}+C)$, improving the result of Lykouris et al. (2021) and answering an open question by Zhang et al. (2021b). Finally, our model selection framework can be easily applied to other settings including linear bandits, linear contextual bandits, and MDPs with general function approximation, leading to several improved or new results.
Singer separation for karaoke content generation
Venue: arXiv cs.SD 2024
Year: 2024

Due to the rapid development of deep learning, we can now successfully separate singing voice from mono audio music. However, this separation can only extract human voices from other musical instruments, which is undesirable for karaoke content generation applications that only require the separation of lead singers. For this karaoke application, we need to separate the music containing male and female duets into two vocals, or extract a single lead vocal from the music containing vocal harmony. For this reason, we propose in this article to use a singer separation system, which generates karaoke content for one or two separated lead singers. In particular, we introduced three models for the singer separation task and designed an automatic model selection scheme to distinguish how many lead singers are in the song. We also collected a large enough data set, MIR-SingerSeparation, which has been publicly released to advance the frontier of this research. Our singer separation is most suitable for sentimental ballads and can be directly applied to karaoke content generation. As far as we know, this is the first singer-separation work for real-world karaoke applications.
Attention-based Estimation and Prediction of Human Intent to augment Haptic Glove aided Control of Robotic Hand
Venue: arXiv cs.RO 2024
Year: 2024

The letter focuses on Haptic Glove (HG) based control of a Robotic Hand (RH) executing in-hand manipulation of certain objects of interest. The high dimensional motion signals in HG and RH possess intrinsic variability of kinematics resulting in difficulty to establish a direct mapping of the motion signals from HG onto the RH. An estimation mechanism is proposed to quantify the motion signal acquired from the human controller in relation to the intended goal pose of the object being held by the robotic hand. A control algorithm is presented to transform the synthesized intent at the RH and allow relocation of the object to the expected goal pose. The lag in synthesis of the intent in the presence of communication delay leads to a requirement of predicting the estimated intent. We leverage an attention-based convolutional neural network encoder to predict the trajectory of intent for a certain lookahead to compensate for the delays. The proposed methodology is evaluated across objects of different shapes, mass, and materials. We present a comparative performance of the estimation and prediction mechanisms on 5G-driven real-world robotic setup against benchmark methodologies. The test-MSE in prediction of human intent is reported to yield ~ 97.3 -98.7% improvement of accuracy in comparison to LSTM-based benchmark
Terminal Embeddings in Sublinear Time
Venue: arXiv cs.DS 2024
Year: 2024

Recently (Elkin, Filtser, Neiman 2017) introduced the concept of a {\it terminal embedding} from one metric space $(X,d_X)$ to another $(Y,d_Y)$ with a set of designated terminals $T\subset X$. Such an embedding $f$ is said to have distortion $ρ\ge 1$ if $ρ$ is the smallest value such that there exists a constant $C>0$ satisfying \begin{equation*} \forall x\in T\ \forall q\in X,\ C d_X(x, q) \le d_Y(f(x), f(q)) \le C ρd_X(x, q) . \end{equation*} When $X,Y$ are both Euclidean metrics with $Y$ being $m$-dimensional, recently (Narayanan, Nelson 2019), following work of (Mahabadi, Makarychev, Makarychev, Razenshteyn 2018), showed that distortion $1+ε$ is achievable via such a terminal embedding with $m = O(ε^{-2}\log n)$ for $n := |T|$. This generalizes the Johnson-Lindenstrauss lemma, which only preserves distances within $T$ and not to $T$ from the rest of space. The downside of prior work is that evaluating their embedding on some $q\in \mathbb{R}^d$ required solving a semidefinite program with $Θ(n)$ constraints in~$m$ variables and thus required some superlinear $\mathrm{poly}(n)$ runtime. Our main contribution in this work is to give a new data structure for computing terminal embeddings. We show how to pre-process $T$ to obtain an almost linear-space data structure that supports computing the terminal embedding image of any $q\in\mathbb{R}^d$ in sublinear time $O^* (n^{1-Θ(ε^2)} + d)$. To accomplish this, we leverage tools developed in the context of approximate nearest neighbor search.
Elevating the future of mobility: UAV-enabled Intelligent Transportation Systems
Venue: arXiv cs.NI 2024
Year: 2024

Intelligent Transportation Systems (ITS) increasingly rely on connectivity for efficient traffic management and enhanced user experience. The existing ITS solutions operate mainly within a 2D domain, thus missing the potential benefits of aerial platforms. This paper envisions 3D ITS by integrating aerial platforms, such as Unmanned Aerial Vehicles (UAVs), to simultaneously improve network coverage and support multi-modal transportation, including Advanced Air Mobility (AAM). Using stochastic models, we investigate how UAV-based Aerial Base Stations (ABSs) can address the limitations of traditional Terrestrial Base Stations (TBSs) by offering superior coverage, particularly in urban environments. Our results demonstrate that ABSs have 106.67% more coverage area than TBS, higher Signal-to-Noise Ratio (SNR) distribution, and are suitable for high-throughput ITS applications.
Distributed Secret Sharing over a Public Channel from Correlated Random Variables
Venue: arXiv cs.IT 2024
Year: 2024

We consider a secret-sharing model where a dealer distributes the shares of a secret among a set of participants with the constraint that only predetermined subsets of participants must be able to reconstruct the secret by pooling their shares. Our study generalizes Shamir's secret-sharing model in three directions. First, we allow a joint design of the protocols for the creation of the shares and the distribution of the shares, instead of constraining the model to independent designs. Second, instead of assuming that the participants and the dealer have access to information-theoretically secure channels at no cost, we assume that they have access to a public channel and correlated randomness. Third, motivated by a wireless network setting where the correlated randomness is obtained from channel gain measurements, we explore a setting where the dealer is an entity made of multiple sub-dealers. Our main results are inner and outer regions for the achievable secret rates that the dealer and the participants can obtain in this model. To this end, we develop two new achievability techniques, a first one to successively handle reliability and security constraints in a distributed setting, and a second one to reduce a multi-dealer setting to multiple single-user dealer settings. Our results yield the capacity region for threshold access structures when the correlated randomness corresponds to pairwise secret keys shared between each sub-dealer and each participant, and the capacity for the all-or-nothing access structure in the presence of a single dealer and arbitrarily correlated randomness.
Morlet wavelet transform using attenuated sliding Fourier transform and kernel integral for graphic processing unit
Venue: arXiv cs.DC 2024
Year: 2024

Morlet or Gabor wavelet transforms as well as Gaussian smoothing, are widely used in signal processing and image processing. However, the computational complexity of their direct calculations is proportional not only to the number of data points in a signal but also to the smoothing size, which is the standard deviation in the Gaussian function in their transform functions. Thus, when the standard deviation is large, its considerable computation time diminishes the advantages of aforementioned transforms. Therefore, it is important to formulate an algorithm to reduce the calculation time of the transformations. In this paper, we first review calculation methods of Gaussian smoothing by using the sliding Fourier transform (SFT) and our proposed attenuated SFT (ASFT) \cite{YamashitaICPR2020}. Based on these methods, we propose two types of calculation methods for Morlet wavelet transforms. We also propose an algorithm to calculate SFT using the kernel integral on graphic processing unit (GPU). When the number of calculation cores in GPU is not less than the number of data points, the order of its calculation time is the logarithm of the smoothing size and does not depend on the number of data points. Using experiments, we compare the two methods for calculating the Morlet wavelet transform and evaluate the calculation time of the proposed algorithm using a kernel integral on GPU. For example, when the number of data points and the standard deviation are 102400 and 8192.0, respectively, the calculation time of the Morlet wavelet transform by the proposed method is 0.545 ms, which 413.6 times faster than a conventional method. (In this version, mistakes in fitures are corrected.)
Temporal Team Semantics Revisited
Venue: arXiv cs.LO 2024
Year: 2024

Temporal logics have been studied as an approach to the specification of hyperproperties, resulting in the conception of "hyperlogics". With a few recent exceptions, the hyperlogics thus far developed can only relate different traces of a transition system synchronously. However, important information is contained in the relation between different points in their asynchronous interaction. To specify such "asynchronous hyperproperties", new trace quantifier based hyperlogics have been developed. Yet, hyperlogics with trace quantification cannot express certain requirements that describe the relationships between all executions of a system. Also, these logics induce model checking problems (MC) with prohibitively high complexity costs in the number of quantifier alternations. We study an alternative approach to asynchronous hyperproperties by introducing a novel foundation of temporal team semantics. Team semantics is a logical framework that specifies properties of sets of traces of unbounded size directly, and thus does not have the same limitation as the quantifier based logics mentioned above. We consider temporal team logics which employ quantification over so-called "time evaluation functions" (TEFs) controlling the asynchronous progress of traces instead of quantification over traces. TEFs constitute a novel approach to defining expressive logics for hyperproperties where diverse asynchronous interactions between computations can be formalised and enforced. We show embeddings of synchronous TeamLTL into our new logics. We show that MC for some TeamCTL fragment is highly undecidable. We present a translation from TeamCTL* to Alternating Asynchronous Büchi Automata, and obtain decidability results for the path checking problem and restrictions of MC and SAT. Our translation constitutes the first approach to team semantics based on automata-theoretic methods.