Miracle-dz/UniFAR-FTUs huggingface dataset:
BBTv2: Towards a Gradient-Free Future with Large Language Models

Most downstream adaptation methods tune all or part of the parameters of pre-trained models (PTMs) through gradient descent, where the tuning cost increases linearly with the growth of the model size.By contrast, gradient-free methods only require the forward computation of the PTM to tune the prompt, retaining the benefits of efficient tuning and deployment.Though, past work on gradient-free tuning often introduces gradient descent to seek a good initialization of prompt and lacks versatility across tasks and PTMs.In this paper, we present BBTv2, an improved version of Black-Box Tuning, to drive PTMs for few-shot learning.We prepend continuous prompts to every layer of the PTM and propose a divide-and-conquer gradient-free algorithm to optimize the prompts at different layers alternately.Extensive experiments across various tasks and PTMs show that BBTv2 can achieve comparable performance to full model tuning and state-of-the-art parameter-efficient methods (e.g., Adapter, LoRA, BitFit, etc.) under few-shot settings while maintaining much fewer tunable parameters.
A Large-Scale Annotated Multivariate Time Series Aviation Maintenance Dataset from the NGAFID

This paper presents the largest publicly available, non-simulated, fleet-wide aircraft flight recording and maintenance log data for use in predicting part failure and maintenance need. We present 31,177 hours of flight data across 28,935 flights, which occur relative to 2,111 unplanned maintenance events clustered into 36 types of maintenance issues. Flights are annotated as before or after maintenance, with some flights occurring on the day of maintenance. Collecting data to evaluate predictive maintenance systems is challenging because it is difficult, dangerous, and unethical to generate data from compromised aircraft. To overcome this, we use the National General Aviation Flight Information Database (NGAFID), which contains flights recorded during regular operation of aircraft, and maintenance logs to construct a part failure dataset. We use a novel framing of Remaining Useful Life (RUL) prediction and consider the probability that the RUL of a part is greater than 2 days. Unlike previous datasets generated with simulations or in laboratory settings, the NGAFID Aviation Maintenance Dataset contains real flight records and maintenance logs from different seasons, weather conditions, pilots, and flight patterns. Additionally, we provide Python code to easily download the dataset and a Colab environment to reproduce our benchmarks on three different models. Our dataset presents a difficult challenge for machine learning researchers and a valuable opportunity to test and develop prognostic health management methods
Learning Algorithms in Static Analysis of Web Applications

Web applications are distributed applications, they are programs that run on more than one computer and communicate through a network or server. This very distributed nature of web applications, combined with the scale and sheer complexity of modern software systems complicate manual security auditing, while also creating a huge attack surface of potential hackers. These factors are making automated analysis a necessity. Static Application Security Testing (SAST) is a method devised to automatically analyze application source code of large code bases without compiling it, and design conditions that are indicative of security vulnerabilities. However, the problem lies in the fact that the most widely used Static Application Security Testing Tools often yield unreliable results, owing to the false positive classification of vulnerabilities grossly outnumbering the classification of true positive vulnerabilities. This is one of the biggest hindrances to the proliferation of SAST testing, which leaves the user to review hundreds, if not thousands, of potential warnings, and classify them as either actionable or spurious. We try to minimize the problem of false positives by introducing a technique to filter the output of SAST tools. The aim of the project is to apply learning algorithms to the output by analyzing the true and false positives classified by OWASP Benchmark, and eliminate, or reduce the number of false positives presented to the user of the SAST Tool.
A Dual Control Variate for doubly stochastic optimization and black-box variational inference

In this paper, we aim at reducing the variance of doubly stochastic optimization, a type of stochastic optimization algorithm that contains two independent sources of randomness: The subsampling of training data and the Monte Carlo estimation of expectations. Such an optimization regime often has the issue of large gradient variance which would lead to a slow rate of convergence. Therefore we propose Dual Control Variate, a new type of control variate capable of reducing gradient variance from both sources jointly. The dual con-trol variate is built upon approximation-based con-trol variates and incremental gradient methods. We show that on doubly stochastic optimization problems, compared with past variance reduction approaches that take only one source of randomness into account, dual control variate leads to a gradient estimator of signiﬁcant smaller variance and demonstrates superior performance on real-world applications, like generalized linear models with dropout and black-box variational inference.
ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentation

Machine learning techniques rely on large and diverse datasets for generalization. Computer vision, natural language processing, and other applications can often reuse public datasets to train many different models. However, due to differences in physical configurations, it is challenging to leverage public datasets for training robotic control policies on new robot platforms or for new tasks. In this work, we propose a novel framework, ExAug to augment the experiences of different robot platforms from multiple datasets in diverse environments. ExAug leverages a simple principle: by extracting 3D information in the form of a point cloud, we can create much more complex and structured augmentations, utilizing both generating synthetic images and geometric-aware penalization that would have been suitable in the same situation for a different robot, with different size, turning radius, and camera placement. The trained policy is evaluated on two new robot platforms with three different cameras in indoor and outdoor environments with obstacles.
Sample Efficient Dynamics Learning for Symmetrical Legged Robots: Leveraging Physics Invariance and Geometric Symmetries

Model generalization of the underlying dynamics is critical for achieving data efficiency when learning for robot control. This paper proposes a novel approach for learning dynamics leveraging the symmetry in the underlying robotic system, which allows for robust extrapolation from fewer samples. Existing frameworks that represent all data in vector space fail to consider the structured information of the robot, such as leg symmetry, rotational symmetry, and physics invariance. As a result, these schemes require vast amounts of training data to learn the system's redundant elements because they are learned independently. Instead, we propose considering the geometric prior by representing the system in symmetrical object groups and designing neural network architecture to assess invariance and equivariance between the objects. Finally, we demonstrate the effectiveness of our approach by comparing the generalization to unseen data of the proposed model and the existing models. We also implement a controller of a climbing robot based on learned inverse dynamics models. The results show that our method generates accurate control inputs that help the robot reach the desired state while requiring less training data than existing methods.
Controlling Bias Exposure for Fair Interpretable Predictions

Recent work on reducing bias in NLP models usually focuses on protecting or isolating information related to a sensitive attribute (like gender or race). However, when sensitive information is semantically entangled with the task information of the input, e.g., gender information is predictive for a profession, a fair trade-off between task performance and bias mitigation is difficult to achieve. Existing approaches perform this trade-off by eliminating bias information from the latent space, lacking control over how much bias is necessarily required to be removed. We argue that a favorable debiasing method should use sensitive information 'fairly', rather than blindly eliminating it (Caliskan et al., 2017; Sun et al., 2019; Bogen et al., 2020). In this work, we provide a novel debiasing algorithm by adjusting the predictive model's belief to (1) ignore the sensitive information if it is not useful for the task; (2) use sensitive information minimally as necessary for the prediction (while also incurring a penalty). Experimental results on two text classification tasks (influenced by gender) and an open-ended generation task (influenced by race) indicate that our model achieves a desirable trade-off between debiasing and task performance along with producing debiased rationales as evidence.
Distributed Distributionally Robust Optimization with Non-Convex Objectives

Distributionally Robust Optimization (DRO), which aims to find an optimal decision that minimizes the worst case cost over the ambiguity set of probability distribution, has been widely applied in diverse applications, e.g., network behavior analysis, risk management, etc. However, existing DRO techniques face three key challenges: 1) how to deal with the asynchronous updating in a distributed environment; 2) how to leverage the prior distribution effectively; 3) how to properly adjust the degree of robustness according to different scenarios. To this end, we propose an asynchronous distributed algorithm, named Asynchronous Single-looP alternatIve gRadient projEction (ASPIRE) algorithm with the itErative Active SEt method (EASE) to tackle the distributed distributionally robust optimization (DDRO) problem. Furthermore, a new uncertainty set, i.e., constrained D-norm uncertainty set, is developed to effectively leverage the prior distribution and flexibly control the degree of robustness. Finally, our theoretical analysis elucidates that the proposed algorithm is guaranteed to converge and the iteration complexity is also analyzed. Extensive empirical studies on real-world datasets demonstrate that the proposed method can not only achieve fast convergence, and remain robust against data heterogeneity as well as malicious attacks, but also tradeoff robustness with performance.
Topological synchronisation or a simple attractor?

A few recent papers introduced the concept of topological synchronisation. We refer in particular to (Lahav et al 2022 Sci. Rep. 12 2508), where the theory was illustrated by means of a skew product system, coupling two logistic maps. In this case, we show that the topological synchronisation could be easily explained as the birth of an attractor for increasing values of the coupling strength and the mutual convergence of two marginal empirical measures. Numerical computations based on a careful analysis of the Lyapunov exponents suggest that the attractor supports an absolutely continuous physical measure (acpm). We finally show that for some unimodal maps such acpm exhibit a multifractal structure.
Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values

Many NLP classification tasks, such as sexism/racism detection or toxicity detection, are based on human values. Yet, human values can vary under diverse cultural conditions. Therefore, we introduce a framework for value-aligned classification that performs prediction based on explicitly written human values in the command. Along with the task, we propose a practical approach that distills value-aligned knowledge from large-scale language models (LLMs) to construct value-aligned classifiers in two steps.First, we generate value-aligned training data from LLMs by prompt-based few-shot learning. Next, we fine-tune smaller classification models with the generated data for the task. Empirical results show that our VA-Models surpass multiple baselines by at least 15.56% on the F1-score, including few-shot learning with OPT-175B and existing text augmentation methods. We suggest that using classifiers with explicit human value input improves both inclusivity & explainability in AI.
Multi-View Photometric Stereo Revisited

Multi-view photometric stereo (MVPS) is a preferred method for detailed and precise 3D acquisition of an object from images. Although popular methods for MVPS can provide outstanding results, they are often complex to execute and limited to isotropic material objects. To address such limitations, we present a simple, practical approach to MVPS, which works well for isotropic as well as other object material types such as anisotropic and glossy. The proposed approach in this paper exploits the benefit of uncertainty modeling in a deep neural network for a reliable fusion of photometric stereo (PS) and multi-view stereo (MVS) network predictions. Yet, contrary to the recently proposed state-of-the-art, we introduce neural volume rendering methodology for a trustworthy fusion of MVS and PS measurements. The advantage of introducing neural volume rendering is that it helps in the reliable modeling of objects with diverse material types, where existing MVS methods, PS methods, or both may fail. Furthermore, it allows us to work on neural 3D shape representation, which has recently shown outstanding results for many geometric processing tasks. Our suggested new loss function aims to fit the zero level set of the implicit neural function using the most certain MVS and PS network predictions coupled with weighted neural volume rendering cost. The proposed approach shows state-of-the-art results when tested extensively on several benchmark datasets.
InterFair: Debiasing with Natural Language Feedback for Fair Interpretable Predictions

Debiasing methods in NLP models traditionally focus on isolating information related to a sensitive attribute (e.g., gender or race). We instead argue that a favorable debiasing method should use sensitive information 'fairly,' with explanations, rather than blindly eliminating it. This fair balance is often subjective and can be challenging to achieve algorithmically. We explore two interactive setups with a frozen predictive model and show that users able to provide feedback can achieve a better and fairer balance between task performance and bias mitigation. In one setup, users, by interacting with test examples, further decreased bias in the explanations (5-8%) while maintaining the same prediction accuracy. In the other setup, human feedback was able to disentangle associated bias and predictive information from the input leading to superior bias mitigation and improved task performance (4-5%) simultaneously.
A new old algorithm, entropy sampling, and/or getting in and out of energy minima

I propose a new algorithm, a free energy Monte Carlo algorithm, for calculations where conventional Monte Carlo simulations struggle with ergodicity problems. The simplest version of the proposed algorithm allows for the determination of the entropy function of statistical systems and/or performs entropy sampling at sufficiently large times. I also mention how this algorithm can be used to explore the system's energy space, in particular for minima.
Surface abnormality detection in medical and inspection systems using energy variations in co-occurrence matrixes

: Detection of surface defects is one of the most important issues in the field of image processing and machine vision. In this article, a method for detecting surface defects based on energy changes in co-occurrence matrices is presented. The presented method consists of two stages of training and testing. In the training phase, the co-occurrence matrix operator is first applied on healthy images and then the amount of output energy is calculated. In the following, according to the changes in the amount of energy, a suitable feature vector is defined, and with the help of it, a suitable threshold for the health of the images is obtained. Then, in the test phase, with the help of the calculated quorum, the defective parts are distinguished from the healthy ones. In the results section, the mentioned method has been applied on stone and ceramic images and its detection accuracy has been calculated and compared with some previous methods. Among the advantages of the presented method, we can mention high accuracy, low calculations and compatibility with all types of levels due to the use of the training stage. The proposed approach can be used in medical applications to detect abnormalities such as diseases. So, the performance is evaluated on 2d-hela dataset to classify cell phenotypes. The proposed approach provides about 89.56 percent accuracy on 2d-hela.
Evaluating Out-of-Distribution Performance on Document Image Classifiers

The ability of a document classifier to handle inputs that are drawn from a distribution different from the training distribution is crucial for robust deployment and generalizability. The RVL-CDIP corpus is the de facto standard benchmark for document classification, yet to our knowledge all studies that use this corpus do not include evaluation on out-of-distribution documents. In this paper, we curate and release a new out-of-distribution benchmark for evaluating out-of-distribution performance for document classifiers. Our new out-of-distribution benchmark consists of two types of documents: those that are not part of any of the 16 in-domain RVL-CDIP categories (RVL-CDIP-O), and those that are one of the 16 in-domain categories yet are drawn from a distribution different from that of the original RVL-CDIP dataset (RVL-CDIP-N). While prior work on document classification for in-domain RVL-CDIP documents reports high accuracy scores, we find that these models exhibit accuracy drops of between roughly 15-30% on our new out-of-domain RVL-CDIP-N benchmark, and further struggle to distinguish between in-domain RVL-CDIP-N and out-of-domain RVL-CDIP-O inputs. Our new benchmark provides researchers with a valuable new resource for analyzing out-of-distribution performance on document classifiers. Our new out-of-distribution data can be found at https://github.com/gxlarson/rvl-cdip-ood.
Gauge fields on $B$-branes over $\mathbb{CP}^n$

Considering the B-branes over the complex projective space P as the objects of the bounded derived category D(P), we prove that the cardinal of the set of holomorphic gauge fields on a given B-brane G• is ≤ 1. Moreover, the cardinal is 1 iff each G is isomorphic to a direct sum of copies of OPn . MSC 2020: 53C05, 18G10
Variational Message Passing-Based Respiratory Motion Estimation and Detection Using Radar Signals

We present a variational message passing (VMP)-based approach to detect the presence of a person based on their respiratory chest motion using multistatic ultra-wideband (UWB) radar. In the process, the respiratory motion is estimated for contact-free vital sign monitoring. The received signal is modeled as a backscatter channel and the respiratory motion and propagation channels are estimated using VMP. We use the evidence lower bound (ELBO) to approximate the model evidence for the detection. Numerical analyses and measurements demonstrate that the proposed method leads to a significant improvement in the detection performance compared to a fast Fourier transform (FFT)-based detector or an estimator-correlator in low-signal-to-noise ratio (SNR) conditions, since the multipath components (MPCs) are better incorporated into the detection procedure. Specifically, the proposed method has a detection probability of 0.95 at −20dB SNR, while the estimator-correlator and FFT-based detector have 0.32 and 0.05, respectively.
Discussion about Attacks and Defenses for Fair and Robust Recommendation System Design

Information has exploded on the Internet and mobile with the advent of the big data era. In particular, recommendation systems are widely used to help consumers who struggle to select the best products among such a large amount of information. However, recommendation systems are vulnerable to malicious user biases, such as fake reviews to promote or demote specific products and attacks that steal personal information. Such biases and attacks compromise the fairness of the recommendation model and infringe the privacy of users and systems by distorting data. Recently, deep-learning collaborative filtering recommendation systems have shown to be more vulnerable to this bias. In this position paper, we examine the effects of bias that cause various ethical and social issues and discuss the need for designing a robust recommendation system for fairness and stability. impact for
Control, Confidentiality, and the Right to be Forgotten

Recent digital rights frameworks give users the right to delete their data from systems that store and process their personal information (e.g., the "right to be forgotten" in the GDPR). How should deletion be formalized in complex systems that interact with many users and store derivative information? We argue that prior approaches fall short. Definitions of machine unlearning[6] are too narrowly scoped and do not apply to general interactive settings. The natural approach of deletion-as-confidentiality[15] is too restrictive: by requiring secrecy of deleted data, it rules out social functionalities. We propose a new formalism: deletion-as-control. It allows users' data to be freely used before deletion, while also imposing a meaningful requirement after deletion--thereby giving users more control. Deletion-as-control provides new ways of achieving deletion in diverse settings. We apply it to social functionalities, and give a new unified view of various machine unlearning definitions from the literature. This is done by way of a new adaptive generalization of history independence. Deletion-as-control also provides a new approach to the goal of machine unlearning, that is, to maintaining a model while honoring users' deletion requests. We show that publishing a sequence of updated models that are differentially private under continual release satisfies deletion-as-control. The accuracy of such an algorithm does not depend on the number of deleted points, in contrast to the machine unlearning literature.
On the existence of closed trajectories and pseudo-trajectories for a family of third order differential equations

The goal of this article is to study the existence of closed trajectories for the differential equation $\dddot{z}+a\ddot{z}+b\dot{z}+abz=\varepsilon F(z,\dot{z},\ddot{z})$ in two situations. In the first situation, we consider $F(z,\dot{z},\ddot{z})=1$ and $b={\rm sgn}(h(z,\dot{z},\ddot{z}))$, where $h(z,\dot{z},\ddot{z})=z^2+(\dot{z})^2+(\ddot{z})^2-1$. We show that the differential equation is equivalent to a piecewise smooth differential system that admits the unit sphere as the discontinuity manifold. We obtain conditions for the existence of a closed pseudo-trajectory in this case. In the second situation, we consider $\varepsilon \neq 0$ sufficiently small, $b>0$, and $F(z,\dot{z},\ddot{z})$ a $n$-degree polynomial. We show that the unperturbed differential equation has a family of isochronous periodic solutions filling an invariant plane. Then, we study the maximum number of limit cycles which bifurcate from this 2-dimensional isochronous using the averaging theory. Thus, within the same family, we have periodic solutions (in the case where the parameters create a smooth equation) and also pseudo-periodic solutions (in the case of Filippov systems).