Miracle-dz/UniFAR-FTUs huggingface dataset:
Inference on quantile processes with a finite number of clusters

I introduce a generic method for inference on entire quantile and regression quantile processes in the presence of a finite number of large and arbitrarily heterogeneous clusters. The method asymptotically controls size by generating statistics that exhibit enough distributional symmetry such that randomization tests can be applied. The randomization test does not require ex-ante matching of clusters, is free of user-chosen parameters, and performs well at conventional significance levels with as few as five clusters. The method tests standard (non-sharp) hypotheses and can even be asymptotically similar in empirically relevant situations. The main focus of the paper is inference on quantile treatment effects but the method applies more broadly. Numerical and empirical examples are provided.
Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly

On robotics computer vision tasks, generating and annotating large amounts of data from real-world for the use of deep learning-based approaches is often difficult or even impossible. A common strategy for solving this problem is to apply simulation-to-reality (sim2real) approaches with the help of simulated scenes. While the majority of current robotics vision sim2real work focuses on image data, we present an industrial application case that uses sim2real transfer learning for point cloud data. We provide insights on how to generate and process synthetic point cloud data in order to achieve better performance when the learned model is transferred to real-world data. The issue of imbalanced learning is investigated using multiple strategies. A novel patch-based attention network is proposed additionally to tackle this problem.
A Programming Model for GPU Load Balancing

We propose a GPU fine-grained load-balancing abstraction that decouples load balancing from work processing and aims to support both static and dynamic schedules with a programmable interface to implement new load-balancing schedules. Prior to our work, the only way to unleash the GPU's potential on irregular problems has been to workload-balance through application-specific, tightly coupled load-balancing techniques. With our open-source framework for load-balancing, we hope to improve programmers' productivity when developing irregular-parallel algorithms on the GPU, and also improve the overall performance characteristics for such applications by allowing a quick path to experimentation with a variety of existing load-balancing techniques. Consequently, we also hope that by separating the concerns of load-balancing from work processing within our abstraction, managing and extending existing code to future architectures becomes easier.
Study of JavaScript Static Analysis Tools for Vulnerability Detection in Node.js Packages

With the emergence of the Node.js ecosystem, JavaScript has become a widely used programming language for implementing server-side web applications. In this article, we present the first empirical study of static code analysis tools for detecting vulnerabilities in Node.js code. To conduct a comprehensive tool evaluation, we created the largest known curated dataset of Node.js code vulnerabilities. We characterized and annotated a set of 957 vulnerabilities by analyzing information contained in npm advisory reports. We tested nine different tools and found that many important vulnerabilities appearing in the OWASP top-10 are not detected by any tool. The three best performing tools combined only detect up to 57.6% of all vulnerabilities in the dataset, but at a very low precision of 0.11%. Our curated dataset offers a new benchmark to help characterize existing Node.js code vulnerabilities and foster the development of better vulnerability detection tools for Node.js code.
Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information

In NLP annotation, it is common to have multiple annotators label the text and then obtain the ground truth labels based on major annotators’ agreement. However, annotators are individuals with different backgrounds and various voices. When annotation tasks become subjective, such as detecting politeness, offense, and social norms, annotators’ voices differ and vary. Their diverse voices may represent the true distribution of people’s opinions on subjective matters. Therefore, it is crucial to study the disagreement from annotation to understand which content is controversial from the annotators. In our research, we extract disagreement labels from five subjective datasets, then fine-tune language models to predict annotators’ disagreement. Our results show that knowing annotators’ demographic information (e.g., gender, ethnicity, education level), in addition to the task text, helps predict the disagreement. To investigate the effect of annotators’ demographics on their disagreement level, we simulate different combinations of their artificial demographics and explore the variance of the prediction to distinguish the disagreement from the inherent controversy from text content and the disagreement in the annotators’ perspective. Overall, we propose an innovative disagreement prediction mechanism for better design of the annotation process that will achieve more accurate and inclusive results for NLP systems. Our code and dataset are publicly available.
Simple Lyapunov spectrum for linear homogeneous differential equations with Lp parameters

In the present paper we prove that densely, with respect to an $L^p$-like topology, the Lyapunov exponents associated to linear continuous-time cocycles $\Phi:\mathbb{R}\times M\to \text{GL}(2,\mathbb{R})$ induced by second order linear homogeneous differential equations $\ddot x+\alpha(\varphi^t(\omega))\dot x+\beta(\varphi^t(\omega))x=0$ are almost everywhere distinct. The coefficients $\alpha,\beta$ evolve along the $\varphi^t$-orbit for $\omega\in M$ and $\varphi^t: M\to M$ is an ergodic flow defined on a probability space. We also obtain the corresponding version for the frictionless equation $\ddot x+\beta(\varphi^t(\omega))x=0$ and for a Schr\"odinger equation $\ddot x+(E-Q(\varphi^t(\omega)))x=0$, inducing a cocycle $\Phi:\mathbb{R}\times M\to \text{SL}(2,\mathbb{R})$.
The Keyword Explorer Suite: A Toolkit for Understanding Online Populations

We have developed a set of Python applications that use large language models to identify and analyze data from social media platforms relevant to a population of interest. Our pipeline begins with using OpenAI’s GPT-3 to generate potential keywords for identifying relevant text content from the target population. The keywords are then validated, and the content downloaded and analyzed using GPT-3 embedding and manifold reduction. Corpora are then created to fine-tune GPT-2 models to explore latent information via prompt-based queries. These tools allow researchers and practitioners to gain valuable insights into population subgroups online.
Think Twice: A Human-like Two-stage Conversational Agent for Emotional Response Generation

Towards human-like dialogue systems, current emotional dialogue approaches jointly model emotion and semantics with a unified neural network. This strategy tends to generate safe responses due to the mutual restriction between emotion and semantics, and requires rare emotion-annotated large-scale dialogue corpus. Inspired by the"think twice"behavior in human dialogue, we propose a two-stage conversational agent for the generation of emotional dialogue. Firstly, a dialogue model trained without the emotion-annotated dialogue corpus generates a prototype response that meets the contextual semantics. Secondly, the first-stage prototype is modified by a controllable emotion refiner with the empathy hypothesis. Experimental results on the DailyDialog and EmpatheticDialogues datasets demonstrate that the proposed conversational outperforms the comparison models in emotion generation and maintains the semantic performance in automatic and human evaluations.
Machine learning methods for prediction of breakthrough curves in reactive porous media

Reactive ﬂows in porous media play an important role in our life and are crucial for many industrial, environmental and biomedical applications. Very often the concentration of the species at the inlet is known, and the so-called breakthrough curves, measured at the outlet, are the quantities which could be measured or computed numerically. The measurements and the simulations could be time-consuming and expensive, and machine learning and Big Data approaches can help to predict breakthrough curves at lower costs. Machine learning (ML) methods, such as Gaussian processes and fully-connected neural networks, and a tensor method, cross approximation, are well suited for predicting breakthrough curves. In this paper, we demonstrate their performance in the case of pore scale reactive ﬂow in catalytic ﬁlters.
On Modeling Network Slicing Communication Resources with SARSA Optimization

Network slicing is a crucial enabler to support the composition and deployment of virtual network infrastructures required by the dynamic behavior of networks like 5G/6G mobile networks, IoT-aware networks, e-health systems, and industry verticals like the internet of vehicles (IoV) and industry 4.0. The communication slices and their allocated communication resources are essential in slicing architectures for resource orchestration and allocation, virtual network function (VNF) deployment, and slice operation functionalities. The communication slices provide the communications capabilities required to support slice operation, SLA guarantees, and QoS/ QoE application requirements. Therefore, this contribution proposes a networking slicing conceptual model to formulate the optimization problem related to the sharing of communication resources among communication slices. First, we present a conceptual model of network slicing, we then formulate analytically some aspects of the model and the optimization problem to address. Next, we proposed to use a SARSA agent to solve the problem and implement a proof of concept prototype. Finally, we present the obtained results and discuss them.
Low PAPR MIMO-OFDM Design Based on Convolutional Autoencoder

An enhanced framework for peak-to-average power ratio ($\mathsf{PAPR}$) reduction and waveform design for Multiple-Input-Multiple-Output ($\mathsf{MIMO}$) orthogonal frequency-division multiplexing ($\mathsf{OFDM}$) systems, based on a convolutional-autoencoder ($\mathsf{CAE}$) architecture, is presented. The end-to-end learning-based autoencoder ($\mathsf{AE}$) for communication networks represents the network by an encoder and decoder, where in between, the learned latent representation goes through a physical communication channel. We introduce a joint learning scheme based on projected gradient descent iteration to optimize the spectral mask behavior and MIMO detection under the influence of a non-linear high power amplifier ($\mathsf{HPA}$) and a multipath fading channel. The offered efficient implementation novel waveform design technique utilizes only a single $\mathsf{PAPR}$ reduction block for all antennas. It is throughput-lossless, as no side information is required at the decoder. Performance is analyzed by examining the bit error rate ($\mathsf{BER}$), the $\mathsf{PAPR}$, and the spectral response and compared with classical $\mathsf{PAPR}$ reduction $\mathsf{MIMO}$ detector methods on 5G simulated data. The suggested system exhibits competitive performance when considering all optimization criteria simultaneously. We apply gradual loss learning for multi-objective optimization and show empirically that a single trained model covers the tasks of $\mathsf{PAPR}$ reduction, spectrum design, and $\mathsf{MIMO}$ detection together over a wide range of SNR levels.
NarrowBERT: Accelerating Masked Language Model Pretraining and Inference

Large-scale language model pretraining is a very successful form of self-supervised learning in natural language processing, but it is increasingly expensive to perform as the models and pretraining corpora have become larger over time. We propose NarrowBERT, a modified transformer encoder that increases the throughput for masked language model pretraining by more than 2x. NarrowBERT sparsifies the transformer model such that the self-attention queries and feedforward layers only operate on the masked tokens of each sentence during pretraining, rather than all of the tokens as with the usual transformer encoder. We also show that NarrowBERT increases the throughput at inference time by as much as 3.5x with minimal (or no) performance degradation on sentence encoding tasks like MNLI. Finally, we examine the performance of NarrowBERT on the IMDB and Amazon reviews classification and CoNLL NER tasks and show that it is also comparable to standard BERT performance.
Dynamics of fluctuating thin sheets under random forcing.

We study the dynamic structure factor of fluctuating elastic thin sheets subject to conservative (athermal) random forcing. In Steinbock et al. [Phys. Rev. Res. 4, 033096 (2022)2643-156410.1103/PhysRevResearch.4. 033096] the static structure factor of such a sheet was studied. In this paper we recap the model developed there and investigate its dynamic properties. Using the self-consistent expansion, the time-dependent two-point function of the height profile is determined and found to decay exponentially in time. Despite strong nonlinear coupling, the decay rate of the dynamic structure factor is found to coincide with the effective coupling constant for the static properties, which suggests that the model under investigation exhibits certain quasilinear behavior. Confirmation of these results by numerical simulations is also presented.
The Accretion Disk and White Dwarf in the U Geminorum type Dwarf Novae UZ Serpentis and SS Aurigae during Quiescence

We have carried out high-gravity model atmosphere and accretion disk synthetic spectral fits for the U Geminorum type dwarf novae during quiescence. For UZ Ser, we find that the far-UV spectrum is best fitted with a log g = 8, Teff = 27,000 K photosphere with abundances near solar. The best-fit accretion disk model for UZ Ser has Mwd = 1.0 M⊙, disk inclination i = 75°, and = 10-9 M⊙ yr-1. For SS Aur, the best-fit model photosphere has Teff = 30,000K, log g = 8, and solar composition abundances, while the best-fit accretion disk model has Mwd = 1.0 M⊙, i = 41°, and = 10-10 M⊙ yr-1. These accretion rates are discussed in connection with the critical accretion rates predicted by the disk instability theory. Whether a white dwarf or a disk accounts for the far-UV spectra, they must be quite hot. If indeed the white dwarf dominates the far-UV light, then UZ Ser and SS Aur add two important temperatures in support of the conclusion that the accreting white dwarfs above the period gap are a factor of 1.5–2 times hotter than the accreting cataclysmic variable degenerates in systems below the period gap.
PINN for Dynamical Partial Differential Equations is Not Training Deeper Networks Rather Learning Advection and Time Variance

The concepts and techniques of physics-informed neural networks (PINNs) is studied and limitations are identified to make it efficient to approximate dynamical equations. Potential working research domains are explored for increasing the robustness of this technique for the solvability of partial differential equations. It is identified that PINNs potentially fails to stronger advection and longer time duration. Also, optimization function and constraint posing needs to be smarter. Even a shallow network is good for a lot of problems while powerful deeper network fails. Reservoir computing based recurrent neural network architecture is recommended to solve dynamical problems.
Evaluating the Fork-Awareness of Coverage-Guided Fuzzers

Fuzz testing (or fuzzing) is an effective technique used to find security vulnerabilities. It consists of feeding a software under test with malformed inputs, waiting for a weird system behaviour (often a crash of the system). Over the years, different approaches have been developed, and among the most popular lies the coverage-based one. It relies on the instrumentation of the system to generate inputs able to cover as much code as possible. The success of this approach is also due to its usability as fuzzing techniques research approaches that do not require (or only partial require) human interactions. Despite the efforts, devising a fully-automated fuzzer still seems to be a challenging task. Target systems may be very complex; they may integrate cryptographic primitives, compute and verify check-sums and employ forks to enhance the system security, achieve better performances or manage different connections at the same time. This paper introduces the fork-awareness property to express the fuzzer ability to manage systems using forks. This property is leveraged to evaluate 14 of the most widely coverage-guided fuzzers and highlight how current fuzzers are ineffective against systems using forks.
Online Hyperparameter Optimization for Class-Incremental Learning

Class-incremental learning (CIL) aims to train a classification model while the number of classes increases phase-by-phase. An inherent challenge of CIL is the stability-plasticity tradeoff, i.e., CIL models should keep stable to retain old knowledge and keep plastic to absorb new knowledge. However, none of the existing CIL models can achieve the optimal tradeoff in different data-receiving settings—where typically the training-from-half (TFH) setting needs more stability, but the training-from-scratch (TFS) needs more plasticity. To this end, we design an online learning method that can adaptively optimize the tradeoff without knowing the setting as a priori. Specifically, we first introduce the key hyperparameters that influence the tradeoff, e.g., knowledge distillation (KD) loss weights, learning rates, and classifier types. Then, we formulate the hyperparameter optimization process as an online Markov Decision Process (MDP) problem and propose a specific algorithm to solve it. We apply local estimated rewards and a classic bandit algorithm Exp3 to address the issues when applying online MDP methods to the CIL protocol. Our method consistently improves top-performing CIL methods in both TFH and TFS settings, e.g., boosting the average accuracy of TFH and TFS by 2.2 percentage points on ImageNet-Full, compared to the state-of-the-art. Code is provided at https://class-il.mpi-inf.mpg. de/online/
Phase-shifted Adversarial Training

Adversarial training has been considered an imperative component for safely deploying neural network-based applications to the real world. To achieve stronger robustness, existing methods primarily focus on how to generate strong attacks by increasing the number of update steps, regularizing the models with the smoothed loss function, and injecting the randomness into the attack. Instead, we analyze the behavior of adversarial training through the lens of response frequency. We empirically discover that adversarial training causes neural networks to have low convergence to high-frequency information, resulting in highly oscillated predictions near each data. To learn high-frequency contents efficiently and effectively, we first prove that a universal phenomenon of frequency principle, i.e., \textit{lower frequencies are learned first}, still holds in adversarial training. Based on that, we propose phase-shifted adversarial training (PhaseAT) in which the model learns high-frequency components by shifting these frequencies to the low-frequency range where the fast convergence occurs. For evaluations, we conduct the experiments on CIFAR-10 and ImageNet with the adaptive attack carefully designed for reliable evaluation. Comprehensive results show that PhaseAT significantly improves the convergence for high-frequency information. This results in improved adversarial robustness by enabling the model to have smoothed predictions near each data.
HADA: A Graph-based Amalgamation Framework in Image-text Retrieval

Many models have been proposed for vision and language tasks, especially the image-text retrieval task. All state-of-the-art (SOTA) models in this challenge contained hundreds of millions of parameters. They also were pretrained on a large external dataset that has been proven to make a big improvement in overall performance. It is not easy to propose a new model with a novel architecture and intensively train it on a massive dataset with many GPUs to surpass many SOTA models, which are already available to use on the Internet. In this paper, we proposed a compact graph-based framework, named HADA, which can combine pretrained models to produce a better result, rather than building from scratch. First, we created a graph structure in which the nodes were the features extracted from the pretrained models and the edges connecting them. The graph structure was employed to capture and fuse the information from every pretrained model with each other. Then a graph neural network was applied to update the connection between the nodes to get the representative embedding vector for an image and text. Finally, we used the cosine similarity to match images with their relevant texts and vice versa to ensure a low inference time. Our experiments showed that, although HADA contained a tiny number of trainable parameters, it could increase baseline performance by more than 3.6% in terms of evaluation metrics in the Flickr30k dataset. Additionally, the proposed model did not train on any external dataset and did not require many GPUs but only 1 to train due to its small number of parameters. The source code is available at https://github.com/m2man/HADA.
Diffusion-based Data Augmentation for Skin Disease Classification: Impact Across Original Medical Datasets to Fully Synthetic Images

Despite continued advancement in recent years, deep neural networks still rely on large amounts of training data to avoid overfitting. However, labeled training data for real-world applications such as healthcare is limited and difficult to access given longstanding privacy, and strict data sharing policies. By manipulating image datasets in the pixel or feature space, existing data augmentation techniques represent one of the effective ways to improve the quantity and diversity of training data. Here, we look to advance augmentation techniques by building upon the emerging success of text-to-image diffusion probabilistic models in augmenting the training samples of our macroscopic skin disease dataset. We do so by enabling fine-grained control of the image generation process via input text prompts. We demonstrate that this generative data augmentation approach successfully maintains a similar classification accuracy of the visual classifier even when trained on a fully synthetic skin disease dataset. Similar to recent applications of generative models, our study suggests that diffusion models are indeed effective in generating high-quality skin images that do not sacrifice the classifier performance, and can improve the augmentation of training datasets after curation.
Measuring Corporate Digital Divide with web scraping: Evidence from Italy

With the increasing pervasiveness of ICTs in the fabric of economic activities, the corporate digital divide has emerged as a new crucial topic to evaluate the IT competencies and the digital gap between ﬁrms and territories. Given the scarcity of available granular data to measure the phenomenon, most studies have used survey data. To bridge the empirical gap, we scrape the website homepage of 182 705 Italian ﬁrms, extracting ten features related to their digital footprint characteristics to develop a new corporate digital assessment index. Our results highlight a signiﬁcant digital divide across dimensions, sectors and geographical locations of Italian ﬁrms, opening up new perspectives on monitoring and near-real-time data-driven analysis.