Miracle-dz/UniFAR-FTUs huggingface dataset:
Secure and Efficient Decentralized Federated Learning with Data Representation Protection

—Federated learning (FL) is a promising technical support to the vision of ubiquitous artiﬁcial intelligence in the sixth generation (6G) wireless communication network. However, traditional FL heavily relies on a trusted centralized server. Besides, FL is vulnerable to poisoning attacks, and the global aggregation of model updates makes the private training data under the risk of being reconstructed. What’s more, FL suffers from efﬁciency problem due to heavy communication cost. Although decentralized FL eliminates the problem of the central dependence of traditional FL, it makes other problems more serious. In this paper, we propose BlockDFL, an efﬁcient fully peer-to-peer (P2P) framework for decentralized FL. It integrates gradient compression and our designed voting mechanism with blockchain to efﬁciently coordinate multiple peer participants without mutual trust to carry out decentralized FL, while preventing data from being reconstructed according to transmitted model updates. Extensive experiments conducted on two real-world datasets exhibit that BlockDFL obtains competitive accuracy compared to centralized FL and can defend against poisoning attacks while achieving efﬁciency and scalability. Especially when the proportion of malicious participants is as high as 40 percent, BlockDFL can still preserve the accuracy of FL, which outperforms existing fully decentralized FL frameworks.
Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering

Federated learning is generally used in tasks where labels are readily available (e.g., next word prediction). Relaxing this constraint requires design of unsupervised learning techniques that can support desirable properties for federated training: robustness to statistical/systems heterogeneity, scalability with number of participants, and communication efficiency. Prior work on this topic has focused on directly extending centralized self-supervised learning techniques, which are not designed to have the properties listed above. To address this situation, we propose Orchestra, a novel unsupervised federated learning technique that exploits the federation's hierarchy to orchestrate a distributed clustering task and enforce a globally consistent partitioning of clients' data into discriminable clusters. We show the algorithmic pipeline in Orchestra guarantees good generalization performance under a linear probe, allowing it to outperform alternative techniques in a broad range of conditions, including variation in heterogeneity, number of clients, participation ratio, and local epochs.
Exposing Outlier Exposure: What Can Be Learned From Few, One, and Zero Outlier Images

Due to the intractability of characterizing everything that looks unlike the normal data, anomaly detection (AD) is traditionally treated as an unsupervised problem utilizing only normal samples. However, it has recently been found that unsupervised image AD can be drastically improved through the utilization of huge corpora of random images to represent anomalousness; a technique which is known as Outlier Exposure. In this paper we show that specialized AD learning methods seem unnecessary for state-of-the-art performance, and furthermore one can achieve strong performance with just a small collection of Outlier Exposure data, contradicting common assumptions in the field of AD. We find that standard classifiers and semi-supervised one-class methods trained to discern between normal samples and relatively few random natural images are able to outperform the current state of the art on an established AD benchmark with ImageNet. Further experiments reveal that even one well-chosen outlier sample is sufficient to achieve decent performance on this benchmark (79.3% AUC). We investigate this phenomenon and find that one-class methods are more robust to the choice of training outliers, indicating that there are scenarios where these are still more useful than standard classifiers. Additionally, we include experiments that delineate the scenarios where our results hold. Lastly, no training samples are necessary when one uses the representations learned by CLIP, a recent foundation model, which achieves state-of-the-art AD results on CIFAR-10 and ImageNet in a zero-shot setting.
Deployment of Energy-Efficient Deep Learning Models on Cortex-M based Microcontrollers using Deep Compression

—Large Deep Neural Networks (DNNs) are the back-bone of today’s artiﬁcial intelligence due to their ability to make accurate predictions when being trained on huge datasets. With advancing technologies, such as the Internet of Things, interpreting large quantities of data generated by sensors is becoming an increasingly important task. However, in many applications not only the predictive performance but also the energy consumption of deep learning models is of major interest. This paper investigates the efﬁcient deployment of deep learning models on resource-constrained microcontroller architectures via network compression. We present a methodology for the systematic exploration of different DNN pruning, quantization, and deployment strategies, targeting different ARM Cortex-M based low-power systems. The exploration allows to analyze trade-offs between key metrics such as accuracy, memory consumption, execution time, and power consumption. We discuss experimental results on three different DNN architectures and show that we can compress them to below 10% of their original parameter count before their predictive quality decreases. This also allows us to deploy and evaluate them on Cortex-M based microcontrollers.
Learning to branch with Tree MDPs

State-of-the-art Mixed Integer Linear Program (MILP) solvers combine systematic tree search with a plethora of hard-coded heuristics, such as the branching rule. The idea of learning branching rules from data has received increasing attention recently, and promising results have been obtained by learning fast approximations of the strong branching expert. In this work, we instead propose to learn branching rules from scratch via Reinforcement Learning (RL). We revisit the work of Etheve et al. (2020) and propose tree Markov Decision Processes, or tree MDPs, a generalization of temporal MDPs that provides a more suitable framework for learning to branch. We derive a tree policy gradient theorem, which exhibits a better credit assignment compared to its temporal counterpart. We demonstrate through computational experiments that tree MDPs improve the learning convergence, and offer a promising framework for tackling the learning-to-branch problem in MILPs.
SQuALITY: Building a Long-Document Summarization Dataset the Hard Way

Summarization datasets are often assembled either by scraping naturally occurring public-domain summaries—which are nearly always in difficult-to-work-with technical domains—or by using approximate heuristics to extract them from everyday text—which frequently yields unfaithful summaries. In this work, we turn to a slower but more straightforward approach to developing summarization benchmark data: We hire highly-qualified contractors to read stories and write original summaries from scratch. To amortize reading time, we collect five summaries per document, with the first giving an overview and the subsequent four addressing specific questions. We use this protocol to collect SQuALITY, a dataset of question-focused summaries built on the same public-domain short stories as the multiple-choice dataset QuALITY (Pang et al., 2021). Experiments with state-of-the-art summarization systems show that our dataset is challenging and that existing automatic evaluation metrics are weak indicators of quality.
High-throughput screening of piezo-photocatalytic materials for hydrogen production

Zhao Liu, 2 Biao Wang, 2 Dewei Chu, and Claudio Cazorla School of Materials Science and Engineering, Dongguan University of Technology, Dongguan 523808, China Research Institute of Interdisciplinary Science, Dongguan University of Technology, Dongguan 523808, China School of Materials Science and Engineering, UNSW Sydney, Sydney, NSW 2052, Australia Departament de F́ısica, Universitat Politècnica de Catalunya, Campus Nord B4-B5, Barcelona E-08034, Spain
Supporting Vision-Language Model Inference with Causality-pruning Knowledge Prompt

Vision-language models are pre-trained by aligning image-text pairs in a common space so that the models can deal with open-set visual concepts by learning semantic information from textual labels. To boost the transferability of these models on downstream tasks in a zero-shot manner, recent works explore generating fixed or learnable prompts, i.e., classification weights are synthesized from natural language describing task-relevant categories, to reduce the gap between tasks in the training and test phases. However, how and what prompts can improve inference performance remains un-clear. In this paper, we explicitly provide exploration and clarify the importance of including semantic information in prompts, while existing prompt methods generate prompts without exploring the semantic information of textual labels. A challenging issue is that manually constructing prompts, with rich semantic information, requires domain expertise and is extremely time-consuming. To this end, we propose Ca usality- p runing K nowledge P rompt (CapKP) for adapting pre-trained vision-language models to downstream image recognition. CapKP retrieves an ontological knowledge graph by treating the textual label as a query to explore task-relevant semantic information. To further refine the derived semantic information, CapKP introduces causality-pruning by following the first principle of Granger causality. Empirically, we conduct extensive evaluations to demonstrate the effectiveness of CapKP, e.g., with 8 shots, CapKP outperforms the manual-prompt method by 12.51% and the learnable-prompt method by 1.39% on average, respectively. Experimental analyses prove the superiority of CapKP in domain generalization compared to benchmark approaches.
Unitarity of some barycentric rational approximants

The exponential function maps the imaginary axis to the unit circle and, for many applications, this unitarity property is also desirable from its approximations. We show that this property is conserved not only by the (k,k)-rational barycentric interpolant of the exponential on the imaginary axis, but also by (k,k)-rational barycentric approximants that minimize a linearized approximation error. These results are a consequence of certain properties of singular vectors of Loewner-type matrices associated to linearized approximation errors. Prominent representatives of this class are rational approximants computed by the adaptive Antoulas--Anderson (AAA) method and the AAA--Lawson method. Our results also lead to a modified procedure with improved numerical stability of the unitarity property and reduced computational cost.
Current-sheet Oscillations Caused by the Kelvin–Helmholtz Instability at the Loop Top of Solar Flares

Current sheets (CSs), long stretching structures of magnetic reconnection above solar flare loops, are usually observed to oscillate; their origins, however, are still puzzled at present. Based on a high-resolution 2.5D MHD simulation of magnetic reconnection, we explore the formation mechanism of CS oscillations. We find that large-amplitude transverse waves are excited by the Kelvin–Helmholtz instability at the highly turbulent cusp-shaped region. The perturbations propagate upward along the CS with a phase speed close to local Alfvén speed thus resulting in the CS oscillations we observe. Though the perturbations damp after propagating for a long distance, the CS oscillations are still detectable. In terms of detected CS oscillations, with a combination of differential emission measure techniques, we propose a new method for measuring the magnetic field strength of the CS and its distribution in height.
Human and Technological Infrastructures of Fact-checking

Increasing demands for fact-checking have led to a growing interest in developing systems and tools to automate the fact-checking process. However, such systems are limited in practice because their system design often does not take into account how fact-checking is done in the real world and ignores the insights and needs of various stakeholder groups core to the fact-checking process. This paper unpacks the fact-checking process by revealing the infrastructures---both human and technological---that support and shape fact-checking work. We interviewed 26 participants belonging to 16 fact-checking teams and organizations with representation from 4 continents. Through these interviews, we describe the human infrastructure of fact-checking by identifying and presenting, in-depth, the roles of six primary stakeholder groups, 1) Editors, 2) External fact-checkers, 3) In-house fact-checkers, 4) Investigators and researchers, 5) Social media managers, and 6) Advocators. Our findings highlight that the fact-checking process is a collaborative effort among various stakeholder groups and associated technological and informational infrastructures. By rendering visibility to the infrastructures, we reveal how fact-checking has evolved to include both short-term claims centric and long-term advocacy centric fact-checking. Our work also identifies key social and technical needs and challenges faced by each stakeholder group. Based on our findings, we suggest that improving the quality of fact-checking requires systematic changes in the civic, informational, and technological contexts.
Deep Learning vs. Gradient Boosting: Benchmarking state-of-the-art machine learning algorithms for credit scoring

Artificial intelligence (AI) and machine learning (ML) have become vital to remain competitive for financial services companies around the globe. The two models currently competing for the pole position in credit risk management are deep learning (DL) and gradient boosting machines (GBM). This paper benchmarked those two algorithms in the context of credit scoring using three distinct datasets with different features to account for the reality that model choice/power is often dependent on the underlying characteristics of the dataset. The experiment has shown that GBM tends to be more powerful than DL and has also the advantage of speed due to lower computational requirements. This makes GBM the winner and choice for credit scoring. However, it was also shown that the outperformance of GBM is not always guaranteed and ultimately the concrete problem scenario or dataset will determine the final model choice. Overall, based on this study both algorithms can be considered state-of-the-art for binary classification tasks on structured datasets, while GBM should be the go-to solution for most problem scenarios due to easier use, significantly faster training time, and superior accuracy.
Human Instance Matting via Mutual Guidance and Multi-Instance Refinement

This paper introduces a new matting task called human instance matting (HIM), which requires the pertinent model to automatically predict a precise alpha matte for each human instance. Straightforward combination of closely related techniques, namely, instance segmentation, soft segmentation and human/conventional matting, will easily fail in complex cases requiring disentangling mingled colors belonging to multiple instances along hairy and thin boundary structures. To tackle these technical challenges, we propose a human instance matting framework, called InstMatt, where a novel mutual guidance strategy working in tandem with a multi-instance refinement module is used, for delineating multi-instance relationship among humans with complex and overlapping boundaries if present. A new instance matting metric called instance matting quality (IMQ) is proposed, which addresses the absence of a unified and fair means of evaluation emphasizing both instance recognition and matting quality. Finally, we construct a HIM benchmark for evaluation, which comprises of both synthetic and natural benchmark images. In addition to thorough experimental results on complex cases with multiple and overlapping human instances each has intricate boundaries, preliminary results are presented on general instance matting. Code and benchmark are available in https://github.com/nowsyn/InstMatt.
The physical factors affecting heavy metals accumulated in the sediment deposited on road surfaces in dry weather: a review

Abstract The main objective of this review is to identify and analyse physical factors (PF) involved in the heavy metal accumulation phenomenon in the road-deposited sediment (RDS) in dry weather. Electronic databases were used to establish an order of prevalence by establishing quartiles (Q-citation frequency) for the PF identified. The results show that the eleven PF can most likely be classified into three categories: climatic-anthropogenic-morphometric. The studies most frequently referenced land use (Q1), rainfall (Q2), particle size (Q2) and vehicles (Q3) as key PF. There is a similar behaviour between the concentrations of Zn-Pb-Cu associated with the RDS in dry weather and the dissolved fraction washed from the surface in road runoff. The size fraction <250 μm is representative of RDS to characterize the metals. The review expands knowledge about the influences of the PF in the processes of deposition-suspension-interception-removal of the metals load in road environments.
The Role of Placebo Samples in Observational Studies

In an observational study, it is common to leverage known null eﬀect to detect bias. One such strategy is to set aside a placebo sample – a subset of data immune from the hypothesized cause-and-eﬀect relationship. Existence of an eﬀect in the placebo sample raises concern of unmeasured confounding bias while absence of it corroborates the causal conclusion. This paper establishes a formal framework for using a placebo sample to detect and remove bias. We state identiﬁcation assumption, and develop estimation and inference methods based on outcome regression, inverse probability weighting, and doubly-robust approaches. Simulation studies and an empirical application illustrate the ﬁnite-sample performance of the proposed methods.
Use of eddy viscosity in resolvent analysis of turbulent channel flow

The predictions obtained from resolvent analysis with and without an eddy viscosity model for turbulent channel ﬂow at Re τ = 550 are compared to direct numerical simulation data to identify the scales and wave speeds for which resolvent analysis provides good predictions. The low-rank behavior of the standard resolvent identiﬁes energetic regions of the ﬂow whereas the eddy resolvent is low rank when the resulting projection of the leading eddy resolvent mode onto the leading mode from spectral proper orthogonal decomposition is maximum. The highest projections are obtained for structures that are associated with the near-wall cycle and structures that are energetic at z = ± 0 . 5. It is argued that these types of structures are likely to be correctly predicted for any friction Reynolds number due to the inner and outer scaling of the Cess eddy viscosity proﬁle. The eddy resolvent also correctly identiﬁes the most energetic wave speed for these two scales. For all other scales, neither analysis reliably predicts the most energetic wave speed or mode shapes. The standard resolvent tends to overestimate the most energetic wave speed while the eddy resolvent underestimates it. The resulting eddy resolvent modes are overly “attached” to the wall since the wall-normal gradient of the eddy viscosity overestimates the transport of energy towards the wall. These observations have direct implications for future work towards estimating turbulent channel ﬂows using resolvent analysis and suggest that the Cess proﬁle can be further optimized for individual scales to provide better low-order models of turbulent channel ﬂows.
Logarithmic regret bounds for continuous-time average-reward Markov decision processes

We consider reinforcement learning for continuous-time Markov decision processes (MDPs) in the infinite-horizon, average-reward setting. In contrast to discrete-time MDPs, a continuous-time process moves to a state and stays there for a random holding time after an action is taken. With unknown transition probabilities and rates of exponential holding times, we derive instance-dependent regret lower bounds that are logarithmic in the time horizon. Moreover, we design a learning algorithm and establish a finite-time regret bound that achieves the logarithmic growth rate. Our analysis builds upon upper confidence reinforcement learning, a delicate estimation of the mean holding times, and stochastic comparison of point processes.
NoisyMix: Boosting Model Robustness to Common Corruptions

For many real-world applications, obtaining stable and robust statistical performance is more important than simply achieving state-of-the-art predictive test accuracy, and thus robustness of neural networks is an increasingly important topic. Relatedly, data augmentation schemes have been shown to improve robustness with respect to input perturbations and domain shifts. Motivated by this, we introduce NoisyMix, a novel training scheme that promotes stability as well as leverages noisy augmentations in input and feature space to improve both model robustness and in-domain accuracy. NoisyMix produces models that are consistently more robust and that provide well-calibrated estimates of class membership probabilities. We demonstrate the benefits of NoisyMix on a range of benchmark datasets, including ImageNet-C, ImageNet-R, and ImageNet-P. Moreover, we provide theory to understand implicit regularization and robustness of NoisyMix.
An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs

Self-supervision based on the information extracted from large knowledge graphs has been shown to improve the generalization of language models, in zero-shot evaluation on various downstream language reasoning tasks. Since these improvements are reported in aggregate, however, little is known about (i) how to select the appropriate knowledge for solid performance across tasks, (ii) how to combine this knowledge with neural language models, and (iii) how these pairings affect granular task performance. In this paper, we study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models. We study the effect of different synthetic datasets on language models with various architectures and sizes. The resulting models are evaluated against four task properties: domain overlap, answer similarity, vocabulary overlap, and answer length. Our experiments show that encoder-decoder models benefit from more data to learn from, whereas sampling strategies that balance across different aspects yield best performance. Most of the improvement occurs on questions with short answers and dissimilar answer candidates, which corresponds to the characteristics of the data used for pre-training.
Improving Long Tailed Document-Level Relation Extraction via Easy Relation Augmentation and Contrastive Learning

Towards real-world information extraction scenario, research of relation extraction is advancing to document-level relation extraction(DocRE). Existing approaches for DocRE aim to extract relation by encoding various information sources in the long context by novel model architectures. However, the inherent long-tailed distribution problem of DocRE is overlooked by prior work. We argue that mitigating the long-tailed distribution problem is crucial for DocRE in the real-world scenario. Motivated by the long-tailed distribution problem, we propose an Easy Relation Augmentation(ERA) method for improving DocRE by enhancing the performance of tailed relations. In addition, we further propose a novel contrastive learning framework based on our ERA, i.e., ERACL, which can further improve the model performance on tailed relations and achieve competitive overall DocRE performance compared to the state-of-arts.