Miracle-dz/UniFAR-FTUs huggingface dataset:
Fault friction under thermal pressurization during large coseismic-slip Part I: Numerical analyses

In this paper, we study the role of thermal pressurization in the frictional response of a fault under large coseismic slip. We investigate the role of the seismic slip velocity, mixture compressibility, characteristic grain size and viscosity parameter in the frictional response of the coupled thermo-hydro-mechanical problem, taking into account the fault’s microstructure. Starting from the mass, energy and momentum balance for Cosserat continua we derive the equations of our model. We complete the mathematical description using perfect plasticity and Perzyna viscoplasticity in the material constitutive behavior. We investigate both the rate independent as well as the rate dependent frictional response and compare with existing models found in literature, namely the rate and state friction law (Dieterich (1992),Ruina (1983a)). We show that our model is capable of predicting strain rate hardening and velocity softening without the assumption of a state variable. We observe traveling instabilities inside the layer that lead to oscillations in the fault’s frictional response, like in the case of Portevin Le Chatelier (PLC) eﬀect. This behavior is not captured by existing numerical analyses presented in Rattez et al. (2018c,b,a) and go beyond the established models of uniform shear (Lachenbruch (1980)) and shear on a mathematical plane (Rice (2006a)), which predict a strictly monotonous behavior during shearing. Recent experimental analyses, which have managed to insulate thermal pressurization from other weakening mechanisms (Badt et al. (2020)), corroborate our numerical results.
A Library Perspective on Nearly-Unsupervised Information Extraction Workflows in Digital Libraries

Information extraction can support novel and effective access paths for digital libraries. Nevertheless, designing reliable extraction workflows can be cost-intensive in practice. On the one hand, suitable extraction methods rely on domain-specific training data. On the other hand, unsupervised and open extraction methods usually produce not-canonicalized extraction results. This paper tackles the question how digital libraries can handle such extractions and if their quality is sufficient in practice. We focus on unsupervised extraction workflows by analyzing them in case studies in the domains of encyclopedias (Wikipedia), pharmacy and political sciences. We report on opportunities and limitations. Finally we discuss best practices for unsupervised extraction workflows.CCS CONCEPTS• Information systems → Information extraction; Data extraction and integration; Document representation.
Reducing the Cost of Training Security Classifier (via Optimized Semi-Supervised Learning)

Background : Most of the existing machine learning models for security tasks, such as spam detection, malware detection, or network intrusion detection, are built on supervised machine learning algorithms. In such a paradigm, models need a large amount of labeled data to learn the useful relationships between selected features and target class. However, such labeled data can be scarce and expensive to acquire. Goal : To help security practitioners train useful security classification models when few labeled training data and many unlabeled training data are available. Method : We propose an adaptive framework called Dapper, which optimizes 1) semi-supervised learning algorithms to assign pseudo-labels to unlabeled data in a propagation paradigm and 2) the machine learning classifier (i.e., random forest). When the dataset class is highly imbalanced, Dapper then adaptively integrates and optimizes a data oversampling method called SMOTE. We use the novel Bayesian Optimization to search a large hyperparameter space of these tuning targets. Result : We evaluate Dapper with three security datasets, i.e., the Twitter spam dataset, the malware URLs dataset, and the CIC-IDS-2017 dataset. Experimental results indicate that we can use as low as 10% of original labeled data but achieve close or even better classification performance than using 100% labeled data in a supervised way. Conclusion : Based on those results, we would recommend using hyperparameter optimization with semi-supervised learning when dealing with shortages of labeled security data.
The mighty force: statistical inference and high-dimensional statistics

Inference is an English noun formed on the verb infer, from the Latin inferre, meaning to carry (fero) in or into (in-) something. That originally concrete meaning can still be felt in the portal quote of this chapter. In modern non-technical use the meaning of inference is more abstract, and rendered either as “A conclusion reached on the basis of evidence and reasoning” or as “The process of reaching such a conclusion” [1]. In scientific language these translate into characteristics of a phenomenon that are not observed directly, but which are arrived at (inferred) from observations with the help of mathematical and/or statistical methods, and those methods themselves. We will discuss three prominent examples of inference in both senses of modern usage, and how they naturally open up new perspectives and possibilities. Statistical physics is played out on the terrain between individual items and distributions over properties of items. The canonical example is the Langevin equation which describes the motion of a Brownian particle interacting with a thermal reservoir, and the Fokker-Planck equation which describes the evolution of the distribution of possible positions and velocities of the particle. In inference the goal can analogously be to reach one conclusion or retrieve one object, or to establish characteristics of a distribution over objects. The second kind of inference is also called statistical inference. We will here discuss inference in this sense. In the “big-data era”, statistical inference of different kinds is routinely performed based on data sets containing millions or even billions of samples, which themselves may live in spaces of tremendously large dimensionality. In this realm, classical statistical wisdom and tools fail: new mathematics and algorithms able to tackle the phenomena emerging in this regime are necessary. In the very same way, phase transitions were understood to emerge from the complexity (i.e. high-dimensionality) of physical systems more than a century ago, whose understanding required to develop statistical mechanics. It turns out that this is more than an analogy as the theory and methods to perform high-dimensional inference are directly connected to statistical mechanics as we will see in this chapter (and others in the book [2]). High-dimensional inference itself is part of a broader statistical theory of complex systems, referred to as high-dimensional statistics, a very active research field at the crossroads of (statistical) physics, computer science, information theory and machine learning, and which is the powerhouse of modern information processing systems.
EasyNLP: A Comprehensive and Easy-to-use Toolkit for Natural Language Processing

The success of Pre-Trained Models (PTMs) has reshaped the development of Natural Language Processing (NLP). Yet, it is not easy to obtain high-performing models and deploy them online for industrial practitioners. To bridge this gap, EasyNLP is designed to make it easy to build NLP applications, which supports a comprehensive suite of NLP algorithms. It further features knowledge-enhanced pre-training, knowledge distillation and few-shot learning functionalities for large-scale PTMs, and provides a unified framework of model training, inference and deployment for real-world applications. Currently, EasyNLP has powered over ten business units within Alibaba Group and is seamlessly integrated to the Platform of AI (PAI) products on Alibaba Cloud. The source code of our EasyNLP toolkit is released at GitHub (https://github.com/alibaba/EasyNLP).
Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering

We present Answer-Me, a task-aware multi-task framework which unifies a variety of question answering tasks, such as, visual question answering, visual entailment, visual reasoning. In contrast to previous works using contrastive or generative captioning training, we propose a novel and simple recipe to pre-train a vision-language joint model, which is multi-task as well. The pre-training uses only noisy image captioning data, and is formulated to use the entire architecture end-to-end with both a strong language encoder and decoder. Our results show state-of-the-art performance, zero-shot generalization, robustness to forgetting, and competitive single-task results across a variety of question answering tasks. Our multi-task mixture training learns from tasks of various question intents and thus generalizes better, including on zero-shot vision-language tasks. We conduct experiments in the challenging multi-task and open-vocabulary settings and across a variety of datasets and tasks, such as VQA2.0, SNLI-VE, NLVR2, GQA. We observe that the proposed approach is able to generalize to unseen tasks and that more diverse mixtures lead to higher accuracy in both known and novel tasks.
OPT: Open Pre-trained Transformer Language Models

Large language models, which are often trained for hundreds of thousands of compute days, have shown remarkable capabilities for zero- and few-shot learning. Given their computational cost, these models are difficult to replicate without significant capital. For the few that are available through APIs, no access is granted to the full model weights, making them difficult to study. We present Open Pre-trained Transformers (OPT), a suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters, which we aim to fully and responsibly share with interested researchers. We show that OPT-175B is comparable to GPT-3, while requiring only 1/7th the carbon footprint to develop. We are also releasing our logbook detailing the infrastructure challenges we faced, along with code for experimenting with all of the released models.
Bayesian Benefit Risk Analysis

The process of approving and assessing new drugs is often quite complicated, mainly due to the fact that multiple criteria need to be considered. A standard way to proceed is with beneﬁt risk analysis, often under the Bayesian paradigm to account for uncertainty and combine data with expert judge-ment, which is operationalised via multi-criteria decision analysis (MCDA) scores. The procedure is based on a suitable model to accommodate key features of the data, which are typically of mixed type and potentially depended, with factor models providing a standard choice. The contribution of this paper is threefold: ﬁrst, we extend the family of existing structured factor models. Second, we provide a framework for choosing between them, which combines ﬁt and out-of-sample predictive performance. Third, we present a sequential estimation framework that can offer multiple beneﬁts: (i) it allows us to efﬁciently re-estimate MCDA scores of different drugs each time new data become available, thus getting an idea on potential ﬂuctuations in differences between them, (ii) it can provide information on potential early stopping in cases of evident conclusions, thus reducing unnecessary further exposure to undesirable treatments; (iii) it can potentially allow to assign treatment groups dynamically based on research objectives. A drawback of sequential estimation is the increased computational time, but this can be mitigated by efﬁcient sequential Monte Carlo schemes which we tailor in this paper to the context of Bayesian beneﬁt risk analysis. The devel-a oped methodology is illustrated on real data on Type II diabetes patients who were administered Metformin (MET), Rosiglitazone (RSG) and a combination of the two (AVM).
ARK: Fully Homomorphic Encryption Accelerator with Runtime Data Generation and Inter-Operation Key Reuse

Homomorphic Encryption (HE) is one of the most promising post-quantum cryptographic schemes that enable privacy-preserving computation on servers. However, noise accumulates as we perform operations on HE-encrypted data, restricting the number of possible operations. Fully HE (FHE) removes this restriction by introducing the bootstrapping operation, which refreshes the data; however, FHE schemes are highly memory-bound. Bootstrapping, in particular, requires loading GBs of evaluation keys and plaintexts from offchip memory, which makes FHE acceleration fundamentally bottlenecked by the off-chip memory bandwidth.In this paper, we propose ARK, an Accelerator for FHE with Runtime data generation and inter-operation Key reuse. ARK enables practical FHE workloads with a novel algorithm-architecture co-design to accelerate bootstrapping. We first eliminate the off-chip memory bandwidth bottleneck through runtime data generation and inter-operation key reuse. This approach enables ARK to fully exploit on-chip memory by substantially reducing the size of the working set. On top of such algorithmic enhancements, we build ARK microarchitecture that minimizes on-chip data movement through an efficient, alternating data distribution policy based on the data access patterns and a streamlined dataflow organization of the tailored functional units – including base conversion, number-theoretic transform, and automorphism units. Overall, our codesign effectively handles the heavy computation and data movement overheads of FHE, drastically reducing the cost of HE operations, including bootstrapping.
Generalization of the basis theorem for the B-type Coxeter groups

The OGS for non-abelian groups is an interesting generalization of the basis of ﬁnite abelian groups. The deﬁnition of OGS states that every element of a group has a unique presentation as a product of some powers of speciﬁc generators of the group, in a speciﬁc given order. In case of the symmetric groups S n there is a paper of R. Shwartz, which demonstrates a strong connection between the OGS and the standard Coxeter presentation of the symmetric group, which is called the standard OGS of S n . In this paper we generalize the standard OGS of S n to the ﬁnite classical Coxeter group B n . We describe the exchange laws for the generalized standard OGS of B n , and we connect it to the Coxeter length and the descent set of B n .
LMTurk: Few-Shot Learners as Crowdsourcing Workers

Vast efforts have been devoted to creating high-performance few-shot learners, i.e., large-scale pretrained language models (PLMs) that perform well with little downstream task training data. Training PLMs has incurred significant cost, but utilizing the few-shot learners is still challenging due to their enormous size. This work focuses on a crucial question: How to make effective use of these few-shot learners? We propose LMTurk, a novel approach that treats few-shot learners as crowdsourcing workers. The rationale is that crowdsourcing workers are in fact few-shot learners: They are shown a few illustrative examples to learn about a task and then start annotating. LMTurk employs few-shot learners built upon PLMs as workers. We show that the resulting annotations can be utilized to train models that solve the task well and are small enough to be deployable in practical scenarios. Active learning is integrated into LMTurk to reduce the amount of queries made to PLMs, minimizing the computational cost of running PLM inference passes. Altogether, LMTurk is an important step towards making effective use of current PLMs.
Configurable Independent Component Analysis Preprocessing Accelerator

An independent component analysis (ICA) has been used in many applications, including self-interference cancellation (SIC) for in-band full-duplex (IBFD) wireless systems and anomaly detection in industrial Internet of Things (IoT). This article presents a high-throughput and highly efficient configurable preprocessing accelerator for the ICA algorithm. The proposed ICA accelerator has three major blocks that perform data centering, covariance matrix for computation, and eigenvalue decomposition (EVD). Specifically, the proposed accelerator is based on a high-performance matrix multiplication array (MMA). The proposed MMA architecture uses time-multiplexed processing, so that the efficiency of hardware utilization is greatly enhanced. Furthermore, the processing flow utilizes parallel processing, such that the centering, the calculation of the covariance matrix, and the EVD are conducted simultaneously and are individually pipelined to maximize throughput. This article presents the architecture, circuit design, and performance estimates based on post-layout extraction of the proposed preprocessing ICA accelerator. The proposed design achieves a throughput of 40.7 kMatrices/s at a complexity of 73.3 kGE.
Geometric Graph Representation with Learnable Graph Structure and Adaptive AU Constraint for Micro-Expression Recognition

Micro-expression recognition (MER) is valuable because micro-expressions (MEs) can reveal genuine emotions. Most works take image sequences as input and cannot effectively explore ME information because subtle ME-related motions are easily submerged in unrelated information. Instead, the facial landmark is a low-dimensional and compact modality, which achieves lower computational cost and potentially concentrates on ME-related movement features. However, the discriminability of facial landmarks for MER is unclear. Thus, this paper explores the contribution of facial landmarks and proposes a novel framework to efficiently recognize MEs. Firstly, a geometric two-stream graph network is constructed to aggregate the low-order and high-order geometric movement information from facial landmarks to obtain discriminative ME representation. Secondly, a self-learning fashion is introduced to automatically model the dynamic relationship between nodes even long-distance nodes. Furthermore, an adaptive action unit loss is proposed to reasonably build the strong correlation between landmarks, facial action units and MEs. Notably, this work provides a novel idea with much higher efficiency to promote MER, only utilizing graph-based geometric features. The experimental results demonstrate that the proposed method achieves competitive performance with a significantly reduced computational cost. Furthermore, facial landmarks significantly contribute to MER and are worth further study for high-efficient ME analysis.
Adaptive Task Planning for Large-Scale Robotized Warehouses

Robotized warehouses are deployed to automatically distribute millions of items brought by the massive logistic orders from e-commerce. A key to automated item distribution is to plan paths for robots, also known as task planning, where each task is to deliver racks with items to pickers for processing and then return the rack back. Prior solutions are unfit for large-scale robotized warehouses due to the inflexibility to time-varying item arrivals and the low efficiency for high throughput. In this paper, we propose a new task planning problem called TPRW, which aims to minimize the end-to-end makespan that incorporates the entire item distribution pipeline, known as a fulfilment cycle. Direct extensions from state-of-the-art path finding methods are ineffective to solve the TPRW problem because they fail to adapt to the bottleneck variations of fulfillment cycles. In response, we propose Efficient Adaptive Task Planning, a framework for large-scale robotized warehouses with time-varying item arrivals. It adaptively selects racks to fulfill at each timestamp via rein-forcement learning, accounting for the time-varying bottleneck of the fulfillment cycles. Then it finds paths for robots to transport the selected racks. The framework adopts a series of efficient optimizations on both time and memory to handle large-scale item throughput. Evaluations on both synthesized and real data show an improvement of 37.1% in effectiveness and 75.5% in efficiency over the state-of-the-arts.
On Dynamic Ray Tracing and Anticipative Channel Prediction for Dynamic Environments

Ray tracing (RT) algorithms, which can simulate multipath radio propagation in the presence of geometric obstacles such as buildings, objects, or vehicles, are becoming quite popular, due to the increasing availability of digital environment databases and high-performance computation platforms, such as multicore computers and cloud computing services. When objects or vehicles are moving, which is the case of industrial or vehicular environments, multiple successive representations of the environment (“snapshots”) and multiple RT runs are often necessary, which require a great human effort and a great deal of computation resources. Recently, the dynamic RT (DRT) approach has been proposed to predict the multipath evolution within a given time lapse on the base of the current multipath geometry, assuming constant speeds and/or accelerations for moving objects, using analytical extrapolation formulas. This is done without rerunning a full RT for every “snapshot” of the environment, therefore with a great reduction in both human labor and computation time. When DRT is embedded in a mobile radio system and used in real time, ahead-of-time (or anticipative) field prediction is possible, which opens the way to interesting applications. In the present work, a full-3-D DRT algorithm is presented, which allows to account for multiple reflections, edge diffraction, and diffuse scattering for the general case where moving objects can translate and rotate. For the purpose of validation, the model is first applied to some ideal cases and then to realistic cases where results are compared with conventional RT simulation and measurements available in the literature.
Adaptive Online Optimization with Predictions: Static and Dynamic Environments

In the past few years, Online Convex Optimization (OCO) has received notable attention in the control literature thanks to its ﬂexible real-time nature and powerful performance guarantees. In this paper, we propose new step-size rules and OCO algorithms that simultaneously exploit gradient predictions , function predictions and dynamics , features particularly pertinent to control applications. The proposed algorithms enjoy static and dynamic regret bounds in terms of the dynamics of the reference action sequence, gradient prediction error and function prediction error, which are generalizations of known regularity measures from the literature. We present results for both convex and strongly convex costs. We validate the performance of the proposed algorithms in a trajectory tracking case study, as well as portfolio optimization using real-world datasets.
Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders

Pretrained multilingual language models (LMs) can be successfully transformed into multilingual sentence encoders (SEs; e.g., LaBSE, xMPNet) via additional fine-tuning or model distillation with parallel data. However, it remains unclear how to best leverage them to represent sub-sentence lexical items (i.e., words and phrases) in cross-lingual lexical tasks. In this work, we probe SEs for the amount of cross-lingual lexical knowledge stored in their parameters, and compare them against the original multilingual LMs. We also devise a simple yet efficient method for exposing the cross-lingual lexical knowledge by means of additional fine-tuning through inexpensive contrastive learning that requires only a small amount of word translation pairs. Using bilingual lexical induction (BLI), cross-lingual lexical semantic similarity, and cross-lingual entity linking as lexical probing tasks, we report substantial gains on standard benchmarks (e.g., +10 Precision@1 points in BLI). The results indicate that the SEs such as LaBSE can be ‘rewired’ into effective cross-lingual lexical encoders via the contrastive learning procedure, and that it is possible to expose more cross-lingual lexical knowledge compared to using them as off-the-shelf SEs. This way, we also provide an effective tool for harnessing ‘covert’ multilingual lexical knowledge hidden in multilingual sentence encoders.