Miracle-dz/UniFAR-FTUs huggingface dataset:
A General and Flexible Multi-concept Parsing Framework for Multilingual Semantic Matching

Sentence semantic matching is a research hotspot in natural language processing, which is considerably significant in various key scenarios, such as community question answering, searching, chatbot, and recommendation. Since most of the advanced models directly model the semantic relevance among words between two sentences while neglecting the \textit{keywords} and \textit{intents} concepts of them, DC-Match is proposed to disentangle keywords from intents and utilizes them to optimize the matching performance. Although DC-Match is a simple yet effective method for semantic matching, it highly depends on the external NER techniques to identify the keywords of sentences, which limits the performance of semantic matching for minor languages since satisfactory NER tools are usually hard to obtain. In this paper, we propose to generally and flexibly resolve the text into multi concepts for multilingual semantic matching to liberate the model from the reliance on NER models. To this end, we devise a \underline{M}ulti-\underline{C}oncept \underline{P}arsed \underline{S}emantic \underline{M}atching framework based on the pre-trained language models, abbreviated as \textbf{MCP-SM}, to extract various concepts and infuse them into the classification tokens. We conduct comprehensive experiments on English datasets QQP and MRPC, and Chinese dataset Medical-SM. Besides, we experiment on Arabic datasets MQ2Q and XNLI, the outstanding performance further prove MCP-SM's applicability in low-resource languages.
Equivariant automorphisms of the Cox construction and applications

In the present paper, we give a complete description of the group of holomorphic automorphisms of the Cox construction of a simplicial fan equivariant with respect to a large enough connected complex Lie subgroup of the large torus acting on the Cox construction. We then apply this description to find the groups of holomorphic automorphisms of rational moment-angle manifolds, including Calabi--Eckmann manifolds and Hopf manifolds. We also calculate the groups of equivariant automorphisms of complete simplicial toric varieties. This paper is the first in a series of two papers dedicated to studying the automorphism groups of toric stacks with applications to complex geometry.
Globally-stable and metastable crystal structure enumeration using polynomial machine learning potentials in elemental As, Bi, Ga, In, La, P, Sb, Sn, and Te

Machine learning potentials (MLPs) have become essential tools for accelerating accurate large-scale atomistic simulations and crystal structure predictions. The polynomial MLPs, which are described by polynomial rotational invariants, have been systematically developed for many elemental and alloy systems. This study presents an efficient procedure for enumerating global minimum and metastable structures using robust polynomial MLPs. To develop MLPs that can handle the structure enumeration, performing random structure searches and updating MLPs are iteratively repeated. The current procedure is systematically applied to the elemental systems of As, Bi, Ga, In, La, P, Sb, Sn, and Te, with many local minimum structures competing with the global minimum structure in terms of energy. This procedure will significantly accelerate global structure searches and expand their search space.
Improving Adversarial Energy-Based Model via Diffusion Process

Generative models have shown strong generation ability while efficient likelihood estimation is less explored. Energy-based models~(EBMs) define a flexible energy function to parameterize unnormalized densities efficiently but are notorious for being difficult to train. Adversarial EBMs introduce a generator to form a minimax training game to avoid expensive MCMC sampling used in traditional EBMs, but a noticeable gap between adversarial EBMs and other strong generative models still exists. Inspired by diffusion-based models, we embedded EBMs into each denoising step to split a long-generated process into several smaller steps. Besides, we employ a symmetric Jeffrey divergence and introduce a variational posterior distribution for the generator's training to address the main challenges that exist in adversarial EBMs. Our experiments show significant improvement in generation compared to existing adversarial EBMs, while also providing a useful energy function for efficient density estimation.
Rethinking LLM Language Adaptation: A Case Study on Chinese Mixtral

Mixtral, a representative sparse mixture of experts (SMoE) language model, has received significant attention due to its unique model design and superior performance. Based on Mixtral-8x7B-v0.1, in this paper, we propose Chinese-Mixtral and Chinese-Mixtral-Instruct with improved Chinese language abilities by adopting further pre-training and instruction fine-tuning. Experimental results show that our Chinese-Mixtral and Chinese-Mixtral-Instruct successfully improve Chinese understanding and generation performance while retaining the original English abilities. Then, we discuss several key questions when performing language adaptation on large language models, including the necessity of extending the language-specific vocabulary and the choice of the initialization model (foundation model v.s. instruction model), by providing empirical results and analysis. We also present the visualizations of each expert to examine their importance on downstream tasks. Our resources are publicly available through \url{https://github.com/ymcui/Chinese-Mixtral}.
An Improved Traditional Chinese Evaluation Suite for Foundation Model

We present TMMLU+, a comprehensive dataset designed for the Traditional Chinese massive multitask language understanding dataset. TMMLU+ is a multiple-choice question-answering dataset with 66 subjects from elementary to professional level. Compared to its predecessor, TMMLU, TMMLU+ is six times larger and boasts a more balanced subject distribution. We included benchmark results in TMMLU+ from closed-source models and 24 open-weight Chinese large language models of parameters ranging from 1.8B to 72B. Our findings reveal that Traditional Chinese models still trail behind their Simplified Chinese counterparts. Additionally, current large language models have yet to outperform human performance in average scores. We publicly release our dataset and the corresponding benchmark source code.
Daedalus: Self-Adaptive Horizontal Autoscaling for Resource Efficiency of Distributed Stream Processing Systems

Distributed Stream Processing (DSP) systems are capable of processing large streams of unbounded data, offering high throughput and low latencies. To maintain a stable Quality of Service (QoS), these systems require a sufficient allocation of resources. At the same time, over-provisioning can result in wasted energy and high operating costs. Therefore, to maximize resource utilization, autoscaling methods have been proposed that aim to efficiently match the resource allocation with the incoming workload. However, determining when and by how much to scale remains a significant challenge. Given the long-running nature of DSP jobs, scaling actions need to be executed at runtime, and to maintain a good QoS, they should be both accurate and infrequent. To address the challenges of autoscaling, the concept of self-adaptive systems is particularly fitting. These systems monitor themselves and their environment, adapting to changes with minimal need for expert involvement. This paper introduces Daedalus, a self-adaptive manager for autoscaling in DSP systems, which draws on the principles of self-adaption to address the challenge of efficient autoscaling. Daedalus monitors a running DSP job and builds performance models, aiming to predict the maximum processing capacity at different scale-outs. When combined with time series forecasting to predict future workloads, Daedalus proactively scales DSP jobs, optimizing for maximum throughput and minimizing both latencies and resource usage. We conducted experiments using Apache Flink and Kafka Streams to evaluate the performance of Daedalus against two state-of-the-art approaches. Daedalus was able to achieve comparable latencies while reducing resource usage by up to 71%.
Quick connect scanning tunneling microscope head with nested piezoelectric coarse walkers

To meet changing research demands, new scanning tunneling microscope (STM) features must constantly evolve. We describe the design, development, and performance of a modular plugin STM which is compact and stable. The STM head is equipped with a quick-connect socket that is matched to a universal connector plug, enabling it to be transferred between systems. This head can be introduced into a vacuum system via a load-lock and transferred to various sites equipped with the connector plug, permitting multi site STM operation. Its design allows reliable operation in a variety of experimental conditions, including a broad temperature range, ultrahigh vacuum, high magnetic fields, and closed-cycle pulse-tube cooling. The compact size of the STM is achieved by a novel nested piezoelectric coarse walker design which allows for large orthogonal travel in the X, Y, and Z directions, ideal for a studying both bulk and thin film samples ranging in size from mm to micrometers. Its stability and noise tolerance is demonstrated by achieving atomic resolution in ambient conditions on a laboratory desktop with no vibrational or acoustic isolation.
Multi-level Product Category Prediction through Text Classification

This article investigates applying advanced machine learning models, specifically LSTM and BERT, for text classification to predict multiple categories in the retail sector. The study demonstrates how applying data augmentation techniques and the focal loss function can significantly enhance accuracy in classifying products into multiple categories using a robust Brazilian retail dataset. The LSTM model, enriched with Brazilian word embedding, and BERT, known for its effectiveness in understanding complex contexts, were adapted and optimized for this specific task. The results showed that the BERT model, with an F1 Macro Score of up to $99\%$ for segments, $96\%$ for categories and subcategories and $93\%$ for name products, outperformed LSTM in more detailed categories. However, LSTM also achieved high performance, especially after applying data augmentation and focal loss techniques. These results underscore the effectiveness of NLP techniques in retail and highlight the importance of the careful selection of modelling and preprocessing strategies. This work contributes significantly to the field of NLP in retail, providing valuable insights for future research and practical applications.
Temporal Knowledge Graph Completion with Time-sensitive Relations in Hypercomplex Space

Temporal knowledge graph completion (TKGC) aims to fill in missing facts within a given temporal knowledge graph at a specific time. Existing methods, operating in real or complex spaces, have demonstrated promising performance in this task. This paper advances beyond conventional approaches by introducing more expressive quaternion representations for TKGC within hypercomplex space. Unlike existing quaternion-based methods, our study focuses on capturing time-sensitive relations rather than time-aware entities. Specifically, we model time-sensitive relations through time-aware rotation and periodic time translation, effectively capturing complex temporal variability. Furthermore, we theoretically demonstrate our method's capability to model symmetric, asymmetric, inverse, compositional, and evolutionary relation patterns. Comprehensive experiments on public datasets validate that our proposed approach achieves state-of-the-art performance in the field of TKGC.
Tsirelson bounds for quantum correlations with indefinite causal order

Quantum theory is in principle compatible with processes that violate causal inequalities, an analogue of Bell inequalities that constrain the correlations observed by a set of parties operating in a definite causal order. Since the introduction of causal inequalities, determining their maximum quantum violation, analogue to Tsirelson's bound for Bell inequalities, has remained an open problem. Here we provide a general method for bounding the violation of arbitrary causal inequalities, establishing limits on the correlations achievable by arbitrary local experiments and by arbitrary quantum processes with indefinite causal order. We prove that the maximum violation is generally smaller than the algebraic maximum, and determine Tsirelson-like bounds for a class of causal inequalities including some of the most paradigmatic examples. Surprisingly, we find that the algebraic maximum of arbitrary causal inequalities can be achieved by a new type of processes that allow for information to flow in an indefinite temporal direction within the parties' laboratories.
Shuffling Momentum Gradient Algorithm for Convex Optimization

The Stochastic Gradient Descent method (SGD) and its stochastic variants have become methods of choice for solving finite-sum optimization problems arising from machine learning and data science thanks to their ability to handle large-scale applications and big datasets. In the last decades, researchers have made substantial effort to study the theoretical performance of SGD and its shuffling variants. However, only limited work has investigated its shuffling momentum variants, including shuffling heavy-ball momentum schemes for non-convex problems and Nesterov's momentum for convex settings. In this work, we extend the analysis of the shuffling momentum gradient method developed in [Tran et al (2021)] to both finite-sum convex and strongly convex optimization problems. We provide the first analysis of shuffling momentum-based methods for the strongly convex setting, attaining a convergence rate of $O(1/nT^2)$, where $n$ is the number of samples and $T$ is the number of training epochs. Our analysis is a state-of-the-art, matching the best rates of existing shuffling stochastic gradient algorithms in the literature.
Deep Reinforcement Learning for Dynamic Algorithm Selection: A Proof-of-Principle Study on Differential Evolution

Evolutionary algorithms, such as Differential Evolution, excel in solving real-parameter optimization challenges. However, the effectiveness of a single algorithm varies across different problem instances, necessitating considerable efforts in algorithm selection or configuration. This paper aims to address the limitation by leveraging the complementary strengths of a group of algorithms and dynamically scheduling them throughout the optimization progress for specific problems. We propose a deep reinforcement learning-based dynamic algorithm selection framework to accomplish this task. Our approach models the dynamic algorithm selection a Markov Decision Process, training an agent in a policy gradient manner to select the most suitable algorithm according to the features observed during the optimization process. To empower the agent with the necessary information, our framework incorporates a thoughtful design of landscape and algorithmic features. Meanwhile, we employ a sophisticated deep neural network model to infer the optimal action, ensuring informed algorithm selections. Additionally, an algorithm context restoration mechanism is embedded to facilitate smooth switching among different algorithms. These mechanisms together enable our framework to seamlessly select and switch algorithms in a dynamic online fashion. Notably, the proposed framework is simple and generic, offering potential improvements across a broad spectrum of evolutionary algorithms. As a proof-of-principle study, we apply this framework to a group of Differential Evolution algorithms. The experimental results showcase the remarkable effectiveness of the proposed framework, not only enhancing the overall optimization performance but also demonstrating favorable generalization ability across different problem classes.
Climatic Aptitude Evaluation for Grapevine Cultivation in Pão de Açúcar, Alagoas

The grapes production in Brazil is comprised in southeastern and southern regions and also in the semi-arid Pernambuco. Environmental, climatic and even human factors influence on quality of grape production, which are sensitive to weather changes. In Alagoas State, a pilot project was carried out for Municipalities of Rio Largo, Pao de Acucar and Delmiro Gouveia between 2013 to 2015 years; but the results were incipient. This work evaluated the climatic aptitude for grapevine cultivation for municipality of Pao de Acucar. For climate characterization, three indices of the Geoviticure Multicriteria Climatic Classification System (MCC) were adopted: Heliothermic (HI), Cold Night (CI) and Dryness (DI), considering different cycles during the year. The Zuluaga Index (IZ) was also used to evaluate the risk of incidence of fungal diseases of the vine, especially in relation at mildew incidence (Plasmoparaviticola), a major disease in humid regions. According on CI, DI and ZI indices, the municipality of Pao de Acucar presented climatic aptitude for vines production with highest quality potential between August to January months, classified as preferential for all indexes analyzed.
Evaluating extensions to LCDM: an application of Bayesian model averaging

We employ Bayesian Model Averaging (BMA) as a powerful statistical framework to address key cosmological questions about the universe's fundamental properties. We explore extensions beyond the standard $\Lambda$CDM model, considering a varying curvature density parameter $\Omega_{\rm k}$, a spectral index $\mathrm{n}_{\rm s}=1$ and a varying $n_{\rm run}$, a constant dark energy equation of state (EOS) $w_0$CDM and a time-dependent one $w_0w_a$CDM. We also test cosmological data against a varying effective number of neutrino species $N_{\rm eff}$. Data from different combinations of cosmic microwave background (CMB) data from the last Planck PR4 analysis, CMB lensing from Planck 2018, baryonic acoustic oscillations (BAO) and the Bicep-KECK 2018 results, are used. We find that the standard $\Lambda$CDM model is favoured when combining CMB data with CMB lensing, BAO and Bicep-KECK 2018 data against $K-\Lambda$CDM model $N_{\rm eff}-\Lambda$CDM with a probability $>80\%$. When investigating the dark energy EOS, we find that this dataset is not able to express a strong preference between the standard $\Lambda$CDM model and the constant dark energy EOS model $w_0$CDM, with an approximately split model posterior probability of $\approx 60\%:40\%$ in favour of $\Lambda$CDM, whereas the time-varying dark energy EOS model is ruled out. Finally, we find that the CMB data alone show a strong preference for a model that includes the running of the spectral index $n_{\rm run}$, with a probability $\approx 90\%$, when compared to the $n_{\rm s}=1$ model and the standard $\Lambda$CDM. Overall, we find that including the model uncertainty in the considered cases does not significantly impact the Hubble tension.
Exploring the Limitations of Large Language Models in Compositional Relation Reasoning

We present a comprehensive evaluation of large language models(LLMs)' ability to reason about composition relations through a benchmark encompassing 1,500 test cases in English, designed to cover six distinct types of composition relations: Positional, Comparative, Personal, Mathematical, Identity, and Other. Acknowledging the significance of multilingual capabilities, we expanded our assessment to include translations of these cases into Chinese, Japanese, French, and Korean. Our Multilingual Composition Relation (MCR) benchmark aims at investigating the robustness and adaptability of LLMs in handling composition relation reasoning across diverse linguistic contexts.
Deployment Challenges of Industrial Intrusion Detection Systems

With the escalating threats posed by cyberattacks on Industrial Control Systems (ICSs), the development of customized Industrial Intrusion Detection Systems (IIDSs) received significant attention in research. While existing literature proposes effective IIDS solutions evaluated in controlled environments, their deployment in real-world industrial settings poses several challenges. This paper highlights two critical yet often overlooked aspects that significantly impact their practical deployment, i.e., the need for sufficient amounts of data to train the IIDS models and the challenges associated with finding suitable hyperparameters, especially for IIDSs training only on genuine ICS data. Through empirical experiments conducted on multiple state-of-the-art IIDSs and diverse datasets, we establish the criticality of these issues in deploying IIDSs. Our findings show the necessity of extensive malicious training data for supervised IIDSs, which can be impractical considering the complexity of recording and labeling attacks in actual industrial environments. Furthermore, while other IIDSs circumvent the previous issue by requiring only benign training data, these can suffer from the difficulty of setting appropriate hyperparameters, which likewise can diminish their performance. By shedding light on these challenges, we aim to enhance the understanding of the limitations and considerations necessary for deploying effective cybersecurity solutions in ICSs, which might be one reason why IIDSs see few deployments.
The Ink Splotch Effect: A Case Study on ChatGPT as a Co-Creative Game Designer

This paper studies how large language models (LLMs) can act as effective, high-level creative collaborators and ``muses'' for game design. We model the design of this study after the exercises artists use by looking at amorphous ink splotches for creative inspiration. Our goal is to determine whether AI-assistance can improve, hinder, or provide an alternative quality to games when compared to the creative intents implemented by human designers. The capabilities of LLMs as game designers are stress tested by placing it at the forefront of the decision making process. Three prototype games are designed across 3 different genres: (1) a minimalist base game, (2) a game with features and game feel elements added by a human game designer, and (3) a game with features and feel elements directly implemented from prompted outputs of the LLM, ChatGPT. A user study was conducted and participants were asked to blindly evaluate the quality and their preference of these games. We discuss both the development process of communicating creative intent to an AI chatbot and the synthesized open feedback of the participants. We use this data to determine both the benefits and shortcomings of AI in a more design-centric role.
Progressive Smoothing for Motion Planning in Real-Time NMPC

Nonlinear model predictive control (NMPC) is a popular strategy for solving motion planning problems, including obstacle avoidance constraints, in autonomous driving applications. Non-smooth obstacle shapes, such as rectangles, introduce additional local minima in the underlying optimization problem. Smooth over-approximations, e.g., ellipsoidal shapes, limit the performance due to their conservativeness. We propose to vary the smoothness and the related over-approximation by a homotopy. Instead of varying the smoothness in consecutive sequential quadratic programming iterations, we use formulations that decrease the smooth over-approximation from the end towards the beginning of the prediction horizon. Thus, the real-time iterations algorithm is applicable to the proposed NMPC formulation. Different formulations are compared in simulation experiments and shown to successfully improve performance indicators without increasing the computation time.