nils-herrmann/clean_scirepeval huggingface dataset:
Weight Friction: A Simple Method to Overcome Catastrophic Forgetting and Enable Continual Learning

In recent years, deep neural networks have found success in replicating human-level cognitive skills, yet they suffer from several major obstacles. One significant limitation is the inability to learn new tasks without forgetting previously learned tasks, a shortcoming known as catastrophic forgetting. In this research, we propose a simple method to overcome catastrophic forgetting and enable continual learning in neural networks. We draw inspiration from principles in neurology and physics to develop the concept of weight friction. Weight friction operates by a modification to the update rule in the gradient descent optimization method. It converges at a rate comparable to that of the stochastic gradient descent algorithm and can operate over multiple task domains. It performs comparably to current methods while offering improvements in computation and memory efficiency.
Quantale semantics of Lambek calculus with subexponential modalities

In this paper, we consider the polymodal version of Lambek calculus with subexponential modalities initially introduced by Kanovich, Kuznetsov, Nigam, and Scedrov and its quantale semantics. In our approach, subexponential modalities have an interpretation in terms of quantic conuclei. We show that this extension of Lambek calculus is complete w.r.t quantales with quantic conuclei. Also, we prove a representation theorem for quantales with quantic conuclei and show that Lambek calculus with subexponentials is relationally complete. Finally, we extend this representation theorem to the category of quantales with quantic conuclei.
A General Theory of Concept Lattice (I): Emergence of General Concept Lattice

As the first part of the treatise on A General Theory of Concept Lattice (I-V), this work develops the general concept lattice for the problem concerning categorization of objects according to their properties. Unlike the conventional approaches, such as the formal concept lattice and the rough set lattice, the general concept lattice is designed to adhere to the general principle that the information content should be invariant regardless how the variables/parameters are presented. Here, one will explicitly demonstrate the existence of such a construction by a sequence of fulfillment compatible with the conventional lattice structure. The general concept lattice promises to be a comprehensive categorization for all the distinctive object classes according to whatever properties they are equipped with. It will be shown that one can always regain the formal concept lattice and rough set lattice from the general concept lattice.
Proposition d'un modèle pour l'optimisation automatique de boucles dans le compilateur Tiramisu : cas d'optimisation de déroulage

Computer architectures become more and more complex. It requires more effort to develop techniques that improve the programs of performance and allow to exploit material resources efficiently. As a result, many transformations are applied on various levels of code abstraction. The first level is the high level, where the representation is close to the high level language. The second one is the low level, where the presentation is close to the machine code. Those transformations are called code optimizations. Optimizing programs requires deep expertise. On one hand, it is a tedious task, because it requires a lot of tests to find out the best combination of optimizations to apply with their best factors. On the other hand, this task is critical, because it may degrade the performance of the program instead of improving it. The automatization of this task can deal with this problem and permit to obtain good results. Our end of study project consists on proposing a novel approach based on neural networks to automatically optimize loops in Tiramisu. Tiramisu is a new language to create a code of high performance. It allows to separate between the algorithm and its optimizations. We have chosen loop unrolling as a study case. Our contribution aims to automate the choice of the best loop unrolling factor for a program written in Tiramisu.
Privacy-Preserving Distributed Machine Learning via Local Randomization and ADMM Perturbation

With the proliferation of training data, distributed machine learning (DML) is becoming more competent for large-scale learning tasks. However, privacy concerns have to be given priority in DML, since training data may contain sensitive information of users. In this paper, we propose a privacy-preserving ADMM-based DML framework with two novel features: First, we remove the assumption commonly made in the literature that the users trust the server collecting their data. Second, the framework provides heterogeneous privacy for users depending on data's sensitive levels and servers’ trust degrees. The challenging issue is to keep the accumulation of privacy losses over ADMM iterations minimal. In the proposed framework, a local randomization approach, which is differentially private, is adopted to provide users with self-controlled privacy guarantee for the most sensitive information. Further, the ADMM algorithm is perturbed through a combined noise-adding method, which simultaneously preserves privacy for users’ less sensitive information and strengthens the privacy protection of the most sensitive information. We provide detailed analyses on the performance of the trained model according to its generalization error. Finally, we conduct extensive experiments using real-world datasets to validate the theoretical results and evaluate the classification performance of the proposed framework.
Classi-Fly: Inferring Aircraft Categories from Open Data

In recent years, air traffic communication data has become easy to access, enabling novel research in many fields. Exploiting this new data source, a wide range of applications have emerged, from weather forecasting to stock market prediction, or the collection of intelligence about military and government movements. Typically, these applications require knowledge about the metadata of the aircraft, specifically its operator and the aircraft category.
 
 armasuisse Science + Technology
 , the R&D agency for the Swiss Armed Forces, has been developing Classi-Fly, a novel approach to obtain metadata about aircraft based on their movement patterns. We validate Classi-Fly using several hundred thousand flights collected through open source means, in conjunction with ground truth from publicly available aircraft registries containing more than 2 million aircraft. We show that we can obtain the correct aircraft category with an accuracy of greater than 88%. In cases, where no metadata is available, this approach can be used to create the data necessary for applications working with air traffic communication. Finally, we show that it is feasible to automatically detect particular sensitive aircraft such as police and surveillance aircraft using this method.
On the modes of convergence of Stochastic Optimistic Mirror Descent (OMD) for saddle point problems

In this article, we study the convergence of Mirror Descent (MD) and Optimistic Mirror Descent (OMD) for saddle point problems satisfying the notion of coherence as proposed in Mertikopoulos et al. We prove convergence of OMD with exact gradients for coherent saddle point problems, and show that monotone convergence only occurs after some sufficiently large number of iterations. This is in contrast to the claim in Mertikopoulos et al. of monotone convergence of OMD with exact gradients for coherent saddle point problems. Besides highlighting this important subtlety, we note that the almost sure convergence guarantees of MD and OMD with stochastic gradients for strictly coherent saddle point problems that are claimed in Mertikopoulos et al. are not fully justified by their proof. As such, we fill out the missing details in the proof and as a result have only been able to prove convergence with high probability. 
We would like to note that our analysis relies heavily on the core ideas and proof techniques introduced in Zhou et al. and Mertikopoulos et al., and we only aim to re-state and correct the results in light of what we were able to prove rigorously while filling in the much needed missing details in their proofs.
Arithmetic and geometry of a K3 surface emerging from virtual corrections to Drell–Yan scattering

We study a K3 surface, which appears in the two-loop mixed electroweak-quantum chromodynamic virtual corrections to Drell--Yan scattering. A detailed analysis of the geometric Picard lattice is presented, computing its rank and discriminant in two independent ways: first using explicit divisors on the surface and then using an explicit elliptic fibration. We also study in detail the elliptic fibrations of the surface and use them to provide an explicit Shioda--Inose structure. Moreover, we point out the physical relevance of our results.
Iterative methods for linear systems of equations: A brief historical journey

This paper presents a brief historical survey of iterative methods for solving linear systems of equations. The journey begins with Gauss who developed the first known method that can be termed iterative. The early 20th century saw good progress of these methods which were initially used to solve least-squares systems, and then linear systems arising from the discretization of partial different equations. Then iterative methods received a big impetus in the 1950s - partly because of the development of computers. The survey does not attempt to be exhaustive. Rather, the aim is to underline the way of thinking at a specific time and to highlight the major ideas that steered the field.
Network Shrinkage Estimation

Networks are a natural representation of complex systems across the sciences, and higher-order dependencies are central to the understanding and modeling of these systems. However, in many practical applications such as online social networks, networks are massive, dynamic, and naturally streaming, where pairwise interactions become available one at a time in some arbitrary order. The massive size and streaming nature of these networks allow only partial observation, since it is infeasible to analyze the entire network. Under such scenarios, it is challenging to study the higher-order structural and connectivity patterns of streaming networks. In this work, we consider the fundamental problem of estimating the higher-order dependencies using adaptive sampling. We propose a novel adaptive, single-pass sampling framework and unbiased estimators for higher-order network analysis of large streaming networks. Our algorithms exploit adaptive techniques to identify edges that are highly informative for efficiently estimating the higher-order structure of streaming networks from small sample data. We also introduce a novel James-Stein-type shrinkage estimator that reduces estimation error. Our approach is fully analytic, computationally efficient, and can be incrementally updated in a streaming setting. Numerical experiments on large networks show that our approach is superior to baseline methods.
Path Length Bounds for Gradient Descent and Flow

We derive bounds on the path length $\zeta$ of gradient descent (GD) and gradient flow (GF) curves for various classes of smooth convex and nonconvex functions. Among other results, we prove that: (a) if the iterates are linearly convergent with factor $(1-c)$, then $\zeta$ is at most $\mathcal{O}(1/c)$; (b) under the Polyak-Kurdyka-Lojasiewicz (PKL) condition, $\zeta$ is at most $\mathcal{O}(\sqrt{\kappa})$, where $\kappa$ is the condition number, and at least $\widetilde\Omega(\sqrt{d} \wedge \kappa^{1/4})$; (c) for quadratics, $\zeta$ is $\Theta(\min\{\sqrt{d},\sqrt{\log \kappa}\})$ and in some cases can be independent of $\kappa$; (d) assuming just convexity, $\zeta$ can be at most $2^{4d\log d}$; (e) for separable quasiconvex functions, $\zeta$ is ${\Theta}(\sqrt{d})$. Thus, we advance current understanding of the properties of GD and GF curves beyond rates of convergence. We expect our techniques to facilitate future studies for other algorithms.
Requirements-Driven Test Generation for Autonomous Vehicles With Machine Learning Components

Autonomous vehicles are complex systems that are challenging to test and debug. A requirements-driven approach to the development process can decrease the resources required to design and test these systems, while simultaneously increasing the reliability. We present a testing framework that uses signal temporal logic (STL), which is a precise and unambiguous requirements language. Our framework evaluates test cases against the STL formulae and additionally uses the requirements to automatically identify test cases that fail to satisfy the requirements. One of the key features of our tool is the support for machine learning (ML) components in the system design, such as deep neural networks. The framework allows evaluation of the control algorithms, including the ML components, and it also includes models of CCD camera, lidar, and radar sensors, as well as the vehicle environment. We use multiple methods to generate test cases, including covering arrays, which is an efficient method to search discrete variable spaces. The resulting test cases can be used to debug the controller design by identifying controller behaviors that do not satisfy requirements. The test cases can also enhance the testing phase of development by identifying critical corner cases that correspond to the limits of the system's allowed behaviors. We present STL requirements for an autonomous vehicle system, which capture both component-level and system-level behaviors. Additionally, we present three driving scenarios and demonstrate how our requirements-driven testing framework can be used to identify critical system behaviors, which can be used to support the development process.
The Use of Binary Choice Forests to Model and Estimate Discrete Choices

We show the equivalence of discrete choice models and the class of binary choice forests, which are random forests based on binary choice trees. This suggests that standard machine learning techniques based on random forests can serve to estimate discrete choice models with an interpretable output. This is confirmed by our data-driven theoretical results which show that random forests can predict the choice probability of any discrete choice model consistently, with its splitting criterion capable of recovering preference rank lists. The framework has unique advantages: it can capture behavioral patterns such as irrationality or sequential searches; it handles nonstandard formats of training data that result from aggregation; it can measure product importance based on how frequently a random customer would make decisions depending on the presence of the product; it can also incorporate price information and customer features. Our numerical results show that using random forests to estimate customer choices represented by binary choice forests can outperform the best parametric models in synthetic and real datasets.
Risk Management via Anomaly Circumvent: Mnemonic Deep Learning for Midterm Stock Prediction

Midterm stock price prediction is crucial for value investments in the stock market. However, most deep learning models are essentially short-term and applying them to midterm predictions encounters large cumulative errors because they cannot avoid anomalies. In this paper, we propose a novel deep neural network Mid-LSTM for midterm stock prediction, which incorporates the market trend as hidden states. First, based on the autoregressive moving average model (ARMA), a midterm ARMA is formulated by taking into consideration both hidden states and the capital asset pricing model. Then, a midterm LSTM-based deep neural network is designed, which consists of three components: LSTM, hidden Markov model and linear regression networks. The proposed Mid-LSTM can avoid anomalies to reduce large prediction errors, and has good explanatory effects on the factors affecting stock prices. Extensive experiments on S&P 500 stocks show that (i) the proposed Mid-LSTM achieves 2-4% improvement in prediction accuracy, and (ii) in portfolio allocation investment, we achieve up to 120.16% annual return and 2.99 average Sharpe ratio.
ABD-Net: Attentive but Diverse Person Re-Identification

Attention mechanisms have been found effective for person re-identification (Re-ID). However, the learned ``attentive'' features are often not naturally uncorrelated or ``diverse'', which compromises the retrieval performance based on the Euclidean distance. We advocate the complementary powers of attention and diversity for Re-ID, by proposing an Attentive but Diverse Network (ABD-Net). ABD-Net seamlessly integrates attention modules and diversity regularizations throughout the entire network to learn features that are representative, robust, and more discriminative. Specifically, we introduce a pair of complementary attention modules, focusing on channel aggregation and position awareness, respectively. Then, we plug in a novel orthogonality constraint that efficiently enforces diversity on both hidden activations and weights. Through an extensive set of ablation study, we verify that the attentive and diverse terms each contributes to the performance boosts of ABD-Net. It consistently outperforms existing state-of-the-art methods on there popular person Re-ID benchmarks.
Machinic Surrogates: Human-Machine Relationships in Computational Creativity

Recent advancements in artificial intelligence (AI) and its sub-branch machine learning (ML) promise machines that go beyond the boundaries of automation and behave autonomously. Applications of these machines in creative practices such as art and design entail relationships between users and machines that have been described as a form of collaboration or co-creation between computational and human agents. This paper uses examples from art and design to argue that this frame is incomplete as it fails to acknowledge the socio-technical nature of AI systems, and the different human agencies involved in their design, implementation, and operation. Situating applications of AI-enabled tools in creative practices in a spectrum between automation and autonomy, this paper distinguishes different kinds of human engagement elicited by systems deemed automated or autonomous. Reviewing models of artistic collaboration during the late 20th century, it suggests that collaboration is at the core of these artistic practices. We build upon the growing literature of machine learning and art to look for the human agencies inscribed in works of computational creativity, and expand the co-creation frame to incorporate emerging forms of human-human collaboration mediated through technical artifacts such as algorithms and data.
Linkages and systemic risk in the European insurance sector: Some new evidence based on dynamic spanning trees

This paper is part of the research on the interlinkages between insurers and their contribution to systemic risk on the insurance market. Its main purpose is to present the results of the analysis of linkage dynamics and systemic risk in the European insurance sector which are obtained using correlation networks. These networks are based on dynamic dependence structures modelled using a copula. Then, we determine minimum spanning trees (MST). Finally, the linkage dynamics is described by means of selected topological network measures.
Relatively growth optimal investment strategies in a market model with competition

We consider a game-theoretic model of a market where investors compete for payoffs yielded by several assets. The main result consists in a proof of existence and uniqueness of a strategy, called relatively growth optimal, such that the logarithm of the share of its wealth in the total wealth of the market is a submartingale for any strategies of the other investors. It is also shown that this strategy is asymptotically optimal in the sense that it achieves the maximal capital growth rate when compared to competing strategies. Based on the obtained results, we study the asymptotic structure of the market when all the investors use the relatively growth optimal strategy.
Permutation-Invariant Feature Restructuring for Correlation-Aware Image Set-Based Recognition

We consider the problem of comparing the similarity of image sets with variable-quantity, quality and un-ordered heterogeneous images. We use feature restructuring to exploit the correlations of both inner$\&$inter-set images. Specifically, the residual self-attention can effectively restructure the features using the other features within a set to emphasize the discriminative images and eliminate the redundancy. Then, a sparse/collaborative learning-based dependency-guided representation scheme reconstructs the probe features conditional to the gallery features in order to adaptively align the two sets. This enables our framework to be compatible with both verification and open-set identification. We show that the parametric self-attention network and non-parametric dictionary learning can be trained end-to-end by a unified alternative optimization scheme, and that the full framework is permutation-invariant. In the numerical experiments we conducted, our method achieves top performance on competitive image set/video-based face recognition and person re-identification benchmarks.
Learning Local Feature Descriptor with Motion Attribute For Vision-based Localization

In recent years, camera-based localization has been widely used for robotic applications, and most proposed algorithms rely on local features extracted from recorded images. For better performance, the features used for open-loop localization are required to be short-term globally static, and the ones used for re-localization or loop closure detection need to be long-term static. Therefore, the motion attribute of a local feature point could be exploited to improve localization performance, e.g., the feature points extracted from moving persons or vehicles can be excluded from these systems due to their unsteadiness. In this paper, we design a fully convolutional network (FCN), named MD-Net, to perform motion attribute estimation and feature description simultaneously. MD-Net has a shared backbone network to extract features from the input image and two network branches to complete each sub-task. With MD-Net, we can obtain the motion attribute while avoiding increasing much more computation. Experimental results demonstrate that the proposed method can learn distinct local feature descriptor along with motion attribute only using an FCN, by outperforming competing methods by a wide margin. We also show that the proposed algorithm can be integrated into a vision-based localization algorithm to improve estimation accuracy significantly.
Searching for Ambiguous Objects in Videos using Relational Referring Expressions

Humans frequently use referring (identifying) expressions to refer to objects. Especially in ambiguous settings, humans prefer expressions (called relational referring expressions) that describe an object with respect to a distinguishing, unique object. Unlike studies on video object search using referring expressions, in this paper, our focus is on (i) relational referring expressions in highly ambiguous settings, and (ii) methods that can both generate and comprehend a referring expression. For this goal, we first introduce a new dataset for video object search with referring expressions that includes numerous copies of the objects, making it difficult to use non-relational expressions. Moreover, we train two baseline deep networks on this dataset, which show promising results. Finally, we propose a deep attention network that significantly outperforms the baselines on our dataset. The dataset and the codes are available at this https URL.