Miracle-dz/UniFAR-FTUs huggingface dataset:
Reviewing the geometric Hamilton–Jacobi theory concerning Jacobi and Leibniz identities

In this survey, we review the classical Hamilton–Jacobi theory from a geometric point of view in different geometric backgrounds. We propose a Hamilton–Jacobi equation for different geometric structures attending to one particular characterization: whether they fulfill the Jacobi and Leibniz identities simultaneously, or if at least they satisfy one of them. This property reviews the work of the present authors in such a way that it is presented according to a hierarchy of nested brackets. This is a new outlook in which we do not only present the geometric structures themselves, but their hierarchical relations and how they affect their dynamic through the Hamilton–Jacobi equation. In this regard, we review the case of time-dependent (t-dependent in the sequel) and dissipative physical systems as systems that fulfill the Jacobi identity but not the Leibnitz identity, as instances of cosymplectic and contact mechanical problems. Let us remark that dissipative here means dissipation provided by a dissipation parameter included in the Hamiltonian function, for instance, the entropy in some thermodynamical systems. Furthermore, we review the contact-evolution Hamilton–Jacobi theory as a split off the regular contact geometry, and that actually satisfies the Leibniz rule instead of Jacobi. Furthermore, we include a novel result, which is the Hamilton–Jacobi equation for conformal Hamiltonian vector fields as a generalization of the well-known Hamilton–Jacobi on a symplectic manifold, that is retrieved in the case of a zero conformal factor. The interest of a geometric Hamilton–Jacobi equation is the primordial observation that if a Hamiltonian vector field X H can be projected into a configuration manifold by means of a one-form dW, then the integral curves of the projected vector field XHdW can be transformed into integral curves of X H provided that W is a solution of the Hamilton–Jacobi equation. Geometrically, the solution of the Hamilton–Jacobi equation plays the role of a Lagrangian submanifold of a certain bundle. Exploiting these features in different geometric scenarios we propose a geometric theory for multiple physical systems depending on the fundamental identities that their dynamic satisfies. Different examples are pictured to reflect the results provided, being all of them new, except for one that is reassessment of a previously considered example. Finally, let us mention that an important issue in the study of the Hamilton–Jacobi equation is separability, relevant for finding conserved quantities. However, we will not deal with this subject in this survey, although we refer the reader to papers [16, 17, 136].
Distributed Sparse Normal Means Estimation with Sublinear Communication

We consider the problem of sparse normal means estimation in a distributed setting with communication constraints. We assume there are $M$ machines, each holding $d$-dimensional observations of a $K$-sparse vector $\mu$ corrupted by additive Gaussian noise. The $M$ machines are connected in a star topology to a fusion center, whose goal is to estimate the vector $\mu$ with a low communication budget. Previous works have shown that to achieve the centralized minimax rate for the $\ell_2$ risk, the total communication must be high - at least linear in the dimension $d$. This phenomenon occurs, however, at very weak signals. We show that at signal-to-noise ratios (SNRs) that are sufficiently high - but not enough for recovery by any individual machine - the support of $\mu$ can be correctly recovered with significantly less communication. Specifically, we present two algorithms for distributed estimation of a sparse mean vector corrupted by either Gaussian or sub-Gaussian noise. We then prove that above certain SNR thresholds, with high probability, these algorithms recover the correct support with total communication that is sublinear in the dimension $d$. Furthermore, the communication decreases exponentially as a function of signal strength. If in addition $KM\ll \tfrac{d}{\log d}$, then with an additional round of sublinear communication, our algorithms achieve the centralized rate for the $\ell_2$ risk. Finally, we present simulations that illustrate the performance of our algorithms in different parameter regimes.
About posets for which no lower cover or no upper cover has the fixed point property

For a finite non-empty set X, let P(X) denote the set of all posets with carrier X, ordered by inclusion of their partial order relations. We investigate properties of posets P ∈ P(X) for which no lower cover or no upper cover in P(X) has the fixed point property. We derive two conditions, one of them sufficient for that no lower cover of P has the fixed point property, the other one sufficient for that no upper cover of P has the fixed point property, and we apply these results on several types of posets. Mathematics Subject Classification: Primary: 06A07. Secondary: 06A06.
On the Evaluation of Neural Code Summarization

Source code summaries are important for program comprehension and maintenance. However, there are plenty of programs with missing, outdated, or mismatched summaries. Recently, deep learning techniques have been exploited to automatically generate summaries for given code snippets. To achieve a profound understanding of how far we are from solving this problem and provide suggestions to future research, in this paper, we conduct a systematic and in-depth analysis of 5 state-of-the-art neural code summarization models on 6 widely used BLEU variants, 4 pre-processing operations and their combinations, and 3 widely used datasets. The evaluation results show that some important factors have a great influence on the model evaluation, especially on the performance of models and the ranking among the models. However, these factors might be easily overlooked. Specifically, (1) the BLEU metric widely used in existing work of evaluating code summarization models has many variants. Ignoring the differences among these variants could greatly affect the validity of the claimed results. Besides, we discover and resolve an important and previously unknown bug in BLEU calculation in a commonly-used software package. Furthermore, we conduct human evaluations and find that the metric BLEU-DC is most correlated to human perception; (2) code pre-processing choices can have a large (from −18% to +25%) impact on the summarization performance and should not be neglected. We also explore the aggregation of pre-processing combinations and boost the performance of models; (3) some important char-acteristics of datasets (corpus sizes, data splitting methods, and duplication ratios) have a significant impact on model evaluation. Based on the experimental results, we give actionable suggestions for evaluating code summarization and choosing the best method in different scenarios. We also build a shared code summarization toolbox to facilitate future research.
Competitive Information Design for Pandora's Box

We study a natural competitive-information-design variant for the Pandora's Box problem (Weitzman 1979), where each box is associated with a strategic information sender who can design what information about the box's prize value to be revealed to the agent when the agent inspects the box. This variant with strategic boxes is motivated by a wide range of real-world economic applications for Pandora's Box. The main contributions of this article are two-fold: (1) we study informational properties of Pandora's Box by showing an intrinsic connection between informativeness of any box's value distribution and the utility order of the search agent; and (2) we fully characterize the pure symmetric equilibrium for the boxes' competitive information revelation, which reveals various insights regarding information competition and the resultant agent utility at equilibrium.
Outstanding thermoelectric performance predicted for out-of-plane p-doped GeSe

The record-breaking thermoelectric performance of tin selenide (SnSe) has motivated the investigation of analogue compounds with the same structure. A promising candidate that emerged recently is germanium selenide (GeSe). Here, using extensive first-principles calculations of the hole-phonon and hole-impurity scattering, we investigate the thermoelectric transport properties of the orthorhombic phase of p-doped GeSe. We predict outstanding thermoelectric performance for GeSe over a broad range of temperatures due to its high Seebeck coefficients, extremely low Lorenz numbers, ultralow total thermal conductivity, and relatively large band gap. In particular, the out-of-plane direction in GeSe presents equivalent or even higher performance than SnSe for temperatures above 500 K. By extending the analysis to 900 K, we obtained an ultrahigh value for the thermoelectric figure of merit (zT = 3.2) at the optimal hole density of 4×10 cm. Our work provides strong motivation for continued experimental work focusing on improving the GeSe doping efficiency in order to achieve this optimal hole density.
Indication as Prior Knowledge for Multimodal Disease Classification in Chest Radiographs with Transformers

When a clinician refers a patient for an imaging exam, they include the reason (e.g. relevant patient history, suspected disease) in the scan request; this appears as the indication field in the radiology report. The interpretation and reporting of the image are substantially influenced by this request text, steering the radiologist to focus on particular aspects of the image. We use the indication field to drive better image classification, by taking a transformer network which is unimodally pre-trained on text (BERT) and fine-tuning it for multimodal classification of a dual image-text input. We evaluate the method on the MIMIC-CXR dataset, and present ablation studies to investigate the effect of the indication field on the classification performance. The experimental results show our approach achieves 87.8 average micro AUROC, outperforming the state-of-the-art methods for unimodal (84.4) and multimodal (86.0) classification. Our code is available at https://github.com/jacenkow/mmbt.
Visual Acoustic Matching

We introduce the visual acoustic matching task, in which an audio clip is transformed to sound like it was recorded in a target environment. Given an image of the target environment and a waveform for the source audio, the goal is to re-synthesize the audio to match the target room acoustics as suggested by its visible geometry and materials. To address this novel task, we propose a cross-modal transformer model that uses audio-visual attention to inject visual properties into the audio and generate realistic audio output. In addition, we devise a self-supervised training objective that can learn acoustic matching from in-the-wild Web videos, despite their lack of acoustically mismatched audio. We demonstrate that our approach successfully translates human speech to a variety of real-world environments depicted in images, outperforming both traditional acoustic matching and more heavily supervised baselines.
A Contrastive Framework for Neural Text Generation

Text generation is of great importance to many natural language processing applications. However, maximization-based decoding methods (e.g. beam search) of neural language models often lead to degenerate solutions -- the generated text is unnatural and contains undesirable repetitions. Existing approaches introduce stochasticity via sampling or modify training objectives to decrease probabilities of certain tokens (e.g., unlikelihood training). However, they often lead to solutions that lack coherence. In this work, we show that an underlying reason for model degeneration is the anisotropic distribution of token representations. We present a contrastive solution: (i) SimCTG, a contrastive training objective to calibrate the model's representation space, and (ii) a decoding method -- contrastive search -- to encourage diversity while maintaining coherence in the generated text. Extensive experiments and analyses on three benchmarks from two languages demonstrate that our proposed approach significantly outperforms current state-of-the-art text generation methods as evaluated by both human and automatic metrics.
Memory Replay with Data Compression for Continual Learning

Continual learning needs to overcome catastrophic forgetting of the past. Memory replay of representative old training samples has been shown as an effective solution, and achieves the state-of-the-art (SOTA) performance. However, existing work is mainly built on a small memory buffer containing a few original data, which cannot fully characterize the old data distribution. In this work, we propose memory replay with data compression (MRDC) to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer. Observing that the trade-off between the quality and quantity of compressed data is highly nontrivial for the efficacy of memory replay, we propose a novel method based on determinantal point processes (DPPs) to efficiently determine an appropriate compression quality for currently-arrived training samples. In this way, using a naive data compression algorithm with a properly selected quality can largely boost recent strong baselines by saving more compressed data in a limited storage space. We extensively validate this across several benchmarks of class-incremental learning and in a realistic scenario of object detection for autonomous driving.
Incremental Learning of Structured Memory via Closed-Loop Transcription

This work proposes a minimal computational model for learning structured memories of multiple object classes in an incremental setting. Our approach is based on establishing a closed-loop transcription between the classes and a corresponding set of subspaces, known as a linear discriminative representation, in a low-dimensional feature space. Our method is simpler than existing approaches for incremental learning, and more efficient in terms of model size, storage, and computation: it requires only a single, fixed-capacity autoencoding network with a feature space that is used for both discriminative and generative purposes. Network parameters are optimized simultaneously without architectural manipulations, by solving a constrained minimax game between the encoding and decoding maps over a single rate reduction-based objective. Experimental results show that our method can effectively alleviate catastrophic forgetting, achieving significantly better performance than prior work of generative replay on MNIST, CIFAR-10, and ImageNet-50, despite requiring fewer resources. Source code can be found at https://github.com/tsb0601/i-CTRL
Optimal Spend Rate Estimation and Pacing for Ad Campaigns with Budgets

Online ad platforms offer budget management tools for advertisers that aim to maximize the number of conversions given a budget constraint. As the volume of impressions, conversion rates and prices vary over time, these budget management systems learn a spend plan (to find the optimal distribution of budget over time) and run a pacing algorithm which follows the spend plan. This paper considers two models for impressions and competition that varies with time: a) an episodic model which exhibits stationarity in each episode, but each episode can be arbitrarily different from the next, and b) a model where the distributions of prices and values change slowly over time. We present the first learning theoretic guarantees on both the accuracy of spend plans and the resulting end-to-end budget management system. We present four main results: 1) for the episodic setting we give sample complexity bounds for the spend rate prediction problem: given $n$ samples from each episode, with high probability we have $|\widehat{\rho}_e - \rho_e| \leq \tilde{O}(\frac{1}{n^{1/3}})$ where $\rho_e$ is the optimal spend rate for the episode, $\widehat{\rho}_e$ is the estimate from our algorithm, 2) we extend the algorithm of Balseiro and Gur (2017) to operate on varying, approximate spend rates and show that the resulting combined system of optimal spend rate estimation and online pacing algorithm for episodic settings has regret that vanishes in number of historic samples $n$ and the number of rounds $T$, 3) for non-episodic but slowly-changing distributions we show that the same approach approximates the optimal bidding strategy up to a factor dependent on the rate-of-change of the distributions and 4) we provide experiments showing that our algorithm outperforms both static spend plans and non-pacing across a wide variety of settings.
Deduplicating Training Data Mitigates Privacy Risks in Language Models

Past work has shown that large language models are susceptible to privacy attacks, where adversaries generate sequences from a trained model and detect which sequences are memorized from the training set. In this work, we show that the success of these attacks is largely due to duplication in commonly used web-scraped training sets. We first show that the rate at which language models regenerate training sequences is superlinearly related to a sequence's count in the training set. For instance, a sequence that is present 10 times in the training data is on average generated ~1000 times more often than a sequence that is present only once. We next show that existing methods for detecting memorized sequences have near-chance accuracy on non-duplicated training sequences. Finally, we find that after applying methods to deduplicate training data, language models are considerably more secure against these types of privacy attacks. Taken together, our results motivate an increased focus on deduplication in privacy-sensitive applications and a reevaluation of the practicality of existing privacy attacks.
Chemical Characterisation of Compression Wood in Pinus Radiata

To determine how the chemistry of compression wood varies in Pinus radiata, we have studied summative composition and lignin structure in a range of different compression wood and opposite wood samples. Wood anatomy was used to characterise the compression wood severity. Lignin structure was studied by three methods based on thioacidolysis. Overall the results showed that, as the severity of compression wood changed, progressively from opposite through mild to severe, all features commonly associated with compression wood changed concurrently. In particular, galactose levels increased and glucose and mannose levels decreased, lignin content increased and the lignin became more condensed. Levels of p-hydroxyphenyl β-O-4 units increased while guaiacyl β-O-4 units decreased with increasing compression wood severity. Similar chemical compositional and lignin structural changes were seen in stems, branches and young wood upon compression wood formation.
RoPGen: Towards Robust Code Authorship Attribution via Automatic Coding Style Transformation

Source code authorship attribution is an important problem often encountered in applications such as software forensics, bug fixing, and software quality analysis. Recent studies show that current source code authorship attribution methods can be compromised by attackers exploiting adversarial examples and coding style ma-nipulation. This calls for robust solutions to the problem of code authorship attribution. In this paper, we initiate the study on making Deep Learning (DL)-based code authorship attribution robust. We propose an innovative framework called Robust coding style Patterns Generation (RoPGen), which essentially learns authors' unique coding style patterns that are hard for attackers to manip-ulate or imitate. The key idea is to combine data augmentation and gradient augmentation at the adversarial training phase. This effectively increases the diversity of training examples, generates meaningful perturbations to gradients of deep neural networks, and learns diversified representations of coding styles. We evaluate the effectiveness of RoPGen using four datasets of programs written in C, C++, and Java. Experimental results show that RoPGen can significantly improve the robustness of DL-based code authorship attribution, by respectively reducing 22.8% and 41.0% of the success rate of targeted and untargeted attacks on average.
CodeFill: Multi-token Code Completion by Jointly learning from Structure and Naming Sequences

Code completion is an essential feature of IDEs, yet current auto-completers are restricted to either grammar-based or NLP-based single token completions. Both approaches have significant draw-backs: grammar-based autocompletion is restricted in dynamically-typed language environments, whereas NLP-based autocompleters struggle to understand the semantics of the programming language and the developer's code context. In this work, we present CodeFill, a language model for autocompletion that combines learned structure and naming information. Using a parallel Transformer architecture and multi-task learning, CodeFill consumes sequences of source code token names and their equivalent AST token types. Uniquely, CodeFill is trained both for single-token and multi-token (statement) prediction, which enables it to learn long-range dependencies among grammatical and naming elements. We train CodeFill on two datasets, consisting of 29M and 425M lines of code, respectively. To make the evaluation more realistic, we develop a method to automatically infer points in the source code at which completion matters. We compare CodeFill against four baselines and two state-of-the-art models, GPT-C and TravTrans+. CodeFill surpasses all baselines in single token prediction (MRR: 70.9% vs. 66.2% and 67.8%) and outperforms the state of the art for multi-token prediction (ROUGE-L: 63.7% vs. 52.4% and 59.2%, for $n=4$ tokens). We publicly release our source code and datasets.
Phase diagrams of Kitaev models for arbitrary magnetic field orientations

The Kitaev model is an exactly solvable quantum spin model within the language of the constrained real fermions. In spite of numerous studies along special magnetic-field orientations, there is a limited amount of knowledge on the complete field-angle characterization, which can provide valuable information on the existence of fractionalized excitations. For this purpose, we first extend previous studies on the field-angle response of the ferromagnetic Kitaev model to its antiferromagnetic version. Yet, the realistic description of the candidate Kitaev materials, within the edge-sharing octahedra paradigm, require additional coupling terms, including a large off-diagonal term Γ along with possible anisotropic corrections Γp. It is therefore not sufficient to depend on the topological properties of the bare Kitaev model as the only source for the observed thermal Hall conductivity signals and an understanding of these extended Kitaev models with a complete field response is demanded. Starting from the zero-field phase diagram of realistic K-Γ-Γp models, we identify antiferromagnetic zig-zag and (partially) polarized phases as well as two unusual Kitaev(-Γ) spin-liquid phases. The magnetic field response of these phases for arbitrary field orientations provides a remarkably rich phase diagram. A partially polarized phase is revealed between two ordered phases with a suppressed magnetization, finite fractionalization and finite Chern number. This phase is characterized as an extended Kitaev-Γ spin-liquid. To comply our findings with experiments, we reproduce the asymmetry in the extent of the intermediate phases specifically for the two different field directions θ = ±60.