Miracle-dz/UniFAR-FTUs huggingface dataset:
Parameter-efficient Zero-shot Transfer for Cross-Language Dense Retrieval with Adapters

A popular approach to creating a zero-shot cross-language retrieval model is to substitute a monolingual pretrained language model in the retrieval model with a multilingual pretrained language model such as Multilingual BERT. This multilingual model is fined-tuned to the retrieval task with monolingual data such as English MS MARCO using the same training recipe as the monolingual retrieval model used. However, such transferred models suffer from mismatches in the languages of the input text during training and inference. In this work, we propose transferring monolingual retrieval models using adapters, a parameter-efficient component for a transformer network. By adding adapters pretrained on language tasks for a specific language with task-specific adapters, prior work has shown that the adapter-enhanced models perform better than fine-tuning the entire model when transferring across languages in various NLP tasks. By constructing dense retrieval models with adapters, we show that models trained with monolingual data are more effective than fine-tuning the entire model when transferring to a Cross Language Information Retrieval (CLIR) setting. However, we found that the prior suggestion of replacing the language adapters to match the target language at inference time is suboptimal for dense retrieval models. We provide an in-depth analysis of this discrepancy between other cross-language NLP tasks and CLIR.
Self-Pair: Synthesizing Changes from Single Source for Object Change Detection in Remote Sensing Imagery

For change detection in remote sensing, constructing a training dataset for deep learning models is difficult due to the requirements of bi-temporal supervision. To overcome this issue, single-temporal supervision which treats change labels as the difference of two semantic masks has been proposed. This novel method trains a change detector using two spatially unrelated images with corresponding semantic labels such as building. However, training on unpaired datasets could confuse the change detector in the case of pixels that are labeled unchanged but are visually significantly different. In order to maintain the visual similarity in unchanged area, in this paper, we emphasize that the change originates from the source image and show that manipulating the source image as an after-image is crucial to the performance of change detection. Extensive experiments demonstrate the importance of maintaining visual information between pre- and post-event images, and our method outperforms existing methods based on single-temporal supervision.
DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization

Dialogue summarization has recently garnered significant attention due to its wide range of applications. However, existing methods for summarizing dialogues have limitations because they do not take into account the inherent structure of dialogue and rely heavily on labeled data, which can lead to poor performance in new domains. In this work, we propose DIONYSUS (dynamic input optimization in pre-training for dialogue summarization), a pre-trained encoder-decoder model for summarizing dialogues in any new domain. To pre-train DIONYSUS, we create two pseudo summaries for each dialogue example: one from a fine-tuned summarization model and the other from important dialogue turns. We then choose one of these pseudo summaries based on information distribution differences in different types of dialogues. This selected pseudo summary serves as the objective for pre-training DIONYSUS using a self-supervised approach on a large dialogue corpus. Our experiments show that DIONYSUS outperforms existing methods on six datasets, as demonstrated by its ROUGE scores in zero-shot and few-shot settings
Human-in-the-loop Abstractive Dialogue Summarization

Abstractive dialogue summarization has received increasing attention recently. Despite the fact that most of the current dialogue summarization systems are trained to maximize the likelihood of human-written summaries and have achieved significant results, there is still a huge gap in generating high-quality summaries as determined by humans, such as coherence and faithfulness, partly due to the misalignment in maximizing a single human-written summary. To this end, we propose to incorporate different levels of human feedback into the training process. This will enable us to guide the models to capture the behaviors humans care about for summaries. Specifically, we ask humans to highlight the salient information to be included in summaries to provide the local feedback , and to make overall comparisons among summaries in terms of coherence, accuracy, coverage, concise and overall quality, as the global feedback. We then combine both local and global feedback to fine-tune the dialog summarization policy with Reinforcement Learning. Experiments conducted on multiple datasets demonstrate the effectiveness and generalization of our methods over the state-of-the-art supervised baselines, especially in terms of human judgments.
Towards Sequence Utility Maximization under Utility Occupancy Measure

The discovery of utility-driven patterns is a useful and difficult research topic. It can extract significant and interesting information from specific and varied databases, increasing the value of the services provided. In practice, the measure of utility is often used to demonstrate the importance, profit, or risk of an object or a pattern. In the database, although utility is a flexible criterion for each pattern, it is a more absolute criterion due to the neglect of utility sharing. This leads to the derived patterns only exploring partial and local knowledge from a database. Utility occupancy is a recently proposed model that considers the problem of mining with high utility but low occupancy. However, existing studies are concentrated on itemsets that do not reveal the temporal relationship of object occurrences. Therefore, this paper towards sequence utility maximization. We first define utility occupancy on sequence data and raise the problem of High Utility-Occupancy Sequential Pattern Mining (HUOSPM). Three dimensions, including frequency, utility, and occupancy, are comprehensively evaluated in HUOSPM. An algorithm called Sequence Utility Maximization with Utility occupancy measure (SUMU) is proposed. Furthermore, two data structures for storing related information about a pattern, Utility-Occupancy-List-Chain (UOL-Chain) and Utility-Occupancy-Table (UO-Table) with six associated upper bounds, are designed to improve efficiency. Empirical experiments are carried out to evaluate the novel algorithm’s efficiency and effectiveness. The influence of different upper bounds and pruning strategies is analyzed and discussed. The comprehensive results suggest that the work of our algorithm is intelligent and effective.
Unleashing the Power of Visual Prompting At the Pixel Level

This paper presents a simple and effective visual prompting method for adapting pre-trained models to downstream recognition tasks. Our method includes two key designs. First, rather than directly adding together the prompt and the image, we treat the prompt as an extra and independent learnable component. We show that the strategy of reconciling the prompt and the image matters, and find that warping the prompt around a properly shrinked image empirically works the best. Second, we re-introduce two"old tricks"commonly used in building transferable adversarial examples, i.e., input diversity and gradient normalization, into visual prompting. These techniques improve optimization and enable the prompt to generalize better. We provide extensive experimental results to demonstrate the effectiveness of our method. Using a CLIP model, our prompting method sets a new record of 82.8% average accuracy across 12 popular classification datasets, substantially surpassing the prior art by +5.6%. It is worth noting that this prompting performance already outperforms linear probing by +2.1% and can even match fully fine-tuning in certain datasets. In addition, our prompting method shows competitive performance across different data scales and against distribution shifts. The code is publicly available at https://github.com/UCSC-VLAA/EVP.
SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers

Diffusion model, a new generative modelling paradigm, has achieved great success in image, audio, and video generation. However, considering the discrete categorical nature of text, it is not trivial to extend continuous diffusion models to natural language, and text diffusion models are less studied. Sequence-to-sequence text generation is one of the essential natural language processing topics. In this work, we apply diffusion models to approach sequence-to-sequence text generation, and explore whether the superiority generation performance of diffusion model can transfer to natural language domain. We propose SeqDiffuSeq, a text diffusion model for sequence-to-sequence generation. SeqDiffuSeq uses an encoder-decoder Transformers architecture to model denoising function. In order to improve generation quality, SeqDiffuSeq combines the self-conditioning technique and a newly proposed adaptive noise schedule technique. The adaptive noise schedule has the difficulty of denoising evenly distributed across time steps, and considers exclusive noise schedules for tokens at different positional order. Experiment results illustrate the good performance on sequence-to-sequence generation in terms of text quality and inference time.
On-orbit operation and performance of MODIS blackbody

MODIS collects data in 36 spectral bands, including 20 reflective solar bands (RSB) and 16 thermal emissive bands (TEB). The TEB on-orbit calibration is performed on a scan-by-scan basis using a quadratic algorithm that relates the detector response with the calibration radiance from the sensor on-board blackbody (BB). The calibration radiance is accurately determined each scan from the BB temperature measured using a set of 12 thermistors. The BB thermistors were calibrated pre-launch with traceability to the NIST temperature standard. Unlike many heritage sensors, the MODIS BB can be operated at a constant temperature or with the temperature continuously varying between instrument ambient (about 270K) and 315K. In this paper, we provide an overview of both Terra and Aqua MODIS on-board BB operations, functions, and on-orbit performance. We also examine the impact of key calibration parameters, such as BB emissivity and temperature (stability and gradient) determined from its thermistors, on the TEB calibration and Level 1 (L1B) data product uncertainty.
Optimizing Serially Concatenated Neural Codes with Classical Decoders

For improving short-length codes, we demonstrate that classic decoders can also be used with real-valued, neural encoders, i.e., deep-learning based codeword sequence generators. Here, the classical decoder can be a valuable tool to gain insights into these neural codes and shed light on weaknesses. Specifically, the turbo-autoencoder is a recently developed channel coding scheme where both encoder and decoder are replaced by neural networks. We first show that the limited receptive field of convolutional neural network (CNN)-based codes enables the application of the BCJR algorithm to optimally decode them with feasible computational complexity. These maximum a posteriori (MAP) component decoders then are used to form classical (iterative) turbo decoders for parallel or serially concatenated CNN encoders, offering a close-to-maximum likelihood (ML) decoding of the learned codes. To the best of our knowledge, this is the first time that a classical decoding algorithm is applied to a non-trivial, real-valued neural code. Furthermore, as the BCJR algorithm is fully differentiable, it is possible to train, or fine-tune, the neural encoder in an end-to-end fashion.
Image quality prediction using synthetic and natural codebooks: comparative results

We investigate a model for image/video quality assessment based on building a set of codevectors representing in a sense some basic properties of images, similar to well-known CORNIA model. We analyze the codebook building method and propose some modifications for it. Also the algorithm is investigated from the point of inference time reduction. Both natural and synthetic images are used for building codebooks and some analysis of synthetic images used for codebooks is provided. It is demonstrated the results on quality assessment may be improves with the use if synthetic images for codebook construction. We also demonstrate regimes of the algorithm in which real time execution on CPU is possible for sufficiently high correlations with mean opinion score (MOS). Various pooling strategies are considered as well as the problem of metric sensitivity to bitrate.
Comparing the Locations of Supernovae to CO (2–1) Emission in Their Host Galaxies

We measure the molecular gas environment near recent (<100 yr old) supernovae (SNe) using ∼1″ or ≤150 pc resolution CO (2–1) maps from the PHANGS–Atacama Large Millimeter/submillimeter Array (ALMA) survey of nearby star-forming galaxies. This is arguably the first such study to approach the scales of individual massive molecular clouds (M mol ≳ 105.3 M ⊙). Using the Open Supernova Catalog, we identify 63 SNe within the PHANGS–ALMA footprint. We detect CO (2–1) emission near ∼60% of the sample at 150 pc resolution, compared to ∼35% of map pixels with CO (2–1) emission, and up to ∼95% of the SNe at 1 kpc resolution, compared to ∼80% of map pixels with CO (2–1) emission. We expect the ∼60% of SNe within the same 150 pc beam, as a giant molecular cloud will likely interact with these clouds in the future, consistent with the observation of widespread SN–molecular gas interaction in the Milky Way, while the other ∼40% of SNe without strong CO (2–1) detections will deposit their energy in the diffuse interstellar medium, perhaps helping drive large-scale turbulence or galactic outflows. Broken down by type, we detect CO (2–1) emission at the sites of ∼85% of our 9 stripped-envelope SNe (SESNe), ∼40% of our 34 Type II SNe, and ∼35% of our 13 Type Ia SNe, indicating that SESNe are most closely associated with the brightest CO (2–1) emitting regions in our sample. Our results confirm that SN explosions are not restricted to only the densest gas, and instead exert feedback across a wide range of molecular gas densities.
Numerical simulation of viscous fingering in porous media

The formation and growth of three-dimensional branching patterns that occur when a less viscous fluid penetrates into a porous media in more viscous surroundings are studied numerically on the basis of a simplified filtration model that corresponds to the zero dispersion and high capillary number limit. The balance equation for the saturation of a porous medium with a less viscous fluid is approximated on the basis of the high-order accurate weighted non-oscillatory (WENO) scheme. Tip-splitting is found to be the dominant way of pattern formation in the numerical solutions. A qualitative illustration of the influence of the numerical scheme is presented.
Matrix recovery from matrix-vector products

Can one recover a matrix eﬃciently from only matrix-vector products? If so, how many are needed? This paper describes algorithms to recover structured matrices, such as tridiagonal, Toeplitz, Toeplitz-like, and hierarchical low-rank, from matrix-vector products. In particular, we derive a randomized algorithm for recovering an N × N unknown hierarchical low-rank matrix from only O (( k + p )log( N )) matrix-vector products with high probability, where k is the rank of the oﬀ-diagonal blocks, and p is a small oversampling parameter. We do this by carefully constructing randomized input vectors for our matrix-vector products that exploit the hierarchical structure of the matrix. While existing algorithms for hierarchical matrix recovery use a recursive “peeling” procedure based on elimination, our approach uses a recursive projection procedure.
High-resolution canopy height map in the Landes forest (France) based on GEDI, Sentinel-1, and Sentinel-2 data with a deep learning approach

In intensively managed forests in Europe, where forests are divided into stands of small size and may show heterogeneity within stands, a high spatial resolution (10 - 20 meters) is arguably needed to capture the differences in canopy height. In this work, we developed a deep learning model based on multi-stream remote sensing measurements to create a high-resolution canopy height map over the"Landes de Gascogne"forest in France, a large maritime pine plantation of 13,000 km$^2$ with flat terrain and intensive management. This area is characterized by even-aged and mono-specific stands, of a typical length of a few hundred meters, harvested every 35 to 50 years. Our deep learning U-Net model uses multi-band images from Sentinel-1 and Sentinel-2 with composite time averages as input to predict tree height derived from GEDI waveforms. The evaluation is performed with external validation data from forest inventory plots and a stereo 3D reconstruction model based on Skysat imagery available at specific locations. We trained seven different U-net models based on a combination of Sentinel-1 and Sentinel-2 bands to evaluate the importance of each instrument in the dominant height retrieval. The model outputs allow us to generate a 10 m resolution canopy height map of the whole"Landes de Gascogne"forest area for 2020 with a mean absolute error of 2.02 m on the Test dataset. The best predictions were obtained using all available satellite layers from Sentinel-1 and Sentinel-2 but using only one satellite source also provided good predictions. For all validation datasets in coniferous forests, our model showed better metrics than previous canopy height models available in the same region.
Bridging necking and shear-banding mediated tensile failure in glasses

The transition between necking-mediated tensile failure of glasses, at elevated temperatures and/or low strain-rates, and shear-banding-mediated tensile failure, at low temperatures and/or high strain-rates, is investigated using tensile experiments on metallic glasses and atomistic simulations. We experimentally and simulationally show that this transition occurs through a sequence of macroscopic failure patterns, parametrized by the ultimate tensile strength. Quantitatively analyzing the spatiotemporal dynamics preceding failure, using large scale atomistic simulations corroborated by experimental fractography, reveals how the collective evolution and mutual interaction of shear-driven plasticity and dilation-driven void formation (cavitation) control the various macroscopic failure modes. In particular, we find that at global failure, the size of the largest cavity in the loading direction exhibits a nonmonotonic dependence on the temperature at a fixed strain rate, which is rationalized in terms of the interplay between shear- and dilation-driven plasticity. We also find that the size of the largest cavity scales with the cross-sectional area of the undeformed sample. These results shed light on tensile failure of glasses, and highlight the need to develop elasto-plastic constitutive models of glasses incorporating both shear- and dilation-driven irreversible processes.
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator

Pre-trained models have achieved remarkable success in natural language processing (NLP). However, existing pre-training methods underutilize the benefits of language understanding for generation. Inspired by the idea of Generative Adversarial Networks (GANs), we propose a GAN-style model for encoder-decoder pre-training by introducing an auxiliary discriminator, unifying the ability of language understanding and generation in a single model. Our model, named as GanLM, is trained with two pre-training objectives: replaced token detection and replaced token denoising. Specifically, given masked source sentences, the generator outputs the target distribution and the discriminator predicts whether the target sampled tokens from distribution are incorrect. The target sentence is replaced with misclassified tokens to construct noisy previous context, which is used to generate the gold sentence. In general, both tasks improve the ability of language understanding and generation by selectively using the denoising data. Extensive experiments in language generation benchmarks show that GanLM with the powerful language understanding capability outperforms various strong pre-trained language models (PLMs) and achieves state-of-the-art performance.
A Physically-Consistent Chemical Dataset for the Simulation of N$_2$-CH$_4$ Shocked Flows Up to T=100,000K

In the previous work carried out in the scope of the \emph{Validation of Aerothermochemistry Models for Re-Entry Applications}, it was verified that the G\"{o}k\c{c}en chemical dataset provided increasingly diverging results from experiments, as one considered shock speeds in excess of 5\kilo\metre\per\second. Namely, for shock velocities between 7 and 9\kilo\metre\per\second, more than one temporal peak in CN Violet radiation were predicted by models considering this kinetic dataset, in contradiction with experiments. This hinted at several of the rates from the dataset not being directly applicable in the temperature range of interest for such applications, often in excess of 10,000\kelvin. Indeed, it has been found that several macroscopic rates from the G\"{o}k\c{c}en chemical dataset reached unphysical values at very high temperatures. Furthermore, many of the ionization rates have been found to be inadequate for the simulation of high-temperature N$_{2}$--CH$_{4}$ shocked flows. Here, we have carried an extensive update of the G\"{o}k\c{c}en chemical dataset, with the aim of at least reaching physically consistent rates for the whole T=100-100,000\kelvin\ temperature range. While it cannot really be claimed that such improved dataset is validated in such an extended temperature range (due to the scarcely available experimental data for such high temperature ranges), it is capable of providing more accurate simulations of high-speed shocked flows for this mixture, when compared to the G\"{o}k\c{c}en chemical dataset.
Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance

Complex and contact-rich robotic manipulation tasks, particularly those that involve multi-fingered hands and underactuated object manipulation, present a significant challenge to any control method. Methods based on reinforcement learning offer an appealing choice for such settings, as they can enable robots to learn to delicately balance contact forces and dexterously reposition objects without strong modeling assumptions. However, running reinforcement learning on real-world dexterous manipulation systems often requires significant manual engineering. This negates the benefits of autonomous data collection and ease of use that reinforcement learning should in principle provide. In this paper, we describe a system for vision-based dexterous manipulation that provides a “programming-free” approach for users to define new tasks and enable robots with complex multi-fingered hands to learn to perform them through interaction. The core principle under-lying our system is that, in a vision-based setting, users should be able to provide high-level intermediate supervision that circumvents challenges in teleoperation or kinesthetic teaching which allows a robot to not only learn a task efficiently but also to autonomously practice. Our system includes a framework for users to define a final task and intermediate sub-tasks with image examples, a reinforcement learning procedure that learns the task autonomously without interventions, and experimental results with a four-finger robotic hand learning multi-stage object manipulation tasks directly in the real world, without simulation, manual modeling, or reward engineering.