Miracle-dz/UniFAR-FTUs huggingface dataset:
CoMave: Contrastive Pre-training with Multi-scale Masking for Attribute Value Extraction

Attribute Value Extraction (AVE) aims to automatically obtain attribute value pairs from product descriptions to aid e-commerce. Despite the progressive performance of existing approaches in e-commerce platforms, they still suffer from two challenges: 1) difficulty in identifying values at different scales simultaneously; 2) easy confusion by some highly similar fine-grained attributes. This paper proposes a pre-training technique for AVE to address these issues. In particular, we first improve the conventional token-level masking strategy, guiding the language model to understand multi-scale values by recovering spans at the phrase and sentence level. Second, we apply clustering to build a challenging negative set for each example and design a pre-training objective based on contrastive learning to force the model to discriminate similar attributes. Comprehensive experiments show that our solution provides a significant improvement over traditional pre-trained models in the AVE task, and achieves state-of-the-art on four benchmarks 1 .
Topic and Style-aware Transformer for Multimodal Emotion Recognition

Understanding emotion expressions in multi-modal signals is key for machines to have a better understanding of human communication. While language, visual and acoustic modalities can provide clues from different perspectives, the visual modality is shown to make minimal contribution to the performance in the emotion recognition field due to its high dimensionality. Therefore, we first leverage the strong multi-modality backbone VATT to project the visual signal to the common space with language and acoustic signals. Also, we propose content-oriented features Topic and Speaking style on top of it to approach the subjectivity issues. Experiments conducted on the benchmark dataset MOSEI show our model can outperform SOTA results and effectively incorporate visual signals and handle subjectivity issues by serving as content "normalization".
Connectivity Patterns are Task Embeddings

Task embeddings are task-specific vectors designed to construct a semantic space of tasks, which can be used to predict the most transferable source task for a given target task via the similarity between task embeddings. However, existing methods use optimized parameters and representations as task embeddings, resulting in substantial computational complexity and storage requirements. In this work, we draw inspiration from the operating mechanism of deep neural networks (DNNs) and biological brains, where neuronal activations are sparse and task-specific, and we use the connectivity patterns of neurons as a unique identifier associated with the task. The proposed method learns to assign importance masks for sub-structures of DNNs, and accordingly indicate the task-specific connectivity patterns. In addition to the storage advantages brought by the binary masking mechanism and structured sparsity, the early-bird nature of the sparse optimization process can deliver an efficient computation advantage. Experiments show that our method consistently outperforms other baselines in predicting inter-task transferability across data regimes and transfer settings, while keeping high efficiency in computation and storage.
Comment on “Dose Mapping Using Multipole Moments”

In the present work, we present a brief comment concerning the paper published by Loussaief et al. 2007 (Nucl. Inst. and Meth in Phys. Resh. A 580. 2007). The reference has used the multipole expansion method to determine the photon flux and the dose distribution in the vicinity of the Tunisian gamma irradiation facility. We have shown that the multipole expansion is useless and wrong for the calculation of the photon flux and the dose distribution in the vicinity of the CNSTN irradiator. We have also determined the dose distribution in the vicinity of the same irradiator, using a straightforward calculation, in order to show that we can abandon the multipole expansion method I our case.
ScaleDB: A Scalable, Asynchronous In-Memory Database

ScaleDB is a serializable in-memory transactional database that achieves excellent scalability on multi-core machines by asynchronously updating range indexes. We find that asynchronous range index updates can significantly improve database scalability by applying updates in batches, reducing contention on critical sections. To avoid stale reads, ScaleDB uses small hash indexlets to hold delayed updates. We use in-dexlets to design ACC, an asynchronous concurrency control protocol providing serializability. With ACC, it is possible to delay range index updates without adverse performance effects on transaction execution in the common case. ACC delivers scalable serializable isolation for transactions, with high throughput and low abort rate. Evaluation on a dual-socket server with 36 cores shows that ScaleDB achieves 9.5 × better query throughput than Peloton on the YCSB benchmark and 1.8 × better transaction throughput than Cicada on the TPC-C benchmark.
Masked Latent Semantic Modeling: an Efficient Pre-training Alternative to Masked Language Modeling

In this paper, we propose an alternative to the classic masked language modeling (MLM) pre-training paradigm, where we modify the objective from the reconstruction of the exact identity of randomly selected masked sub-words to the prediction of their latent semantic properties. We coin the proposed pre-training technique masked latent semantic modeling (MLSM for short). In order to make the contextualized determination of the latent semantic properties of the masked subwords possible, we rely on an unsupervised technique using sparse coding. Our experimental results reveal that the fine-tuned performance of those models that we pre-trained via MLSM is consistently and significantly better compared to the use of vanilla MLM pre-training and other strong baselines.
Contrastive Token-Wise Meta-Learning for Unseen Performer Visual Temporal-Aligned Translation

Visual temporal-aligned translation aims to transform the visual sequence into natural words, including important applicable tasks such as lipreading and fingerspelling recognition. However, various performance habits of specific words by different speakers or signers can lead to visual ambiguity, which has become a major obstacle to the development of current methods. Considering the constraints above, the generalization ability of the translation system is supposed to be further explored through the evaluation results on unseen per-formers. In this paper, we develop a novel generalizable framework named C ontrastive To ken-Wise M eta-l earning (CtoML), which strives to transfer recognition skills to unseen performers. To the best of our knowledge, employing meta-learning methods directly in the image domain poses two main challenges, and we propose corresponding strategies. First, sequence prediction in visual temporal-aligned translation, which aims to generate multiple words autoregressively, is different from the vanilla classification. Thus, we devise the token-wise diversity-aware weights for the meta-train stage, which encourages the model to make efforts on those ambiguously recognized tokens. Second, considering the consistency of word-visual prototypes across different domains, we develop two complementary global and local contrastive losses to maintain inter-class relationships and promote domain-independence. We conduct extensive experiments on the widely-used lipreading dataset GRID and the fingerspelling dataset ChicagoF-SWild, and the experimental results show the effectiveness of our proposed CtoML over existing state-of-the-art methods.
Sentiment Knowledge Enhanced Self-supervised Learning for Multimodal Sentiment Analysis

Multimodal Sentiment Analysis (MSA) has made great progress that benefits from extraordinary fusion scheme. However, there is a lack of labeled data, resulting in severe over-fitting and poor generalization for supervised models applied in this field. In this paper, we propose Sentiment Knowledge Enhanced Self-supervised Learning (SKESL) to capture common sentimental patterns in unlabeled videos, which facilitates further learning on limited labeled data. Specifically, with the help of sentiment knowledge and non-verbal behavior, SKESL conducts sentiment word masking and predicts fine-grained word sentiment intensity, so as to embed sentiment information at the word level into pre-trained multimodal representation. In addition, a non-verbal injection method is also proposed to integrate non-verbal information into the word semantics. Experiments on two standard benchmarks of MSA clearly show that SKESL significantly outperforms the baseline, and achieves new State-Of-The-Art (SOTA) results.
Beware of Fragmentation: Scheduling GPU-Sharing Workloads with Fragmentation Gradient Descent

Large tech companies are piling up a massive number of GPUs in their server fleets to run diverse machine learning (ML) workloads. However, these expensive devices often suffer from significant underutilization. To tackle this issue, GPU sharing techniques have been developed to enable multiple ML tasks to run on a single GPU. Nevertheless, our analysis of Alibaba production traces reveals that allocating partial GPUs can result in severe GPU fragmentation in large clusters, leaving hundreds of GPUs unable to be allocated. Existing resource packing algorithms fall short in addressing this problem, as GPU sharing mandates a new scheduling formulation beyond the classic bin packing. In this paper, we propose a novel measure of fragmentation to statistically quantify the extent of GPU fragmentation caused by different sources. Building upon this measure, we propose to schedule GPU-sharing workloads towards the direction of the steepest descent of fragmentation , an approach we call Fragmentation Gradient Descent (FGD). Intuitively, FGD packs tasks to minimize the growth of GPU fragmentation, thereby achieving the maximum GPU allocation rate. We have implemented FGD as a new scheduler in Kubernetes and evaluated its performance using production traces on an emulated cluster comprising more than 6,200 GPUs. Compared to the existing packing-based schedulers, FGD reduces unallocated GPUs by up to 49%, resulting in the utilization of additional 290 GPUs.
Disagreement Matters: Preserving Label Diversity by Jointly Modeling Item and Annotator Label Distributions with DisCo

This paper contains content that can be offensive or disturbing. Annotator disagreement is common whenever human judgment is needed for supervised learning. It is conventional to assume that one label per item represents ground truth. However, this obscures minority opinions, if present. We regard “ground truth” as the distribution of all labels that a population of annotators could produce, if asked (and of which we only have a small sample). We next introduce DisCo (Distribution from Context), a simple neural model that learns to predict this distribution. The model takes annotator-item pairs, rather than items alone, as input, and performs inference by aggregating over all annotators. Despite its simplicity, our experiments show that, on six benchmark datasets, our model is competitive with, and frequently outperforms, other, more complex models that either do not model specific annotators or were not designed for label distribution learning.
Early Exit with Disentangled Representation and Equiangular Tight Frame

Dynamic early exit has demonstrated great potential in coping with the sharply increasing number of pre-trained language model parameters, which can achieve a good trade-off be-tween performance and efficiency. The existing early exit paradigm relies on training paramet-rical internal classifiers at each intermediate layer to complete specific tasks. Based on the predictions of these internal classifiers, different methods are designed to decide when to exit. Under this circumstance, each intermediate layer takes on both generic language representation learning and task-specific feature extraction, which makes each intermediate layer struggle to balance two types of backward loss signals during training. To break this dilemma, we propose an adapter method to decouple the two distinct types of representation and further introduce a non-parametric simplex equiangu-lar tight frame classifier (ETF) for improvement. Extensive experiments on monolingual and multilingual tasks demonstrate that our method gains significant improvements over strong PLM backbones and early exit methods.
Lexical Translation Inconsistency-Aware Document-Level Translation Repair

Following the idea of “one translation per dis-course”, in this paper we aim to improve translation consistency via document-level translation repair (DocRepair), i.e., automatic post-editing on translations of documents. To this end, we propose a lexical translation inconsistency-aware DocRepair to explicitly model translation inconsistency. First we locate the inconsistency in automatic translation. Then we properly provide translation candidates for those inconsistency. Finally, we pro-pose lattice-like input to properly model inconsistent phrases and their candidates. Experimental results on three document-level translation datasets show that based on G-Transformer, a state-of-the-art document-to-document (Doc2Doc) translation model, our Doc2Doc DocRepair not only achieves improvement in translation quality in BLEU scores, but also greatly improves lexical translation consistency.
Sentiment Analysis using the Relationship between Users and Products

In product reviews, user and product aspects are useful in sentiment analysis. Neverthe-less, previous studies mainly focus on modeling user and product aspects without considering the relationship between users and products. The relationship between users and products is typically helpful in estimating the bias of a user toward a product. In this paper, we, therefore, introduce the Graph Neural Network-based model with the pre-trained Language Model (GNNLM), where the relationship between users and products is incorporated. We conducted experiments on three well-known benchmarks for sentiment classification with the user and product information. The experimental results show that the relationship between users and products improves the performance of sentiment analysis. Furthermore, GNNLM achieves state-of-the-art results on yelp-2013 and yelp-2014 datasets.
MicroRNA-375 is induced during astrocyte-to-neuron reprogramming and promotes survival of reprogrammed neurons when overexpressed

While astrocyte-to-neuron (AtN) reprogramming holds great promise in regenerative medicine, the molecular mechanisms that govern this unique biological process remain elusive. MicroRNAs (miRNAs), as post-transcriptional regulators of gene expression, play crucial roles during development and under various pathological conditions. To understand the function of miRNAs during AtN reprogramming process, we performed RNA-seq of both mRNAs and miRNAs on human astrocyte (HA) cultures upon NeuroD1 overexpression. Bioinformatics analyses showed that NeuroD1 not only activates essential neuronal genes to initiate reprogramming process but also induces miRNA changes in HA. Among the upregulated miRNAs, we identified miR-375 and its targets, neuronal ELAVL genes (nELAVLs), which encode a family of RNA-binding proteins and are also upregulated by NeuroD1. We further showed that manipulating miR-375 level regulates nELAVLs expression during NeuroD1-mediated reprogramming. Interestingly, miR-375/nELAVLs are also induced by reprogramming factors Neurog2 and ASCL1 in HA suggesting a conserved function to neuronal reprogramming, and by NeuroD1 in the mouse astrocyte culture and spinal cord. Functionally, we showed that miR-375 overexpression improves NeuroD1-mediated reprogramming efficiency by promoting cell survival at early stages in HA even in cultures treated with the chemotherapy drug Cisplatin. Moreover, miR-375 overexpression doesn’t appear to compromise maturation of the reprogrammed neurons in long term HA cultures. Lastly, overexpression of miR-375-refractory ELAVL4 induces apoptosis and reverses the cell survival-promoting effect of miR-375 during AtN reprogramming. Together, we demonstrate a neuro-protective role of miR-375 during NeuroD1-mediated AtN reprogramming and suggest a strategy of combinatory overexpression of NeuroD1 and miR-375 for improving neuronal reprogramming efficiency.
A test for equality of two distributions via integrating characteristic functions

In this paper, we investigate the problem of testing the equality of two distributions by integrating the squared norm of the difference between two corresponding empirical characteristic functions. This results in a linear combination of three different U-statistics. So the original testing problem is reduced to testing whether this linear combination is zero or not. We apply the Jackknife Empirical Likelihood (JEL) method to the new hypothesis testing problem. It has been proved that, under multivariate case, the log JEL statistic after scaling tends to a chi-square distribution with degree of freedom 1. Simulation studies are presented to assess the finite-sample performance of our method.
IDENTITY, MEMORY OF THE CIVIL WAR, AND SUPPORT FOR REPARATIONS IN GUATEMALA

The mutually constitutive relationship between constructions of social identity and collective memory is mediated by cultural tools. Across four studies, I explored the relationship between ethnic and national identity and engagement with the final report of the Commission for Historical Clarification as a cultural tool for the memory of the Civil War in Guatemala. I also investigated the implications of social identification and engagement with the report for support of social justice policies aimed at repairing harm associated with the war. In Study 1, identification with the Indigenous social category and a tendency towards exploration of national identity were associated with positive attitudes towards the CEH report and with greater support for reparative policy. In Study 2, identification with the Indigenous category—particularly a sense of solidarity and commitment towards Indigenous identity—was again related to more positive report attitudes and support for reparations. In the final two studies, I measured (Study 3) and manipulated (Study 4) a sense of shared heritage with Indigenous Peoples. Such sense of shared heritage was associated with increased Indigenous identification, which in turn predicted positive attitudes towards the report as well as increased support for reparations. These findings have implications for our understanding of social identities and their relationship with collective memory. I discuss these implications for the Guatemalan context with reference to local struggles for transitional justice.
different approaches to quantifying land surface temperature impacts of afforestation using satellite observations

This study conducted an interesting research about three influential approaches in evaluating the climatic effects induced by afforestation over China. So far, no such studies have ever compared the three methods simultaneously and investigated the underlying mechanisms that lead to their discrepancies and more importantly, whether the discrepancies can be mitigated or reconciled. I'm happy to see that the authors filled this knowledge gap and gave us a good reference. As far as I know, in previous studies involving both the actual and potential effects (Li Yan, 2016, JGR-A, Shen Wenjuan, 2019, AFM), the two effects, characterized by LST changes (or cooling) were comparable and consistent in magnitude. As a result, their discrepancies attracted less attention. Fortunately, this research emphasized this point by applying the afforestation experiment over China. Coincidentally, I have a pending research (in prepare for subscription) in support of the result (actual effect is largely less than potential effect) in this study.
The SPARC water vapour assessment II: biases and drifts of water vapour satellite data records with respect to frost point hygrometer records

Abstract. Satellite data records of stratospheric water vapour have been compared to balloon-borne frost point hygrometer (FP) profiles that are coincident in space and time. The satellite data records of 15 different instruments cover water vapour data available from January 2000 through December 2016. The hygrometer data are from 27 stations all over the world in the same period. For the comparison, real or constructed averaging kernels have been applied to the hygrometer profiles to adjust them to the measurement characteristics of the satellite instruments. For bias evaluation, we have compared satellite profiles averaged over the available temporal coverage to the means of coincident FP profiles for individual stations. For drift determinations, we analysed time series of relative differences between spatiotemporally coincident satellite and hygrometer profiles at individual stations. In a synopsis we have also calculated the mean biases and drifts (and their respective uncertainties) for each satellite record over all applicable hygrometer stations in three altitude ranges (10–30 hPa, 30–100 hPa, and 100 hPa to tropopause). Most of the satellite data have biases <10 % and average drifts <1 % yr−1 in at least one of the respective altitude ranges. Virtually all biases are significant in the sense that their uncertainty range in terms of twice the standard error of the mean does not include zero. Statistically significant drifts (95 % confidence) are detected for 35 % of the ≈ 1200 time series of relative differences between satellites and hygrometers.
Towards Comprehensive Veriﬁcation of Hardware and Software for RISC-V based Embedded Systems *

In this extended abstract, we present several approaches for the veriﬁcation of embedded systems. First, we present four cross-level approaches for processor veriﬁcation at the Register-Transfer Level (RTL) using a cross-level setting with an Instruction Set Simulator (ISS) as a reference model. Second, we present a comprehensive veriﬁcation approach for verifying embedded software with symbolic execution
Explaining the long reach of prenatal behaviors and attitudes in unmarried men at birth on father engagement in early and middle childhood and adolescence.

The current study examined three research questions: (a) Are unmarried at birth fathers' prenatal and birth-related behavioral, attitudinal, and identity adjustments directly related to father engagement in child-related activities during early childhood and father-child closeness in middle childhood and adolescence? (b) Do father engagement in child-related activities during early childhood, coresidence, and coparenting at age 5 mediate the association between unmarried fathers' prenatal and birth-related variables and father-child relationship in middle childhood and adolescence? (c) Do father-child closeness, coresidence, and coparenting in middle childhood mediate the association between fathers' prenatal and birth-related variables and father-child relationship during adolescence? Using a subsample of Fragile Families and Child Wellbeing data (N = 2,647), we found support for our hypotheses that fathers' prenatal and birth-related variables significantly predicted father-child engagement during early childhood and father-child closeness during middle childhood and adolescence, although not all prenatal and birth-related variables are related to outcomes during each stage of childhood development. Father involvement and coparenting cooperation significantly mediated the associations among fathers' prenatal and birth-related variables and father-child closeness at ages 9 and 15. Our findings indicate that researchers, practitioners, and policymakers should take advantage of the prenatal period and direct resources to facilitate and strengthen prospective unmarried fathers' early relationships with their partners and children. (PsycInfo Database Record (c) 2022 APA, all rights reserved).