Reviewerly/exHarmony huggingface dataset:
KGTN: Knowledge Graph Transformer Network for explainable multi-category item recommendation

Most existing methods for recommendation assume that all the items are of the same category. However, single-category recommendations are no longer sufficient to fulfill the diverse needs of users in the real world. Existing methods cannot be directly applied to recommend multiple categories of items due to the different properties and complex relationships among them. However, a drawback of these approaches is their assumption of a fixed and homogeneous data format, making them incapable of addressing the challenge of recommending multiple categories of items. Recommendation of multiple categories of items to users concurrently has become a challenging task. To tackle this problem, we design a novel method Knowledge Graph Transformer Network (KGTN) for explainable multi-category item recommendation, inspired by advances in knowledge graph in the field of recommendation. Knowledge graph and neural network methods have shown advantages in addressing recommendation problems in heterogeneous graph structures. This is because different categories of items have unique attributes and dimensions, which can be effectively represented and integrated by incorporating knowledge graph and graph neural network techniques. That is, our approach can handle heterogeneous collaborative knowledge graphs composed of users and items, and can mine hidden path relationships without defining original paths. In addition, its algorithmic process has interpretability. Specifically, graph transformer layer converts the heterogeneous input graph into a useful meta-path graph for recommended task; graph convolution layer learns the node representation on the new meta-path graph. Finally, we calculate the inner product of user and item vector representations to output the probability for the recommendation. At the same time, we use the critical path in the learned useful meta-path graph as the explanation. Comprehensive experiments on three datasets demonstrate that KGTN achieves state-of-the-art performance over existing baselines.
B-CANF: Adaptive B-frame Coding with Conditional Augmented Normalizing Flows

Over the past few years, learning-based video compression has become an active research area. However, most works focus on P-frame coding. Learned B-frame coding is under-explored and more challenging. This work introduces a novel B-frame coding framework, termed B-CANF, that exploits conditional augmented normalizing flows for B-frame coding. B-CANF additionally features two novel elements: frame-type adaptive coding and B*-frames. Our frame-type adaptive coding learns better bit allocation for hierarchical B-frame coding by dynamically adapting the feature distributions according to the B-frame type. Our B*-frames allow greater flexibility in specifying the group-of-pictures (GOP) structure by reusing the B-frame codec to mimic P-frame coding, without the need for an additional, separate P-frame codec. On commonly used datasets, B-CANF achieves the state-of-the-art compression performance as compared to the other learned B-frame codecs and shows comparable BD-rate results to HM-16.23 under the random access configuration in terms of PSNR. When evaluated on different GOP structures, our B*-frames achieve similar performance to the additional use of a separate P-frame codec.
Unified Demonstration Retriever for In-Context Learning

In-context learning is a new learning paradigm where a language model conditions on a few input-output pairs (demonstrations) and a test input, and directly outputs the prediction.It has been shown highly dependent on the provided demonstrations and thus promotes the research of demonstration retrieval: given a test input, relevant examples are retrieved from the training set to serve as informative demonstrations for in-context learning.While previous works focus on training task-specific retrievers for several tasks separately, these methods are often hard to transfer and scale on various tasks, and separately trained retrievers incur a lot of parameter storage and deployment cost.In this paper, we propose Unified Demonstration Retriever (UDR), a single model to retrieve demonstrations for a wide range of tasks.To train UDR, we cast various tasks' training signals into a unified list-wise ranking formulation by language model's feedback.Then we propose a multi-task list-wise ranking training framework, with an iterative mining strategy to find high-quality candidates, which can help UDR fully incorporate various tasks' signals.Experiments on 30+ tasks across 13 task families and multiple data domains show that UDR significantly outperforms baselines.Further analyses show the effectiveness of each proposed component and UDR's strong ability in various scenarios including different LMs (1.3B ∼ 175B), unseen datasets, varying demonstration quantities, etc.
Automated evaluation of written discourse coherence using GPT-4

The popularization of large language models (LLMs) such as OpenAI's GPT-3 and GPT-4 have led to numerous innovations in the field of AI in education. With respect to automated writing evaluation (AWE), LLMs have reduced challenges associated with assessing writing quality characteristics that are difficult to identify automatically, such as discourse coherence. In addition, LLMs can provide rationales for their evaluations (ratings) which increases score interpretability and transparency. This paper investigates one approach to producing ratings by training GPT-4 to assess discourse coherence in a manner consistent with expert human raters. The findings of the study suggest that GPT-4 has strong potential to produce discourse coherence ratings that are comparable to human ratings, accompanied by clear rationales. Furthermore, the GPT-4 ratings outperform traditional NLP coherence metrics with respect to agreement with human ratings. These results have implications for advancing AWE technology for learning and assessment.
Salience Allocation as Guidance for Abstractive Summarization

Abstractive summarization models typically learn to capture the salient information from scratch implicitly.Recent literature adds extractive summaries as guidance for abstractive summarization models to provide hints of salient content and achieves better performance.However, extractive summaries as guidance could be over strict, leading to information loss or noisy signals.Furthermore, it cannot easily adapt to documents with various abstractiveness.As the number and allocation of salience content pieces vary, it is hard to find a fixed threshold deciding which content should be included in the guidance.In this paper, we propose a novel summarization approach with a flexible and reliable salience guidance, namely SEASON (SaliencE Allocation as Guidance for Abstractive SummarizatiON).SEASON utilizes the allocation of salience expectation to guide abstractive summarization and adapts well to articles in different abstractiveness.Automatic and human evaluations on two benchmark datasets show that the proposed method is effective and reliable.Empirical results on more than one million news articles demonstrate a natural fifteen-fifty salience split for news article sentences, providing a useful insight for composing news articles.
SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition

Few-shot Named Entity Recognition (NER) aims to identify named entities with very little annotated data.Previous methods solve this problem based on token-wise classification, which ignores the information of entity boundaries, and inevitably the performance is affected by the massive non-entity tokens.To this end, we propose a seminal span-based prototypical network (SpanProto) that tackles few-shot NER via a two-stage approach, including span extraction and mention classification.In the span extraction stage, we transform the sequential tags into a global boundary matrix, enabling the model to focus on the explicit boundary information.For mention classification, we leverage prototypical learning to capture the semantic representations for each labeled span and make the model better adapt to novel-class entities.To further improve the model performance, we split out the false positives generated by the span extractor but not labeled in the current episode set, and then present a margin-based loss to separate them from each prototype region.Experiments over multiple benchmarks demonstrate that our model outperforms strong baselines by a large margin.
CEFR-Based Sentence Difficulty Annotation and Assessment

Controllable text simplification is a crucial assistive technique for language learning and teaching.One of the primary factors hindering its advancement is the lack of a corpus annotated with sentence difficulty levels based on language ability descriptions.To address this problem, we created the CEFR-based Sentence Profile (CEFR-SP) corpus, containing 17k English sentences annotated with the levels based on the Common European Framework of Reference for Languages assigned by English-education professionals.In addition, we propose a sentence-level assessment model to handle unbalanced level distribution because the most basic and highly proficient sentences are naturally scarce.In the experiments in this study, our method achieved a macro-F1 score of 84.5% in the level assessment, thus outperforming strong baselines employed in readability assessment.
Prompting for Multimodal Hateful Meme Classification

Hateful meme classification is a challenging multimodal task that requires complex reasoning and contextual background knowledge.Ideally, we could leverage an explicit external knowledge base to supplement contextual and cultural information in hateful memes.However, there is no known explicit external knowledge base that could provide such hate speech contextual information.To address this gap, we propose PromptHate, a simple yet effective prompt-based model that prompts pre-trained language models (PLMs) for hateful meme classification.Specifically, we construct simple prompts and provide a few in-context examples to exploit the implicit knowledge in the pretrained RoBERTa language model for hateful meme classification.We conduct extensive experiments on two publicly available hateful and offensive meme datasets.Our experimental results show that PromptHate is able to achieve a high AUC of 90.96, outperforming state-ofthe-art baselines on the hateful meme classification task.We also perform fine-grained analyses and case studies on various prompt settings and demonstrate the effectiveness of the prompts on hateful meme classification.
Towards Unified Prompt Tuning for Few-shot Text Classification

Prompt-based fine-tuning has boosted the performance of Pre-trained Language Models (PLMs) on few-shot text classification by employing task-specific prompts.Yet, PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few-shot learning performance on downstream tasks.It would be desirable if the models can acquire some prompting knowledge before adapting to specific NLP tasks.We present the Unified Prompt Tuning (UPT) framework, leading to better few-shot text classification for BERTstyle models by explicitly capturing prompting semantics from non-target NLP datasets.In UPT, a novel paradigm Prompt-Options-Verbalizer is proposed for joint prompt learning across different NLP tasks, forcing PLMs to capture task-invariant prompting knowledge.We further design a self-supervised task named Knowledge-enhanced Selective Masked Language Modeling to improve the PLM's generalization abilities for accurate adaptation to previously unseen tasks.After multi-task learning across multiple tasks, the PLM can be better prompt-tuned towards any dissimilar target tasks in low-resourced settings.Experiments over a variety of NLP tasks show that UPT consistently outperforms state-of-the-arts for prompt-based fine-tuning.
Two-Fluxonium Cross-Resonance Gate

The superconducting fluxonium qubit has a great potential for high-fidelity quantum gates with its long coherence times and strong anharmonicity at the half-flux-quantum sweet spot. However, current implementations of two-qubit gates compromise fluxonium's coherence properties by requiring either a temporary population of the noncomputational states or tuning the magnetic flux off the sweet spot. Here we realize a fast all-microwave cross-resonance gate between two capacitively coupled fluxoniums with the qubit dynamics well confined to the computational space. We demonstrate a direct controlled not (cnot) gate in 70 ns with fidelity up to $\mathcal{F}=0.9949(6)$ despite the limitations of a suboptimal measurement setup and device coherence. Our results project a possible pathway towards reducing the two-qubit error rate near to or below ${10}^{\ensuremath{-}4}$ with present-day technologies.
Hybrid Transformer and CNN Attention Network for Stereo Image Super-resolution

Multi-stage strategies are frequently employed in image restoration tasks. While transformer-based methods have exhibited high efficiency in single-image super-resolution tasks, they have not yet shown significant advantages over CNN-based methods in stereo super-resolution tasks. This can be attributed to two key factors: first, current single-image super-resolution transformers are unable to leverage the complementary stereo information during the process; second, the performance of transformers is typically reliant on sufficient data, which is absent in common stereo-image super-resolution algorithms. To address these issues, we propose a Hybrid Transformer and CNN Attention Network (HTCAN), which utilizes a transformer-based network for single-image enhancement and a CNN-based network for stereo information fusion. Furthermore, we employ a multi-patch training strategy and larger window sizes to activate more input pixels for super-resolution. We also revisit other advanced techniques, such as data augmentation, data ensemble, and model ensemble to reduce overfitting and data bias. Finally, our approach achieved a score of 23.90dB and emerged as the winner in Track 1 of the NTIRE 2023 Stereo Image Super-Resolution Challenge.
Garbage in, garbage out: An analysis of HTML text extractors and their impact on NLP performance

Technological advancement has significantly facilitated the research and development of Artificial Intelligence, with particular emphasis on Natural Language Processing (NLP). High-quality data is crucial to achieving success in this area. This aspect becomes particularly important considering the recent widespread adoption of large language models trained on a considerable amount of text from the Internet. This research expands on the issue of data quality in NLP by examining the impact of automated text extraction techniques from HTML on the performance of specific NLP tasks. For this purpose, an empirical evaluation was conducted to assess the efficacy of various automated techniques for HTML text extraction using 300 news articles written in English, Romanian, and French. The evaluation was conducted by comparing the results of the most popular automated text extraction technologies (i.e., “boiler”, “justext”, “newspaper”, “readability”, and “trafilatura”) against the results of human-validated texts. Both extracted texts, automated and human-validated, were subjected to three NLP tasks: named entity recognition, sentiment analysis, and text summarization. Our analysis of the NLP results indicates that text from Romanian online news articles should be extracted with “newspaper”, whereas “trafilatura” should be used for English and French articles, regardless of the NLP task. Overall, our study provides a comprehensive understanding of the performance of the selected technologies for extracting the text of online news articles by language and NLP task.
PLA: Language-Driven Open-Vocabulary 3D Scene Understanding

Open-vocabulary scene understanding aims to localize and recognize unseen categories beyond the annotated label space. The recent breakthrough of 2D open-vocabulary perception is largely driven by Internet-scale paired image-text data with rich vocabulary concepts. However, this success cannot be directly transferred to 3D scenarios due to the inaccessibility of large-scale 3D-text pairs. To this end, we propose to distill knowledge encoded in pretrained vision-language (VL) foundation models through captioning multi-view images from 3D, which allows explicitly associating 3D and semantic-rich captions. Further, to foster coarse-to-fine visual-semantic representation learning from captions, we design hierarchical 3D-caption pairs, leveraging geometric constraints between 3D scenes and multi-view images. Finally, by employing contrastive learning, the model learns language-aware embeddings that connect 3D and text for open-vocabulary tasks. Our method not only remarkably outperforms baseline methods by 25.8% ~ 44.7% hIoU and 14.5% ~ 50.4% hAP <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">50</inf> in open-vocabulary semantic and instance segmentation, but also shows robust transferability on challenging zero-shot domain transfer tasks. See the project website at https://dingry.github.io/projects/PLA.
A Unified Pyramid Recurrent Network for Video Frame Interpolation

Flow-guided synthesis provides a common framework for frame interpolation, where optical flow is estimated to guide the synthesis of intermediate frames between consecutive inputs. In this paper, we present UPR-Net, a novel Unified Pyramid Recurrent Network for frame interpolation. Cast in a flexible pyramid framework, UPR-Net exploits lightweight recurrent modules for both bi-directional flow estimation and intermediate frame synthesis. At each pyramid level, it leverages estimated bi-directional flow to generate forward-warped representations for frame synthesis; across pyramid levels, it enables iterative refinement for both optical flow and intermediate frame. In particular, we show that our iterative synthesis strategy can significantly improve the robustness of frame interpolation on large motion cases. Despite being extremely lightweight (1.7M parameters), our base version of UPR-Net achieves excellent performance on a large range of benchmarks. Code and trained models of our UPR-Net series are available at: https://github.com/srcn-iv1/UPR-Net.
MixTeacher: Mining Promising Labels with Mixed Scale Teacher for Semi-Supervised Object Detection

Scale variation across object instances remains a key challenge in object detection task. Despite the remarkable progress made by modern detection models, this challenge is particularly evident in the semi-supervised case. While existing semi-supervised object detection methods rely on strict conditions to filter high-quality pseudo labels from network predictions, we observe that objects with extreme scale tend to have low confidence, resulting in a lack of positive supervision for these objects. In this paper, we propose a novel framework that addresses the scale variation problem by introducing a mixed scale teacher to improve pseudo label generation and scale-invariant learning. Additionally, we propose mining pseudo labels using score promotion of predictions across scales, which benefits from better predictions from mixed scale features. Our extensive experiments on MS COCO and PASCAL VOC benchmarks under various semi-supervised settings demonstrate that our method achieves new state-of-the-art performance. The code and models are available at https://github.com/lliuz/MixTeacher.
Comprehensive Multiview Representation Learning via Deep Autoencoder-Like Nonnegative Matrix Factorization

Learning a comprehensive representation from multiview data is crucial in many real-world applications. Multiview representation learning (MRL) based on nonnegative matrix factorization (NMF) has been widely adopted by projecting high-dimensional space into a lower order dimensional space with great interpretability. However, most prior NMF-based MRL techniques are shallow models that ignore hierarchical information. Although deep matrix factorization (DMF)-based methods have been proposed recently, most of them only focus on the consistency of multiple views and have cumbersome clustering steps. To address the above issues, in this article, we propose a novel model termed deep autoencoder-like NMF for MRL (DANMF-MRL), which obtains the representation matrix through the deep encoding stage and decodes it back to the original data. In this way, through a DANMF-based framework, we can simultaneously consider the multiview consistency and complementarity, allowing for a more comprehensive representation. We further propose a one-step DANMF-MRL, which learns the latent representation and final clustering labels matrix in a unified framework. In this approach, the two steps can negotiate with each other to fully exploit the latent clustering structure, avoid previous tedious clustering steps, and achieve optimal clustering performance. Furthermore, two efficient iterative optimization algorithms are developed to solve the proposed models both with theoretical convergence analysis. Extensive experiments on five benchmark datasets demonstrate the superiority of our approaches against other state-of-the-art MRL methods.
A Cipher-Agnostic Neural Training Pipeline with Automated Finding of Good Input Differences

Neural cryptanalysis is the study of cryptographic primitives through machine learning techniques. Following Gohr’s seminal paper at CRYPTO 2019, a focus has been placed on improving the accuracy of such distinguishers against specific primitives, using dedicated training schemes, in order to obtain better key recovery attacks based on machine learning. These distinguishers are highly specialized and not trivially applicable to other primitives. In this paper, we focus on the opposite problem: building a generic pipeline for neural cryptanalysis. Our tool is composed of two parts. The first part is an evolutionary algorithm for the search of good input differences for neural distinguishers. The second part is DBitNet, a neural distinguisher architecture agnostic to the structure of the cipher. We show that this fully automated pipeline is competitive with a highly specialized approach, in particular for SPECK32, and SIMON32. We provide new neural distinguishers for several primitives (XTEA, LEA, HIGHT, SIMON128, SPECK128) and improve over the state-of-the-art for PRESENT, KATAN, TEA and GIMLI.
AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment

With the rapid advancements of the text-to-image generative model, AI-generated images (AGIs) have been widely applied to entertainment, education, social media, etc. However, considering the large quality variance among different AGIs, there is an urgent need for quality models that are consistent with human subjective ratings. To address this issue, we extensively consider various popular AGI models, generated AGI through different prompts and model parameters, and collected subjective scores at the perceptual quality and text-to-image alignment, thus building the most comprehensive AGI subjective quality database AGIQA-3K so far. Furthermore, we conduct a benchmark experiment on this database to evaluate the consistency between the current Image Quality Assessment (IQA) model and human perception, while proposing StairReward that significantly improves the assessment performance of subjective text-to-image alignment. We believe that the fine-grained subjective scores in AGIQA-3K will inspire subsequent AGI quality models to fit human subjective perception mechanisms at both perception and alignment levels and to optimize the generation result of future AGI models. The database is released on https://github.com/lcysyzxdxc/AGIQA-3k-Database.