Reviewerly/exHarmony huggingface dataset:
CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs

We present CrossSum, a large-scale crosslingual summarization dataset comprising 1.68 million article-summary samples in 1,500+ language pairs.We create CrossSum by aligning parallel articles written in different languages via cross-lingual retrieval from a multilingual abstractive summarization dataset and perform a controlled human evaluation to validate its quality.We propose a multistage data sampling algorithm to effectively train a cross-lingual summarization model capable of summarizing an article in any target language.We also introduce LaSE, an embedding-based metric for automatically evaluating model-generated summaries.LaSE is strongly correlated with ROUGE and, unlike ROUGE, can be reliably measured even in the absence of references in the target language.Performance on ROUGE and LaSE indicate that our proposed model consistently outperforms baseline models.To the best of our knowledge, CrossSum is the largest cross-lingual summarization dataset and the first ever that is not centered around English.We are releasing the dataset, training and evaluation scripts, and models to spur future research on cross-lingual summarization.The resources can be found at https: //github.com/csebuetnlp/CrossSum.
Boosting Text Augmentation via Hybrid Instance Filtering Framework

Text augmentation is an effective technique for addressing the problem of insufficient data in natural language processing.However, existing text augmentation methods tend to focus on few-shot scenarios and usually perform poorly on large public datasets.Our research indicates that existing augmentation methods often generate instances with shifted feature spaces, which leads to a drop in performance on the augmented data (for example, EDA generally loses ≈ 2% in aspect-based sentiment classification).To address this problem, we propose a hybrid instance-filtering framework (BO O S TAU G) based on pre-trained language models that can maintain a similar feature space with natural datasets.BO O S TAU G is transferable to existing text augmentation methods (such as synonym substitution and back translation) and significantly improves the augmentation performance by ≈ 2 -3% in classification accuracy.Our experimental results on three classification tasks and nine public datasets show that BO O S TAU G addresses the performance drop problem and outperforms state-ofthe-art text augmentation methods.Additionally, we release the code to help improve existing augmentation methods on large datasets.
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval

With the development of large language models (LLMs), zero-shot learning has attracted much attention for various NLP tasks.Different from prior works that generate training data with billion-scale natural language generation (NLG) models, we propose a retrievalenhanced framework to create training data from a general-domain unlabeled corpus.To realize this, we first conduct contrastive pretraining to learn an unsupervised dense retriever for extracting most relevant documents using classdescriptive verbalizers.We then further propose two simple strategies, namely Verbalizer Augmentation with Demonstrations and Selfconsistency Guided Filtering to improve the topic coverage of the dataset while removing noisy examples.Experiments on nine datasets demonstrate that REGEN achieves 4.3% gain over strongest baselines and saves around 70% of the time when compared with baselines using large NLG models.Besides, REGEN can be naturally integrated with recently proposed large language models to boost performance 1 .
Cold-Start Data Selection for Better Few-shot Language Model Fine-tuning: A Prompt-based Uncertainty Propagation Approach

We present PATRON, a prompt-based data selection method for pre-trained language model fine-tuning under cold-start scenarios, i.e., no initial labeled data are available. In PATRON, we design (1) a prompt-based uncertainty propagation approach to estimate the importance of data points and (2) a partition-then-rewrite (PTR) strategy to promote sample diversity when querying for annotations. Experiments on six text classification datasets show that PATRON outperforms the strongest cold-start data selection baselines by up to 6.9%. Besides, with 128 labels only, PATRON achieves 91.0% and 92.1% of the fully supervised performance based on vanilla fine-tuning and prompt-based learning respectively. Our implementation of PATRON will be published upon acceptance.
Zero-shot Visual Question Answering with Language Model Feedback

In this paper, we propose a novel language model guided captioning approach, LAMOC, for knowledge-based visual question answering (VQA).Our approach employs the generated captions by a captioning model as the context of an answer prediction model, which is a Pre-trained Language model (PLM).As the major contribution, we leverage the guidance and feedback of the prediction model to improve the capability of the captioning model.In this way, the captioning model can become aware of the task goal and information need from the PLM.To develop our approach, we design two specific training stages, where the first stage adapts the captioning model to the prediction model (selecting more suitable caption propositions for training) and the second stage tunes the captioning model according to the task goal (learning from feedback of the PLM).Extensive experiments demonstrate the effectiveness of the proposed approach on the knowledgebased VQA task.Specifically, on the challenging A-OKVQA dataset, LAMOC outperforms several competitive zero-shot methods and even achieves comparable results to a fine-tuned VLP model.Our code is publicly available at https://github.com/RUCAIBox/LAMOC.
On the Calibration of Massively Multilingual Language Models

Massively Multilingual Language Models (MMLMs) have recently gained popularity due to their surprising effectiveness in cross-lingual transfer.While there has been much work in evaluating these models for their performance on a variety of tasks and languages, little attention has been paid on how well calibrated these models are with respect to the confidence in their predictions.We first investigate the calibration of MMLMs in the zero-shot setting and observe a clear case of miscalibration in low-resource languages or those which are typologically diverse from English.Next, we empirically show that calibration methods like temperature scaling and label smoothing do reasonably well in improving calibration in the zero-shot scenario.We also find that fewshot examples in the language can further help reduce calibration errors, often substantially.Overall, our work contributes towards building more reliable multilingual models by highlighting the issue of their miscalibration, understanding what language and model-specific factors influence it, and pointing out the strategies to improve the same.
Extending Phrase Grounding with Pronouns in Visual Dialogues

Conventional phrase grounding aims to localize noun phrases mentioned in a given caption to their corresponding image regions, which has achieved great success recently.Apparently, sole noun phrase grounding is not enough for cross-modal visual language understanding.Here we extend the task by considering pronouns as well.First, we construct a dataset of phrase grounding with both noun phrases and pronouns to image regions.Based on the dataset, we test the performance of phrase grounding by using a state-of-the-art literature model of this line.Then, we enhance the baseline grounding model with coreference information which should help our task potentially, modeling the coreference structures with graph convolutional networks.Experiments on our dataset, interestingly, show that pronouns are easier to ground than noun phrases, where the possible reason might be that these pronouns are much less ambiguous.Additionally, our final model with coreference information can significantly boost the grounding performance of both noun phrases and pronouns.
Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering

A common thread of open-domain question answering (QA) models employs a retrieverreader pipeline that first retrieves a handful of relevant passages from Wikipedia and then peruses the passages to produce an answer.However, even state-of-the-art readers fail to capture the complex relationships between entities appearing in questions and retrieved passages, leading to answers that contradict the facts.In light of this, we propose a novel knowledge Graph enhanced passage reader, namely GRAPE , to improve the reader performance for open-domain QA.Specifically, for each pair of question and retrieved passage, we first construct a localized bipartite graph, attributed to entity embeddings extracted from the intermediate layer of the reader model.Then, a graph neural network learns relational knowledge while fusing graph and contextual representations into the hidden states of the reader model.Experiments on three open-domain QA benchmarks show GRAPE can improve the state-of-the-art performance by up to 2.2 exact match score with a negligible overhead increase, with the same retriever and retrieved passages.Our code is publicly available at https://github.com/jumxglhf/GRAPE. * Equal contribution.Question: What is the primary language of China?Retrieved Wikipedia passage (ID: 253645): Language of the People's Republic of China and of the Republic of China (Taiwan), and one of the official languages of Singapore.Standard Mandarin Chinese now dominates public life in mainland China.Outside China and Taiwan, the only varieties of Chinese commonly taught in university courses are Mandarin and Cantonese.
MSViT: Training Multiscale Vision Transformers for Image Retrieval

The recently developed vision transformer (ViT) has achieved promising results on image retrieval compared to convolutional neural networks. However, most of these vision transformer-based image retrieval methods use the original ViT model to extract global features, ignoring the importance of local features for image retrieval. In this work, we propose a vision transformer-based multiscale feature fusion image retrieval method (MSViT) to achieve the fusion of global features with local features. The challenge of this research work is how to learn the feature representation ability of transformer model, so as to improve the performance of image retrieval model. First, a transformer-based two-branch network structure is proposed to obtain different scale features by processing image patches with different granularities. Second, we present a multiscale feature fusion strategy, which can efficiently and effectively fuse the feature information of different sizes on two branches. Finally, to more fully utilize the label information to supervise the network training process, we optimize the construction rules for the triplet data. The comparison of experimental results with ten CNN-based and six transformer-based image retrieval methods on four publicly available image datasets shows that our method outperforms the state-of-the-art methods. And ablation experiments show that the designed multiscale feature fusion strategy and improved triplet loss function have an implicit improvement on the performance of MSViT. The source codes are available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/x-28/msvit-image-retrieval</uri> .
Prediction of Solar Irradiation in Africa using Linear-Nonlinear Hybrid Models

Solar irradiation prediction including Global Horizontal Irradiation (GHI) and Direct Normal Irradiation (DNI) is a useful technique for assessing the solar energy potential at specific locations. This study used five Artificial Neural Network (ANN) models and Multiple Linear Regression (MLR) to predict GHI and DNI in Africa. Additionally, a hybrid model combining MLR and ANNs was proposed to predict both GHI and DNI and improve the accuracy of individual ANN models. Solar radiation (GHI and DNI) and global meteorological data from 85 cities with different climatic conditions over Africa during 2001-2020 were used to train and test the models developed. The Pearson correlation coefficient was used to identify the most influential input variables to predict GHI and DNI. Two scenarios were proposed to achieve the goal, each with different input variables. The first scenario used influential input parameters, while the second incorporated geographical coordinates to assess their impact on solar radiation prediction accuracy. The results revealed that the suggested linear-nonlinear hybrid models outperformed all other models in terms of prediction accuracy. Moreover, the investigation revealed that geographical coordinates have a minimal impact on the prediction of solar radiation.
Diffusion Models for Non-autoregressive Text Generation: A Survey

Non-autoregressive (NAR) text generation has attracted much attention in the field of natural language processing, which greatly reduces the inference latency but has to sacrifice the generation accuracy. Recently, diffusion models, a class of latent variable generative models, have been introduced into NAR text generation, showing an improved text generation quality. In this survey, we review the recent progress in diffusion models for NAR text generation. As the background, we first present the general definition of diffusion models and the text diffusion models, and then discuss their merits for NAR generation. As the core content, we further introduce two mainstream diffusion models in existing work of text diffusion, and review the key designs of the diffusion process. Moreover, we discuss the utilization of pre-trained language models (PLMs) for text diffusion models and introduce optimization techniques for text data. Finally, we discuss several promising directions and conclude this paper. Our survey aims to provide researchers with a systematic reference of related research on text diffusion models for NAR generation. We also demonstrate our collection of text diffusion models at https://github.com/RUCAIBox/Awesome-Text-Diffusion-Models.
The OpenCitations blog posts are now archived on Rogue Scholar with DOIs

Last April, Martin Fenner launched Rogue Scholar, an archive of science blogs aiming to index full-text of blog posts, establish a full-text search, and register DOIs and metadata for all posts. Rogue Scholar works with all blogging platforms that publish scholarly content and have an RSS or Atom feed with full-text content distributed under a Creative Commons Attribution (CC-BY 4.0) license.
Packet Header Recognition Utilizing an All-Optical Reservoir Based on Reinforcement-Learning- Optimized Double-Ring Resonators

Optical packet header recognition is an important signal processing task of optical communication networks. In this work, we propose an all-optical reservoir, consisting of integrated double-ring resonators (DRRs) as nodes, for fast and accurate optical packet header recognition. As the delay-bandwidth product (DBP) of the node is a key figure-of-merit in the reservoir, we adopt a deep reinforcement learning algorithm to maximize the DBPs for various types of DRRs, which has the advantage of full parameter space optimization and fast convergence speed. Intriguingly, the optimized DBPs of the DRRs in cascaded, parallel, and embedded configurations reach the same maximum value, which is believed to be the global maximum. Finally, 3-bit and 6-bit packet header recognition tasks are performed with the all-optical reservoir consisting of the optimized cascaded rings, which have greatly reduced chip size and the desired “flat-top” delay spectra. Using this optical computing scheme, word-error rates as low as 5×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-4</sup> and 9×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-4</sup> are achieved for 3-bit and 6-bit packet header recognition tasks, respectively, which are one order of magnitude better than the previously reported values.
A Phenomenological Exploration of Students’ Perceptions of AI Chatbots in Higher Education

This study examines the impact of AI chatbots as a communication medium on student engagement and support in higher education. The qualitative method and Interpretative Phenomenological Analysis (IPA) were employed as the research approach, utilizing in-depth semi-structured interviews. Purposive sampling was used to select 11 participants from the state of Kerala, India, in higher education. Data analysis followed the Systematic Text Consideration (STC), a five-step process, including framing meaning units, condensing meaning units, coding, creating sub-themes, and deriving themes. By exploring themes aligned with the UTAUT2 constructs, a comprehensive understanding of the factors influencing student engagement and support was achieved. A total of eight themes were identified, encompassing “Effectiveness and Limitations,” “Beyond,” “Enrichment,” “Optimization,” “Synergize,” “Streamlining Communication,” “Engage+AI,” and “Refine.” These themes provided compelling evidence of the transformative potential of AI chatbots in facilitating effective communication, enhancing engagement, and offering timely support. The study’s results carry significant practical implications for higher education institutions. Embracing AI chatbots, universities and institutions can enhance student engagement and support through efficient communication, personalized recommendations, and streamlined interactions. These chatbots offer a balance between quick assistance and human expertise, optimizing both routine tasks and complex inquiries. Additionally, addressing security and privacy concerns is crucial to fostering trust and successful integration. Overall, embracing AI chatbots can transform the educational experience, making it more efficient, engaging, and supportive for students in higher education.
A fast, dense Chebyshev solver for electronic structure on GPUs

Matrix diagonalization is almost always involved in computing the density matrix needed in quantum chemistry calculations. In the case of modest matrix sizes (≲4000), performance of traditional dense diagonalization algorithms on modern GPUs is underwhelming compared to the peak performance of these devices. This motivates the exploration of alternative algorithms better suited to these types of architectures. We newly derive, and present in detail, an existing Chebyshev expansion algorithm [Liang et al., J. Chem. Phys. 119, 4117–4125 (2003)] whose number of required matrix multiplications scales with the square root of the number of terms in the expansion. Focusing on dense matrices of modest size, our implementation on GPUs results in large speed ups when compared to diagonalization. Additionally, we improve upon this existing method by capitalizing on the inherent task parallelism and concurrency in the algorithm. This improvement is implemented on GPUs by using CUDA and HIP streams via the MAGMA library and leads to a significant speed up over the serial-only approach for smaller (≲1000) matrix sizes. Finally, we apply our technique to a model system with a high density of states around the Fermi level, which typically presents significant challenges.
The Development of Islamic Religious Education and Character Materials Online Based in Elementary Schools

Purpose: The purpose of the study is to find and develop teaching materials for improving the outcomes of student learning in the subjects of Islamic Religious Education and Moral Character (PAdB) in Elementary Schools (SD). Theoretical reference: The study utilizes the research and development (RnD) method as its theoretical framework. This method encompasses various research stages, including result examination, product development, testing, and iterative refinement. The study aligns with contemporary educational theories emphasizing the use of innovative teaching materials and online-based resources to enhance student engagement and understanding. Method: The research employs the RnD method to design and validate new teaching materials. The process involves multiple stages, starting with result analysis, leading to the creation of online-based teaching materials for the PAdB subjects in elementary schools. These materials are rigorously tested and refined based on feedback and identified weaknesses. Results and Conclusion: The following research findings have been identified: 1) students are actively involved in variations and innovations of online-based teaching materials in PAdB subjects in elementary schools in order to deliver intriguing experiences for students; 2) it is straightforward to access online-based teaching materials; 3) it is simple to understand and study material products in online-based teaching thoroughly; 4) providing learning opportunities for students through collaborative principles and the development of online-based PAdB learning material in elementary schools. Implications of research: The study's implications are far-reaching. It highlights the effectiveness of online-based teaching materials in improving student outcomes in PAdB subjects. This finding has practical implications for educators and curriculum designers, emphasizing the importance of integrating digital resources into elementary school education. Additionally, the collaborative learning approach introduced through online materials opens new avenues for enhancing student engagement. Originality/value: This research contributes to the field of education by presenting an innovative approach to teaching materials development and implementation. The use of online-based resources, combined with collaborative learning principles, is a novel and valuable strategy for improving student performance in PAdB subjects. This study's originality lies in its innovative pedagogical approach and its potential to transform elementary school education positively.
Variational log‐<scp>Gaussian</scp> point‐process methods for grid cells

We present practical solutions to applying Gaussian-process (GP) methods to calculate spatial statistics for grid cells in large environments. GPs are a data efficient approach to inferring neural tuning as a function of time, space, and other variables. We discuss how to design appropriate kernels for grid cells, and show that a variational Bayesian approach to log-Gaussian Poisson models can be calculated quickly. This class of models has closed-form expressions for the evidence lower-bound, and can be estimated rapidly for certain parameterizations of the posterior covariance. We provide an implementation that operates in a low-rank spatial frequency subspace for further acceleration, and demonstrate these methods on experimental data.
Quantum repeaters and teleportation via entangled phase-modulated multimode coherent states

We present a scheme of quantum repeater that uses entangled multimode coherent states that are obtained by electro-optic modulation of symmetric and antisymmetric Schr\"odinger cat states. Part of the generated entangled frequency modes is sent to a symmetric beam splitter at the central node, while the remaining modes are stored locally in quantum memories. The entangled coherent states between remote quantum memories are conditionally prepared by photon-counting measurements at the output channels of the beam splitter. We study how the effects of decoherence in the quantum channel affect statistics of photocounts and, for the heralding outcomes determined by the parity of photocounts, evaluate the probability of success and the fidelity of the prepared entanglement depending on the symmetry of input cat states. It is demonstrated that the generated entanglement can be employed for teleportation of the phase information from the modulated states that are utilized in quantum key distributions with subcarrier wave encoding.