Reviewerly/exHarmony huggingface dataset:
Thrust: Adaptively Propels Large Language Models with External Knowledge

Although large-scale pre-trained language models (PTLMs) are shown to encode rich knowledge in their model parameters, the inherent knowledge in PTLMs can be opaque or static, making external knowledge necessary. However, the existing information retrieval techniques could be costly and may even introduce noisy and sometimes misleading knowledge. To address these challenges, we propose the instance-level adaptive propulsion of external knowledge (IAPEK), where we only conduct the retrieval when necessary. To achieve this goal, we propose measuring whether a PTLM contains enough knowledge to solve an instance with a novel metric, Thrust, which leverages the representation distribution of a small number of seen instances. Extensive experiments demonstrate that thrust is a good measurement of PTLM models' instance-level knowledgeability. Moreover, we can achieve significantly higher cost-efficiency with the Thrust score as the retrieval indicator than the naive usage of external knowledge on 88% of the evaluated tasks with 26% average performance improvement. Such findings shed light on the real-world practice of knowledge-enhanced LMs with a limited knowledge-seeking budget due to computation latency or costs.
Integrating a Heterogeneous Graph with Entity-aware Self-attention using Relative Position Labels for Reading Comprehension Model

Despite the significant progress made by transformer models in machine reading comprehension tasks, they still fall short in handling complex reasoning tasks due to the absence of explicit knowledge in the input sequence. To address this limitation, many recent works have proposed injecting external knowledge into the model. However, selecting relevant external knowledge, ensuring its availability, and requiring additional processing steps remain challenging. In this paper, we introduce a novel attention pattern that integrates reasoning knowledge derived from a heterogeneous graph into the transformer architecture without relying on external knowledge. The proposed attention pattern comprises three key elements: global-local attention for word tokens, graph attention for entity tokens that exhibit strong attention towards tokens connected in the graph as opposed to those unconnected, and the consideration of the type of relationship between each entity token and word token. This results in optimized attention between the two if a relationship exists. The pattern is coupled with special relative position labels, allowing it to integrate with LUKE's entity-aware self-attention mechanism. The experimental findings corroborate that our model outperforms both the cutting-edge LUKE-Graph and the baseline LUKE model across two distinct datasets: ReCoRD, emphasizing commonsense reasoning, and WikiHop, focusing on multi-hop reasoning challenges.
Confidence Estimation Using Unlabeled Data

Overconfidence is a common issue for deep neural networks, limiting their deployment in real-world applications. To better estimate confidence, existing methods mostly focus on fully-supervised scenarios and rely on training labels. In this paper, we propose the first confidence estimation method for a semi-supervised setting, when most training labels are unavailable. We stipulate that even with limited training labels, we can still reasonably approximate the confidence of model on unlabeled samples by inspecting the prediction consistency through the training process. We use training consistency as a surrogate function and propose a consistency ranking loss for confidence estimation. On both image classification and segmentation tasks, our method achieves state-of-the-art performances in confidence estimation. Furthermore, we show the benefit of the proposed method through a downstream active learning task. The code is available at https://github.com/TopoXLab/consistency-ranking-loss
Fast Approximate Nearest Neighbor Search with a Dynamic Exploration Graph using Continuous Refinement

For approximate nearest neighbor search, graph-based algorithms have shown to offer the best trade-off between accuracy and search time. We propose the Dynamic Exploration Graph (DEG) which significantly outperforms existing algorithms in terms of search and exploration efficiency by combining two new ideas: First, a single undirected even regular graph is incrementally built by partially replacing existing edges to integrate new vertices and to update old neighborhoods at the same time. Secondly, an edge optimization algorithm is used to continuously improve the quality of the graph. Combining this ongoing refinement with the graph construction process leads to a well-organized graph structure at all times, resulting in: (1) increased search efficiency, (2) predictable index size, (3) guaranteed connectivity and therefore reachability of all vertices, and (4) a dynamic graph structure. In addition we investigate how well existing graph-based search systems can handle indexed queries where the seed vertex of a search is the query itself. Such exploration tasks, despite their good starting point, are not necessarily easy. High efficiency in approximate nearest neighbor search (ANNS) does not automatically imply good performance in exploratory search. Extensive experiments show that our new Dynamic Exploration Graph outperforms existing algorithms significantly for indexed and unindexed queries.
What can we learn from Data Leakage and Unlearning for Law?

Large Language Models (LLMs) have a privacy concern because they memorize training data (including personally identifiable information (PII) like emails and phone numbers) and leak it during inference. A company can train an LLM on its domain-customized data which can potentially also include their users' PII. In order to comply with privacy laws such as the "right to be forgotten", the data points of users that are most vulnerable to extraction could be deleted. We find that once the most vulnerable points are deleted, a new set of points become vulnerable to extraction. So far, little attention has been given to understanding memorization for fine-tuned models. In this work, we also show that not only do fine-tuned models leak their training data but they also leak the pre-training data (and PII) memorized during the pre-training phase. The property of new data points becoming vulnerable to extraction after unlearning and leakage of pre-training data through fine-tuned models can pose significant privacy and legal concerns for companies that use LLMs to offer services. We hope this work will start an interdisciplinary discussion within AI and law communities regarding the need for policies to tackle these issues.
Interactive Segmentation for Diverse Gesture Types Without Context

Interactive segmentation entails a human marking an image to guide how a model either creates or edits a segmentation. Our work addresses limitations of existing methods: they either only support one gesture type for marking an image (e.g., either clicks or scribbles) or require knowledge of the gesture type being employed, and require specifying whether marked regions should be included versus excluded in the final segmentation. We instead propose a simplified interactive segmentation task where a user only must mark an image, where the input can be of any gesture type without specifying the gesture type. We support this new task by introducing the first interactive segmentation dataset with multiple gesture types as well as a new evaluation metric capable of holistically evaluating interactive segmentation algorithms. We then analyze numerous interactive segmentation algorithms, including ones adapted for our novel task. While we observe promising performance overall, we also highlight areas for future improvement. To facilitate further extensions of this work, we publicly share our new dataset at https://github.com/joshmyersdean/dig.
FedSoup: Improving Generalization and Personalization in Federated Learning via Selective Model Interpolation

Cross-silo federated learning (FL) enables the development of machine learning models on datasets distributed across data centers such as hospitals and clinical research laboratories. However, recent research has found that current FL algorithms face a trade-off between local and global performance when confronted with distribution shifts. Specifically, personalized FL methods have a tendency to overfit to local data, leading to a sharp valley in the local model and inhibiting its ability to generalize to out-of-distribution data. In this paper, we propose a novel federated model soup method (i.e., selective interpolation of model parameters) to optimize the trade-off between local and global performance. Specifically, during the federated training phase, each client maintains its own global model pool by monitoring the performance of the interpolated model between the local and global models. This allows us to alleviate overfitting and seek flat minima, which can significantly improve the model's generalization performance. We evaluate our method on retinal and pathological image classification tasks, and our proposed method achieves significant improvements for out-of-distribution generalization. Our code is available at https://github.com/ubc-tea/FedSoup.
Identifying Interpretable Subspaces in Image Representations

We propose Automatic Feature Explanation using Contrasting Concepts (FALCON), an interpretability framework to explain features of image representations. For a target feature, FALCON captions its highly activating cropped images using a large captioning dataset (like LAION-400m) and a pre-trained vision-language model like CLIP. Each word among the captions is scored and ranked leading to a small number of shared, human-understandable concepts that closely describe the target feature. FALCON also applies contrastive interpretation using lowly activating (counterfactual) images, to eliminate spurious concepts. Although many existing approaches interpret features independently, we observe in state-of-the-art self-supervised and supervised models, that less than 20% of the representation space can be explained by individual features. We show that features in larger spaces become more interpretable when studied in groups and can be explained with high-order scoring concepts through FALCON. We discuss how extracted concepts can be used to explain and debug failures in downstream tasks. Finally, we present a technique to transfer concepts from one (explainable) representation space to another unseen representation space by learning a simple linear transformation. Code available at https://github.com/NehaKalibhat/falcon-explain.
Building Socio-culturally Inclusive Stereotype Resources with Community Engagement

With rapid development and deployment of generative language models in global settings, there is an urgent need to also scale our measurements of harm, not just in the number and types of harms covered, but also how well they account for local cultural contexts, including marginalized identities and the social biases experienced by them. Current evaluation paradigms are limited in their abilities to address this, as they are not representative of diverse, locally situated but global, socio-cultural perspectives. It is imperative that our evaluation resources are enhanced and calibrated by including people and experiences from different cultures and societies worldwide, in order to prevent gross underestimations or skews in measurements of harm. In this work, we demonstrate a socio-culturally aware expansion of evaluation resources in the Indian societal context, specifically for the harm of stereotyping. We devise a community engaged effort to build a resource which contains stereotypes for axes of disparity that are uniquely present in India. The resultant resource increases the number of stereotypes known for and in the Indian context by over 1000 stereotypes across many unique identities. We also demonstrate the utility and effectiveness of such expanded resources for evaluations of language models. CONTENT WARNING: This paper contains examples of stereotypes that may be offensive.
Hughes-Hassell, Sandra (2020). Collection management for youth: equity, inclusion, and learning (2nd ed.). Chicago, IL: ALA Editions. (Review)

Book Review
Simulation Study of Imbalanced Classification on High-Dimensional Gene Expression Data

Purpose: Classification of gene expression helps study disease. However, it faces two obstacles: an imbalanced class and a high dimension. The motivation of this study is to examine the effectiveness of undersampling before feature selection on high-dimensional data with imbalanced classes.Methods: Least Absolute Shrinkage and Selection Operator (Lasso), which can select features, can handle high-dimensional data modeling. Random undersampling (RUS) can be used to deal with imbalanced classes. The Classification and Decision Tree (CART) algorithm is used to construct a classification model because it can produce an interpretable model. Thirty simulated datasets with varying imbalance ratios are used to test the proposed approaches, which are Lasso-CART and RUS-Lasso-CART. The simulated data are generated from parameters of real gene expression data.Results: The simulation study results show that when the minority class accounts for more than 25% of the observation size, the Lasso-CART method is appropriate. Meanwhile, RUS-Lasso-CART is effective when the minority class size is at least 20 observations.Novelty: The novelty of this simulation study is using the RUS-Lasso-CART hybrid method to address the classification problem of high-dimensional gene expression data with imbalanced classes.
Monthly Rainfall Prediction Using the Backpropagation Neural Network (BPNN) Algorithm in Maros Regency

Purpose: This study aims to identify the right combination of network architecture, learning rate, and epoch in making predictions at each rainfall post in Maros Regency. In addition, this study also predicts the monthly rainfall profile in 2021-2025 in Maros Regency.Methods: The method in this study is the backpropagation neural network algorithm to learn and predict the data. BPNN is one of the most commonly used non-linear methods in making predictions recently. The data used in this study is monthly rainfall data from 2000-2020 as training and testing data at four rainfall stations including BPP Batubassi, Staklim Maros, Stamet Hasanuddin, and BPP Tanralili.Result: The results showed that the combination of network architecture, learning rate, and epoch obtained at each rainfall post was different. The highest level of prediction accuracy was obtained on 5 layers rather than 3 or 4 layers of network architecture with prediction accuracy at each rainfall post respectively 76.91%, 72.47%, 75.24%, and 76.53%. The predictions of rainfall from 2021-2025 are following the monsoon rain pattern with the highest rainfall in January 2025 of 964.1 mm, while the largest annual rainfall is obtained in 2023 with a total of 3359.6 mm.Novelty: In this study, various combinations of network architecture parameters consisting of learning rate, epoch, and architecture at each rainfall post obtained different results. Particularly in the Maros Regency, the combination that is most suitable for use in predicting monthly rainfall at the Batubassi BPP post is learning rate 0.7, epoch 50000, and network architecture 11-6-10-7-5, at Staklim Maros post is learning rate 0.5, epoch 50000, and network architecture 11-5-9-10-5, at Stamet Hasanuddin post is learning rate 0.8, epoch 20000, and network architecture 11-5-8-6-5, and at BPP Tanralili post is learning rate 0.5, epoch 10000, and 11-5-9-9-5 network architecture.
A Review of Research on Second Language Learning Strategies (2000-2022)

Research on second language learning strategies is a hot topic in the fields of linguistics and education.This paper employs CiteSpace software to conduct visualized analysis of the temporal distribution, knowledge base, hot topics, and emerging trends in second language learning strategy research literature from 2000 to 2022 in the Web of Science database.The results reveal that: 1) the annual publication volume in this field shows an overall upward trend; 2) the author collaboration network exhibits a dispersed pattern, with low density of collaboration among individual authors and limited collaboration among research teams; 3) cluster analysis of high-frequency keywords indicates that personalized learning strategy research, autonomous and collaborative learning research, and multimodal learning strategy research are the hot topics in this field; 4) keyword centrality and burstness analysis of recent research suggest that research on learning strategies in intelligent learning environments and interdisciplinary language learning strategy research are the frontier areas in this field.Future research on second language learning strategies should focus on improving and advancing research methods, expanding the scope of research, promoting interdisciplinary collaboration, and exploring practical applications.
Can cyber insurance combat cyber crime?

In the 17th century, pirates would kidnap wealthy merchants and demand money in exchange for their safe release.Today, the ransoming of digital data by cyber criminals is far more likely.Cyber security researchers Dr Jason Nurse, from the University of Kent's Institute of Cyber Security for Society (iCSS), and Jamie MacColl, fromIn a world where so much information is stored digitally, cyber attacks that hold computers hostage can be incredibly damaging.Dr Jason Nurse, from the Institute of
Enhanced Document Retrieval System Using Suffix Tree Clustering Algorithm

Most search engines in use today present the user with a single-ordered list of documents matching the search query leading to lexical ambiguity.An alternative to a single-ordered list is to cluster the search results and present a list of clusters to the user.This study implements the suffix tree clustering algorithm to optimize search.The user selects which cluster appears most relevant and the search results in that cluster are then displayed in a list under the assumption that similar documents are likely to be relevant to the same query.The proposed system clusters search results from the file system.The proposed system allows the user to issue a search query and we return the results as a set of coherent clusters.The suffix tree clustering algorithm efficiently determined documents that share common phrases.The nodes in the suffix tree define the initial cluster and to increase the number of documents in each cluster, clusters that are sufficiently similar are merged.The proposed system adopted web technologies such as hypertext markup language (HTML), and cascade styling sheet (CSS), to design the interface, while Javascript programming language was used to implement the entire system.The proposed system was implemented using PHP5 and MySQL database.The experimental results show that the suffix tree clustering algorithm can be used to cluster documents efficiently.The resulting system demonstrated an optimized search of 4.1 trillion search results of the word "Electricity" whereas a total result of 4.3 trillion was retrieved by the conventional Google Search Engine.
Discrete clusters formulation through the exploitation of optimized k-modes algorithm for hypotheses validation in social work research: the case of greek social workers working with refugees

This article focuses on the results of self-funded quantitative research conducted by social workers working in the “refugee” crisis and social services in Greece (1). The research, among other findings, argues that front-line professionals possess specific characteristics regarding their working profile. Statistical methods in the research performed significance tests to validate the initial hypotheses concerning the correlation between dataset variables. On the contrary of this concept, in this work, we present an alternative approach for validating initial hypotheses through the exploitation of clustering algorithms. Toward that goal, we evaluated several frequently used clustering algorithms regarding their efficiency in feature selection processes, and we finally propose a modified k-Modes algorithm for efficient feature subset selection.
Heterogeneous Graph Contrastive Learning with Augmentation Graph

Heterogeneous graph neural networks (HGNNs) have demonstrated promising capabilities in addressing various problems defined on heterogeneous graphs containing multiple types of nodes or edges. However, traditional HGNN models depend on label information and capture the local structural information of the original graph. In this paper, we propose a novel Heterogeneous Graph Contrastive Learning method with Augmentation Graph (AHGCL). Specifically, we construct an augmentation graph by calculating the feature similarity of nodes to capture latent structural information. For the original graph and the augmentation graph, we employ a shared Graph Neural Network (GNN) encoder to extract the semantic features of nodes with different meta-paths. The feature information is aggregated through a semantic-level attention mechanism to generate final node embeddings, which capture latent high-order semantic structural information. Considering the problems of label information for the real-world datasets, we adopt contrastive learning to train the GNN encoder for maximizing the common information between similar nodes from the original graph and the augmentation graph views. We conduct node classification experiments on four real-world datasets, AMiner, Freebase, DBLP, and ACM, to evaluate the performance of AHGCL. The results show that the proposed AHGCL demonstrates excellent stability and capability compared to existing graph representation learning methods.
A Synchronic Assessment of Google Translate, Microsoft Translator and Reverso in English&lt;&gt;Arabic Translation

The study aimed to evaluate three multilingual MT systems that provide free English&lt;&gt; Arabic translation services. It assessed the effectiveness of the three systems in dealing with WHO, UN, Petra News Agency, and Arab League documents. The study adopted (LDC, 2002) intelligibility and fidelity scales to highlight the limitations and strengths of the systems understudy. The study found that Google Translate outperformed the three systems in providing intelligible and fluent output without flaws with 230 segments, equaling 57% of the corpus size, followed by Microsoft Translator with 48% and Reverso with 45% translation free of errors. On the other hand, the study found that Google Translate outperformed the other systems in terms of adequacy and fluency scales, followed by Microsoft Translator and Reverso interchangeably. However, the three systems are still unable to substitute human translators. Therefore, the study recommends conducting a diachronic study to trace the three systems' development over five years scientifically.