Reviewerly/exHarmony huggingface dataset:
Moving beyond the challenges of learning through technologies: The current status of ICT integration in South African schools

Authors:
raw_author_name: Thokozani Isaac Mtshali
raw_affiliation_string: Department of Technology and Vocational Education, Tshwane University of Technology, South Africa.
raw_author_name: Makgato Moses
raw_affiliation_string: Department of Technology and Vocational Education, Tshwane University of Technology, South Africa.

Abstract: The purpose of this study was to understand the current status of ICT integration in South African schools. Yet, this study focused on the Tshwane West District schools in Gauteng Province, South Africa. This study employed a descriptive research design to understand the aforesaid phenomenon. Data was collected through interviews and open-ended questionnaires with 10 teachers and one curriculum specialist. This study employed thematic analysis as a qualitative research method. Furthermore, Roger's Diffusion of Innovations is used as a framework. Despite the existence of policies, this study revealed that the integration of ICT remained a significant challenge for several instructors in South Africa. The most interesting part was that schools had improved on most ICT tools to enable access to learning with technologies, but to a greater extent, insufficient internet coverage in classrooms made it impossible for most teachers to appreciate the benefits that come with using tools such as laptops, whiteboards, and online videos. This research proposes the installation of pocket Wi-Fi devices in every classroom in schools to provide comprehensive internet access. This will facilitate interaction between instructors and students through up-to-date learning management systems subscribed to by schools, as well as films. This will promote a digitally viable learning environment that promotes creative opportunities for learners.
Examining Pathological Bias in a Generative Adversarial Network  Discriminator: A Case Study on a StyleGAN3 Model

Authors:
raw_author_name: Grissom II, Alvin
raw_author_name: Neto, Jeova Farias Sales Rocha
raw_author_name: Lin, Bailey

Abstract: Generative adversarial networks generate photorealistic faces that are often indistinguishable by humans from real faces. We find that the discriminator in the pre-trained StyleGAN3 model, a popular GAN network, systematically stratifies scores by both image- and face-level qualities and that this disproportionately affects images across gender, race, and other categories. We examine the discriminator's bias for color and luminance across axes perceived race and gender; we then examine axes common in research on stereotyping in social psychology.
NutePrune: Efficient Progressive Pruning with Numerous Teachers for  Large Language Models

Authors:
raw_author_name: Li, Shengrui
raw_author_name: Han, Xueting

Abstract: The considerable size of Large Language Models (LLMs) presents notable deployment challenges, particularly on resource-constrained hardware. Structured pruning, offers an effective means to compress LLMs, thereby reducing storage costs and enhancing inference speed for more efficient utilization. In this work, we study data-efficient and resource-efficient structure pruning methods to obtain smaller yet still powerful models. Knowledge Distillation is well-suited for pruning, as the intact model can serve as an excellent teacher for pruned students. However, it becomes challenging in the context of LLMs due to memory constraints. To address this, we propose an efficient progressive Numerous-teacher pruning method (NutePrune). NutePrune mitigates excessive memory costs by loading only one intact model and integrating it with various masks and LoRA modules, enabling it to seamlessly switch between teacher and student roles. This approach allows us to leverage numerous teachers with varying capacities to progressively guide the pruned model, enhancing overall performance. Extensive experiments across various tasks demonstrate the effectiveness of NutePrune. In LLaMA-7B zero-shot experiments, NutePrune retains 97.17% of the performance of the original model at 20% sparsity and 95.07% at 25% sparsity.
Sequential Recommendation on Temporal Proximities with Contrastive  Learning and Self-Attention

Authors:
raw_author_name: Lim, ChieHyeon

Abstract: Sequential recommender systems identify user preferences from their past interactions to predict subsequent items optimally. Although traditional deep-learning-based models and modern transformer-based models in previous studies capture unidirectional and bidirectional patterns within user-item interactions, the importance of temporal contexts, such as individual behavioral and societal trend patterns, remains underexplored. Notably, recent models often neglect similarities in users' actions that occur implicitly among users during analogous timeframes-a concept we term vertical temporal proximity. These models primarily adapt the self-attention mechanisms of the transformer to consider the temporal context in individual user actions. Meanwhile, this adaptation still remains limited in considering the horizontal temporal proximity within item interactions, like distinguishing between subsequent item purchases within a week versus a month. To address these gaps, we propose a sequential recommendation model called TemProxRec, which includes contrastive learning and self-attention methods to consider temporal proximities both across and within user-item interactions. The proposed contrastive learning method learns representations of items selected in close temporal periods across different users to be close. Simultaneously, the proposed self-attention mechanism encodes temporal and positional contexts in a user sequence using both absolute and relative embeddings. This way, our TemProxRec accurately predicts the relevant items based on the user-item interactions within a specific timeframe. We validate this work through comprehensive experiments on TemProxRec, consistently outperforming existing models on benchmark datasets as well as showing the significance of considering the vertical and horizontal temporal proximities into sequential recommendation.
Quickest Detection of False Data Injection Attack in Distributed Process  Tracking

Authors:
raw_author_name: Chattopadhyay, Arpan

Abstract: This paper addresses the problem of detecting false data injection (FDI) attacks in a distributed network without a fusion center, represented by a connected graph among multiple agent nodes. Each agent node is equipped with a sensor, and uses a Kalman consensus information filter (KCIF) to track a discrete time global process with linear dynamics and additive Gaussian noise. The state estimate of the global process at any sensor is computed from the local observation history and the information received by that agent node from its neighbors. At an unknown time, an attacker starts altering the local observation of one agent node. In the Bayesian setting where there is a known prior distribution of the attack beginning instant, we formulate a Bayesian quickest change detection (QCD) problem for FDI detection in order to minimize the mean detection delay subject to a false alarm probability constraint. While it is well-known that the optimal Bayesian QCD rule involves checking the Shriyaev's statistic against a threshold, we demonstrate how to compute the Shriyaev's statistic at each node in a recursive fashion given our non-i.i.d. observations. Next, we consider non-Bayesian QCD where the attack begins at an arbitrary and unknown time, and the detector seeks to minimize the worst case detection delay subject to a constraint on the mean time to false alarm and probability of misidentification. We use the multiple hypothesis sequential probability ratio test for attack detection and identification at each sensor. For unknown attack strategy, we use the window-limited generalized likelihood ratio (WL-GLR) algorithm to solve the QCD problem. Numerical results demonstrate the performances and trade-offs of the proposed algorithms.
System-level Impact of Non-Ideal Program-Time of Charge Trap Flash (CTF)  on Deep Neural Network

Authors:
raw_author_name: Saraswat, V.
raw_author_name: Ganguly, U.

Abstract: Learning of deep neural networks (DNN) using Resistive Processing Unit (RPU) architecture is energy-efficient as it utilizes dedicated neuromorphic hardware and stochastic computation of weight updates for in-memory computing. Charge Trap Flash (CTF) devices can implement RPU-based weight updates in DNNs. However, prior work has shown that the weight updates (V_T) in CTF-based RPU are impacted by the non-ideal program time of CTF. The non-ideal program time is affected by two factors of CTF. Firstly, the effects of the number of input pulses (N) or pulse width (pw), and secondly, the gap between successive update pulses (t_gap) used for the stochastic computation of weight updates. Therefore, the impact of this non-ideal program time must be studied for neural network training simulations. In this study, Firstly, we propose a pulse-train design compensation technique to reduce the total error caused by non-ideal program time of CTF and stochastic variance of a network. Secondly, we simulate RPU-based DNN with non-ideal program time of CTF on MNIST and Fashion-MNIST datasets. We find that for larger N (~1000), learning performance approaches the ideal (software-level) training level and, therefore, is not much impacted by the choice of t_gap used to implement RPU-based weight updates. However, for lower N (<500), learning performance depends on T_gap of the pulses. Finally, we also performed an ablation study to isolate the causal factor of the improved learning performance. We conclude that the lower noise level in the weight updates is the most likely significant factor to improve the learning performance of DNN. Thus, our study attempts to compensate for the error caused by non-ideal program time and standardize the pulse length (N) and pulse gap (t_gap) specifications for CTF-based RPUs for accurate system-level on-chip training.
Text Summarization: Taking Advantage of Structural Syntax, Term Expansion and Process Refinement

Authors:
raw_author_name: Mohamed Taybe Elhadi
raw_affiliation_string: Department of Computer Science, University of Zawia, Zawia, Libya.

Abstract: This chapter is dedicated to the development and application of an extractive summarization procedure and is a detailed account of the set of experiments performed to study the utility of applying a combined structural property of a text’s sentences and term expansion using WordNet along with a local thesaurus all combined and used in the selection of the most appropriate extractive text summarization in a particular document.&nbsp; Sentences were tagged and normalized. Next, they were subjected to the well-known Longest Common Subsequence (LCS) algorithm for the selection of the most similar subset of sentences. Calculated similarity was based on LCS of pairs of sentences making up the document. A normalized score was calculated and used to rank the sentences. A selected top subset of the most similar sentences was then tokenized to produce a set of important keywords or terms. The produced terms were further expanded into two subsets using 1) WorldNet; and 2) a local electronic dictionary/thesaurus.&nbsp; The three sets obtained (the original and the expanded two) were then re-cycled to further refine and expand the list of selected sentences from the original document. The process was repeated a number of times in order to find the best representative set of sentences. A final set of the top (best) sentences was selected as candidate sentences for summarization.&nbsp; In order to verify the utility of the procedure, a number of experiments were conducted using an email corpus. The results were compared to those produced by human annotators as well as to those results produced using basic sentences similarity calculation method. Produced results were very encouraging and compared well to those of human annotators and Jacquard sentences similarity.
Lester: rotoscope animation through video object segmentation and  tracking

Authors:
raw_author_name: Tous, Ruben

Abstract: This article introduces Lester, a novel method to automatically synthetise retro-style 2D animations from videos. The method approaches the challenge mainly as an object segmentation and tracking problem. Video frames are processed with the Segment Anything Model (SAM) and the resulting masks are tracked through subsequent frames with DeAOT, a method of hierarchical propagation for semi-supervised video object segmentation. The geometry of the masks' contours is simplified with the Douglas-Peucker algorithm. Finally, facial traits, pixelation and a basic shadow effect can be optionally added. The results show that the method exhibits an excellent temporal consistency and can correctly process videos with different poses and appearances, dynamic shots, partial shots and diverse backgrounds. The proposed method provides a more simple and deterministic approach than diffusion models based video-to-video translation pipelines, which suffer from temporal consistency problems and do not cope well with pixelated and schematic outputs. The method is also much most practical than techniques based on 3D human pose estimation, which require custom handcrafted 3D models and are very limited with respect to the type of scenes they can process.
A Quantum Approach to News Verification from the Perspective of a News  Aggregator

Authors:
raw_author_name: Andronikos, Theodore
raw_author_name: Sirokofskich, Alla

Abstract: In the dynamic landscape of digital information, the rise of misinformation and fake news presents a pressing challenge. This paper takes a completely new approach to verifying news, inspired by how quantum actors can reach agreement even when they are spatially spread out. We propose a radically new, to the best of our knowledge, algorithm that uses quantum ``entanglement'' (think of it as a special connection) to help news aggregators sniff out bad actors, whether they be other news sources or even fact-checkers trying to spread misinformation. This algorithm doesn't rely on quantum signatures, it just uses basic quantum technology we already have, in particular, special pairs of particles called ``EPR pairs'' that are much easier to create than other options. More complex entangled states are like juggling too many balls - they're hard to make and slow things down, especially when many players are involved. For instance, bigger, more complex states like ``GHZ states'' work for small groups, but they become messy with larger numbers. So, we stick with Bell states, the simplest form of entanglement, which are easy to generate no matter how many players are in the game. This means our algorithm is faster to set up, works for any number of participants, and is more practical for real-world use. Bonus points: it finishes in a fixed number of steps, regardless of how many players are involved, making it even more scalable. This new approach may lead to a powerful and efficient way to fight misinformation in the digital age, using the weird and wonderful world of quantum mechanics.
OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large  Language Models

Authors:
raw_author_name: AhmadiTeshnizi, Ali
raw_author_name: Gao, Wenzhi
raw_author_name: Udell, Madeleine

Abstract: Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than optimally by state-of-the-art solvers because the expertise required to formulate and solve these problems limits the widespread adoption of optimization tools and techniques. This paper introduces OptiMUS, a Large Language Model (LLM)-based agent designed to formulate and solve (mixed integer) linear programming problems from their natural language descriptions. OptiMUS can develop mathematical models, write and debug solver code, evaluate the generated solutions, and improve its model and code based on these evaluations. OptiMUS utilizes a modular structure to process problems, allowing it to handle problems with long descriptions and complex data without long prompts. Experiments demonstrate that OptiMUS outperforms existing state-of-the-art methods on easy datasets by more than $20\%$ and on hard datasets (including a new dataset, NLP4LP, released with this paper that features long and complex problems) by more than $30\%$.
Bridging the Empirical-Theoretical Gap in Neural Network Formal Language  Learning Using Minimum Description Length

Authors:
raw_author_name: Lan, Nur
raw_author_name: Chemla, Emmanuel
raw_author_name: Katzir, Roni

Abstract: Neural networks offer good approximation to many tasks but consistently fail to reach perfect generalization, even when theoretical work shows that such perfect solutions can be expressed by certain architectures. Using the task of formal language learning, we focus on one simple formal language and show that the theoretically correct solution is in fact not an optimum of commonly used objectives -- even with regularization techniques that according to common wisdom should lead to simple weights and good generalization (L1, L2) or other meta-heuristics (early-stopping, dropout). However, replacing standard targets with the Minimum Description Length objective (MDL) results in the correct solution being an optimum.
Towards Safer Large Language Models through Machine Unlearning

Authors:
raw_author_name: Liu, Zheyuan
raw_author_name: Tan, Zhaoxuan
raw_author_name: Tian, Yijun
raw_author_name: Jiang, Meng

Abstract: The rapid advancement of Large Language Models (LLMs) has demonstrated their vast potential across various domains, attributed to their extensive pretraining knowledge and exceptional generalizability. However, LLMs often encounter challenges in generating harmful content when faced with problematic prompts. To address this problem, existing work attempted to implement a gradient ascent based approach to prevent LLMs from producing harmful output. While these methods can be effective, they frequently impact the model utility in responding to normal prompts. To address this gap, we introduce Selective Knowledge negation Unlearning (SKU), a novel unlearning framework for LLMs, designed to eliminate harmful knowledge while preserving utility on normal prompts. Specifically, SKU is consisted of two stages: harmful knowledge acquisition stage and knowledge negation stage. The first stage aims to identify and acquire harmful knowledge within the model, whereas the second is dedicated to remove this knowledge. SKU selectively isolates and removes harmful knowledge in model parameters, ensuring the model's performance remains robust on normal prompts. Our experiments conducted across various LLM architectures demonstrate that SKU identifies a good balance point between removing harmful information and preserving utility.
Few-Shot Object Detection with Sparse Context Transformers

Authors:
raw_author_name: Mei, Jie
raw_author_name: Jiu, Mingyuan
raw_author_name: Sahbi, Hichem
raw_author_name: Jiang, Xiaoheng
raw_author_name: Xu, Mingliang

Abstract: Few-shot detection is a major task in pattern recognition which seeks to localize objects using models trained with few labeled data. One of the mainstream few-shot methods is transfer learning which consists in pretraining a detection model in a source domain prior to its fine-tuning in a target domain. However, it is challenging for fine-tuned models to effectively identify new classes in the target domain, particularly when the underlying labeled training data are scarce. In this paper, we devise a novel sparse context transformer (SCT) that effectively leverages object knowledge in the source domain, and automatically learns a sparse context from only few training images in the target domain. As a result, it combines different relevant clues in order to enhance the discrimination power of the learned detectors and reduce class confusion. We evaluate the proposed method on two challenging few-shot object detection benchmarks, and empirical results show that the proposed method obtains competitive performance compared to the related state-of-the-art.
Self-Augmented In-Context Learning for Unsupervised Word Translation

Authors:
raw_author_name: Li, Yaoyiran
raw_author_name: Korhonen, Anna
raw_author_name: Vulić, Ivan

Abstract: Recent work has shown that, while large language models (LLMs) demonstrate strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, they still cannot match the performance of 'traditional' mapping-based approaches in the unsupervised scenario where no seed translation pairs are available, especially for lower-resource languages. To address this challenge with LLMs, we propose self-augmented in-context learning (SAIL) for unsupervised BLI: starting from a zero-shot prompt, SAIL iteratively induces a set of high-confidence word translation pairs for in-context learning (ICL) from an LLM, which it then reapplies to the same LLM in the ICL fashion. Our method shows substantial gains over zero-shot prompting of LLMs on two established BLI benchmarks spanning a wide range of language pairs, also outperforming mapping-based baselines across the board. In addition to achieving state-of-the-art unsupervised BLI performance, we also conduct comprehensive analyses on SAIL and discuss its limitations.
Benchmarking federated strategies in Peer-to-Peer Federated learning for  biomedical data

Authors:
raw_author_name: Salmeron, Jose L.
raw_author_name: Arévalo, Irina
raw_author_name: Ruiz-Celma, Antonio

Abstract: The increasing requirements for data protection and privacy has attracted a huge research interest on distributed artificial intelligence and specifically on federated learning, an emerging machine learning approach that allows the construction of a model between several participants who hold their own private data. In the initial proposal of federated learning the architecture was centralised and the aggregation was done with federated averaging, meaning that a central server will orchestrate the federation using the most straightforward averaging strategy. This research is focused on testing different federated strategies in a peer-to-peer environment. The authors propose various aggregation strategies for federated learning, including weighted averaging aggregation, using different factors and strategies based on participant contribution. The strategies are tested with varying data sizes to identify the most robust ones. This research tests the strategies with several biomedical datasets and the results of the experiments show that the accuracy-based weighted average outperforms the classical federated averaging method.