Reviewerly/exHarmony huggingface dataset:
Demystifying Embedding Spaces using Large Language Models

Embeddings have become a pivotal means to represent complex, multi-faceted information about entities, concepts, and relationships in a condensed and useful format. Nevertheless, they often preclude direct interpretation. While downstream tasks make use of these compressed representations, meaningful interpretation usually requires visualization using dimensionality reduction or specialized machine learning interpretability methods. This paper addresses the challenge of making such embeddings more interpretable and broadly useful, by employing Large Language Models (LLMs) to directly interact with embeddings -- transforming abstract vectors into understandable narratives. By injecting embeddings into LLMs, we enable querying and exploration of complex embedding data. We demonstrate our approach on a variety of diverse tasks, including: enhancing concept activation vectors (CAVs), communicating novel embedded entities, and decoding user preferences in recommender systems. Our work couples the immense information potential of embeddings with the interpretative power of LLMs.
A School-Based Program Focused on Initial Education Teachers to Strengthen Cognitive and Socio-Emotional Skills in Children

This study describes a Program to Strengthen Cognitive and Socio-Emotional Skills (PCSES) applied by teachers to their initial-level students during the COVID-19 pandemic. This work has two objectives. The first is to explore the grade of acceptance of teachers regarding the training process on cognitive and socio-emotional skills carried out by our team during the COVID-19 pandemic period, according to the school context, urban vs. urban-marginal. The second is to analyse whether teachers perceived improvements in the cognitive and socio-emotional performance of their students after the implementation of the PCSE, according to the school context. The sample consists of 36 initial-level teachers, in charge of classrooms of 4-year-olds and 5-year-olds. During the pandemic, 2020-2021, 36 teachers responded to the survey: 20 teachers from urban schools, and 16 teachers from urban-marginalized schools located in socially-vulnerable sectors. Despite the difficulties experienced during the pandemic, 94.45% of the teachers indicated that they were satisfied with the training process received. In addition, teachers from both school contexts perceived improvements in the cognitive and socio-emotional performance of their students after the implementation of the PCSES. These results suggest that the training of initial education teachers made it possible to implement a stimulation program during the pandemic period that aimed to favor cognitive, emotional, and social functions in students. The main contribution of this work was to increase the evidence of the importance of teacher training and knowledge for the development of strategies that can be used to scaffold socio-cognitive and emotional processes in the context of teaching-learning.
Chaos-based Image Encryption in Embedded Systems using Lorenz-Rossler System

Digital data is increasing rapidly in the world day by day. Information security is important during data exchange over the Internet. The way to securely transmit images over the network is through the image encryption technique. In the proposed cryptography system, the hybridization of Lorenz-Rossler chaotic systems is used, and a random number sequence is generated. The security analyses such as histogram, correlation, differential attack, information entropy, and duration analysis of the study are performed. It is seen that the proposed system performs well, especially in terms of correlation. Additionally, the performance of the developed embedded system platforms is compared after testing on Nvidia Jetson Nano and Xilinx PYNQ Z1 boards. The Nvidia Jetson Nano board is more performant than the Xilinx PYNQ Z1 board. The safety and feasibility of the proposed system have been demonstrated.
DISCOVER: Making Vision Networks Interpretable via Competition and Dissection

Modern deep networks are highly complex and their inferential outcome very hard to interpret. This is a serious obstacle to their transparent deployment in safety-critical or bias-aware applications. This work contributes to post-hoc interpretability, and specifically Network Dissection. Our goal is to present a framework that makes it easier to discover the individual functionality of each neuron in a network trained on a vision task; discovery is performed in terms of textual description generation. To achieve this objective, we leverage: (i) recent advances in multimodal vision-text models and (ii) network layers founded upon the novel concept of stochastic local competition between linear units. In this setting, only a small subset of layer neurons are activated for a given input, leading to extremely high activation sparsity (as low as only $\approx 4\%$). Crucially, our proposed method infers (sparse) neuron activation patterns that enables the neurons to activate/specialize to inputs with specific characteristics, diversifying their individual functionality. This capacity of our method supercharges the potential of dissection processes: human understandable descriptions are generated only for the very few active neurons, thus facilitating the direct investigation of the network's decision process. As we experimentally show, our approach: (i) yields Vision Networks that retain or improve classification performance, and (ii) realizes a principled framework for text-based description and examination of the generated neuronal representations.
Digital Twin-assisted Fuzzy Logic-inspired Intelligent Approach for Flood Prediction

Natural hazards causing catastrophic damage and infrastructure destruction have increased in recent decades, with floods being a serious problem that leads to crop damage, population loss, infrastructure degradation, and public service collapse. Digital Twin (DT) technology is a promising solution for alerting communities of oncoming floods and providing sufficient time for evacuation and property protection. This research introduces a digital twin-inspired intelligent framework that analyzes hydrological and meteorological parameters causing floods, validated using data from the Indian Meteorological Department (IMD). Artificial intelligence (AI) algorithms improve situational analysis and decision-making for flood forecasting, while advanced blockchain security features keep recorded and analyzed data secure. A case study demonstrates the proposed approach’s efficacy in smart catastrophe management with the best training and testing accuracy of 97.23% and 95.58%, respectively.
Bridging the Gap between Human Motion and Action Semantics via Kinematic Phrases

The goal of motion understanding is to establish a reliable mapping between motion and action semantics, while it is a challenging many-to-many problem. An abstract action semantic (i.e., walk forwards) could be conveyed by perceptually diverse motions (walk with arms up or swinging), while a motion could carry different semantics w.r.t. its context and intention. This makes an elegant mapping between them difficult. Previous attempts adopted direct-mapping paradigms with limited reliability. Also, current automatic metrics fail to provide reliable assessments of the consistency between motions and action semantics. We identify the source of these problems as the significant gap between the two modalities. To alleviate this gap, we propose Kinematic Phrases (KP) that take the objective kinematic facts of human motion with proper abstraction, interpretability, and generality characteristics. Based on KP as a mediator, we can unify a motion knowledge base and build a motion understanding system. Meanwhile, KP can be automatically converted from motions and to text descriptions with no subjective bias, inspiring Kinematic Prompt Generation (KPG) as a novel automatic motion generation benchmark. In extensive experiments, our approach shows superiority over other methods. Our code and data would be made publicly available at https://foruck.github.io/KP.
Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference

We propose a method to improve the efficiency and accuracy of amortized Bayesian inference by leveraging universal symmetries in the joint probabilistic model of parameters and data. In a nutshell, we invert Bayes' theorem and estimate the marginal likelihood based on approximate representations of the joint model. Upon perfect approximation, the marginal likelihood is constant across all parameter values by definition. However, errors in approximate inference lead to undesirable variance in the marginal likelihood estimates across different parameter values. We penalize violations of this symmetry with a \textit{self-consistency loss} which significantly improves the quality of approximate inference in low data regimes and can be used to augment the training of popular neural density estimators. We apply our method to a number of synthetic problems and realistic scientific models, discovering notable advantages in the context of both neural posterior and likelihood approximation.
On Double Descent in Reinforcement Learning with LSTD and Random Features

Temporal Difference (TD) algorithms are widely used in Deep Reinforcement Learning (RL). Their performance is heavily influenced by the size of the neural network. While in supervised learning, the regime of over-parameterization and its benefits are well understood, the situation in RL is much less clear. In this paper, we present a theoretical analysis of the influence of network size and $l_2$-regularization on performance. We identify the ratio between the number of parameters and the number of visited states as a crucial factor and define over-parameterization as the regime when it is larger than one. Furthermore, we observe a double descent phenomenon, i.e., a sudden drop in performance around the parameter/state ratio of one. Leveraging random features and the lazy training regime, we study the regularized Least-Square Temporal Difference (LSTD) algorithm in an asymptotic regime, as both the number of parameters and states go to infinity, maintaining a constant ratio. We derive deterministic limits of both the empirical and the true Mean-Squared Bellman Error (MSBE) that feature correction terms responsible for the double descent. Correction terms vanish when the $l_2$-regularization is increased or the number of unvisited states goes to zero. Numerical experiments with synthetic and small real-world environments closely match the theoretical predictions.
BRAINTEASER: Lateral Thinking Puzzles for Large Language Models

The success of language models has inspired the NLP community to attend to tasks that require implicit and complex reasoning, relying on human-like commonsense mechanisms. While such vertical thinking tasks have been relatively popular, lateral thinking puzzles have received little attention. To bridge this gap, we devise BRAINTEASER: a multiple-choice Question Answering task designed to test the model's ability to exhibit lateral thinking and defy default commonsense associations. We design a three-step procedure for creating the first lateral thinking benchmark, consisting of data collection, distractor generation, and generation of adversarial examples, leading to 1,100 puzzles with high-quality annotations. To assess the consistency of lateral reasoning by models, we enrich BRAINTEASER based on a semantic and contextual reconstruction of its questions. Our experiments with state-of-the-art instruction- and commonsense language models reveal a significant gap between human and model performance, which is further widened when consistency across adversarial formats is considered. We make all of our code and data available to stimulate work on developing and evaluating lateral thinking models.
Revisiting Large Language Models as Zero-shot Relation Extractors

Relation extraction (RE) consistently involves a certain degree of labeled or unlabeled data even if under zero-shot setting. Recent studies have shown that large language models (LLMs) transfer well to new tasks out-of-the-box simply given a natural language prompt, which provides the possibility of extracting relations from text without any data and parameter tuning. This work focuses on the study of exploring LLMs, such as ChatGPT, as zero-shot relation extractors. On the one hand, we analyze the drawbacks of existing RE prompts and attempt to incorporate recent prompt techniques such as chain-of-thought (CoT) to improve zero-shot RE. We propose the summarize-and-ask (\textsc{SumAsk}) prompting, a simple prompt recursively using LLMs to transform RE inputs to the effective question answering (QA) format. On the other hand, we conduct comprehensive experiments on various benchmarks and settings to investigate the capabilities of LLMs on zero-shot RE. Specifically, we have the following findings: (i) \textsc{SumAsk} consistently and significantly improves LLMs performance on different model sizes, benchmarks and settings; (ii) Zero-shot prompting with ChatGPT achieves competitive or superior results compared with zero-shot and fully supervised methods; (iii) LLMs deliver promising performance in extracting overlapping relations; (iv) The performance varies greatly regarding different relations. Different from small language models, LLMs are effective in handling challenge none-of-the-above (NoTA) relation.
FedFed: Feature Distillation against Data Heterogeneity in Federated Learning

Federated learning (FL) typically faces data heterogeneity, i.e., distribution shifting among clients. Sharing clients' information has shown great potentiality in mitigating data heterogeneity, yet incurs a dilemma in preserving privacy and promoting model performance. To alleviate the dilemma, we raise a fundamental question: \textit{Is it possible to share partial features in the data to tackle data heterogeneity?} In this work, we give an affirmative answer to this question by proposing a novel approach called {\textbf{Fed}erated \textbf{Fe}ature \textbf{d}istillation} (FedFed). Specifically, FedFed partitions data into performance-sensitive features (i.e., greatly contributing to model performance) and performance-robust features (i.e., limitedly contributing to model performance). The performance-sensitive features are globally shared to mitigate data heterogeneity, while the performance-robust features are kept locally. FedFed enables clients to train models over local and shared data. Comprehensive experiments demonstrate the efficacy of FedFed in promoting model performance.
Exploiting Unfair Advantages: Investigating Opportunistic Trading in the NFT Market

As cryptocurrency evolved, new financial instruments, such as lending and borrowing protocols, currency exchanges, fungible and non-fungible tokens (NFT), staking and mining protocols have emerged. A financial ecosystem built on top of a blockchain is supposed to be fair and transparent for each participating actor. Yet, there are sophisticated actors who turn their domain knowledge and market inefficiencies to their strategic advantage; thus extracting value from trades not accessible to others. This situation is further exacerbated by the fact that blockchain-based markets and decentralized finance (DeFi) instruments are mostly unregulated. Though a large body of work has already studied the unfairness of different aspects of DeFi and cryptocurrency trading, the economic intricacies of non-fungible token (NFT) trades necessitate further analysis and academic scrutiny. The trading volume of NFTs has skyrocketed in recent years. A single NFT trade worth over a million US dollars, or marketplaces making billions in revenue is not uncommon nowadays. While previous research indicated the presence of wrongdoings in the NFT market, to our knowledge, we are the first to study predatory trading practices, what we call opportunistic trading, in depth. Opportunistic traders are sophisticated actors who employ automated, high-frequency NFT trading strategies, which, oftentimes, are malicious, deceptive, or, at the very least, unfair. Such attackers weaponize their advanced technical knowledge and superior understanding of DeFi protocols to disrupt trades of unsuspecting users, and collect profits from economic situations that are inaccessible to ordinary users, in a "supposedly" fair market. In this paper, we explore three such broad classes of opportunistic strategies aiming to realize three distinct trading objectives, viz., acquire, instant profit generation, and loss minimization.
Improving Contrastive Learning of Sentence Embeddings with Focal-InfoNCE

The recent success of SimCSE has greatly advanced state-of-the-art sentence representations. However, the original formulation of SimCSE does not fully exploit the potential of hard negative samples in contrastive learning. This study introduces an unsupervised contrastive learning framework that combines SimCSE with hard negative mining, aiming to enhance the quality of sentence embeddings. The proposed focal-InfoNCE function introduces self-paced modulation terms in the contrastive objective, downweighting the loss associated with easy negatives and encouraging the model focusing on hard negatives. Experimentation on various STS benchmarks shows that our method improves sentence embeddings in terms of Spearman's correlation and representation alignment and uniformity.
Monsters in the Dark: Sanitizing Hidden Threats with Diffusion Models

Steganography is the art of hiding information in plain sight. This form of covert communication can be used by bad actors to propagate malware, exfiltrate victim data, and communicate with other bad actors. Current image steganography defenses rely upon steganalysis, or the detection of hidden messages. These methods, however, are non-blind as they require information about known steganography techniques and are easily bypassed. Recent work has instead focused on a defense mechanism known as sanitization, which eliminates hidden information from images. In this work, we introduce a novel blind deep learning steganography sanitization method that utilizes a diffusion model framework to sanitize universal and dependent steganography (DM-SUDS), which both sanitizes and preserves image quality. We evaluate this approach against state-of-the-art deep learning sanitization frameworks and provide further detailed analysis through an ablation study. DM-SUDS outperforms previous sanitization methods and improves image preservation MSE by 71.32%, PSNR by 22.43% and SSIM by 17.30%. This is the first blind deep learning image sanitization framework to meet these image quality results.
Answer Candidate Type Selection: Text-to-Text Language Model for Closed Book Question Answering Meets Knowledge Graphs

Pre-trained Text-to-Text Language Models (LMs), such as T5 or BART yield promising results in the Knowledge Graph Question Answering (KGQA) task. However, the capacity of the models is limited and the quality decreases for questions with less popular entities. In this paper, we present a novel approach which works on top of the pre-trained Text-to-Text QA system to address this issue. Our simple yet effective method performs filtering and re-ranking of generated candidates based on their types derived from Wikidata "instance_of" property.
Secure Decentralized Learning with Blockchain

Federated Learning (FL) is a well-known paradigm of distributed machine learning on mobile and IoT devices, which preserves data privacy and optimizes communication efficiency. To avoid the single point of failure problem in FL, decentralized federated learning (DFL) has been proposed to use peer-to-peer communication for model aggregation, which has been considered an attractive solution for machine learning tasks on distributed personal devices. However, this process is vulnerable to attackers who share false models and data. If there exists a group of malicious clients, they might harm the performance of the model by carrying out a poisoning attack. In addition, in DFL, clients often lack the incentives to contribute their computing powers to do model training. In this paper, we proposed Blockchain-based Decentralized Federated Learning (BDFL), which leverages a blockchain for decentralized model verification and auditing. BDFL includes an auditor committee for model verification, an incentive mechanism to encourage the participation of clients, a reputation model to evaluate the trustworthiness of clients, and a protocol suite for dynamic network updates. Evaluation results show that, with the reputation mechanism, BDFL achieves fast model convergence and high accuracy on real datasets even if there exist 30\% malicious clients in the system.
Why bother with geometry? On the relevance of linear decompositions of Transformer embeddings

A recent body of work has demonstrated that Transformer embeddings can be linearly decomposed into well-defined sums of factors, that can in turn be related to specific network inputs or components. There is however still a dearth of work studying whether these mathematical reformulations are empirically meaningful. In the present work, we study representations from machine-translation decoders using two of such embedding decomposition methods. Our results indicate that, while decomposition-derived indicators effectively correlate with model performance, variation across different runs suggests a more nuanced take on this question. The high variability of our measurements indicate that geometry reflects model-specific characteristics more than it does sentence-specific computations, and that similar training conditions do not guarantee similar vector spaces.
SparseCoder: Advancing Source Code Analysis with Sparse Attention and Learned Token Pruning

As software projects rapidly evolve, software artifacts become more complex and defects behind get harder to identify. The emerging Transformer-based approaches, though achieving remarkable performance, struggle with long code sequences due to their self-attention mechanism, which scales quadratically with the sequence length. This paper introduces SparseCoder, an innovative approach incorporating sparse attention and learned token pruning (LTP) method (adapted from natural language processing) to address this limitation. Extensive experiments carried out on a large-scale dataset for vulnerability detection demonstrate the effectiveness and efficiency of SparseCoder, scaling from quadratically to linearly on long code sequence analysis in comparison to CodeBERT and RoBERTa. We further achieve 50% FLOPs reduction with a negligible performance drop of less than 1% comparing to Transformer leveraging sparse attention. Moverover, SparseCoder goes beyond making "black-box" decisions by elucidating the rationale behind those decisions. Code segments that contribute to the final decision can be highlighted with importance scores, offering an interpretable, transparent analysis tool for the software engineering landscape.
Heisenberg-scaling Enhancement in Quantum Parameter Estimation with Indefinite Time Direction

The manipulation and metrology of light beams are pivotal for optical science and applications. In particular, achieving ultra-high precision in the measurement of light beam rotations has been a long-standing challenge. Instead of utilizing quantum probes like entangled photons, we address this challenge by incorporating a quantum strategy called "indefinite time direction" into the parameterizing process of quantum parameter estimation. Leveraging this quantum property of the parameterizing dynamics allows us to maximize the utilization of OAM resources for measuring ultra-small angular rotations of beam profile. Notably, a nrad-scale precision of light rotation measurement is finally achieved in the experiment, which is the highest precision by far to our best knowledge. Furthermore, this scheme holds promise in various optical applications due to the diverse range of manipulable resources offered by photons.