Reviewerly/exHarmony huggingface dataset:
Why are hyperbolic neural networks effective? A study on hierarchical  representation capability

Authors:
raw_author_name: Tan, Shicheng
raw_author_name: Zhao, Huanjing
raw_author_name: Zhao, Shu
raw_author_name: Zhang, Yanping

Abstract: Hyperbolic Neural Networks (HNNs), operating in hyperbolic space, have been widely applied in recent years, motivated by the existence of an optimal embedding in hyperbolic space that can preserve data hierarchical relationships (termed Hierarchical Representation Capability, HRC) more accurately than Euclidean space. However, there is no evidence to suggest that HNNs can achieve this theoretical optimal embedding, leading to much research being built on flawed motivations. In this paper, we propose a benchmark for evaluating HRC and conduct a comprehensive analysis of why HNNs are effective through large-scale experiments. Inspired by the analysis results, we propose several pre-training strategies to enhance HRC and improve the performance of downstream tasks, further validating the reliability of the analysis. Experiments show that HNNs cannot achieve the theoretical optimal embedding. The HRC is significantly affected by the optimization objectives and hierarchical structures, and enhancing HRC through pre-training strategies can significantly improve the performance of HNNs.
A Survey on Blockchain in E-Government Services: Status and Challenges

Authors:
raw_author_name: Salama, May

Abstract: Blockchain technology is referred to as a very secure decentralized, distributed ledger that records the history of any digital asset. It is being used in numerous governmental and private sector organizations across numerous nations. Surveying the current state of blockchain applications and difficulties in e-government services is the goal of this review. Held to the account are use cases for current facilities that use blockchain. Finally, it examines the research gap in blockchain deployment and makes suggestions for future work for additional research.
Quantum Secret Sharing Enhanced: Utilizing W States for Anonymous and  Secure Communication

Authors:
raw_author_name: Li, Guo-Dong
raw_author_name: Wang, Qing-Le
raw_author_name: Mao, Ying
raw_author_name: Jia, Heng-Yue

Abstract: Quantum secret sharing (QSS) is the result of merging the principles of quantum mechanics with secret information sharing. It enables a sender to share a secret among receivers, and the receivers can then collectively recover the secret when the need arises. To enhance the practicality of these quantum protocols, an innovative concept of quantum anonymous secret sharing (QASS) is advanced. In this paper, we propose a QASS protocol via W states, which can share secrets while ensuring recover-ability, recover-security, and recover-anonymity. We have rigorously evaluated our protocols, verifying their accuracy and fortifying their security against scenarios involving the active adversary. This includes considerations for dishonest receivers and non-receivers. Moreover, acknowledging the imperfections inherent in real-world communication channels, we have also undertaken an exhaustive analysis of our protocol's security and effectiveness in a quantum network where some form of noise is present. Our investigations reveal that W states exhibit good performance in mitigating noise interference, making them apt for practical applications.
eXplainable Bayesian Multi-Perspective Generative Retrieval

Authors:
raw_author_name: Song, EuiYul
raw_author_name: Oh, Philhoon
raw_author_name: Thorne, James

Abstract: Modern deterministic retrieval pipelines prioritize achieving state-of-the-art performance but often lack interpretability in decision-making. These models face challenges in assessing uncertainty, leading to overconfident predictions. To overcome these limitations, we integrate uncertainty calibration and interpretability into a retrieval pipeline. Specifically, we introduce Bayesian methodologies and multi-perspective retrieval to calibrate uncertainty within a retrieval pipeline. We incorporate techniques such as LIME and SHAP to analyze the behavior of a black-box reranker model. The importance scores derived from these explanation methodologies serve as supplementary relevance scores to enhance the base reranker model. We evaluate the resulting performance enhancements achieved through uncertainty calibration and interpretable reranking on Question Answering and Fact Checking tasks. Our methods demonstrate substantial performance improvements across three KILT datasets.
Quantum teleportation based on the elegant joint measurement

Authors:
raw_author_name: Ding, Dong
raw_author_name: He, Ying-Qiu
raw_author_name: Gao, Ting
raw_author_name: Yan, Feng-Li

Abstract: As a generalization of the well-known Bell state measurement (BSM), the elegant joint measurement (EJM) is a kind of novel two-qubit joint measurement, parameterized by a subtle phase factor $\theta \in [0,\pi/2]$. We explore quantum teleportation based on the EJM, inspired by Gisin's idea that quantum entanglement not only provides quantum channel and also quantum joint measurement for quantum teleportation. It is a probabilistic teleportation caused by undesired nonunitary quantum evolution. There are two interesting features in the present scenario. First, it goes beyond the conventional teleportation scenario, which can be included in the present scenario. Second, different from the BSM being single input and four outcomes, it can provide an adjustable input setting or even multiple measurement settings for the sender (or the controller). Moreover, we show in detail the feasible quantum circuits to realize the present scenario, where a few unitary operations and a nonunitary quantum gate are being utilized.
VM-UNet: Vision Mamba UNet for Medical Image Segmentation

Authors:
raw_author_name: Xiang, Suncheng

Abstract: In the realm of medical image segmentation, both CNN-based and Transformer-based models have been extensively explored. However, CNNs exhibit limitations in long-range modeling capabilities, whereas Transformers are hampered by their quadratic computational complexity. Recently, State Space Models (SSMs), exemplified by Mamba, have emerged as a promising approach. They not only excel in modeling long-range interactions but also maintain a linear computational complexity. In this paper, leveraging state space models, we propose a U-shape architecture model for medical image segmentation, named Vision Mamba UNet (VM-UNet). Specifically, the Visual State Space (VSS) block is introduced as the foundation block to capture extensive contextual information, and an asymmetrical encoder-decoder structure is constructed. We conduct comprehensive experiments on the ISIC17, ISIC18, and Synapse datasets, and the results indicate that VM-UNet performs competitively in medical image segmentation tasks. To our best knowledge, this is the first medical image segmentation model constructed based on the pure SSM-based model. We aim to establish a baseline and provide valuable insights for the future development of more efficient and effective SSM-based segmentation systems. Our code is available at https://github.com/JCruan519/VM-UNet.
Weisfeiler Leman for Euclidean Equivariant Machine Learning

Authors:
raw_author_name: Dym, Nadav

Abstract: The $k$-Weifeiler-Leman ($k$-WL) graph isomorphism test hierarchy is a common method for assessing the expressive power of graph neural networks (GNNs). Recently, the $2$-WL test was proven to be complete on weighted graphs which encode $3\mathrm{D}$ point cloud data. Consequently, GNNs whose expressive power is equivalent to the $2$-WL test are provably universal on point clouds. Yet, this result is limited to invariant continuous functions on point clouds. In this paper we extend this result in three ways: Firstly, we show that $2$-WL tests can be extended to point clouds which include both positions and velocity, a scenario often encountered in applications. Secondly, we show that PPGN (Maron et al., 2019) can simulate $2$-WL uniformly on all point clouds with low complexity. Finally, we show that a simple modification of this PPGN architecture can be used to obtain a universal equivariant architecture that can approximate all continuous equivariant functions uniformly. Building on our results, we develop our WeLNet architecture, which can process position-velocity pairs, compute functions fully equivariant to permutations and rigid motions, and is provably complete and universal. Remarkably, WeLNet is provably complete precisely in the setting in which it is implemented in practice. Our theoretical results are complemented by experiments showing WeLNet sets new state-of-the-art results on the N-Body dynamics task and the GEOM-QM9 molecular conformation generation task.
Integrated Magneto-photonic Non-Volatile Multi-Bit Memory

Authors:
raw_author_name: Heck, Martijn
raw_author_name: Koopmans, Bert

Abstract: We present an integrated magneto-photonic device for all-optical switching of non-volatile multi-bit spintronic memory. The bits are based on stand-alone magneto-tunnel junctions which are perpendicularly magnetized with all-optically switchable free layers, coupled onto photonic crystal nanobeam cavities on an indium phosphide based platform. This device enables switching of the magnetization state of the bits by locally increasing the power absorption of light at resonance with the cavity. We design an add/drop network of cavities to grant random access to multiple bits via a wavelength-division multiplexing scheme. Based on a three-dimensional finite-difference time-domain method, we numerically illustrate a compact device capable of switching and accessing 8 bits in different cavities with a 5 nm wavelength spacing in the conventional (C) telecommunication band. Our multi-bit device holds promise as a new paradigm for developing an ultrafast photonically-addressable spintronic memory and may also empower novel opportunities for photonically-driven spintronic-based neuromorphic computing.
Quantum reversal: a general theory of coherent quantum absorbers

Authors:
raw_author_name: Tsang, Mankei

Abstract: The fascinating concept of coherent quantum absorber -- which can absorb any photon emitted by another system while maintaining entanglement with that system -- has found diverse implications in open quantum system theory and quantum metrology. This work generalizes the concept by proposing the so-called reversal conditions for the two systems, in which a "reverser" coherently reverses any effect of the other system on a field. The reversal conditions are rigorously boiled down to concise formulas involving the Petz recovery map and Kraus operators, thereby generalizing as well as streamlining the existing treatments of coherent absorbers.
Neur2BiLO: Neural Bilevel Optimization

Authors:
raw_author_name: Dumouchelle, Justin
raw_author_name: Julien, Esther
raw_author_name: Kurtz, Jannis
raw_author_name: Khalil, Elias B.

Abstract: Bilevel optimization deals with nested problems in which a leader takes the first decision to minimize their objective function while accounting for a follower's best-response reaction. Constrained bilevel problems with integer variables are particularly notorious for their hardness. While exact solvers have been proposed for mixed-integer linear bilevel optimization, they tend to scale poorly with problem size and are hard to generalize to the non-linear case. On the other hand, problem-specific algorithms (exact and heuristic) are limited in scope. Under a data-driven setting in which similar instances of a bilevel problem are solved routinely, our proposed framework, Neur2BiLO, embeds a neural network approximation of the leader's or follower's value function, trained via supervised regression, into an easy-to-solve mixed-integer program. Neur2BiLO serves as a heuristic that produces high-quality solutions extremely fast for the bilevel knapsack interdiction problem, the "critical node game" from network security, a donor-recipient healthcare problem, and discrete network design from transportation planning. These problems are diverse in that they have linear or non-linear objectives/constraints and integer or mixed-integer variables, making Neur2BiLO unique in its versatility.
Are Large Language Models Table-based Fact-Checkers?

Authors:
raw_author_name: Si, Qingyi
raw_author_name: Fu, Peng
raw_author_name: Lin, Zheng
raw_author_name: Wang, Weiping

Abstract: Table-based Fact Verification (TFV) aims to extract the entailment relation between statements and structured tables. Existing TFV methods based on small-scaled models suffer from insufficient labeled data and weak zero-shot ability. Recently, the appearance of Large Language Models (LLMs) has gained lots of attraction in research fields. They have shown powerful zero-shot and in-context learning abilities on several NLP tasks, but their potential on TFV is still unknown. In this work, we implement a preliminary study about whether LLMs are table-based fact-checkers. In detail, we design diverse prompts to explore how the in-context learning can help LLMs in TFV, i.e., zero-shot and few-shot TFV capability. Besides, we carefully design and construct TFV instructions to study the performance gain brought by the instruction tuning of LLMs. Experimental results demonstrate that LLMs can achieve acceptable results on zero-shot and few-shot TFV with prompt engineering, while instruction-tuning can stimulate the TFV capability significantly. We also make some valuable findings about the format of zero-shot prompts and the number of in-context examples. Finally, we analyze some possible directions to promote the accuracy of TFV via LLMs, which is beneficial to further research of table reasoning.
DeSparsify: Adversarial Attack Against Token Sparsification Mechanisms  in Vision Transformers

Authors:
raw_author_name: Zolfi, Alon
raw_author_name: Baras, Amit
raw_author_name: Elovici, Yuval
raw_author_name: Shabtai, Asaf

Abstract: Vision transformers have contributed greatly to advancements in the computer vision domain, demonstrating state-of-the-art performance in diverse tasks (e.g., image classification, object detection). However, their high computational requirements grow quadratically with the number of tokens used. Token sparsification techniques have been proposed to address this issue. These techniques employ an input-dependent strategy, in which uninformative tokens are discarded from the computation pipeline, improving the model's efficiency. However, their dynamism and average-case assumption makes them vulnerable to a new threat vector - carefully crafted adversarial examples capable of fooling the sparsification mechanism, resulting in worst-case performance. In this paper, we present DeSparsify, an attack targeting the availability of vision transformers that use token sparsification mechanisms. The attack aims to exhaust the operating system's resources, while maintaining its stealthiness. Our evaluation demonstrates the attack's effectiveness on three token sparsification techniques and examines the attack's transferability between them and its effect on the GPU resources. To mitigate the impact of the attack, we propose various countermeasures.
"What's my model inside of?": Exploring the role of environments for  grounded natural language understanding

Authors:
raw_author_name: Tamari, Ronen

Abstract: In contrast to classical cognitive science which studied brains in isolation, ecological approaches focused on the role of the body and environment in shaping cognition. Similarly, in this thesis we adopt an ecological approach to grounded natural language understanding (NLU) research. Grounded language understanding studies language understanding systems situated in the context of events, actions and precepts in naturalistic/simulated virtual environments. Where classic research tends to focus on designing new models and optimization methods while treating environments as given, we explore the potential of environment design for improving data collection and model development. We developed novel training and annotation approaches for procedural text understanding based on text-based game environments. We also drew upon embodied cognitive linguistics literature to propose a roadmap for grounded NLP research, and to inform the development of a new benchmark for measuring the progress of large language models on challenging commonsense reasoning tasks. We leveraged the richer supervision provided by text-based game environments to develop Breakpoint Transformers, a novel approach to modeling intermediate semantic information in long narrative or procedural texts. Finally, we integrated theories on the role of environments in collective human intelligence to propose a design for AI-augmented "social thinking environments" for knowledge workers like scientists.
Device Scheduling and Assignment in Hierarchical Federated Learning for  Internet of Things

Authors:
raw_author_name: Zhang, Tinghao
raw_author_name: Lam, Kwok-Yan
raw_author_name: Zhao, Jun

Abstract: Federated Learning (FL) is a promising machine learning approach for Internet of Things (IoT), but it has to address network congestion problems when the population of IoT devices grows. Hierarchical FL (HFL) alleviates this issue by distributing model aggregation to multiple edge servers. Nevertheless, the challenge of communication overhead remains, especially in scenarios where all IoT devices simultaneously join the training process. For scalability, practical HFL schemes select a subset of IoT devices to participate in the training, hence the notion of device scheduling. In this setting, only selected IoT devices are scheduled to participate in the global training, with each of them being assigned to one edge server. Existing HFL assignment methods are primarily based on search mechanisms, which suffer from high latency in finding the optimal assignment. This paper proposes an improved K-Center algorithm for device scheduling and introduces a deep reinforcement learning-based approach for assigning IoT devices to edge servers. Experiments show that scheduling 50% of IoT devices is generally adequate for achieving convergence in HFL with much lower time delay and energy consumption. In cases where reduction in energy consumption (such as in Green AI) and reduction of messages (to avoid burst traffic) are key objectives, scheduling 30% IoT devices allows a substantial reduction in energy and messages with similar model accuracy.
Leveraging Continuously Differentiable Activation Functions for Learning  in Quantized Noisy Environments

Authors:
raw_author_name: Shah, Vivswan
raw_author_name: Youngblood, Nathan

Abstract: Real-world analog systems intrinsically suffer from noise that can impede model convergence and accuracy on a variety of deep learning models. We demonstrate that differentiable activations like GELU and SiLU enable robust propagation of gradients which help to mitigate analog quantization error that is ubiquitous to all analog systems. We perform analysis and training of convolutional, linear, and transformer networks in the presence of quantized noise. Here, we are able to demonstrate that continuously differentiable activation functions are significantly more noise resilient over conventional rectified activations. As in the case of ReLU, the error in gradients are 100x higher than those in GELU near zero. Our findings provide guidance for selecting appropriate activations to realize performant and reliable hardware implementations across several machine learning domains such as computer vision, signal processing, and beyond.
Foundation Model Makes Clustering a Better Initialization for Active  Learning

Authors:
raw_author_name: Hong, Chuan

Abstract: Active learning selects the most informative samples from the unlabeled dataset to annotate in the context of a limited annotation budget. While numerous methods have been proposed for subsequent sample selection based on an initialized model, scant attention has been paid to the indispensable phase of active learning: selecting samples for model initialization. Most of the previous studies resort to random sampling or naive clustering. However, random sampling is prone to fluctuation, and naive clustering suffers from convergence speed, particularly when dealing with high-dimensional data such as imaging data. In this work, we propose to integrate foundation models with clustering methods to select samples for active learning initialization. Foundation models refer to those trained on massive datasets by the self-supervised paradigm and capable of generating informative and compacted embeddings for various downstream tasks. Leveraging these embeddings to replace raw features such as pixel values, clustering quickly converges and identifies better initial samples. For a comprehensive comparison, we included a classic ImageNet-supervised model to acquire embeddings. Experiments on two clinical tasks of image classification and segmentation demonstrated that foundation model-based clustering efficiently pinpointed informative initial samples, leading to models showcasing enhanced performance than the baseline methods. We envisage that this study provides an effective paradigm for future active learning.
A Truly Joint Neural Architecture for Segmentation and Parsing

Authors:
raw_author_name: Tsarfaty, Reut

Abstract: Contemporary multilingual dependency parsers can parse a diverse set of languages, but for Morphologically Rich Languages (MRLs), performance is attested to be lower than other languages. The key challenge is that, due to high morphological complexity and ambiguity of the space-delimited input tokens, the linguistic units that act as nodes in the tree are not known in advance. Pre-neural dependency parsers for MRLs subscribed to the joint morpho-syntactic hypothesis, stating that morphological segmentation and syntactic parsing should be solved jointly, rather than as a pipeline where segmentation precedes parsing. However, neural state-of-the-art parsers to date use a strict pipeline. In this paper we introduce a joint neural architecture where a lattice-based representation preserving all morphological ambiguity of the input is provided to an arc-factored model, which then solves the morphological segmentation and syntactic parsing tasks at once. Our experiments on Hebrew, a rich and highly ambiguous MRL, demonstrate state-of-the-art performance on parsing, tagging and segmentation of the Hebrew section of UD, using a single model. This proposed architecture is LLM-based and language agnostic, providing a solid foundation for MRLs to obtain further performance improvements and bridge the gap with other languages.
A new approach for imprecise probabilities

Authors:
raw_author_name: Pratelli, Luca

Abstract: This paper introduces a novel concept of interval probability measures that enables the representation of imprecise probabilities, or uncertainty, in a natural and coherent manner. Within an algebra of sets, we introduce a notion of weak complementation denoted as $\psi$. The interval probability measure of an event $H$ is defined with respect to the set of indecisive eventualities $(\psi(H))^c$, which is included in the standard complement $H^c$. We characterize a broad class of interval probability measures and define their properties. Additionally, we establish an updating rule with respect to $H$, incorporating concepts of statistical independence and dependence. The interval distribution of a random variable is formulated, and a corresponding definition of stochastic dominance between two random variables is introduced. As a byproduct, a formal solution to the century-old Keynes-Ramsey controversy is presented.