Reviewerly/exHarmony huggingface dataset:
Large Language Models and Their Current Use Cases

Large language models, powered by advancements in artificial intelligence and deep learning, have emerged as powerful tools for natural language processing tasks. These models, exemplified by OpenAI's GPT-3, possess the ability to generate coherent and contextually relevant text, rendering them invaluable across a wide range of applications. This paper provides an overview of large language models and their current use cases across various domains. The introduction elucidates the concept of large language models, elucidating their architectures, training methodologies, and capabilities. From an application standpoint, these models find utility in tasks such as sentiment analysis, question answering, language generation, and translation. Finally, the chapter concludes by identifying future directions for the development and application of large language models. It emphasizes the significance of ongoing research in areas including model interpretability, multi-modal learning, and domain adaptation, with the aim of further enhancing the capabilities and versatility of these models.
SQUIP: Exploiting the Scheduler Queue Contention Side Channel

Modern superscalar CPUs have multiple execution units that independently execute operations from the instruction stream. Previous work has shown that numerous side channels exist around these out-of-order execution pipelines, particularly for an attacker running on an SMT core.In this paper, we present the SQUIP attack, the first side-channel attack on scheduler queues, which are critical for deciding the schedule of instructions to be executed in superscalar CPUs. Scheduler queues have not been explored as a side channel so far, as Intel CPUs only have a single scheduler queue, and contention thereof would be virtually the same as contention of the reorder buffer. However, the Apple M1, AMD Zen 2, and Zen 3 microarchitectures have separate scheduler queues per execution unit. We first reverse-engineer the behavior of the scheduler queues on these CPUs and show that they can be primed and probed. The SQUIP attack observes the occupancy level from within the same hardware core and across SMT threads. We evaluate the performance of the SQUIP attack in a covert channel, exfiltrating 0.89 Mbit/s from a co-located virtual machine at an error rate below 0.8 %, and 2.70 Mbit/s from a co-located process at an error rate below 0.8 %. We then demonstrate the side channel on an mbedTLS RSA signature process in a co-located process and in a co-located virtual machine. Our attack recovers full RSA-4096 keys with only 50 500 traces and less than 5 to 18 bit errors on average. Finally, we discuss mitigations necessary, especially for Zen 2 and Zen 3 systems, to prevent our attacks.
DVFS Frequently Leaks Secrets: Hertzbleed Attacks Beyond SIKE, Cryptography, and CPU-Only Data

The recent Hertzbleed disclosure demonstrates how remote-timing analysis can reveal secret information previously only accessible to local-power analysis. At worst, this constitutes a fundamental break in the constant-time programming principles and the many deployed programs that rely on them. But all hope is not lost. Hertzbleed relies on a coarse-grained, noisy channel that is difficult to exploit. Indeed, the Hertzbleed paper required a bespoke cryptanalysis to attack a specific cryptosystem (SIKE). Thus, it remains unclear if Hertzbleed represents a threat to the broader security ecosystem.In this paper, we demonstrate that Hertzbleed’s effects are wide ranging, not only affecting cryptosystems beyond SIKE, but also programs beyond cryptography, and even computations occurring outside the CPU cores. First, we demonstrate how latent gadgets in other cryptosystem implementations— specifically "constant-time" ECDSA and Classic McEliece— can be combined with existing cryptanalysis to bootstrap Hertzbleed attacks on those cryptosystems. Second, we demonstrate how power consumption on the integrated GPU influences frequency on the CPU—and how this can be used to perform the first cross-origin pixel stealing attacks leveraging "constant-time" SVG filters on Google Chrome.
Intelligent layout generation based on deep generative models: A comprehensive survey

Layout generation, as a crucial stage in Artificial Intelligence-Generated Content (AIGC) task information presentation generation, is a fascinating and significant research topic. Automatically generating layouts intrigued the computer vision community dating back to the 1990s, and the advancement was sluggish for a time due to the inherent complexity of the issue. Thanks to the success of deep learning, layout generation is leading to a surge in both academic literature and business applications recently. Yet a systematic and comprehensive literature review of this domain is still missing. In this study, we provide a comprehensive overview of the state-of-the-art layout generation methods. Primarily, we discuss the problem of layout generation and associated background knowledge including layout representation, layout aesthetics, and generative models. Subsequently, recent advances in the field of intelligent layout generation are organized and summarized from three perspectives: innovations, models & algorithms, and applications. Furthermore, the paper outlines the datasets and evaluation methodology, both qualitative and quantitive, and briefly analyzes the selection of them for specific tasks. To sum up, we conclude the current challenges of intelligent layout generation and suggest possible directions for future endeavors.
Scale-Arbitrary Invertible Image Downscaling

Conventional social media platforms usually downscale high-resolution (HR) images to restrict their resolution to a specific size for saving transmission/storage cost, which makes those visual details inaccessible to other users. To bypass this obstacle, recent invertible image downscaling methods jointly model the downscaling/upscaling problems and achieve impressive performance. However, they only consider fixed integer scale factors and may be inapplicable to generic downscaling tasks towards resolution restriction as posed by social media platforms. In this paper, we propose an effective and universal Scale-Arbitrary Invertible Image Downscaling Network (AIDN), to downscale HR images with arbitrary scale factors in an invertible manner. Particularly, the HR information is embedded in the downscaled low-resolution (LR) counterparts in a nearly imperceptible form such that our AIDN can further restore the original HR images solely from the LR images. The key to supporting arbitrary scale factors is our proposed Conditional Resampling Module (CRM) that conditions the downscaling/upscaling kernels and sampling locations on both scale factors and image content. Extensive experimental results demonstrate that our AIDN achieves top performance for invertible downscaling with both arbitrary integer and non-integer scale factors. Also, both quantitative and qualitative evaluations show our AIDN is robust to the lossy image compression standard. The source code and trained models are publicly available at https://github.com/Doubiiu/AIDN.
Self-Enhanced Mixed Attention Network for Three-Modal Images Few-Shot Semantic Segmentation

As an important computer vision technique, image segmentation has been widely used in various tasks. However, in some extreme cases, the insufficient illumination would result in a great impact on the performance of the model. So more and more fully supervised methods use multi-modal images as their input. The dense annotated large datasets are difficult to obtain, but the few-shot methods still can have satisfactory results with few pixel-annotated samples. Therefore, we propose the Visible-Depth-Thermal (three-modal) images few-shot semantic segmentation method. It utilizes the homogeneous information of three-modal images and the complementary information of different modal images, which can improve the performance of few-shot segmentation tasks. We constructed a novel indoor dataset VDT-2048-5i for the three-modal images few-shot semantic segmentation task. We also proposed a Self-Enhanced Mixed Attention Network (SEMANet), which consists of a Self-Enhanced module (SE) and a Mixed Attention module (MA). The SE module amplifies the difference between the different kinds of features and strengthens the weak connection for the foreground features. The MA module fuses the three-modal feature to obtain a better feature. Compared with the most advanced methods before, our model improves mIoU by 3.8% and 3.3% in 1-shot and 5-shot settings, respectively, which achieves state-of-the-art performance. In the future, we will solve failure cases by obtaining more discriminative and robust feature representations, and explore achieving high performance with fewer parameters and computational costs.
Particle Filter Based on Harris Hawks Optimization Algorithm for Underwater Visual Tracking

Due to the complexity of the underwater environment, tracking underwater targets via traditional particle filters is a challenging task. To resolve the problem that the tracking accuracy of a traditional particle filter is low due to the sample impoverishment caused by resampling, in this paper, a new tracking algorithm using Harris-hawks-optimized particle filters (HHOPF) is proposed. At the same time, the problem of particle filter underwater target feature construction and underwater target scale transformation is addressed, the corrected background-weighted histogram method is introduced into underwater target feature recognition, and the scale filter is combined to realize target scaling transformation during tracking. In addition, to enhance the computational speed of underwater target tracking, this paper constructs a nonlinear escape energy using the Harris hawks algorithm in order to balance the exploration and exploitation processes. Based on the proposed HHOPF tracker, we performed detection and evaluation using the Underwater Object Tracking (UOT100) vision database. The proposed method is compared with evolution-based tracking algorithms and particle filters, as well as with recent tracker-based correlation filters and some other state-of-the-art tracking methods. By comparing the results of tracking using the test data sets, it is determined that the presented algorithm improves the overlap accuracy and tracking accuracy by 11% compared with other algorithms. The experiments demonstrate that the presented HHOPF visual tracking provides better tracking results.
Vertical Federated Unlearning on the Logistic Regression Model

Vertical federated learning is designed to protect user privacy by building local models over disparate datasets and transferring intermediate parameters without directly revealing the underlying data. However, the intermediate parameters uploaded by participants may memorize information about the training data. With the recent legislation on the“right to be forgotten”, it is crucial for vertical federated learning systems to have the ability to forget or remove previous training information of any client. For the first time, this work fills in this research gap by proposing a vertical federated unlearning method on logistic regression model. The proposed method is achieved by imposing constraints on intermediate parameters during the training process and then subtracting target client updates from the global model. The proposed method boasts the advantages that it does not need any new clients for training and requires only one extra round of updates to recover the performance of the previous model. Moreover, data-poisoning attacks are introduced to evaluate the effectiveness of the unlearning process. The effectiveness of the method is demonstrated through experiments conducted on four benchmark datasets. Compared to the conventional unlearning by retraining from scratch, the proposed unlearning method has a negligible decrease in accuracy but can improve training efficiency by over 400%.
ClueReader: Heterogeneous Graph Attention Network for Multi-Hop Machine Reading Comprehension

Multi-hop machine reading comprehension is a challenging task in natural language processing as it requires more reasoning ability across multiple documents. Spectral models based on graph convolutional networks have shown good inferring abilities and lead to competitive results. However, the analysis and reasoning of some are inconsistent with those of humans. Inspired by the concept of grandmother cells in cognitive neuroscience, we propose a heterogeneous graph attention network model named ClueReader to imitate the grandmother cell concept. The model is designed to assemble the semantic features in multi-level representations and automatically concentrate or alleviate information for reasoning through the attention mechanism. The name ClueReader is a metaphor for the pattern of the model: it regards the subjects of queries as the starting points of clues, takes the reasoning entities as bridge points, considers the latent candidate entities as grandmother cells, and the clues end up in candidate entities. The proposed model enables the visualization of the reasoning graph, making it possible to analyze the importance of edges connecting entities and the selectivity in the mention and candidate nodes, which is easier to comprehend empirically. Evaluations on the open-domain multi-hop reading dataset WikiHop and drug–drug interaction dataset MedHop proved the validity of ClueReader and showed the feasibility of its application of the model in the molecular biology domain.
AUQantO: Actionable Uncertainty Quantification Optimization in deep learning architectures for medical image classification

Deep learning algorithms have the potential to automate the examination of medical images obtained in clinical practice. Using digitized medical images, convolution neural networks (CNNs) have demonstrated their ability and promise to discriminate among different image classes. As an initial step towards explainability in clinical diagnosis, deep learning models must be exceedingly precise, offering a measure of uncertainty for their predictions. Such uncertainty-aware models can help medical professionals in detecting complicated and corrupted samples for re-annotation or exclusion. This paper proposes a new model and data-agnostic mechanism, called Actionable Uncertainty Quantification Optimization (AUQantO) to improve the performance of deep learning architectures for medical image classification. This is achieved by optimizing the hyperparameters of the proposed entropy-based and Monte Carlo (MC) dropout uncertainty quantification techniques escorted by single- and multi-objective optimization methods, abstaining from the classification of images with a high level of uncertainty. This helps in improving the overall accuracy and reliability of deep learning models. To support the above claim, AUQantO has been validated with four deep learning architectures on four medical image datasets and using various performance metric measures such as precision, recall, Area Under the Receiver Operating Characteristic (ROC) Curve score (AUC), and accuracy. The study demonstrated notable enhancements in deep learning performance, with average accuracy improvements of 1.76% and 2.02% for breast cancer histology and 5.67% and 4.24% for skin cancer datasets, utilizing two uncertainty quantification techniques, and AUQantO further improved accuracy by 1.41% and 1.31% for brain tumor and 4.73% and 1.83% for chest cancer datasets while allowing exclusion of images based on confidence levels.
Artificial Intuition for Automated Decision-Making

Formulae display:?Mathematical formulae have been encoded as MathML and are displayed in this HTML version using MathJax in order to improve their display. Uncheck the box to turn MathJax off. This feature requires Javascript. Click on a formula to zoom.
Large Language Models

Large Language ModelsIn the latest edition of Stats, STAT!, Fralick and colleagues explain the statistics behind large language models - used in chat bots like ChatGPT and Bard. While these new tools may seem remarkably intelligent, at their core they just assemble sentences based on statistics from large amounts of text.
Scaling Blockchain Consensus via a Robust Shared Mempool

Leader-based Byzantine fault-tolerant (BFT) consensus protocols used by permissioned blockchains have limited scalability and robustness. To alleviate the leader bottleneck in BFT consensus, we introduce Stratus, a robust shared mempool protocol that decouples transaction distribution from consensus. Our idea is to have replicas disseminate transactions in a distributed manner and have the leader only propose transaction ids. Stratus uses a provably available broadcast (PAB) protocol to ensure the availability of the referenced transactions. To deal with unbalanced load across replicas, Stratus adopts a distributed load balancing protocol.We implemented and evaluated Stratus by integrating it with state-of-the-art BFT-based blockchain protocols. Our evaluation of these protocols in both LAN and WAN settings shows that Stratus-based protocols achieve 5× to 20× higher throughput than their native counterparts in a network with hundreds of replicas. In addition, the performance of Stratus degrades gracefully in the presence of network asynchrony, Byzantine attackers, and unbalanced workloads.
Bayesian Negative Sampling for Recommendation

How to sample high quality negative instances from unlabeled data, i.e., negative sampling, is important for training implicit collaborative filtering and contrastive learning models. Although previous studies have proposed some approaches to sample informative instances, discriminating false negative from true negative for unbiased negative sampling remains an unsolved problem. On the basis of our order relation analysis of negatives’ scores, we first derive the class conditional density of true negatives and that of false negatives. We next design a Bayesian classifier for negative classification, from which we define a model-agnostic posterior probability estimate of an instance being true negative as a quantitative negative signal measure. We also propose a Bayesian optimal sampling rule to sample high-quality negatives. The proposed Bayesian Negative Sampling (BNS) algorithm has a linear time complexity. Experimental studies validate the superiority of BNS over the peers in terms of better sampling quality and better recommendation performance. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Online Anomalous Subtrajectory Detection on Road Networks with Deep Reinforcement Learning

Detecting anomalous trajectories has become an important task in many location-based applications. While many approaches have been proposed for this task, they suffer from various issues including (1) incapability of detecting anomalous subtrajectories, which are finer-grained anomalies in trajectory data, and/or (2) non-data driven, and/or (3) requirement of suﬃcient supervision labels which are costly to collect. In this paper, we propose a novel reinforcement learning based solution called RL4OASD, which avoids all aforementioned issues of existing approaches. RL4OASD involves two networks, one responsible for learning features of road networks and trajectories and the other responsible for detecting anomalous subtrajectories based on the learned features, and the two networks can be trained iteratively without labeled data. Extensive experiments are conducted on two real datasets, and the results show that our solution can significantly outperform the state-of-the-art methods (with 20-30% improvement) and is eﬃcient for online detection (it takes less than 0.1ms to process each newly generated data point).
Modelling High-Order Social Relations for Item Recommendation (Extended Abstract)

Personalized recommendation is becoming increasingly important in online information systems in the current era of information explosion. In real-world scenarios, when a user considers which items to consume, the decision choice may be affected by her friends. For example, she may ask her friends for suggestions or be attracted by products purchased by one friend. As such, to provide satisfactory recommendation service, it is important to account for the evidence in social relations when they are available to use. Several prior efforts have been made to leverage social relations to build the recommender system and verified their utility. However, most existing methods, such as the well-known TrustSVD, leverage only first-order social relations, i.e., the direct neighbors that are connected to the target user. The high-order social relations, e.g., the friends of friends, which are very informative to reveal user preference, have been largely ignored.
Collaborative Graph Neural Networks for Attributed Network Embedding

Graph neural networks (GNNs) have shown prominent performance on attributed network embedding. However, existing efforts mainly focus on exploiting network structures, while the exploitation of node attributes is rather limited as they only serve as node features at the initial layer. This simple strategy impedes the potential of node attributes in augmenting node connections, leading to limited receptive field for inactive nodes with few or even no neighbors. Furthermore, the training objectives (i.e., reconstructing network structures) of most GNNs also do not include node attributes, although studies have shown that reconstructing node attributes is beneficial. Thus, it is encouraging to deeply involve node attributes in the key components of GNNs, including graph convolution operations and training objectives. However, this is a nontrivial task since an appropriate way of integration is required to maintain the merits of GNNs. To bridge the gap, in this paper, we propose COllaborative graph Neural Networks–CONN, a tailored GNN architecture for attribute network embedding. It improves model capacity by 1) selectively diffusing messages from neighboring nodes and involved attribute categories, and 2) jointly reconstructing node-to-node and node-to-attribute-category interactions via cross-correlation. Experiments on real-world networks demonstrate that CONN excels state-of-the-art embedding algorithms with a great margin.
PMR-CNN: Prototype Mixture R-CNN for Few-Shot Object Detection

Few-shot object detection is a challenging task because of the limited annotation data. Under the limitation of few-shot samples, images from the same class may differ significantly in appearance and pose. Although the research has progressed considerably since adding the prototype vector to few-shot object detection, the previous paradigm is still constrained by several factors: (1) using a single prototype to represent the support image tends to cause semantic ambiguity; (2) the way of extracting prototypes is too simple, like global average pooling, which makes prototypes not representative enough. In this work, we design PMR-CNN to address the above limitations. PMR-CNN proposes a new method of prototype generation and enhances the representative information by using multiple prototypes to represent support images. For experiments, we not only evaluate our method on general image dataset MS COCO, but also evaluate on SiTi (a real-world autonomous driving dataset collected by us). Experiment on the few-shot object detection benchmark shows that we have a significant advantage over the previous methods. Code is available at: https://github.com/Chientsung-Chou/PMR-CNN.