Reviewerly/exHarmony huggingface dataset:
Feature Mixing for Writer Retrieval and Identification on Papyri Fragments

This paper proposes a deep-learning-based approach to writer retrieval and identification for papyri, with a focus on identifying fragments associated with a specific writer and those corresponding to the same image. We present a novel neural network architecture that combines a residual backbone with a feature mixing stage to improve retrieval performance, and the final descriptor is derived from a projection layer. The methodology is evaluated on two benchmarks: PapyRow, where we achieve a mAP of 26.6 % and 24.9 % on writer and page retrieval, and HisFragIR20, showing state-of-the-art performance (44.0 % and 29.3 % mAP). Furthermore, our network has an accuracy of 28.7 % for writer identification. Additionally, we conduct experiments on the influence of two binarization techniques on fragments and show that binarizing does not enhance performance. Our code and models are available to the community at https://github.com/marco-peer/hip23.
DIVA-DAF: A Deep Learning Framework for Historical Document Image Analysis

Deep learning methods have shown strong performance in solving tasks for historical document image analysis. However, despite current libraries and frameworks, programming an experiment or a set of experiments and executing them can be time-consuming. This is why we propose an open-source deep learning framework, DIVA-DAF, which is based on PyTorch Lightning and specifically designed for historical document analysis. Pre-implemented tasks such as segmentation and classification can be easily used or customized. It is also easy to create one’s own tasks with the benefit of powerful modules for loading data, even large data sets, and different forms of ground truth. The applications conducted have demonstrated time savings for the programming of a document analysis task, as well as for different scenarios such as pre-training or changing the architecture. Thanks to its data module, the framework also allows to reduce the time of model training significantly.
MultiIB‐TransUNet: Transformer with multiple information bottleneck blocks for CT and ultrasound image segmentation

Accurate medical image segmentation is crucial for disease diagnosis and surgical planning. Transformer networks offer a promising alternative for medical image segmentation as they can learn global features through self-attention mechanisms. To further enhance performance, many researchers have incorporated more Transformer layers into their models. However, this approach often results in the model parameters increasing significantly, causing a potential rise in complexity. Moreover, the datasets of medical image segmentation usually have fewer samples, which leads to the risk of overfitting of the model.This paper aims to design a medical image segmentation model that has fewer parameters and can effectively alleviate overfitting.We design a MultiIB-Transformer structure consisting of a single Transformer layer and multiple information bottleneck (IB) blocks. The Transformer layer is used to capture long-distance spatial relationships to extract global feature information. The IB block is used to compress noise and improve model robustness. The advantage of this structure is that it only needs one Transformer layer to achieve the state-of-the-art (SOTA) performance, significantly reducing the number of model parameters. In addition, we designed a new skip connection structure. It only needs two 1× 1 convolutions, the high-resolution feature map can effectively have both semantic and spatial information, thereby alleviating the semantic gap.The proposed model is on the Breast UltraSound Images (BUSI) dataset, and the IoU and F1 evaluation indicators are 67.75 and 87.78. On the Synapse multi-organ segmentation dataset, the Param, Hausdorff Distance (HD) and Dice Similarity Cofficient (DSC) evaluation indicators are 22.30, 20.04 and 81.83.Our proposed model (MultiIB-TransUNet) achieved superior results with fewer parameters compared to other models.
The Exploration of Knowledge-Preserving Prompts for Document Summarisation

Despite the great development of document summarisation techniques nowadays, factual inconsistencies between the generated summaries and the original texts still occur from time to time. This study explores the possibility of adopting prompts to incorporate factual knowledge into generated summaries. We specifically study prefix-tuning that uses a set of trainable continuous prefix prompts together with discrete natural language prompts to aid summary generation. Experimental results demonstrate that the trainable prefixes can help the summarisation model extract information from discrete prompts precisely, thus generating knowledge-preserving summaries that are factually consistent with the discrete prompts. The ROUGE improvements of the generated summaries indicate that explicitly adding factual knowledge into the summarisation process could boost the overall performance, showing great potential for applying it to other natural language processing tasks.
Prompt-Learning for Cross-Lingual Relation Extraction

Relation Extraction (RE) is a crucial task in Information Extraction, which entails predicting relationships between entities within a given sentence. However, extending pre-trained RE models to other languages is challenging, particularly in real-world scenarios where Cross-Lingual Relation Extraction (XRE) is required. Despite recent advancements in Prompt-Learning, which involves transferring knowledge from Multilingual Pre-trained Language Models (PLMs) to diverse downstream tasks, there is limited research on the effective use of multilingual PLMs with prompts to improve XRE. In this paper, we present a novel XRE algorithm based on Prompt-Tuning, referred to as Prompt-Xre. To evaluate its effectiveness, we design and implement several prompt templates, including hard, soft, and hybrid prompts, and empirically test their performance on competitive multilingual PLMs, specifically mBART. Our extensive experiments, conducted on the low-resource ACE05 benchmark across multiple languages, demonstrate that our Prompt-Xre algorithm significantly outperforms both vanilla multilingual PLMs and other existing models, achieving state-of-the-art performance in XRE. To further show the generalization of our Prompt-XRE on larger data scales, we construct and release a new XRE dataset-WMTI7-EnZh XRE, containing 0.9M English-Chinese pairs extracted from WMT 2017 parallel corpus. Experiments on WMTI7-EnZh XRE also show the effectiveness of our Prompt-XRE against other competitive baselines. The code and newly constructed dataset are freely available at httus://2ithub.com/HSU-CHIA-MING/Promut-XRE.
Quantum Machine Learning in Disease Detection and Prediction: a survey of applications and future possibilities

Quantum machine learning (QML) in the field of disease detection and prediction use quantum computing techniques and algorithms to analyze and classify large datasets of medical information, by identifying subtle patterns and predict the occurrence or progression of diseases. It involves applying machine learning techniques to data from biological and medical research, such as-genomic and proteomic data, medical imaging, electronic health records, and clinical trial data, using quantum computing algorithms and architectures to perform these analyses more efficiently and accurately than classical computing methods. This approach has the potential to provide new insights into complex biological systems and facilitate the development of more effective treatments and personalized medicine. In this paper, a systematic review of the use of QML algorithms has been conducted, which focuses on the detection and prediction of diseases among patients. The current essence of the field along with the challenges and limitations of current works have also been discussed. After evaluating the implemented and proposed methods of data analysis, algorithm development, usefulness and efficiency of the system in various disease detection and prediction, a recommendation was made on the open research scopes in this field at the end of the paper.
Cell Attention Networks

Since their introduction, graph attention networks achieved outstanding results in graph representation learning tasks. However, these networks consider only pairwise relations between features associated to the nodes and then are unable to fully exploit higher-order and long-range interactions present in many real world data-sets. In this paper, we introduce a neural architecture operating on data defined over the nodes and the edges of a graph, represented as the 1-skeleton of a regular cell complex, able to capture insightful higher-order and long-range interactions. In particular, we exploit the lower and upper neighborhoods, as encoded in the cell complex, to design two independent masked self-attention mechanisms, thus generalizing the conventional graph attention strategy. The approach used is hierarchical and it incorporates the following steps: i) a lifting algorithm that learns (additional) edge features from node features; ii) a cell attention mechanism to find the optimal combination of edge features over both lower and upper neighbors; iii) a hierarchical edge pooling mechanism to extract a compact meaningful set of features. The experimental results show that this method compares favorably with state of the art results on graph-based learning tasks while maintaining a low complexity.
Application of Machine Learning for Anomaly Detection in Printed Circuit Boards Imbalance Date Set

The detection of anomalies in printed circuit boards (PCBs) is an important challenge in the electronics manufacturing industry. Traditional anomaly detection methods often struggle to handle imbalanced datasets, which are common in real-world PCB production. In recent years, machine learning (ML) algorithms have emerged as a promising solution to this problem. This study investigates the use of ML algorithms for anomaly detection in PCBs, with a particular focus on addressing the issue of imbalanced data. We propose a data-level technique to balance the dataset and improve the performance of the ML algorithm. Our results show that our approach outperforms traditional methods in terms of precision, recall, and F1 score. Overall, this study demonstrates the potential of ML in addressing the challenge of anomaly detection in PCBs and highlights the importance of considering imbalanced data in such applications.
Leveraging the Power of Artificial Intelligence and Blockchain in Recruitment using Beetle Platform

The recruitment process can be arduous and time-consuming for both recruiters and job seekers. To address this issue, we propose the Beetle platform that leverages artificial intelligence (AI) and Blockchain technology to provide seamless and efficient solutions for the recruitment journey. Beetle platform includes a ChatGPT styled prompt interface, a Tinder-styled matching interface, an AI-powered video based digital twin bot for interview preparation, as well as a Blockchain-driven vetting system. The searching/shortlisting phase utilizes continuous machine learning algorithms to enhance the matching of job seekers with relevant recruiters and/or positions. The digital twin bot integrates Natural Language Processing to prepare candidates for interviews and provide feedback. The vetting phase uses Blockchain technology to store information to improve Beetle’s reliability and security. Information stored on it is immutable, transparent, and easily verifiable, thus making the vetting process faster and more reliable.
METEOR Guided Divergence for Video Captioning

Automatic video captioning aims for a holistic visual scene understanding. It requires a mechanism for capturing temporal context in video frames and the ability to comprehend the actions and associations of objects in a given timeframe. Such a system should also learn to abstract video sequences into sensible representations and generate natural written language. While the majority of captioning models focus solely on the visual inputs, little attention has been paid to the audiovisual modality. To tackle this issue, we propose a novel two-fold approach. First, we implement a reward-guided KL Divergence to train a video captioning model which is resilient towards token permutations. Second, we utilise a Bi-Modal Hierarchical Reinforcement Learning (BMHRL) Transformer architecture to capture long-term temporal dependencies of the input data as a foundation for our hierarchical captioning module. Using our BMHRL, we show the suitability of the HRL agent in the generation of content-complete and grammatically sound sentences by achieving 4.91, 2.23, and 10.80 in BLEU3, BLEU4, and METEOR scores, respectively on the ActivityNet Captions dataset.
Exploring Blockchain Technology for Chain of Custody Control in Physical Evidence: A Systematic Literature Review

Blockchain technology, initially known for its applications in the financial industry, has emerged as a promising solution for various other domains. One prominent area for the use of blockchain-based solutions is forensics, specifically the chain of custody maintenance and control. While there have been numerous research projects exploring the use of blockchain technology in digital forensics, limited attention has been given to its application in controlling of the physical evidence chain of custody. In this research, we aim to explore the literature on the use of blockchain technology to solve problems related to the physical evidence chain of custody. Through a systematic literature review (SLR), we analyzed 26 resources discussing blockchain-based solutions for evidence chain of custody issues, based on requirements that could be applied to both physical and digital evidence. The results showed that there is a lack of studies involving the use of blockchain technology to solve problems related to the physical evidence chain of custody, and future research should focus on solving the issue.
SRRM: Semantic Region Relation Model for Indoor Scene Recognition

Despite the remarkable success of convolutional neural networks in various computer vision tasks, recognizing indoor scenes still presents a significant challenge due to their complex composition. Consequently, effectively leveraging semantic information in the scene has been a key issue in advancing indoor scene recognition. Unfortunately, the accuracy of semantic segmentation has limited the effectiveness of existing approaches for leveraging semantic information. As a result, many of these approaches remain at the stage of auxiliary labeling or co-occurrence statistics, with few exploring the contextual relationships between the semantic elements directly within the scene. In this paper, we propose the Semantic Region Relationship Model (SRRM), which starts directly from the semantic information inside the scene. Specifically, SRRM adopts an adaptive and efficient approach to mitigate the negative impact of semantic ambiguity and then models the semantic region relationship to perform scene recognition. Additionally, to more comprehensively exploit the information contained in the scene, we combine the proposed SRRM with the PlacesCNN module to create the Combined Semantic Region Relation Model (CSRRM), and propose a novel information combining approach to effectively explore the complementary contents between them. CSRRM significantly outperforms the SOTA methods on the MIT Indoor 67, reduced Places365 dataset, and SUN RGB-D without retraining. The code is available at: https://github.com/ChuanxinSong/SRRM
RBPSum: An extractive summarization approach using Bi-Stream Attention and Position Residual Connection

Extractive text summarization is a well-studied downstream task of natural language processing that aims to select sentences as a summary of the document's critical information. For the systems built based on pre-trained language models, the large volume of parameters can cause the model performance gets significantly degradation when the training data is insufficient. However, acquiring high-quality labeled data is a time-consuming and laborious task. Previous works mainly focus on accuracy, and few of them pay attention to the robustness of the model. Hence training a robust and high-quality model is the concern of this work. In this work, we propose RBPSum, a robust extractive summarization model based on the pre-trained language model. Through the experiments, we find that under the situation of restricted data size, sentence position information plays a critical role in extractive summarization. In terms of ROUGE metrics, our model outperforms the previous state-of-the-art approaches when using the entire training set, and around a third of the training set produces competitive results.
Pruning of Dendritic Neuron Model with Significance Constraints for Classification

The dendritic neural model (DNM) simulates the information-processing mechanisms and procedures of neurons by mimicking the nonlinearity of synapses in the human brain. This allows for a better understanding of biological nervous systems and has been used to a wonderful effect in various fields. However, there are some problems with the existing DNM, such as the high complexity and the limited generalization capability. Pruning is one of the most common and crucial approaches to model compression. It is a model optimization technique that involves removing redundant values from a weight tensor to develop smaller and more efficient neural networks. A compressed neural network can enable faster running and reduced computational costs in network training. To improve the model performance, this work proposes a DNM pruning method with dendrite layer significance constraints. Our proposed method not only calculates the significance of dendrite layers but also makes the significance of a few dendrite layers in the trained model concentrated in a few dendrite layers so that the dendrite layers with low significance can be pruned away. The results of the simulation experiment on classification problems show that our proposed method outperforms the existing pruning methods in terms of network size and generalization performance.
FG-UAP: Feature-Gathering Universal Adversarial Perturbation

Deep Neural Networks (DNNs) are susceptible to elaborately designed perturbations, whether such perturbations are dependent or independent of images. The latter one, called Universal Adversarial Perturbation (UAP), is very attractive for model robustness analysis, since its independence of input reveals the intrinsic characteristics of the model. Relatively, another interesting observation is Neural Collapse (NC), which means the feature variability may collapse during the terminal phase of training. Motivated by this, we propose to generate UAP by attacking the layer where NC phenomenon happens. Because of NC, the proposed attack could gather all the natural images’ features to its surrounding, which is hence called Feature-Gathering UAP (FG-UAP). We evaluate the effectiveness our proposed algorithm on abundant experiments, including untargeted and targeted universal attacks, attacks under limited dataset, and transfer-based black-box attacks among different architectures including Vision Transformers, which are believed to be more robust. After that, we empirically verify the effectiveness of NC's conclusion on UAP by attacking on only 10% of the dataset while keeping comparable performance. Finally, we investigate FG-UAP in the view of NC by analyzing the labels and extracted features of adversarial examples, finding that collapse phenomenon becomes stronger after the model is corrupted. Codes for the project are available at https://2ithub.com/yzx1213/FG-UAP.
FEAMNet: Light Field Depth Estimation Network Based On Feature Extraction and Attention Mechanism

With the rich 4D visual information of light rays, light field (LF) applications contribute to the development of immersive multimedia and virtual reality, where LF depth estimation is the critical problem. However, existing light field depth estimation algorithms deliver weak performance in edge and textureless regions. In this paper, we propose the feature-extraction and attention mechanism-based network (FEAMNet), which can effectively handle edges and textureless regions in depth maps. The FEAMNet contains a dilated-convolution and average-pooling (DCAP) feature extraction module with a large receptive field to acquire multi-scale features. And the channel attention-based disparity regression (CADR) module is introduced to measure the importance weights of different feature channels for high accuracy. The experimental results show that the FEAMNet outperforms state-of-the-art algorithms, such as OACC and DistgDisp. The implementation of our FEAMNet with the mentioned dataset is open sourced at https://github.com/lymwxq/FEAMNet.
Deformable Feature Interaction Network and Graph Structure Reasoning for 3D Dense Alignment and Face Reconstruction

3D face reconstruction from large pose images is a long-standing and challenging problem. Existing methods based on 3DMM parameter regression with strong constraints only consider reducing the error between 68 landmarks and their ground truth, which may lead to insufficient learning about the landmark. In this paper, we propose a deformable feature interaction network (DFIN) for 3D dense alignment and face reconstruction. We design shallow and deep modules to extract semantic information at different levels, perform feature interaction, and improve the ability of the network to perform local detail feature extraction. We use deformable convolution properties to increase the convolution kernel's acceptance domain to learn the effective face area and the weights of the facial feature distribution, and the output features are enriched. Moreover, we propose a graph structure reasoning method by introducing a facial geometric consistency loss. Specifically, we selected the leftmost and rightmost cheeks, the chin, the tip of the nose, and the landmarks of the eyes. We triangulate the landmarks of the chin, eyes, and cheeks and calculate the geometric relationship between the center of gravity of the triangle and the tip of the nose to constrain the entire face. Extensive experiments on AFLW2000-3D and AFLW datasets demonstrate the validity of our method. Source codes are available at https://github.com/wricked520/DFIN.
Verification-friendly Networks: the Case for Parametric ReLUs

It has increasingly been recognised that verification can contribute to the validation and debugging of neural networks before deployment, particularly in safety-critical areas. While progress has been made in the area of verification of neural networks, present techniques still do not scale to large ReLU-based neural networks used in many applications. In this paper we show that considerable progress can be made by employing Parametric ReLU activation functions in lieu of plain ReLU functions. We give training procedures that produce networks which achieve one order of magnitude gain in verification overheads and 30-100% fewer timeouts with VeriNet, a SoA Symbolic Interval Propagation-based verification toolkit, while not compromising the resulting accuracy. Furthermore, we show that adversarial training combined with our approach improves certified robustness up to 36% compared to adversarial training performed on baseline ReLU networks.