Reviewerly/exHarmony huggingface dataset:
Using YOLO Network for Automatic Processing of Finite Automata Images with Application to Bit-Strings Recognition

The recognition of handwritten diagrams has drawn attention in recent years because of their potential applications in many areas, especially when it can be used for educational purposes. Although there are many online approaches, the advances of deep object detector networks have made offline recognition an attractive option, allowing simple inputs such as paper-drawn diagrams. In this paper, we have tested the YOLO network, including its version with fewer parameters, YOLO-Tiny, for the recognition of images of finite automata. This recognition was applied to the development of an application that recognizes bit-strings used as input to the automaton: given an image of a transition diagram, the user inserts a sequence of bits and the system analyzes whether the automaton recognizes the sequence or not. Using two bases of finite automata, we have evaluated the detection and recognition of finite automata symbols as well as bit-string processing. With regard to the diagram symbol detection task, experiments on a handwritten finite automata image dataset returned 82.04% and 97.20% for average precision and recall, respectively.
Static Pruning for Multi-Representation Dense Retrieval

Dense retrieval approaches are challenging the prevalence of inverted index-based sparse representation approaches for information retrieval systems. Different families have arisen: single representations for each query or passage (such as ANCE or DPR), or multiple representations (usually one per token) as exemplified by the ColBERT model. While ColBERT is effective, it requires significant storage space for each token's embedding. In this work, we aim to prune the embeddings for tokens that are not important for effectiveness. Indeed, we show that, by adapting standard uniform and document-centric static pruning methods to embedding-based indexes, but retaining their focus on low-IDF tokens, we can attain large improvements in space efficiency while maintaining high effectiveness. Indeed, on experiments conducted on the MSMARCO passage ranking task, by removing all embeddings corresponding to the 100 most frequent BERT tokens, the index size is reduced by 45%, with limited impact on effectiveness (e.g. no statistically significant degradation of NDCG@10 or MAP on the TREC 2020 queryset). Similarly, on TREC Covid, we observed a 1.3% reduction in nDCG@10 for a 38% reduction in total index size.
Subjective Quality Evaluation of Alternative Imaging Techniques for Microfiche Digitization

Before the advent of digital formats, microfiche, a type of microform, was widely utilized for archiving and preserving historical documents. As a result, numerous historical collections and documents can only be found in microfiche format, transforming them into valuable artifacts and indispensable aspects of our cultural heritage. Although microfiche can last a long time, it is still susceptible to damage and requires digitization for preservation and broader accessibility. In addition, traditional microfiche readers are not always available and are primarily designed for reading rather than digitization. In this study, we evaluated the performance of two alternative imaging devices compared to a traditional microfiche reader and the impact of enhancement on image quality using subjective image quality assessment. The experiments were carried out in a controlled environment with twenty-one participants, including an expert. Our results showed that the reproduction of alternative devices was preferred over that of a traditional microfiche reader. Furthermore, our results demonstrate that image enhancement techniques significantly improved image quality. This study suggests that alternative imaging devices may be a viable option for digitizing microfiche and improving access to historical collections.
Enhancing conversational agents for successful operation: A multi-perspective evaluation approach for continuous improvement

Abstract Contemporary organizations increasingly adopt conversational agents (CAs) as intelligent and natural language-based solutions for providing services and information. CAs offer new forms of personalization, speed, (cost-)effectiveness, and automation. However, despite their hype in research and practice, many organizations still fail to seize CAs’ potential because they lack knowledge of how to evaluate and improve the quality of CAs to sustain them in organizational operations. We aim to fill this knowledge gap by conducting a design science research project in which we aggregate insights from the literature and practice to derive an applicable set of quality criteria for CAs. Our article contributes to CA research and guides practitioners by providing a blueprint to structure the evaluation of CAs and to discover areas for systematic improvement.
An adaptive biogeography-based optimization with integrated covariance matrix learning for robust visual object tracking

Biogeography-based optimization (BBO) has gained significant popularity as a population-based metaheuristic optimization algorithm. However, the existing variants of BBO encounter difficulties when tackling complex optimization problems with variable coupling features. This is primarily attributed to the rotational variability of the migration operator, which hampers its ability to accurately capture and utilize variable coupling features among decision variables. To overcome this challenge, we propose a solution called integrated covariance matrix learning (ICML), which utilizes distribution information from subsets of current population to capture variable coupling features. ICML employs local distribution information to guide BBO towards a globally optimal solution by building eigen coordinate systems. This enables individuals to identify a more optimal value based on their nearby distribution. A new class of BBOs, referred to as ICML-BBOs, is presented by embedding ICML into existing BBO variants. The performance of ICML is evaluated by applying it to original BBO and three BBO variants, enabling a comprehensive performance comparison. Experimental results on the CEC2005 and CEC2017, as well as a real-world robust visual object tracking optimization problem, showcase the effectiveness of ICML.
Audio-driven talking face generation with diverse yet realistic facial animations

Audio-driven talking face generation, which aims to synthesize talking faces with realistic facial animations (including accurate lip movements, vivid facial expression details and natural head poses) corresponding to the audio, has achieved rapid progress in recent years. However, most existing work focuses on generating lip movements only without handling the closely correlated facial expressions, which degrades the realism of the generated faces greatly. This paper presents DIRFA, a novel method that can generate talking faces with diverse yet realistic facial animations from the same driving audio. To accommodate fair variation of plausible facial animations for the same audio, we design a transformer-based probabilistic mapping network that can model the variational facial animation distribution conditioned upon the input audio and autoregressively convert the audio signals into a facial animation sequence. In addition, we introduce a temporally-biased mask into the mapping network, which allows to model the temporal dependency of facial animations and produce temporally smooth facial animation sequence. With the generated facial animation sequence and a source image, photo-realistic talking faces can be synthesized with a generic generation network. Extensive experiments show that DIRFA can generate talking faces with realistic facial animations effectively.
PipePar: Enabling fast DNN pipeline parallel training in heterogeneous GPU clusters

Recently, pipeline parallelism for large-scale Deep Neural Network (DNN) training has been developed, which partitions the DNN model across multiple devices (e.g., GPUs) and improves the training efficiency by processing data divided into minibatches as a pipeline. However, existing model partitioning algorithms are mostly designed for homogeneous clusters with the same GPU devices and network connections (e.g., bandwidths), while heterogeneous GPU clusters are widely used in mainstream computing infrastructures. In heterogeneous environment, devices are equipped with different GPUs and network connections, and the efficiency of previous approaches is unsatisfactory due to the unbalanced load of the pipeline stages. In this paper, we propose PipePar, a model partitioning and task placement algorithm for pipeline parallel DNN training in heterogeneous GPU clusters. PipePar is based on dynamic programming with search space pruning that takes into consideration both the heterogeneity of GPUs and network bandwidth. PipePar can profile the DNN model for each type of GPU and conduct model partitioning and task placement based on given GPUs and network connections, which can optimize pipeline load balancing in heterogeneous environments and thus improve training efficiency. We design and implement a pipeline-based distributed deep learning training system in a heterogeneous GPU cluster and show through extensive experiments that PipePar outperforms the baseline approaches in the speed of large-scale DNN training.
TSDTVOS: Target-guided spatiotemporal dual-stream transformers for video object segmentation

Video object segmentation automatically separates the interested objects from the background across a video sequence and was an active research area in recent years. The crucial challenge lies in investigating an effective architecture to fully exploit spatiotemporal correlation in a given video sequence for achieving accurate segmentation results. In this paper, we propose a novel semi-supervised Transformer-based framework called Target-guided Spatiotemporal Dual-stream Transformers (TSDT) with two separate streams to enable effective spatiotemporal context propagation. Technically, the temporal stream is used to aggregate rich temporal cues from past frames, while the spatial stream is trained to encode object location and appearance information stored in the current frame. To compress and integrate temporal features, a target guidance block (TGB) is designed to retrieve target information in the past video flow under the guidance of the current frame. The experimental results on video object segmentation benchmarks demonstrate the feasibility and effectiveness of the proposed framework. Codes and trained models are available at https://github.com/zhouweii234/TSDTVOS.
Application of Interpretable Machine Learning for Production Feasibility Prediction of Gold Mine Project

In the context of globalization in the mining industry, assessing the production feasibility of mining projects by smart technology is crucial for the improvement of mining development efficiency. However, evaluating the feasibility of such projects faces significant challenges due to incomplete data and complex variables. In recent years, the development of big data technology has offered new possibilities for rapidly evaluating mining projects. This study conducts an intelligent evaluation of gold mines based on global mineral resources data to estimate whether a gold mine project can be put into production. A technical workflow is constructed, including data filling, evaluation model construction, and production feasibility evaluation. Based on the workflow, the missing data is filled in by the Miceforest imputation algorithm first. The evaluation model is established based on the Random Forest model to quantitatively predict the feasibility of the mining project being put into production, and important features of the model are extracted using Shapley Additive explanation(SHAP). This workflow may enhance the efficiency and accuracy of quantitative production feasibility evaluation for mining projects, with an accuracy rate increased from 93.80% to 95.99%. Results suggest that the features of estimated mine life and gold ore grade have the most significant impact on production feasibility.
Implementation of fuzzy associative memory toward optimizing a neural network model to predict total iron binding capacity

Determining the optimal number of hidden layer neurons in a neural network (NN) is an important issue. In this research, a fuzzy associative memory (FAM) model will be developed to determine the hidden neuron of neural network architecture. The material for this research is data from chronic kidney disease (CKD) patients on hemodialysis to predict total iron binding capacity (TIBC). The research begins by choosing the most appropriate backpropagation algorithm. The FAM system was developed using the product-correlation coding operator, the product-max composition relation, and a combination of winner-take-all and weighted average. Testing is done by calculating the average error (MSE) of training and testing, as well as the average error for gradients. We selected 12 alternative network architectures. The Levenberg-Marquardt backpropagation algorithm was chosen for its best performance. In testing the training data, the average percentage of errors was 7.16% for MSE and 12.93% for gradients. The test results of the FAM system show an average error of 0%. The test results on the test data show an average error for MSE of 7.67% and an average error for gradients of 8.55%. Besides being able to predict the MSE and gradient for a particular NN architecture, FAM can also determine the NN architectural model that provides the optimal combination of MSE and gradient. Further research will improve the performance of the FAM system.
Real-time arrival picking of rock microfracture signals based on convolutional-recurrent neural network and its engineering application

Accurately picking P- and S-wave arrivals of microseismic (MS) signals in real-time directly influences the early warning of rock mass failure. A common contradiction between accuracy and computation exists in the current arrival picking methods. Thus, a real-time arrival picking method of MS signals is constructed based on a convolutional-recurrent neural network (CRNN). This method fully utilizes the advantages of convolutional layers and gated recurrent units (GRU) in extracting short- and long-term features, in order to create a precise and lightweight arrival picking structure. Then, the synthetic signals with field noises are used to evaluate the hyperparameters of the CRNN model and obtain an optimal CRNN model. The actual operation on various devices indicates that compared with the U-Net method, the CRNN method achieves faster arrival picking with less performance consumption. An application of large underground caverns in the Yebatan hydropower station (YBT) project shows that compared with the short-term average/long-term average (STA/LTA), Akaike information criterion (AIC) and U-Net methods, the CRNN method has the highest accuracy within four sampling points, which is 87.44% for P-wave and 91.29% for S-wave, respectively. The sum of mean absolute errors (MAESUM) of the CRNN method is 4.22 sampling points, which is lower than that of the other methods. Among the four methods, the MS sources location calculated based on the CRNN method shows the best consistency with the actual failure, which occurs at the junction of the shaft and the second gallery. Thus, the proposed method can pick up P- and S-arrival accurately and rapidly, providing a reference for rock failure analysis and evaluation in engineering applications.
Random projection forest initialization for graph convolutional networks

Graph convolutional networks (GCNs) were a great step towards extending deep learning to graphs. GCN uses the graph G and the feature matrix X as inputs. However, in most cases the graph G is missing and we are only provided with the feature matrix X. To solve this problem, classical graphs such as k-nearest neighbor (k-nn) are usually used to construct the graph G and initialize the GCN. Although it is computationally efficient to construct k-nn graphs, the constructed graph might not be very useful for learning. In a k-nn graph, points are restricted to have a fixed number of edges, and all edges in the graph have equal weights. Our contribution is Initializing GCN using a graph with varying weights on edges, which provides better performance compared to k-nn initialization. Our proposed method is based on random projection forest (rpForest). rpForest enables us to assign varying weights on edges indicating varying importance, which enhanced the learning. The number of trees is a hyperparameter in rpForest. We performed spectral analysis to help us setting this parameter in the right range. In the experiments, initializing the GCN using rpForest provides better results compared to k-nn initialization.•Constructing the graph G using rpForest sets varying weights on edges, which represents the similarity between a pair of samples.Unlike k-nearest neighbor graph where all weights are equal.•Using rpForest graph to initialize GCN provides better results compared to k-nn initialization. The varying weights in rpForest graph quantify the similarity between samples, which guided the GCN training to deliver better results.•The rpForest graph involves the tuning of the hyperparameter (number of trees T). We provided an informative way to set this hyperparameter through spectral analysis.
Identification of IT Governance and Management Objectives and Target Process Capability Level in Government Institution

This study attempts to review whether the organization's IT governance and management practices are appropriate, whether other practices are needed, and the level of capability required for the practices implemented. This study uses COBIT 2019 (Control Objectives for Information and Related Technology 2019) as a reference for guideline practices and COBIT 2019 design factors as a method to identify organizational IT governance and management objectives. Based on this research, it shows that organizational IT governance and management are in accordance with national policies. However, in implementation, it still needs improvement with several other IT governance and management practices according to COBIT 2019. Based on the final results of this study after refinement of the design factor, the proposed IT objectives and practices that meet compliance with a policy are APO 13, APO 12, BAI 08, BAI 06, BAI 09, APO 14, DSS 02, and APO 07. However, adjustments still need to be made by adding some of the practices that are in line with the characteristics of the organization, which are MEA 03, DSS 05, EDM 03, and BAI 11. This study identifies appropriate IT governance and management objectives in the Province of West Java, Republic of Indonesia.
Attention-based multimodal fusion with contrast for robust clinical prediction in the face of missing modalities

With the increasing amount and growing variety of healthcare data, multimodal machine learning supporting integrated modeling of structured and unstructured data is an increasingly important tool for clinical machine learning tasks. However, it is non-trivial to manage the differences in dimensionality, volume, and temporal characteristics of data modalities in the context of a shared target task. Furthermore, patients can have substantial variations in the availability of data, while existing multimodal modeling methods typically assume data completeness and lack a mechanism to handle missing modalities.We propose a Transformer-based fusion model with modality-specific tokens that summarize the corresponding modalities to achieve effective cross-modal interaction accommodating missing modalities in the clinical context. The model is further refined by inter-modal, inter-sample contrastive learning to improve the representations for better predictive performance. We denote the model as Attention-based cRoss-MOdal fUsion with contRast (ARMOUR). We evaluate ARMOUR using two input modalities (structured measurements and unstructured text), six clinical prediction tasks, and two evaluation regimes, either including or excluding samples with missing modalities.Our model shows improved performances over unimodal or multimodal baselines in both evaluation regimes, including or excluding patients with missing modalities in the input. The contrastive learning improves the representation power and is shown to be essential for better results. The simple setup of modality-specific tokens enables ARMOUR to handle patients with missing modalities and allows comparison with existing unimodal benchmark results.We propose a multimodal model for robust clinical prediction to achieve improved performance while accommodating patients with missing modalities. This work could inspire future research to study the effective incorporation of multiple, more complex modalities of clinical data into a single model.
Mixed-precision quantization-aware training for photonic neural networks

Abstract The energy demanding nature of deep learning (DL) has fueled the immense attention for neuromorphic architectures due to their ability to operate in a very high frequencies in a very low energy consumption. To this end, neuromorphic photonics are among the most promising research directions, since they are able to achieve femtojoule per MAC efficiency. Although electrooptical substances provide a fast and efficient platform for DL, they also introduce various noise sources that impact the effective bit resolution, introducing new challenges to DL quantization. In this work, we propose a quantization-aware training method that gradually performs bit reduction to layers in a mixed-precision manner, enabling us to operate lower-precision networks during deployment and further increase the computational rate of the developed accelerators while keeping the energy consumption low. Exploiting the observation that intermediate layers have lower-precision requirements, we propose to gradually reduce layers’ bit resolutions, by normally distributing the reduction probability of each layer. We experimentally demonstrate the advantages of mixed-precision quantization in both performance and inference time. Furthermore, we experimentally evaluate the proposed method in different tasks, architectures, and photonic configurations, highlighting its immense capabilities to reduce the average bit resolution of DL models while significantly outperforming the evaluated baselines.
A Hierarchical Motion Planning Method for Mobile Manipulator

This paper focuses on motion planning for mobile manipulators, which includes planning for both the mobile base and the manipulator. A hierarchical motion planner is proposed that allows the manipulator to change its configuration autonomously in real time as needed. The planner has two levels: global planning for the mobile base in two dimensions and local planning for both the mobile base and the manipulator in three dimensions. The planner first generates a path for the mobile base using an optimized A* algorithm. As the mobile base moves along the path with the manipulator configuration unchanged, potential collisions between the manipulator and the environment are checked using the environment data obtained from the on-board sensors. If the current manipulator configuration is in a potential collision, a new manipulator configuration is searched. A sampling-based heuristic algorithm is used to effectively find a collision-free configuration for the manipulator. The experimental results in simulation environments proved that our heuristic sampling-based algorithm outperforms the conservative random sampling-based method in terms of computation time, percentage of successful attempts, and the quality of the generated configuration. Compared with traditional methods, our motion planning method could deal with 3D obstacles, avoid large memory requirements, and does not require a long time to generate a global plan.