Reviewerly/exHarmony huggingface dataset:
User Feedback-based Online Learning for Intent Classification

Intent classification is a key task in natural language processing (NLP) that aims to infer the goal or intention behind a user’s query. Most existing intent classification methods rely on supervised deep models trained on large annotated datasets of text-intent pairs. However, obtaining such datasets is often expensive and impractical in real-world settings. Furthermore, supervised models may overfit or face distributional shifts when new intents, utterances, or data distributions emerge over time, requiring frequent retraining. Online learning methods based on user feedback can overcome this limitation, as they do not need access to intents while collecting data and adapting the model continuously. In this paper, we propose a novel multi-armed contextual bandit framework that leverages a text encoder based on a large language model (LLM) to extract the latent features of a given utterance and jointly learn multimodal representations of encoded text features and intents. Our framework consists of two stages: offline pretraining and online fine-tuning. In the offline stage, we train the policy on a small labeled dataset using a contextual bandit approach. In the online stage, we fine-tune the policy parameters using the REINFORCE algorithm with a user feedback-based objective, without relying on the true intents. We further introduce a sliding window strategy for simulating the retrieval of data samples during online training. This novel two-phase approach enables our method to efficiently adapt to dynamic user preferences and data distributions with improved performance. An extensive set of empirical studies indicate that our method significantly outperforms policies that omit either offline pretraining or online fine-tuning, while achieving competitive performance to a supervised benchmark trained on an order of magnitude larger labeled dataset.
Underwater Image Translation via Multi-Scale Generative Adversarial Network

The role that underwater image translation plays assists in generating rare images for marine applications. However, such translation tasks are still challenging due to data lacking, insufficient feature extraction ability, and the loss of content details. To address these issues, we propose a novel multi-scale image translation model based on style-independent discriminators and attention modules (SID-AM-MSITM), which learns the mapping relationship between two unpaired images for translation. We introduce Convolution Block Attention Modules (CBAM) to the generators and discriminators of SID-AM-MSITM to improve its feature extraction ability. Moreover, we construct style-independent discriminators that enable the discriminant results of SID-AM-MSITM to be not affected by the style of images and retain content details. Through ablation experiments and comparative experiments, we demonstrate that attention modules and style-independent discriminators are introduced reasonably and SID-AM-MSITM performs better than multiple baseline methods.
Multimodal Turn Analysis and Prediction for Multi-party Conversations

This paper presents a computational study to analyze and predict turns (i.e., turn-taking and turn-keeping) in multiparty conversations. Specifically, we use a high-fidelity hybrid data acquisition system to capture a large-scale set of multi-modal natural conversational behaviors of interlocutors in three-party conversations, including gazes, head movements, body movements, speech, etc. Based on the inter-pausal units (IPUs) extracted from the in-house acquired dataset, we propose a transformer-based computational model to predict the turns based on the interlocutor states (speaking/back-channeling/silence) and the gaze targets. Our model can robustly achieve more than 80% accuracy, and the generalizability of our model was extensively validated through cross-group experiments. Also, we introduce a novel computational metric called “relative engagement level" (REL) of IPUs, and further validate its statistical significance between turn-keeping IPUs and turn-taking IPUs, and between different conversational groups. Our experimental results also found that the patterns of the interlocutor states can be used as a more effective cue than their gaze behaviors for predicting turns in multiparty conversations.
Augmented Immersive Viewing and Listening Experience Based on Arbitrarily Angled Interactive Audiovisual Representation

We propose an arbitrarily angled interactive audiovisual representation technique that combines a unique sound field synthesis with visual representation in order to augment the possibility of interactive immersive viewing experiences on mobile devices. This technique can synthesize two-channel stereo sound with constant stereo width having an arbitrary angle range from minimum 30 to maximum 360 degrees centering on an arbitrary direction from multi-channel surround sound. The visual representation can be chosen either equirectangular projection or stereographic projection. The developed video player app allows users to enjoy arbitrarily angled 360-degree videos by manipulating the touchscreen, and the stereo sound and the visual representation changes in terms of its spatial synchronization depending on the view. The app was released as a demonstration, and its acceptability and worth were investigated through interviews and subjective assessment tests. The app has been well received, and to date, more than 30 pieces of content have been produced in multiple genres, with a total of more than 200,000 views.
Toward Fair Facial Expression Recognition with Improved Distribution Alignment

We present a novel approach to mitigate bias in facial expression recognition (FER) models. Our method aims to reduce sensitive attribute information such as gender, age, or race, in the embeddings produced by FER models. We employ a kernel mean shrinkage estimator to estimate the kernel mean of the distributions of the embeddings associated with different sensitive attribute groups, such as young and old, in the Hilbert space. Using this estimation, we calculate the maximum mean discrepancy (MMD) distance between the distributions and incorporate it in the classifier loss along with an adversarial loss, which is then minimized through the learning process to improve the distribution alignment. Our method makes sensitive attributes less recognizable for the model, which in turn promotes fairness. Additionally, for the first time, we analyze the notion of attractiveness as an important sensitive attribute in FER models and demonstrate that FER models can indeed exhibit biases towards more attractive faces. To prove the efficacy of our model in reducing bias regarding different sensitive attributes (including the newly proposed attractiveness attribute), we perform several experiments on two widely used datasets, CelebA and RAF-DB. The results in terms of both accuracy and fairness measures outperform the state-of-the-art in most cases, demonstrating the effectiveness of the proposed method.
Weaving Augmented Reality Markers

This paper presents the use of weaving as a technique to create functional augmented reality (AR) markers using different textile structures and colors. We conducted experiments with plain, twill, and satin weaves, as well as varying colors in the warp, to test the effectiveness of the markers. Our findings show that weaving is a viable method for creating AR markers, and the software can detect markers even with varying colors and slightly misaligned quadrants. This work opens up new possibilities for weaving and textile structures in AR design.
Estimation of Violin Bow Pressure Using Photo-Reflective Sensors

The violin is one of the most popular instruments, but it is hard to learn. The bowing of the right hand is a crucial factor in determining the tone quality, but it is too complex to master, teach, and reproduce. Therefore, many studies have attempted to measure and analyze the bowing of the violin to help record performances and support practice. This work aimed to measure bow pressure, one of the parameters of bowing motion.
An ODE-based neural network with Bayesian optimization

An application of the Bayesian optimization to an ordinary differential equation-based neural network is proposed. The loss function was considered as a black box function of the coefficients, and Bayesian optimization was applied to obtain desirable parameter values. The proposed method drastically simplifies the implementation because the adjoint method-based updating of coefficients is not required. Numerical experiments demonstrate that the performance of the proposed method is comparable to that of existing methods.
Robot Duck Debugging: Can Attentive Listening Improve Problem Solving?

While thinking aloud has been reported to positively affect problem-solving, the effects of the presence of an embodied entity (e.g., a social robot) to whom words can be directed remain mostly unexplored. In this work, we investigated the role of a robot in a "rubber duck debugging" setting, by analyzing how a robot's listening behaviors could support a thinking-aloud problem-solving session. Participants completed two different tasks while speaking their thoughts aloud to either a robot or an inanimate object (a giant rubber duck). We implemented and tested two types of listener behavior in the robot: a rule-based heuristic and a deep-learning-based model. In a between-subject user study with 101 participants, we evaluated how the presence of a robot affected users' engagement in thinking aloud, behavior during the task, and self-reported user experience. In addition, we explored the impact of the two robot listening behaviors on those measures. In contrast to prior work, our results indicate that neither the rule-based heuristic nor the deep learning robot conditions improved performance or perception of the task, compared to an inanimate object. We discuss potential explanations and shed light on the feasibility of designing social robots as assistive tools in thinking-aloud problem-solving tasks.
Geometric model in visual space

The images that our eyes perceive are slightly different from the objects. Images in visual space are illusions. However, they are indeed based on optics. In physical space, parallel lines never intersect. In visual space, however, parallel lines intersect at the vanishing point. This is not limited to the human eye; camera lenses produce images similarly. If visual space is spherical, the phenomenon of intersecting parallel lines is no mystery. This paper provides a new geometric model considering the intersection of parallel lines at vanishing points.
Secure Registration Protocol for the Internet of Drones Using Blockchain and Physical Unclonable Function Technology

Unmanned aerial vehicles (UAVs) have become increasingly popular in recent years and are applied in various fields, from commercial and scientific to military and humanitarian operations. However, their usage presents many challenges, including limited resources, scalability issues, insecure communication, and inefficient solutions. We developed a secure and scalable registration protocol to address these issues using LoRa technology. Our solution involves the usage of the physical unclonable function (PUF) and blockchain technology for key exchange. PUF also ensures security against physical tampering, and blockchain is applied to share the symmetric key among the base stations. After the registration, the later communication messages are encrypted with AES-GCM to provide authentication and confidentiality between the parties. We conducted a security analysis of the registration protocol using the ProVerif tool, and our solution meets the security requirements, including the mutual authentication of entities, key freshness, key secrecy and also key confirmation properties. Besides the Proverif-based analysis, an informal security analysis is also provided that shows that the registration is protected against a variety of well-known active and passive security attacks. As drone resources are limited, we also prepared a proof of concept to test our solution under real-life conditions, focusing on efficiency and lightweight operations.
Cervical Cancer Diagnosis Using Stacked Ensemble Model and Optimized Feature Selection: An Explainable Artificial Intelligence Approach

Cervical cancer affects more than half a million women worldwide each year and causes over 300,000 deaths. The main goals of this paper are to study the effect of applying feature selection methods with stacking models for the prediction of cervical cancer, propose stacking ensemble learning that combines different models with meta-learners to predict cervical cancer, and explore the black-box of the stacking model with the best-optimized features using explainable artificial intelligence (XAI). A cervical cancer dataset from the machine learning repository (UCI) that is highly imbalanced and contains missing values is used. Therefore, SMOTE-Tomek was used to combine under-sampling and over-sampling to handle imbalanced data, and pre-processing steps are implemented to hold missing values. Bayesian optimization optimizes models and selects the best model architecture. Chi-square scores, recursive feature removal, and tree-based feature selection are three feature selection techniques that are applied to the dataset For determining the factors that are most crucial for predicting cervical cancer, the stacking model is extended to multiple levels: Level 1 (multiple base learners) and Level 2 (meta-learner). At Level 1, stacking (training and testing stacking) is employed for combining the output of multi-base models, while training stacking is used to train meta-learner models at level 2. Testing stacking is used to evaluate meta-learner models. The results showed that based on the selected features from recursive feature elimination (RFE), the stacking model has higher accuracy, precision, recall, f1-score, and AUC. Furthermore, To assure the efficiency, efficacy, and reliability of the produced model, local and global explanations are provided.
Adaptive Marginal Multi-Target Bayes Filter without Need for Clutter Density for Object Detection and Tracking

The random finite set (RFS) approach for multi-target tracking is widely researched because it has a rigorous theoretical basis. However, many prior parameters such as the clutter density, survival probability and detection probability of the target, pruning threshold, merging threshold, initial state of the birth object and its error covariance matrix are required in the standard RFS-based filters. In real application scenes, it is difficult to obtain these prior parameters. To address this problem, an adaptive marginal multi-target Bayes filter without the need for clutter density is proposed. This filter obviates the need for prior clutter density and survival probability. Instead of using the prior initial states of newborn targets and their error covariance matrices, it uses two scans of observations to generate the initial states of potential birth targets and their error covariance matrices according to the least squares technique. Simulation results reveal that the proposed adaptive filter has smaller OSPA and OSPA(2) errors as well as less cardinality error than the adaptive RFS-based filters. The OSPA and OSPA(2) errors have been reduced by more than 20% compared to those of the adaptive RFS-based filters.
Research on the Identification of a Winter Olympic Multi-Intent Chinese Problem Based on Multi-Model Fusion

Aiming at addressing the inability of traditional web technologies to effectively respond to Winter-Olympics-related user questions containing multiple intentions, this paper explores a multi-model fusion-based multi-intention recognition model BCNBLMATT to solve this problem. The model is proposed to address the characteristics of complex semantics, strong contextual relevance, and a large number of informative features of the Chinese problem text related to the Winter Olympics, as well as the limitations of the traditional word vector model, such as insufficient expression in the textual representation and the relative concern mechanism of feature expression. The BCNBLMATT model first obtains a comprehensive feature vector representation of the problem text through BERT. Then, a multi-scale text convolutional neural network model and a BiLSTM-Multi-heads attention model (a joint model combining a bidirectional long- and short-term attention network with a multi-head attention mechanism) are used to capture local features at more scales and contextually critical information features at more levels. Finally, the two obtained kinds of features are concatenated and fused to obtain richer and more comprehensive information about the problem text features, which improves the model’s performance in the multi-attention recognition task. Comparative experiments on the Winter Olympics Chinese question dataset and the MixATIS question dataset show that the BCNBLMATT model significantly improves the three metrics of macro-averaged precision, macro-averaged recall, and macro-averaged F1 value and exhibits better generalization. This study provides an effective solution to the multi-intent recognition task for Winter Olympic problems, overcomes the limitations of traditional models, and provides new ideas for improving the performance of multi-intent recognition.
Affordable Graph Neural Network Framework Using Topological Graph Contraction

Graph neural networks (GNNs) have many applications, including the medical field (e.g., neuroscience), thanks to their flexibility and expressiveness in handling unstructured graph-based data. However, one major issue in applying GNNs for medical applications is their unscalability in handling large and complex graphs, primarily due to their high memory usage. This high memory usage is controlled mainly by the data points’ interconnectivity and the neighbourhood expansion problem, which causes non-trivial mini-batching, unlike standard neural networks. While many recent publications are focusing on mini-batching GNN using sub-graph sampling/partitioning methods, there are many other directions to take in terms of reducing memory usage, such as quantisation and graph reduction methods. In this paper, we propose a novel topological graph contraction method using centrality measure and a memory-efficient GNN training framework (C-QSIGN), which incorporates our proposed contraction method along with several other state-of-the-art (SOTA) methods. Furthermore, we benchmarked our proposed model performance in terms of prediction quality and GPU usage against other SOTA methods. We show that C-QSIGN requires the least GPU memory compared to benchmark SOTA models (54% of SIGN and 18% of GCN on the Organ C dataset), allowing more efficient learning on complex, large-scale graphs. Our codes are publicly available at https://github.com/basiralab/CQSIGN .
Multi-task Learning for Few-Shot Differential Diagnosis of Breast Cancer Histopathology Images

Deep learning models may be useful for the differential diagnosis of breast cancer histopathology images. However, most modern deep learning methods are data-hungry. But, large annotated dataset of breast cancer histopathology images are elusive. As a result, the application of such deep learning methods for the differential diagnosis of breast cancer is limited. To deal with this problem, we propose a few-shot learning approach for the differential diagnosis of the histopathology images of breast tissue. Our model is trained through two stages. We initially train our model for a binary classification task of identifying benign and malignant tissues. Subsequently, we propose a multi-task learning strategy for the few-shot differential diagnosis of breast tissues. Experiments on publicly available breast cancer histopathology image datasets show the efficacy of the proposed method.
Combining Weakly Supervised Segmentation with Multitask Learning for Improved 3D MRI Brain Tumour Classification

Segmentation plays a crucial role in computer-aided medical image diagnosis, as it enables the models to focus on the region of interest (ROI) and improve classification performance. However, medical image datasets often lack segmentation masks due to cost. In this work, we have introduced a novel end-to-end pipeline that enhances classification by incorporating ROIs generated by a weakly supervised method as an auxiliary task in a deep multitask learning framework. We have demonstrated that our approach outperforms conventional methods that only rely on classification. By using weakly supervised segmentation, we are able to leverage pixel-wise information, which ultimately leads to improved classification performance.
Exigent Examiner and Mean Teacher: An Advanced 3D CNN-Based Semi-Supervised Brain Tumor Segmentation Framework

With the rise of deep learning applications to medical imaging, there has been a growing appetite for large and well-annotated datasets, yet annotation is time-consuming and hard to come by. In this work, we train a 3D semantic segmentation model in an advanced semi-supervised learning fashion. The proposed SSL framework consists of three models: a Student model that learns from annotated data and a large amount of raw data, a Teacher model with the same architecture as the student, updated by self-ensembling and which supervises the student through pseudo-labels, and an Examiner model that assesses the quality of the student’s inferences. All three models are built with 3D convolutional operations. The overall framework mimics a collaboration between a consistency training Student $$\leftrightarrow $$ Teacher module and an adversarial training Examiner $$\leftrightarrow $$ Student module. The proposed method is validated with various evaluation metrics on a public benchmarking 3D MRI brain tumor segmentation dataset. The experimental results of the proposed method outperform pre-existing semi-supervised methods. The source code, baseline methods, and dataset are available at https://github.com/ziyangwang007/CV-SSL-MIS .
Dynamic Canonical Data Model: An Architecture Proposal for the External and Data Loose Coupling for the Integration of Software Units

Integrating third-party and legacy systems has become a critical necessity for companies, driven by the need to exchange information with various entities such as banks, suppliers, customers, and partners. Ensuring data integrity, keeping integrations up-to-date, reducing transaction risks, and preventing data loss are all vital aspects of this complex task. Achieving success in this endeavor, which involves both technological and business challenges, necessitates the implementation of a well-suited architecture. This article introduces an architecture known as the Dynamic Canonical Data Model through Agnostic Messages. The proposal addresses the integration of loosely coupled software units, mainly when dealing with internal and external data integration. To illustrate the architecture’s components, a case study from the Mexican Logistics Company Paquetexpress is presented. This organization manages integrations across several platforms, including SalesForce and Oracle ERP, with clients like Amazon, Mercado Libre, Grainger, and Afull. Each of these incurs costs ranging from USD 30,000 to USD 36,000, with consultants from firms such as Quanam, K&amp;F, TSOL, and TekSi playing a crucial role in their execution. This consumes much time, making maintenance costs considerably high when clients request data transmission or type changes, particularly when utilizing tools like Oracle Integration Cloud (OIC) or Oracle Service Bus (OSB). The article provides insights into the architecture’s design and implementation in a real-world scenario within the delivery company. The proposed architecture significantly reduces integration and maintenance times and costs while maximizing scalability and encouraging the reuse of components. The source code for this implementation has been registered in the National Registry of Copyrights in Mexico.