Reviewerly/exHarmony huggingface dataset:
A Novel Approach for Pill-Prescription Matching with GNN Assistance and Contrastive Learning

Medication mistaking is one of the risks that can result in unpredictable consequences for patients. To mitigate this risk, we develop an automatic system that correctly identifies pill-prescription from mobile images. Specifically, we define a so-called pill-prescription matching task, which attempts to match the images of the pills taken with the pills’ names in the prescription. We then propose PIMA, a novel approach using Graph Neural Network (GNN) and contrastive learning to address the targeted problem. In particular, GNN is used to learn the spatial correlation between the text boxes in the prescription and thereby highlight the text boxes carrying the pill names. In addition, contrastive learning is employed to facilitate the modeling of cross-modal similarity between textual representations of pill names and visual representations of pill images. We conducted extensive experiments and demonstrated that PIMA outperforms baseline models on a real-world dataset of pill and prescription images that we constructed. Specifically, PIMA improves the accuracy from $$19.09\%$$ to $$46.95\%$$ compared to other baselines. We believe our work can open up new opportunities to build new clinical applications and improve medication safety and patient care.
Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learning

Well-annotated medical datasets enable deep neural networks (DNNs) to gain strong power in extracting lesion-related features. Building such large and well-designed medical datasets is costly due to the need for high-level expertise. Model pre-training based on ImageNet is a common practice to gain better generalization when the data amount is limited. However, it suffers from the domain gap between natural and medical images. In this work, we pre-train DNNs on ultrasound (US) domains instead of ImageNet to reduce the domain gap in medical US applications. To learn US image representations based on unlabeled US videos, we propose a novel meta-learning-based contrastive learning method, namely Meta Ultrasound Contrastive Learning (Meta-USCL). To tackle the key challenge of obtaining semantically consistent sample pairs for contrastive learning, we present a positive pair generation module along with an automatic sample weighting module based on meta-learning. Experimental results on multiple computer-aided diagnosis (CAD) problems, including pneumonia detection, breast cancer classification, and breast tumor segmentation, show that the proposed self-supervised method reaches state-of-the-art (SOTA). The codes are available at https://github.com/Schuture/Meta-USCL.
Dynamic Ensemble Selection Using Fuzzy Hyperboxes

Most dynamic ensemble selection (DES) methods utilize the K-Nearest Neighbors (KNN) algorithm to estimate the competence of classifiers in a small region surrounding the query sample. However, KNN is very sensitive to the local distribution of the data. Moreover, it also has a high computational cost as it requires storing the whole data in memory and performing multiple distance calculations during inference. Hence, the de-pendency on the KNN algorithm ends up limiting the use of DES techniques for large-scale problems. This paper presents a new DES framework based on fuzzy hyperboxes called FH-DES. Each hyperbox can represent a group of samples using only two data points (Min and Max corners). Thus, the hyperbox-based system will have less computational complexity than other dynamic se-lection methods. In addition, despite the KNN-based approaches, the fuzzy hyperbox is not sensitive to the local data distribution. Therefore, the local distribution of the samples does not affect the system's performance. Furthermore, in this research, for the first time, misclassified samples are used to estimate the competence of the classifiers, which has not been observed in previous fusion approaches. Experimental results demonstrate that the proposed method has high classification accuracy while having a lower complexity when compared with the state-of-the-art dynamic selection methods. The implemented code is available at https://github.com/redavtalab/FH-DES_IJCNN.git.
Cardiovascular System Pathologies Studying Using Virtual Reality Tools

Accelerated growth of the online studying mode tended to the lack of practical skills development. Though many specialties were covered with newly developed courses and educational platforms not all of them provided the necessary level of the hard skills gathering. In some areas such as medicine, engineering, human interaction practice is crucial since it is a basis of the specialists’ knowledge.The article covers the development of the software application to provide medical students with a tool that will show pathologies of the cardiovascular system. The complex consists of two modules. The first contains a scene consisting of a 3D heart model. It shows four different pathologies of the heart. The second scene is dedicated to laparoscopic surgery with stenting of coronary vessels. To simulate interaction during laparoscopic surgery Leap Motion controller was used. The instrumental basis of the project for 3d modeling is Blender, the project’s development environment is Unity 3D game engine. All the scripts describing the application’s logic were written in the C# programming language.
Open Information Extraction System For Extracting Relations in Legal Documents

Open Information Extraction system (OIE) extracts textual tuples consisting of arguments and its relation within the input sentence. OIE can also be considered as an unrestricted variant of information extraction. Throughout the years these types of systems have become more and more popular. The main factor for considering such a system is that they are made for speed, then any other factor. Also, one of the major advantages of using such a system is that the relations that are extracted are human-readable. Relation extraction in the legal domain has always remained a challenge because there is no structured data available. hence making this process more challenging. Since we are working in the legal domain, extracting the relations that are present in the document is very important. This leads us to a system that is capable of assisting people in the domain. The introduction of such a system would be really important for professionals like judges, advocates and also for the common people if they want to have a quick glance at certain documents. The outcome of these systems can be really helpful for future downstream applications like question answering, sentence similarity etc.
UAV Image Geo-Localization by Point-Line-Patch Feature Matching and ICLK Optimization

Estimating the coordinate positions corresponding to each pixel in the Unmanned Aerial Vehicles (UAV) image is a prerequisite in actual fields. However, GPS devices are vulnerable to interference or attack, making it impossible to obtain accurate position and attitude parameters. Therefore, UAV image geo-localization based on image content has become a research hotspot. This paper proposes a UAV image geo-Localization method in a GPS-denied environment based on deep-learning building semantic feature extraction and point-line-patch (PLP) feature matching, which could correct the longitude and latitude coordinates of each image pixel within the range of several km <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> areas. First, we employ an enhanced edge and multi-scale (EEMS) building extraction network to retain building edges and extract multi-scale buildings. Second, we design the PLP feature using the characteristics of buildings and the distribution of adjacent buildings. Then, we design a semantic matching algorithm based on the PLP feature and optimize the result using the inverse compositional Lucas-Kanade (ICLK) method for the UAV image geo-Iocalization. Experiments show that our proposed method can better locate each UAV image pixel's position, benefiting urban and regional spatio-temporal big data fusion and update.
Neural-Network-Based Models Ensemble for Identification in Distributed-Parameter Systems with Application to Elastic Materials Modeling

The purpose of the paper is to develop the methodology of training procedures for neural modeling of distributed-parameter systems with special attention given to systems whose dynamics are described by a fourth-order partial differential equation. The work is motivated by applications from control of elastic materials, such as deformable mirrors, vibrating beams, smart materials, etc. The numerical methods for solving such systems usually are characterized by large-scale and heavy computational cost, therefore they cannot be used for real-time prediction and/or estimation. Here we propose the data-driven approach for modeling distributed-parameter system based on the state-space neural networks forming the ensemble imitating a spatial discretization of the finite element method for solving the partial differential equations. Then, for each spatial subdomain the networks are trained with local temporal data reconstructing the dynamic system response and maintaining the accuracy of original method. Thus, we obtain the efficient on-line prediction model with a possibility for reduction of system scale owing to generalization property of neural networks. Finally, implementability of the proposed approach is confirmed by numerical simulations on the dynamic displacements estimation of elastic membrane.
Fairness-aware Adaptive Network Link Prediction

Network link prediction has attracted increasing attention due to its capability of extracting missing information, and evaluating network-evolving mechanisms. Despite the increasing popularity, fairness is widely under-explored in the area. Motivated by this, this study proposes novel fairness-aware graph augmentation designs to mitigate the bias in graph data while creating node representations. Different fairness notions on graphs are introduced to guide the designs of the adaptive structural and attributive augmentation schemes. Experimental results on real-world networks demonstrate that the introduced augmentation frameworks can improve group fairness measures for link prediction while providing comparable utility to state-of-the-art contrastive learning algorithms.
Radar: Reliable Resource Scheduling for Composable/Disaggregated Data Centers

Hardware disaggregation decouples resources (e.g., processors and memory) from monolithic servers, potentially improving service reliability. However, from another perspective, directly exposing resource modules to a shared network may adversely affect service reliability. In this article, we study a reliable resource allocation problem in disaggregated data centers, considering network impact and different disaggregation scales. We provide a mixed-integer linear programming formulation and a resource allocation framework named Radar for this problem. The numerical results demonstrate that the benefits of hardware disaggregation may be adversely affected by an imperfect network. It also shows that both the hardware backup and a proposed migration-based restoration can be applied to overcome this potential adverse effect.
Cyberbullying Detection in Code-Mixed Languages: Dataset and Techniques

The advent of the Internet is a boon to society. However, many of its banes cannot be undermined, and cyberbullying is one of them. In this work, we have created a benchmark corpus for cyberbullying detection in code-mixed languages. In India, most communications on different social media platforms are based on Hindi and English languages, and language switching is a common practice in digital communication. To investigate how code-mixed data can be handled effectively, BERT language model and VecMap based bilingual embedding along with a two-channel convolutional neural network model, namely BERT+VecMap-CNN, have been used. The input to one channel is the BERT language model and that to the other is the bilingual word embedding based on VecMap. As a baseline, we used standard machine learning models, as well as deep neural network models like CNN and LSTM. Our proposed model outperforms the baselines with overall accuracy and F1-measure values of 81.12%, and 81.03%, respectively. Furthermore, a different benchmark code-mixed dataset has been considered to show the robustness of our proposed model. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Reverse Image Filtering Using Total Derivative Approximation and Accelerated Gradient Descent

In this paper, a new problem of reverse image filtering is addressed. The problem is to reverse the effect of an image filter given the observation <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</i> = <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">g(x)</i> . The filter <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">g</i> is modelled as an available black box. An inverse method is proposed to recover the input image <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</i> . The key idea is to formulate this inverse problem as minimizing a local patch-based cost function. To use gradient descent for the minimization, total derivative is used to approximate the unknown gradient. This paper presents a study of factors affecting the convergence and quality of the output in the Fourier domain when the filter is linear. It discusses the convergence property for nonlinear filters by using contraction mapping as a tool. It also presents applications of the accelerated gradient descent algorithms to three gradient-free reverse filters, including the one proposed in this paper. Results from extensive experiments are used to evaluate the complexity and effectiveness of the proposed algorithm. The proposed algorithm outperforms the state-of-the-art in two aspects. (1) It is at the same level of complexity as that of the fastest reverse filter, but it can reverse a larger number of filters. (2) It can reverse the same list of filters as that of a very complex reverse filter, but its complexity is much lower.
Efficient Video Understanding-Based Random Hand Gesture Authentication

Hand gesture is an emerging biometric trait that is receiving increasing attention. In recent years, there have emerged several video understanding-based hand gesture authentication methods. However, their parameter number is too large to be deployed directly on mobile devices. Thus, we introduce the advanced Knowledge Distillation via Knowledge Review (KDKR) to compress the Temporal Difference Symbiotic Neural Network (TDS-Net). Compared with the teacher model, whose average equal error rates are 2.846% and 6.953% in single and cross sessions, the parameter number of the student model is reduced by 77.29%, while the average equal error rates only increase by 1.757% and 4.229%, respectively. Besides, we investigate some attention modules in depth to improve the KDKR and obtain a suitable fusion method in the KDKR for random hand gesture authentication networks.
Improving Multiple Machine Vision Tasks in the Compressed Domain

There is a growing number of images that are analyzed by machines rather than just humans. Recently, most machine vision tasks are based on decoded images which require an image compression (encoding/decoding) framework. However, using the decoded images in the pixel-domain has two drawbacks: 1) the complexity is high for the decoder part, 2) the accuracy (e.g., mIoU, mean absolute error, and average precision) of machine vision tasks will be degraded since decoded images only aim to optimize the human perceived quality (e.g., PSNR) so that information required for machine vision tasks will be lost during the decoding process. In this paper, we improve the machine vision tasks in the compressed domain. 1) A gate module is utilized to effectively select some compressed-domain features. 2) Knowledge distillation is introduced to improve the accuracy. 3) A training strategy is explored to support multiple tasks including the image compression. The experimental results show that we can achieve better rate-accuracy/distortion and lower complexity compared with the state-of-the-art pixel-domain work that can take both machine and human vision tasks.
High-Dimensional Signal Processing using Classical-Quantum Machine Learning Pipelines with the TensorFlow Stack, Cirq-NISQ, and Vertica

Modeling High Dimensional Signal Data (HDSD) is challenging due to statistical complexities and an imbalance of noise to signal data, which creates ground truth problems. Likewise, enabling recognition and detection analytics on the information-bearing portion of the noise-obscured HDSD is computationally cost-prohibitive due to the unprecedented size, speed, and scale of HDSD. Consequently, human experts often out-perform Deep Learning (DL) algorithms at HDSD recognition. This is problematic due to the spectrum of applications reliant on accurate HDSD detection. These include multi-sensor and biologic signal detection, virus outbreak surveillance, genetic association identification, image, audio, and sonar processing, source separation for speech identification, astrophysical source detection, financial time series, and satellite payload processing. To solve this problem and facilitate favorable DL outcomes, synthetic data possessing enough positive samples is needed to balance the HDSD dataset. Research reveals an entangled Quantum Generative Adversarial Network (QGAN) utilizing parameterized quantum circuits for the QGAN Generator (QGEN) and Discriminator (QDIS) exhibit an exponential advantage over a classical-GAN when generating highly accurate and cost-effective synthetic data. Using this method, an unsupervised TensorFlow-Quantum QGAN is prepared from Cirq Noisy Intermediate-Scale Quantum (NISQ) circuits. The QGEN and QDIS create new statistically consistent synthetic HDSD from real data; thereby solving the ground truth problem. Next, TensorFlow-GPU facilitates classical-DL to train the new synthetic HDSD dataset. Once validated, the TensorFlow model is exported to the production Vertica analytics tier for fast and accurate ML inference, where the HDSD is instantly identified and classified at ingest.
Deep Learning for Arbitrary-Shaped Water Pooling Region Detection on Aerial Images

Recent rapid development in Unmanned Aerial Vehicles (UAVs) have extensively promoted several types of civilian tasks. In this paper, we propose and compare two different deep learning and convolutional neural network methods to detect and extract the region of water pooling areas, such as gutters, abandoned ponds, tires, and other water retaining areas on rooftops, using UAVs based aerial images. The performance comparison between the YOLOv4 algorithm and the Mask-RCNN algorithm was explored in the case study to identify the best deep learning method for detecting these uneven regions of water pooling. Experimental results show that the Mask-RCNN approach efficiently detects these uneven areas in an aerial image while simultaneously generating a high-quality segmentation mask for each instance. On the other hand, YOLOv4 detects the best bounding box for the area of interest. The mean average precision (mAP) scores for Mask-RCNN and YOLOv4 are 71.67% and 57.9% respectively. The Mask-RCNN system has shown promising results on test images and video clips. Such real-time detection systems would eventually help to identify mosquito breeding sites to assist the dengue eradication as well as to identify suitable water resources for daily uses, thereby facilitating a better community health system.
Commonsense Knowledge Construction with Concept and Pretrained Model

Commonsense knowledge (CSK) is the information that people use in daily life but do not often mention. It summarizes the practical knowledge about how the world works. Existing machines have knowledge but lack commonsense because they do not understand and master commonsense knowledge in the same way that humans do. In the latest works, crowdsourcing-based method is costly and has low coverage, knowledge base completion method can highly fit samples, and methods extracted from unstructured data have the defects of low quality. CG &BF is commonsense knowledge construction with a concept-based generator and a BERT-based filter. We utilize semantic search for node matching and entropy encoder for filtering triples with high abstraction. Two algorithms based on concept aggregation and path credibility are proposed to obtain high-quality CSK triples. We subsequently finetuning a BERT to filter incorrect triples. We obtain 500,000 CSK triples based on ConceptNet, which is superior to other construction methods in novelty and quality. In the reading comprehension task, the three-way attention network is selected as the basic model and the knowledge we generate enables the base model to perform better, which proves that the output of CG &BF has higher quality and ease of use.
Do learned representations respect causal relationships?

Data often has many semantic attributes that are causally associated with each other. But do attribute-specific learned representations of data also respect the same causal relations? We answer this question in three steps. First, we introduce NCINet, an approach for obser-vational causal discovery from high-dimensional data. It is trained purely on synthetically generated representations and can be applied to real representations, and is specif-ically designed to mitigate the domain gap between the two. Second, we apply NCINet to identify the causal relations between image representations of different pairs of at-tributes with known and unknown causal relations between the labels. For this purpose, we consider image represen-tations learned for predicting attributes on the 3D Shapes, CelebA, and the CASIA-WebFace datasets, which we an-notate with multiple multi-class attributes. Third, we an-alyze the effect on the underlying causal relation between learned representations induced by various design choices in representation learning. Our experiments indicate that (1) NCINet significantly outperforms existing observational causal discovery approaches for estimating the causal relation between pairs of random samples, both in the presence and absence of an unobserved confounder, (2) under controlled scenarios, learned representations can indeed satisfy the underlying causal relations between their respective labels, and (3) the causal relations are positively correlated with the predictive capability of the representations. Code and annotations are available at: https://github.com/human-analysis/causal-relations-between-representations.