Reviewerly/exHarmony huggingface dataset:
Multi-View Visual Semantic Embedding

Visual Semantic Embedding (VSE) is a dominant method for cross-modal vision-language retrieval. Its purpose is to learn an embedding space so that visual data can be embedded in a position close to the corresponding text description. However, there are large intra-class variations in the vision-language data. For example, multiple texts describing the same image may be described from different views, and the descriptions of different views are often dissimilar. The mainstream VSE method embeds samples from the same class in similar positions, which will suppress intra-class variations and lead to inferior generalization performance. This paper proposes a Multi-View Visual Semantic Embedding (MV-VSE) framework, which learns multiple embeddings for one visual data and explicitly models intra-class variations. To optimize MV-VSE, a multi-view upper bound loss is proposed, and the multi-view embeddings are jointly optimized while retaining intra-class variations. MV-VSE is plug-and-play and can be applied to various VSE models and loss functions without excessively increasing model complexity. Experimental results on the Flickr30K and MS-COCO datasets demonstrate the superior performance of our framework.
A Model Predictive Control Approach for Reach Redirection in Virtual Reality

Reach redirection is an illusion-based virtual reality (VR) interaction technique where a user’s virtual hand is shifted during a reach in order to guide their real hand to a physical location. Prior works have not considered the underlying sensorimotor processes driving redirection. In this work, we propose adapting a sensorimotor model for goal-directed reach to obtain a model for visually-redirected reach, specifically by incorporating redirection as a sensory bias in the state estimate used by a minimum jerk motion controller. We validate and then leverage this model to develop a Model Predictive Control (MPC) approach for reach redirection, enabling the real-time generation of spatial warping according to desired optimization criteria (e.g., redirection goals) and constraints (e.g., sensory thresholds). We illustrate this approach with two example criteria – redirection to a desired point and redirection along a desired path – and compare our approach against existing techniques in a user evaluation.
EmoBalloon - Conveying Emotional Arousal in Text Chats with Speech Balloons

Text chat applications are an integral part of daily social and professional communication. However, messages sent over text chat applications do not convey vocal or nonverbal information from the sender, and detecting the emotional tone in text-only messages is challenging. In this paper, we explore the effects of speech balloon shapes on the sender-receiver agreement regarding the emotionality of a text message. We first investigated the relationship between the shape of a speech balloon and the emotionality of speech text in Japanese manga. Based on these results, we created a system that automatically generates speech balloons matching linear emotional arousal intensity by Auxiliary Classifier Generative Adversarial Networks (ACGAN). Our evaluation results from a controlled experiment suggested that the use of emotional speech balloons outperforms the use of emoticons in decreasing the differences between message senders' and receivers' perceptions about the level of emotional arousal in text messages.
Unmaking as Agonism: Using Participatory Design with Youth to Surface Difference in an Intergenerational Urban Context

Design has been used to contest existing socio-technical arrangements, provoke conversations around matters of concern, and operationalize radical theories such as agonism, which embraces difference and contention. However, the focus is usually on creating something new: a product, interface or artifact. In this paper, we investigate what happens when critical unmaking is deployed as a deliberate design strategy in an intergenerational, agonistic urban context. Specifically, we report on how youth in a six-week design internship used unmaking as a design move to subvert conventional narratives about their surrounding urban context. We analyze how this led to conflictual encounters at the local senior center, and compare it to the other, making-centric proposals which received favorable feedback but failed to raise the same important discussions. Through this ethnographic account, we argue that critical unmaking is important yet overlooked, and should be in the repertoire of design moves available for agonism and provocation.
Improving quantum simulation efficiency of final state radiation with dynamic quantum circuits

B. Nachman et al. [Phys. Rev. Lett. 126, 062001 (2021)] recently introduced an algorithm (QPS) for simulating parton showers with intermediate flavor states using polynomial resources on a digital quantum computer. We make use of a new quantum hardware capability called dynamical quantum computing to improve the scaling of this algorithm, which significantly improves the method precision. In particular, we modify the quantum parton shower circuit to incorporate midcircuit qubit measurements, resets, and quantum operations conditioned on classical information. This reduces the computational depth from $\mathcal{O}({N}^{5}{\mathrm{log}}_{2}(N{)}^{2})$ to $\mathcal{O}({N}^{3}{\mathrm{log}}_{2}(N{)}^{2})$ and the qubit requirements from $\mathcal{O}(N{\mathrm{log}}_{2}(N))$ to $\mathcal{O}(N)$. Using ``matrix product state'' state vector simulators, we demonstrate that the improved algorithm yields expected results for 2, 3, 4, and 5-steps of the algorithm. We compare absolute costs with the original QPS algorithm, and show that dynamical quantum computing can significantly reduce costs in the class of digital quantum algorithms representing quantum walks (which includes the QPS). Python code that implements QPS, both with and without dynamic gates, is publicly available on Github.
Operator training framework for hybrid environments: An Augmented Reality module using machine learning object recognition

As market demands are characterized by more customized products with shorter lifecycles, it is obligatory for modern operators to manage recurrent product or manufacturing system changes. In contrary to previous years, adaptation to such changes prerequires memorization of more information, and familiarization with more complex systems and resources in a shorter period of time. This manuscript presents a novel operator training framework based on Augmented Reality (AR) technology. More specifically, intuitive instructions enhanced with machine learning-based physical object detection are used for making steeper learning curves and providing hands-on experience to operators. The implemented application also supports a walkthrough mode where users can get familiarized with Information and Communication Technologies (ICT) data streams besides fenceless Human-Robot coexistence in collaborative schemes. An automotive case study is used for evaluating the performance of the training framework through a Human-Robot Collaboration (HRC) assembly scenario.
Two dimensional quaternion valued singular spectrum analysis with application to image denoising

Compared to the traditional single color plane based image denoising methods, the quaternion valued singular value decomposition (QSVD) exploits the relationship among different color planes. Hence, it has been applied to the color image denoising. On the other hand, compared to the non-overlapping based image denoising methods, the two dimensional real valued singular spectrum analysis (2DSSA) constructs the trajectory matrix with many elements in the matrix being overlapped. Since the 2DSSA exploits the local information within each color plane, it has also been applied to the single color plane based image denoising. However, neither these two image denoising methods can exploit the relationship among the color planes and the local information within each color plane simultaneously. Therefore, this paper proposes a two dimensional quaternion valued singular spectrum analysis (2DQSSA) based method for performing the color image denoising. Our proposed method can enjoy the advantages of both methods. However, the most critical issue for the 2DQSSA is on the selection of these 2DQSSA components. This paper finds that the optimal total number of the 2DQSSA components used for performing the reconstruction is monotonic decreasing with respect to the power of the noise in the image. Therefore, the polynomial fitting approach is proposed to model this relationship. Since the 2DQSSA based denoising method exploits the relationship among the red color plane, the green color plane and the blue color plane, the 2DQSSA based denoising method outperforms the conventional single color plane based denoising methods. Moreover, since the 2DQSSA based denoising method also exploits the local relationship within each color plane, the 2DQSSA based denoising method outperforms the non-overlapping based methods.
Lung Cancer Subtype Diagnosis by Fusing Image-genomics Data and Hybrid Deep Networks

Accurate diagnosis of cancer subtypes is crucial for precise treatment, because different cancer subtypes are involved with different pathology and require different therapies. Although deep learning techniques have made great success in computer vision and other fields, they do not work well on Lung cancer subtype diagnosis, due to the distinction of slide images between different cancer subtypes is ambiguous. Furthermore, they often over-fit to high-dimensional genomics data with limited samples, and do not fuse the image and genomics data in a sensible way. In this paper, we propose a hybrid deep network based approach LungDIG for Lung cancer subtype Diagnosis by fusing Image-Genomics data. LungDIG first tiles the tissue slide image into small patches and extracts the patch-level features by fine-tuning an Inception-V3 model. Since the patches may contain some false positives in non-diagnostic regions, it further designs a patch-level feature combination strategy to integrate the extracted patch features and maintain the diversity between different cancer subtypes. At the same time, it extracts the genomics features from Copy Number Variation data by an attention based nonlinear extractor. Next, it fuses the image and genomics features by an attention based multilayer perceptron (MLP) to diagnose cancer subtype. Experiments on TCGA lung cancer data show that LungDIG can not only achieve higher accuracy for cancer subtype diagnosis than state-of-the-art methods, but also have a high authenticity and good interpretability.
Dynamic Hand Gesture Recognition Using Improved Spatio-Temporal Graph Convolutional Network

Hand gesture recognition is essential to human-computer interaction as the most natural way of communicating. Furthermore, with the development of 3D hand pose estimation technology and the performance improvement of low-cost depth cameras, skeleton-based dynamic hand gesture recognition has received much attention. This paper proposes a novel multi-stream improved spatio-temporal graph convolutional network (MS-ISTGCN) for skeleton-based dynamic hand gesture recognition. We adopt an adaptive spatial graph convolution that can learn the relationship between distant hand joints and propose an extended temporal graph convolution with multiple dilation rates that can extract informative temporal features from short to long periods. Furthermore, we add a new attention layer consisting of effective spatio-temporal attention and channel attention between the spatial and temporal graph convolution layers to find and focus on key features. Finally, we propose a multi-stream structure that feeds multiple data modalities (i.e., joints, bones, and motions) as inputs to improve performance using the ensemble technique. Each of the three-stream networks is independently trained and fused to predict the final hand gesture. The performance of the proposed method is verified through extensive experiments with two widely used public dynamic hand gesture datasets: SHREC’17 Track and DHG-14/28. Our proposed method achieves the highest recognition accuracy in various gesture categories for both datasets compared with state-of-the-art methods.
A novel ensemble approach for heterogeneous data with active learning

At present, millions of internet users are contributing a huge amount of data. This data is extremely heterogeneous, and so, it is hard to analyze and derive information from this data that is considered an indispensable source for decision-makers. Due to this massive growth, the classification of data and analysis has become an important research subject. Extracting information from this data has become a necessity. As a result, it was necessary to process these enormous volumes of data to uncover hidden information and therefore improve data analysis and, in turn, classification accuracy. In this paper, firstly, we focus on developing an ensemble machine-learning model based on active learning which identifies the most effective feature extraction strategy for heterogeneous data analysis, and compare it with traditional machine-learning algorithms. Secondly, we evaluate the proposed model during the experiments; five heterogeneous datasets from various domains were used, such as a Health Care Reform dataset, Sander Frandsen dataset, Financial Phrase Bank dataset, SMS Spam Collection dataset, and Textbook sales dataset. According to the results, the novel approach for data analysis performed better than conventional methods. Finally, the study’s findings confirmed the validity of the suggested technique, meeting the study’s goal of using ensemble methods with active learning to raise the model’s overall accuracy for effectively classifying and analyzing heterogeneous data, reducing the time and money spent training the model, and delivering superior analysis performance as well as insights into other elements of extracting information from heterogeneous data.
Adaptive and structured graph learning for semi-supervised clustering

This paper proposes a new method for semi-supervised clustering of data that only contains pairwise relational information. Specifically, our method simultaneously learns two similarity matrices in feature space and label space, in which similarity matrix in feature space learned by adopting adaptive neighbor strategy while another one obtained through tactful label propagation approach. Moreover, the above two learned matrices explore the local structure (i.e., learned from feature space) and global structure (i.e., learned from label space) of data respectively. Furthermore, most of the existing clustering methods do not fully consider the graph structure, they can not achieve the optimal clustering performance. Therefore, our method forcibly divides the data into c clusters by adding a low rank restriction on the graphical Laplacian matrix. Finally, a restriction of alignment between two similarity matrices is imposed and all items are combined into a unified framework, and an iterative optimization strategy is leveraged to solve the proposed model. Experiments in practical data show that our method has achieved brilliant performance compared with some other state-of-the-art methods.
A facial expression recognition model using hybrid feature selection and support vector machines

Facial expression recognition is a challenging issue in the field of computer vision. Due to the limited feature extraction capability of a single feature descriptor, in this paper, a hybrid feature extraction is utilised. The proposed methodology includes local and global feature extractions that is done by local binary pattern (LBP) and histogram orientation gradient (HOG) respectively. Before applying the feature extraction process, pre-processing and face detection is applied on the face image to extract the useful features. The Viola and Jones algorithm is utilised for face detection and the hybrid Laplacian of Gaussian (HLOG) is used for pre-processing stage. The orthogonal local preserving projection (OLPP)-based dimension reduction algorithm is applied to the extracted features to minimise the computational complexity of the classification algorithm. The SVM classification algorithm is utilised for identifying the facial expression. Here, standard CK+ facial expression dataset is used for evaluating the proposed methodology. The proposed methodology performed well in terms of accuracy compared to the existing PCA + Gabor and PCA + LBP methodology.
Detection of Drive-by Download Attacks Using Machine Learning Approach

Drive-by download refers to attacks that automatically download malwares to user's computer without his knowledge or consent. This type of attack is accomplished by exploiting web browsers and plugins vulnerabilities. The damage may include data leakage leading to financial loss. Traditional antivirus and intrusion detection systems are not efficient against such attacks. Researchers proposed plenty of detection approaches mostly passive blacklisting. However, a few proposed dynamic classification techniques, which suffer from clear shortcomings. In this paper, we propose a novel approach to detect drive-by download infected web pages based on extracted features from their source code. We test 23 different machine learning classifiers using data set of 5435 webpages and based on the detection accuracy we selected the top five to build our detection model. The approach is expected to serve as a base for implementing and developing anti drive-by download programs. We develop a graphical user interface program to allow the end user to examine the URL before visiting the website. The Bagged Trees classifier exhibited the highest accuracy of 90.1% and reported 96.24% true positive and 26.07% false positive rate.
Segmentation of Blood cell Images using Hybrid K-means with Cluster Center Estimation Technique

Image segmentation plays a predominant role in the field of image processing. k- Means clustering is one of the most powerful algorithms for medical image segmentation. However, the randomly selected cluster number and initial centroids cause inconsistency in the image segmentation results. To overcome this limitation we have proposed a combined approach namely Hybrid K-Means with Cluster Center Estimation (HKMCCE) for image segmentation. The proposed technique use histogram peaks of the image to find the cluster number and initial cluster centers automatically.Moreover, it requires lessuser interaction to determine k-means initialization parameters. The performance of the proposed technique is compared with traditional segmentationmethods and it yields better results with less execution time.
Machine Learning Techniques and Extreme Learning Machine for Early Breast Cancer Prediction

Breast Cancer is one of the most deadly disease and most of the women are infected by this vital disease in many parts of the world. Medical tests conducted in hospitals for determining the disease are very much expensive as well as time-consuming. The problem can be resolved by diagnosing the problem in early spam of time and by providing results with more accuracy.In this paper, different machine learning and neural network algorithmhave been studied and compared to predict cancer in early stages so that life can be saved. The dataset available publically for Breast Cancer has been used. Different algorithms compared includeSupport Vector Machine Classification (SVM), K-Nearest NeighborClassification (KNN), Decision tree Classification (DT), Random Forest Classification (RF) and Extreme Learning Machine (ELM).All are compared on the basis of Accuracy and processing time are considered as the parameters for comparing analysis. The results reveal that extreme learning machine comes to be the better algorithm.
The Potential of AR Solutions for Behavioral Learning: A Scoping Review

In recent years, educational researchers and practitioners have become increasingly interested in new technologies for teaching and learning, including augmented reality (AR). The literature has already highlighted the benefit of AR in enhancing learners’ outcomes in natural sciences, with a limited number of studies exploring the support of AR in social sciences. Specifically, there have been a number of systematic and scoping reviews in the AR field, but no peer-reviewed review studies on the contribution of AR within interventions aimed at teaching or training behavioral skills have been published to date. In addition, most AR research focuses on technological or development issues. However, limited studies have explored how technology affects social experiences and, in particular, the impact of using AR on social behavior. To address these research gaps, a scoping review was conducted to identify and analyze studies on the use of AR within interventions to teach behavioral skills. These studies were conducted across several intervention settings. In addition to this research question, the review reports an investigation of the literature regarding the impact of AR technology on social behavior. The state of the art of AR solutions designed for interventions in behavioral teaching and learning is presented, with an emphasis on educational and clinical settings. Moreover, some relevant dimensions of the impact of AR on social behavior are discussed in more detail. Limitations of the reviewed AR solutions and implications for future research and development efforts are finally discussed.
ShapePU: A New PU Learning Framework Regularized by Global Consistency for Scribble Supervised Cardiac Segmentation

Cardiac segmentation is an essential step for the diagnosis of cardiovascular diseases. However, pixel-wise dense labeling is both costly and time-consuming. Scribble, as a form of sparse annotation, is more accessible than full annotations. However, it’s particularly challenging to train a segmentation network with weak supervision from scribbles. To tackle this problem, we propose a new scribble-guided method for cardiac segmentation, based on the Positive-Unlabeled (PU) learning framework and global consistency regularization, and termed as ShapePU. To leverage unlabeled pixels via PU learning, we first present an Expectation-Maximization (EM) algorithm to estimate the proportion of each class in the unlabeled pixels. Given the estimated ratios, we then introduce the marginal probability maximization to identify the classes of unlabeled pixels. To exploit shape knowledge, we apply cutout operations to training images, and penalize the inconsistent segmentation results. Evaluated on two open datasets, i.e., ACDC and MSCMRseg, our scribble-supervised ShapePU surpassed the fully supervised approach respectively by 1.4% and 9.8% in average Dice, and outperformed the state-of-the-art weakly supervised and PU learning methods by large margins. Our code is available at https://github.com/BWGZK/ShapePU .
Rediscovering orbital mechanics with machine learning

Abstract We present an approach for using machine learning to automatically discover the governing equations and unknown properties (in this case, masses) of real physical systems from observations. We train a ‘graph neural network’ to simulate the dynamics of our Solar System’s Sun, planets, and large moons from 30 years of trajectory data. We then use symbolic regression to correctly infer an analytical expression for the force law implicitly learned by the neural network, which our results showed is equivalent to Newton’s law of gravitation. The key assumptions our method makes are translational and rotational equivariance, and Newton’s second and third laws of motion. It did not, however, require any assumptions about the masses of planets and moons or physical constants, but nonetheless, they, too, were accurately inferred with our method. Naturally, the classical law of gravitation has been known since Isaac Newton, but our results demonstrate that our method can discover unknown laws and hidden properties from observed data.