Reviewerly/exHarmony huggingface dataset:
Towards Interpretable Deep Networks for Monocular Depth Estimation

Deep networks for Monocular Depth Estimation (MDE) have achieved promising performance recently and it is of great importance to further understand the interpretability of these networks. Existing methods attempt to provide post-hoc explanations by investigating visual cues, which may not explore the internal representations learned by deep networks. In this paper, we find that some hidden units of the network are selective to certain ranges of depth, and thus such behavior can be served as a way to interpret the internal representations. Based on our observations, we quantify the interpretability of a deep MDE network by the depth selectivity of its hidden units. Moreover, we then propose a method to train interpretable MDE deep networks without changing their original architectures, by assigning a depth range for each unit to select. Experimental results demonstrate that our method is able to enhance the interpretability of deep MDE networks by largely improving the depth selectivity of their units, while not harming or even improving the depth estimation accuracy. We further provide comprehensive analysis to show the reliability of selective units, the applicability of our method on different layers, models, and datasets, and a demonstration on analysis of model error. Source code and models are available at https://github.com/youzunzhi/InterpretableMDE.
Robust optimal classification trees under noisy labels

Abstract In this paper we propose a novel methodology to construct Optimal Classification Trees that takes into account that noisy labels may occur in the training sample. The motivation of this new methodology is based on the superaditive effect of combining together margin based classifiers and outlier detection techniques. Our approach rests on two main elements: (1) the splitting rules for the classification trees are designed to maximize the separation margin between classes applying the paradigm of SVM; and (2) some of the labels of the training sample are allowed to be changed during the construction of the tree trying to detect the label noise. Both features are considered and integrated together to design the resulting Optimal Classification Tree. We present a Mixed Integer Non Linear Programming formulation for the problem, suitable to be solved using any of the available off-the-shelf solvers. The model is analyzed and tested on a battery of standard datasets taken from UCI Machine Learning repository, showing the effectiveness of our approach. Our computational results show that in most cases the new methodology outperforms both in accuracy and AUC the results of the benchmarks provided by OCT and OCT-H.
Controllable Cardiac Synthesis via Disentangled Anatomy Arithmetic

Acquiring annotated data at scale with rare diseases or conditions remains a challenge. It would be extremely useful to have a method that controllably synthesizes images that can correct such underrepresentation. Assuming a proper latent representation, the idea of a "latent vector arithmetic" could offer the means of achieving such synthesis. A proper representation must encode the fidelity of the input data, preserve invariance and equivariance, and permit arithmetic operations. Motivated by the ability to disentangle images into spatial anatomy (tensor) factors and accompanying imaging (vector) representations, we propose a framework termed "disentangled anatomy arithmetic", in which a generative model learns to combine anatomical factors of different input images such that when they are re-entangled with the desired imaging modality (e.g. MRI), plausible new cardiac images are created with the target characteristics. To encourage a realistic combination of anatomy factors after the arithmetic step, we propose a localized noise injection network that precedes the generator. Our model is used to generate realistic images, pathology labels, and segmentation masks that are used to augment the existing datasets and subsequently improve post-hoc classification and segmentation tasks. Code is publicly available at https://github.com/vios-s/DAA-GAN.
A new method for detection and prediction of occluded text in natural scene images

Text detection from natural scene images is an active research area for computer vision, signal, and image processing because of several real-time applications such as driving vehicles automatically and tracing person behaviors during sports or marathon events. In these situations, there is a high probability of missing text information due to the occlusion of different objects/persons while capturing images. Unlike most of the existing methods, which focus only on text detection by ignoring the effect of missing texts, this work detects and predicts missing texts so that the performance of the OCR improves. The proposed method exploits the property of DCT for finding significant information in images by selecting multiple channels. For chosen DCT channels, the proposed method studies texture distribution based on statistical measurement to extract features. We propose to adopt Bayesian classifier for categorizing text pixels using extracted features. Then a deep learning model is proposed for eliminating false positives to improve text detection performance. Further, the proposed method employs a Natural Language Processing (NLP) model for predicting missing text information by using detected and recognition texts. Experimental results on our dataset, which contains texts occluded by objects, show that the proposed method is effective in predicting missing text information. To demonstrate the effectiveness and objectiveness of the proposed method, we also tested it on the standard datasets of natural scene images, namely, ICDAR 2017-MLT, Total-Text, and CTW1500.
Hybrid Supervision Learning for Pathology Whole Slide Image Classification

Weak supervision learning on classification labels has demonstrated high performance in various tasks, while a few pixel-level fine annotations are also affordable. Naturally a question comes to us that whether the combination of pixel-level (e.g., segmentation) and image level (e.g., classification) annotation can introduce further improvement. However in computational pathology this is a difficult task for this reason: High resolution of whole slide images makes it difficult to do end-to-end classification model training, which is challenging to research of weak or hybrid supervision learning in the past. To handle this problem, we propose a hybrid supervision learning framework for this kind of high resolution images with sufficient image-level coarse annotations and a few pixel-level fine labels. This framework, when applied in training patch model, can carefully make use of coarse image-level labels to refine generated pixel-level pseudo labels. Complete strategy is proposed to suppress pixel-level false positives and false negatives. A large hybrid annotated dataset is used to evaluate the effectiveness of hybrid supervision learning. By extracting pixel-level pseudo labels in initially image-level labeled samples, we achieve 5.2% higher specificity than purely training on existing labels while retaining 100% sensitivity, in the task of image-level classification to be positive or negative.
A Super-Resolution Generative Adversarial Network with Siamese CNN Based on Low Quality for Breast Cancer Identification

Breast cancer is a chronic illness leading to the death of millions of people yearly. Despite the fact that successful identification of benign and malignant images is dependent on radiologists' long-term knowledge, specialists occasionally disagree with their decisions. An automatic system provides an alternative choice for the image diagnosis, thereby helping the expert to make more reliable decisions efficiently, less prone to errors and make diagnosis more scalable. Another issue based with the diagnosis of breast cancer identification is the poor quality of the image which poses a challenge in identification performance. An enhanced super-resolution generative adversarial network has been implemented in this paper to produce super-resolution images of breast cancer from a low-resolution counterpart with higher quality and finer details using an upscale factor of 4. Additionally, siamese convolutional neural network was utilized for the features extraction and classification of breast cancer. The proposed model provides an effective classification performance in terms of accuracy and ROC-AUC scores of 98.87% and 98.76% respectively as compared to other existing approaches.
Transfer Learning Improves Accelerometer-Based Child Activity Recognition via Subject-Independent Adult-Domain Adaption

Wearable activity recognition can collate the type, intensity, and duration of each child's physical activity profile, which is important for exploring underlying adolescent health mechanisms. Traditional machine-learning-based approaches require large labeled data sets; however, child activity data sets are typically small and insufficient. Thus, we proposed a transfer learning approach that adapts adult-domain data to train a high-fidelity, subject-independent model for child activity recognition. Twenty children and twenty adults wore an accelerometer wristband while performing walking, running, sitting, and rope skipping activities. Activity classification accuracy was determined via the traditional machine learning approach without transfer learning and with the proposed subject-independent transfer learning approach. Results showed that transfer learning increased classification accuracy to 91.4% as compared to 80.6% without transfer learning. These results suggest that subject-independent transfer learning can improve accuracy and potentially reduce the size of the required child data sets to enable physical activity monitoring systems to be adopted more widely, quickly, and economically for children and provide deeper insights into injury prevention and health promotion strategies.
Global Context for Improving Recognition of Online Handwritten Mathematical Expressions

This paper presents a temporal classification method for all three subtasks of symbol segmentation, symbol recognition and relation classification in online handwritten mathematical expressions (HMEs). The classification model is trained by multiple paths of symbols and spatial relations derived from the Symbol Relation Tree (SRT) representation of HMEs. The method benefits from global context of a deep bidirectional Long Short-term Memory network, which learns the temporal classification directly from online handwriting by the Connectionist Temporal Classification loss. To recognize an online HME, a symbol-level parse tree with Context-Free Grammar is constructed, where symbols and spatial relations are obtained from the temporal classification results. We show the effectiveness of the proposed method on the two latest CROHME datasets.
A Novel Overlapping Method to Alleviate the Cold-Start Problem in Recommendation Systems

Recommendation systems (RSs) are tools for interacting with large and complex information spaces. They provide a personalized view of such spaces, prioritizing items likely to be of interest to the user. The main objective of RSs is to tool up users with desired items that meet their preferences. A major problem in RSs is called: “cold-start”; it is a potential problem called so in computer-based information systems which comprises a degree of automated data modeling. Particularly, it concerns the issue in which the system cannot draw any inferences nor have it yet gathered sufficient information about users or items. Since RSs performance is substantially limited by cold-start users and cold-start items problems; this research study takes the route for a major aim to attenuate users’ cold-start problem. Still in the process of researching, sundry studies have been conducted to tackle this issue by using clustering techniques to group users according to their social relations, their ratings or both. However, a clustering technique disregards a variety of users’ tastes. In this case, the researcher has adopted the overlapping technique as a tool to deal with the clustering technique’s defects. The advantage of the overlapping technique excels over others by allowing users to belong to multi-clusters at the same time according to their behavior in the social network and ratings feedback. On that account, a novel overlapping method is presented and applied. This latter is executed by using the partitioning around medoids (PAM) algorithm to implement the clustering, which is achieved by means of exploiting social relations and confidence values. After acquiring users’ clusters, the average distances are computed in each cluster. Thereafter, a content comparison is made regarding the distances between every user and the computed distances of the clusters. If the comparison result is less than or equal to the average distance of a cluster, a new user is added to this cluster. The singular value decomposition plus (SVD[Formula: see text]) method is then applied to every cluster to compute predictions values. The outcome is calculated by computing the average of mean absolute error (MAE) and root mean square error (RMSE) for every cluster. The model is tested by two real world datasets: Ciao and FilmTrust. Ultimately, findings have exhibited a great deal of insights on how the proposed model outperformed a number of the state-of-the-art studies in terms of prediction accuracy.
3D Hand Gestures Segmentation and Optimized Classification Using Deep Learning

Hand gestures recognition system has massive applications which are mainly utilized in robotics and computer vision specially to control Unmanned Aerial Vehicles (UAV). These methods bypass the presence of electronic control to UAVs and provide an ease to the operators. In this paper, we present a method for 3D hand gestures segmentation and classification by combining MASK-RCNN with Grass Hopper Optimization. We created a private 3D and RGB hand gestures dataset using Intel Kinetic and Intel Real sense d435i camera, then proposed a model for RGB hand gestures to estimate the key points using human kinematics, the key points later then utilize to get the best degree of freedom (DoF). The grass hopper optimization besides minimum distance function was applied to achieve the finest deep features from the 3D hand gestures dataset. The ResNet50 network is used as the backbone to calculate the Overlap Coefficient (OC) for segmentation and the ResNet50, ResNet101 networks to calculate the classification for 3D hand gestures. The classification accuracy achieved on the private dataset is 99.05% and 99.29% on public Microsoft Kinect and Leap Motion dataset where the OC are 88.16%. and 88.19% respectively.
Attack as the Best Defense: Nullifying Image-to-image Translation GANs via Limit-aware Adversarial Attack

Due to the great success of image-to-image (Img2Img) translation GANs, many applications with ethics issues arise, e.g., DeepFake and DeepNude, presenting a challenging problem to prevent the misuse of these techniques. In this work, we tackle the problem by a new adversarial attack scheme, namely the Nullifying Attack, which cancels the image translation process and proposes a corresponding framework, the Limit-Aware Self-Guiding Gradient Sliding Attack (LaS-GSA) under a black-box setting. In other words, by processing the image with the proposed LaS-GSA before publishing, any image translation functions can be nullified, which prevents the images from malicious manipulations. First, we introduce the limit-aware RGF and the gradient sliding mechanism to estimate the gradient that adheres to the adversarial limit, i.e., the pixel value limitations of the adversarial example. We theoretically prove that our model is able to avoid the error caused by the projection in both the direction and the length. Then, an effective self-guiding prior is extracted solely from the threat model and the target image to efficiently leverage the prior information and guide the gradient estimation process. Extensive experiments demonstrate that LaS-GSA requires fewer queries to nullify the image translation process with higher success rates than 4 state-of-the-art methods.
Byte embeddings for file fragment classification

In digital forensics, file carving is the process of recovering files on a storage media in part or in whole without any file system information. An important problem in file carving is the identification of fragment types. Many fragment classification studies in the literature employ inflexible and indiscernible feature selection methods such as different statistics of byte frequency distributions. Moreover, assessing the strengths and weaknesses of some approaches is difficult as they are specific to certain file types such as graphics. In this paper, we propose a novel feature generation model using byte embeddings (Byte2Vec) which map fragments to dense vector representations. The proposed model extends the word2vec and doc2vec document embedding models to bytes and fragments, respectively. We use Byte2Vec for feature extraction and k-Nearest Neighbors (kNN) for classification. We present effectiveness of Byte2Vec+kNN in file fragment classification using a publicly available digital forensics dataset and a random web search dataset. Our experimental results show that Byte2Vec+kNN reaches an accuracy rate of 72% along with 74% precision and 72% recall. Compared to the other feature extraction techniques such as n-gram, byte distributions, byte statistics, byte distances, and sparse dictionaries for byte n-gram along with different classifiers, Byte2Vec+kNN achieves an absolute improvement of 3% and 12% in accuracy and precision, respectively.
Adaptive Recognition of Motion Posture in Sports Video Based on Evolution Equation

In the field of sports, the formulation of existing training plans mainly relies on the manual observation and personal experience of coaches. This method is inevitably subjective. The application of artificial intelligence technology to the training of athletes to recognize athletes’ posture can help coaches assist in decision-making and greatly enhance athletes’ competitive ability. The human body movements embodied in sports are more complicated, and the accurate recognition of sports postures plays an active and important role in sports competitions and training. In this paper, inertial sensor technology is applied to attitude recognition in motion. First, in order to improve the accuracy of attitude calculation and reduce the noise interference in the preparation process, this article uses differential evolution algorithm to apply attitude calculation to realize multisensor data fusion. Secondly, a two-level neural network intelligent motion gesture recognition algorithm is proposed. The two-level neural network intelligent recognition algorithm effectively recognizes similar actions by splitting the traditional single-level neural network into two-level neural networks. Experiments show that the experimental method designed in this article for the posture in motion can obtain the motion information of the examinee in real time, realize the accurate extraction of individual motion data, and complete the recognition of the motion posture. The average accuracy rate can reach 98.85%. There is a certain practical value in gesture recognition.
Quantitative Evaluation of Gymnastics Based on Multiple MEMS Sensors

The quantitative evaluation of sports motions is a precondition for scientific training and data-driven professional athletic motion analyses. However, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">high accuracy</i> and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">portability</i> of the evaluation schemes are contradictory and it is difficult to simultaneously improve both metrics in one device. In this work, we propose an effective gymnastics motion recognition and evaluation system to solve this problem. The system is a sensor network composed of 11 acceleration and angular velocity sensors. The gymnastics data obtained from each sensor node can be uploaded in real-time via Wi-Fi. The data from different sensor nodes on the body is used to form the corresponding gymnastics motion recognition data sets. Using these sensor node data and the linear discriminant analysis (LDA) algorithm, 6 different sets gymnastics motions could be recognized at an accuracy of 99.18%. Experimental results also showed that, by combining only the data from the left arm sensor node and the right foot sensor node as the gymnastic motion evaluation data set, the LDA algorithm could be used to evaluate three gymnastics motions, all of which delivered an accuracy of no less than 96.0%. This evaluation system meets the needs for accuracy and portability at the same time, and potentially can be extended to evaluate other physical activities, such as yoga and Tai Chi.
Analyzing The Robustness of Unsupervised Speech Recognition

Unsupervised speech recognition (unsupervised ASR) aims to learn the ASR system with non-parallel speech and text corpus only. Wav2vec-U [1] has shown promising results in unsupervised ASR by self-supervised speech representations coupled with Generative Adversarial Network (GAN) training, but the robustness of the unsupervised ASR framework is unknown. In this work, we further analyze the training robustness of unsupervised ASR on the domain mismatch scenarios in which the domains of unpaired speech and text are different. Three domain mismatch scenarios include: (1) using speech and text from different datasets, (2) utilizing noisy/spontaneous speech, and (3) adjusting the amount of speech and text data. We also quantify the degree of the domain mismatch by calculating the JS-divergence of phoneme n-gram be-tween the transcription of speech and text. This metric correlates with the performance highly. Experimental results show that do-main mismatch leads to inferior performance, but a self-supervised model pre-trained on the targeted speech domain can extract better representation to alleviate the performance drop.
The elliptic scalar multiplication graph and its application in elliptic curve cryptography

In this work, a new graph has been defined as a main point to design a new version of an asymmetric encryption scheme. This graph is formed based on the scalar multiplication operation on elliptic curve defined over a prime field, which is called an elliptic scalar multiplication (ESM) graph. Some mathematical facts to show the ESM graph is a complete graph are proved theoretically. On the complete elliptic scalar multiplication (CESM) graph, an asymmetric encryption algorithm has been implemented. Fast computations on the proposed algorithm are Z obtained through the matrices representation and determination of the minimum spanning tree (MST) to compute the ciphertext. More secure computations with the CESM graphic encryption algorithm in comparison with the previous asymmetric encryption algorithms. So, the CESM graphic asymmetric encryption algorithm is considered as a new sight for elliptic curve cryptographic usages.