Reviewerly/exHarmony huggingface dataset:
Cross-Domain Lossy Compression as Entropy Constrained Optimal Transport

We study an extension of lossy compression where the reconstruction is subject to a distribution constraint which can be different from the source distribution. We formulate our setting as a generalization of optimal transport with an entropy bottleneck to account for the rate constraint due to compression. We provide expressions for the tradeoff between compression rate and the achievable distortion with and without shared common randomness between the encoder and decoder. We study the examples of binary, uniform and Gaussian sources (in an asymptotic setting) in detail and demonstrate that shared randomness can strictly improve the tradeoff. For the case without common randomness and squared-Euclidean distortion, we show that the optimal solution partially decouples into the problem of optimal compression and transport and also characterize the penalty associated with fully decoupling them. We provide experimental results by training deep learning end-to-end compression systems for performing denoising on SVHN (The Street View House Numbers) and super-resolution on MNIST (Modified National Institute of Standards and Technology) datasets suggesting consistency with our theoretical results. Our code is available at https://github.com/liuh127/CrossdomainLC.
Enhanced Video Caption Model Based on Text Attention Mechanism

In the current video caption model, there are two main problems: only the word of the most recent previous time step is used to represent the contextual information, resulting in insufficient expression of text information affecting the quality of the generated captions; and the use of the word-level cross-entropy loss optimization model makes the training and test inconsistent, which may easily lead to the problems of exposure bias and insufficient model optimization. Considering the above two problems, we first propose a kind of attention mechanism based on historical textual information. Soft selection of the generated historical classification distribution is guided by using the current text encoding to enhance contextual information on the current moment. Then the maximum reward via the multi-sampling method is used to improve the strategic gradient algorithm, which exploits a sentence-level loss to directly optimize the evaluation metric. The Experiments on the Microsoft Research Video Description Corpus (MSVD) dataset shows that our proposed method is comparable with state-of-the-art algorithms.
Bilingual Terminology Extraction from Comparable E-Commerce Corpora

Bilingual terminologies are important machine translation resources in the field of e-commerce, which are usually either manually translated or automatically extracted from parallel data. The human translation is costly and e-commerce parallel corpora is very scarce. However, the comparable data in different languages in the same commodity field is abundant. In this paper, we propose a novel framework of extracting e-commercial bilingual terminologies from comparable data. Benefiting from the cross-lingual pre-training in e-commerce, our framework can make full use of the deep semantic relationship between source-side terminology and target-side sentence to extract corresponding target terminology. Experimental results on various language pairs show that our approaches achieve significantly better performance than various strong baselines.
Swapping Semantic Contents for Mixing Images

Deep architecture have proven capable of solving many tasks provided a sufficient amount of labeled data. In fact, the amount of available labeled data has become the principal bottleneck in low label settings such as Semi-Supervised Learning. Mixing Data Augmentations do not typically yield new labeled samples, as indiscriminately mixing contents creates between-class samples. In this work, we introduce the SciMix framework that can learn to replace the global semantic content from one sample. By teaching a StyleGan generator to embed a semantic style code into image backgrounds, we obtain new mixing scheme for data augmentation. We then demonstrate that SciMix yields novel mixed samples that inherit many characteristics from their non-semantic parents. Afterwards, we verify those samples can be used to improve the performance semi-supervised frameworks like Mean Teacher or Fixmatch, and even fully supervised learning on a small labeled dataset.
Content Selection Methods Using User Interest Prediction Based on Similarities of Web Activities

This research predicts the current user’s interests based on the characteristics of the user’s recently acquired web activities and uses it for content selection. In conventional research, user interests are predicted using cookies for identifying individuals on a web service, and are obtained from the similarity of behavior patterns between current and previous users. In this case, the web advertising company can acquire information including personal information such as web activities, acquired contents on the web service, and inputted information. The proposed approach analyzes the characteristics of web content acquired by users for each web service, constructs a combinable feature model named fog model, and then combines these models by adapting to the user’s activities. The combined fog model can be considered a feature model learned from the characteristics of the user’s interests. Then, from the candidate contents, the most likely content to be output from this fog model is chosen. Thus, information adapted to the user’s interests can be selected and provided. The results of the evaluation to show that the proposed method is effective for approximately $$57\%$$ users.
Effect of End Point Detection on Fixed Phrase Speaker Verification

Speech endpoint detection is the crucial part of fixed phrase speaker verification as the performance of the system is largely depends on the proper start and terminus point identification. In this paper, we have analyzed the effect of endpoint detection on fixed phrase speaker verification. For this experimental study, we have selected three endpoint detection techniques from the literature. These are energy-based voice activity detection (VAD), glottal activity detection (GAD), and pitch and energy-based detection (PED). Our fixed phrase speaker verification model is developed using Mel Frequency Cepstral Coefficients (MFCC) features and for template matching dynamic time warping (DTW) is used. The experimentation for speaker matching is done by speaker-specific threshold and cohort speaker-based approach. We have performed our analysis on the RSR2015 database, which was collected in a controlled environment, and the NITS database that was recorded in a real-life scenario. The result of our experimental evaluation resembles that the PED gives better performance compared to VAD and GAD. An average improvement of 5.2% and 3.9% in Equal Error Rate (EER) is achieved with PED in comparison to VAD and GAD, respectively.
Multi-grained Syntactic Dependency-aware Graph Convolution for Aspect-based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) aims to identify the sentiment of one or more aspects in the text. Existing methods pay attention to the syntactic structure, and significant progress has been achieved by using graph convolutional network (GCN). However, they ignore internal connections between different types of syntactic structure from a fine-grained perspective, which may lead to underexploring critical syntactic information of sentences. Additionally, the aspect-oriented syntactic dependency is omitted that generally provides penetrating insights of the corresponding sentiment. To tackle these problems, we propose a multi-grained (both coarse-grained and fine-grained) syntactic dependency-aware graph convolutional network model (named MSD-GCN). Particularly, in the initial representation layer, we redesign the aspect-enhanced coarse-grained dependency graph and construct five fine-grained dependency graphs by taking into account the types of syntactic structure. Moreover, we explore a multi-grained syntactic enhancement layer, which employs GCN and attention mechanism over multi-grained dependency graphs to capture more abundant syntactic information. Experimental results on five datasets illustrate that our proposed MSD-GCN model outperforms other representative ones in terms of Accuracy and Macro-Averaged F1.
Augment-Connect-Explore: a Paradigm for Visual Action Planning with Data Scarcity

Visual action planning particularly excels in applications where the state of the system cannot be computed explicitly, such as manipulation of deformable objects, as it enables planning directly from raw images. Even though the field has been significantly accelerated by deep learning techniques, a crucial requirement for their success is the availability of a large amount of data. In this work, we propose the Augment-Connect-Explore (ACE) paradigm to enable visual action planning in cases of data scarcity. We build upon the Latent Space Roadmap (LSR) framework which performs planning with a graph built in a low dimensional latent space. In particular, ACE is used to i) Augment the available training dataset by autonomously creating new pairs of datapoints, ii) create new unobserved Connections among representations of states in the latent graph, and iii) Explore new regions of the latent space in a targeted manner. We validate the proposed approach on both simulated box stacking and real-world folding task showing the applicability for rigid and deformable object manipulation tasks, respectively.
Face Recognition Attendance System for Online Classes

Due to the increasing need for online lectures due to situations like COVID-19 and various online learning platforms, there is a need for a reliable attendance system for online classes. Our system is developed for deploying an easy and secure way of taking attendance without tedious roll-calls and inaccurate participant lists. The teachers will take screenshots of students in the online meets with their video cameras on. This screenshot will be uploaded to our system which will recognize the students and generate a report of their attendance. Our system will allow students and faculty to view the attendance of each lecture and give feedback if any discrepancy is found. We collected the dataset of face images of 94 students which are then augmented to increase the dataset and then used HOG for face detection. We then applied four algorithms namely VGG-16, MobileNet, InceptionV3, and our own CNN model for face recognition. MobileNet gave us the highest accuracy of 97.14%. We, therefore, deployed this model for our website to recognize faces and generate the attendance report.
Machine Learning Algorithm Based Emotion Detection System

Emotion detection System is generally use to detect the thoughts of one that is either positive (1) or negative (0). So here, the text classification will be used to detect the emotion from the text either it’s on the social media platform or any day to day survey organized by the various companies. Hence, text classification in a machine learning plays a really important role in our day to day life to best classify the information or text in correct emotion we’ve used BERT algorithm (basically a language model for classification).
Increasing the Accuracy of a Neural Network Using Frequency Selective Mesh-to-Grid Resampling

Neural networks are widely used for almost any task of recognizing image content. Even though much effort has been put into investigating efficient network architectures, optimizers, and training strategies, the influence of image interpolation on the performance of neural networks is not well studied. Furthermore, research has shown that neural networks are often sensitive to minor changes in the input image leading to drastic drops of their performance. Therefore, we propose the use of keypoint agnostic frequency selective mesh-to-grid resampling (FSMR) for the processing of input data for neural networks in this paper. This model-based interpolation method already showed that it is capable of outperforming common interpolation methods in terms of PSNR. Using an extensive experimental evaluation we show that depending on the network architecture and classification task the application of FSMR during training aids the learning process. Furthermore, we show that the usage of FSMR in the application phase is beneficial. The classification accuracy can be increased by up to 4.31 percentage points for ResNet50 and the Oxtlower17 dataset.
A Source-Level Instrumentation Framework for the Dynamic Analysis of Memory Safety

Low-level control makes C unsafe, resulting in memory errors that can lead to data corruption, security vulnerabilities or program crashes. Dynamic analysis tools, which have been widely used for detecting memory errors at runtime, usually perform instrumentation at the IR or binary level. However, these non-source-level instrumentation frameworks and tools suffer from two inherent drawbacks: optimization sensitivity and platform dependence. Due to optimization sensitivity, the user of these tools must trade either performance for effectiveness by compiling the program at <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-O0</monospace> or effectiveness for performance by compiling the program at a higher optimization level, say, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-O3</monospace> . In this paper, we propose a new source-level instrumentation framework to overcome these two drawbacks, and implement it in a new dynamic analysis tool, called <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Movec</small> , that adopts a pointer-based monitoring algorithm. We have evaluated <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Movec</small> comprehensively by using the NIST's SARD benchmark suite (1152 programs), a set of 126 microbenchmarks (with ground truth), a set of 20 MiBench benchmarks and 5 pure-C SPEC CPU 2017 benchmarks. In terms of effectiveness, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Movec</small> outperforms three state-of-the-art dynamic analysis tools, AddressSanitizer, SoftBoundCETS and Valgrind, for all the standard optimization levels (from <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-O0</monospace> to <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-O3</monospace> ). In terms of performance, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Movec</small> outperforms SoftBoundCETS and Valgrind, and is slower than AddressSanitizer but consumes less memory.
RSINet: Inpainting Remotely Sensed Images Using Triple GAN Framework

We tackle the problem of image inpainting in the remote sensing domain. Remote sensing images possess high resolution and geographical variations, that render the conventional inpainting methods less effective. This further entails the requirement of models with high complexity to sufficiently capture the spectral, spatial and textural nuances within an image, emerging from its high spatial variability. To this end, we propose a novel inpainting method that individually focuses on each aspect of an image such as edges, colour and texture using a task specific GAN. Moreover, each individual GAN also incorporates the attention mechanism that explicitly extracts the spectral and spatial features. To ensure consistent gradient flow, the model uses residual learning paradigm, thus simultaneously working with high and low level features. We evaluate our model, alongwith previous state of the art models, on the two well known remote sensing datasets, Open Cities AI and Earth on Canvas, and achieve competitive performance. The code can be referred here: https://github.com/advaitkumar3107/RSINet.
Towards a Bayesian Analysis of Migration Pathways Using Chain Event Graphs of Agent Based Models

Strong, Peter McAlpine, Alys Smith, Jim Q.Agent-Based Models (ABMs) are often used to model migration and are increasingly used to simulate individual migrant decision-making and unfolding events through a sequence of heuristic if-then rules. However, ABMs lack the methods to embed more principled strategies of performing inference to estimate and validate the models, both of which are of significant importance for real-world case studies. Chain Event Graphs (CEGs) can fill this need: they can be used to provide a Bayesian framework which represents an ABM accurately. Through the use of the CEG, we illustrate how to transform an elicited ABM into a Bayesian framework and outline the benefits of this approach.
3-D Refuse-Derived Fuel Particle Tracking-by-Detection Using a Plenoptic Camera System

Multiple particle tracking-by-detection is a widely investigated issue in image processing. The paper presents approaches to detecting and tracking various refuse-derived fuel particles in a industrial environment using a plenoptic camera system, which is able to yield 2D gray value information and 3D point clouds with noticeable fluctuations. The presented approaches, including an innovative combined detection method and a post-processing framework for multiple particle tracking, aim at making the most of the acquired 2D and 3D information to deal with the fluctuations of the measuring system. The proposed novel detection method fuses the captured 2D gray value information and 3D point clouds, which is superior to applying single information. Subsequently, the particles are tracked by the linear Kalman filter and 2.5D global nearest neighbor (GNN) and joint probabilistic data association (JPDA) approach, respectively. As a result of several inaccurate detection results caused by the measuring system, the initial tracking results contain faulty and incomplete tracklets that entail a post-processing process. The developed post-processing approach based merely on particle motion similarity benefits a precise tracking performance by eliminating faulty tracklets, deleting outliers, connecting tracklets, and fusing trajectories. The proposed approaches are quantitatively assessed with manuelly labeled ground truth datasets to prove their availability and adequacy as well. The presented combined detection method provides the highest F <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -score, and the proposed post-processing framework enhances the tracking performance significantly with regard to several recommended evaluation indices.
Sparsity Fuzzy C-Means Clustering With Principal Component Analysis Embedding

The clustering method has been widely used in data mining, pattern recognition, and image identification. Fuzzy c-means (FCM) is a soft clustering method that introduces the concept of membership. In this method, the fuzzy membership matrix is obtained by calculating the distance between data points in the original space. However, these methods may yield suboptimal results owing to the influence of redundant features. Moreover, FCM is always sensitive to noise points and heavily subject to outliers. In this article, we propose a method called sparsity FCM clustering with principal component analysis embedding (P_SFCM). We simultaneously conduct principal component analysis and membership learning, and then add an additional weighting factor for each data point. The goal of this operation is to identify the noise or outliers. Overall, the benefit of our framework is that it retains most of the information in the subspace while improving the robustness of the noise. In this article, we employ an iterative optimization algorithm to efficiently solve our model. To verify the reliability of the proposed method, we conduct a convergence analysis, noise robustness analysis, and multicluster experiments. Furthermore, comparative experiments are conducted on both synthetic and real benchmark datasets. The experimental results show that the P_SFCM is competitive with comparable methods.
A Variational Bayesian Clustering Approach to Acoustic Emission Interpretation Including Soft Labels

We investigate Gaussian Mixture Models (GMM) with uncertain parameters to evaluate whether this model can help in interpreting acoustic emission data used in non-destructive testing. This model, called VBGMM (variational Bayesian GMM) allows the end-user to automatically determine the number of clusters which makes it relevant for this type of application where clusters are related to damages. In this work, we modify the training procedure to include prior knowledge about clusters. Experiments are made on a recently published benchmark, ORION-AE, that aims at estimating the tightening levels in a bolted structure under vibrations. Preliminary results of the VBGMM with soft priors (VBGMM-SOFT) show good improvement over the standard VBGMM.
SE3: Sequential Semantic Segmentation of Large Images with Minimized Memory

Semantic segmentation results in pixel-wise perception accompanied with GPU computation and expensive memory, which makes trained models hard to apply to small devices in testing. Assuming the availability of hardware in training CNN backbones, this work converts them to a linear architecture enabling the inference on edge devices. Keeping the same accuracy as patch-mode testing, we segment images using a scanning line with the minimum memory. Exploring periods of pyramid network shifting on image, we perform such sequential semantic segmentation (SE3) with a circular memory to avoid redundant computation and preserve the same receptive field as patches for spatial dependency. In the experiments on large drone images and panoramas, we examine this approach in terms of accuracy, parameter memory, and testing speed. Benchmark evaluations demonstrate that, with only one-line computation in linear time, our designed SE3 network consumes a small fraction of memory to maintain an equivalent accuracy as the image segmentation in patches. Considering semantic segmentation for high-resolution images, particularly for data streamed from sensors, this method is significant to the real-time applications of CNN based networks on light-weighted edge devices.
CCFE: A Few-Shot Learning Model for Earthquake Detection and Phase Identification

Earthquake detection and phase identification are fundamental and challenging tasks in observational seismology. Deep learning has achieved many achievements in these two tasks. To overcome the limitations of existing methods, mainly becauseof the lack oflarge labeled seismic datasets and the separation of detection and identification tasks, we introducedthe CWT-CNN Few-shot learning Earthquake model (CCFE), a deep learning model for simultaneous earthquake detection and phase identification. CCFE can performfew-shot learning with minimal labeled seismic data by utilizing continuous wavelet transform and lightweight convolutional neural networks with fewer layers. We tested our model in the Huoshan area of southern Chinaand foundthat CCFE outperformed both traditional and published representative deep learning models for detection and identification in this area and that combining detection and identification tasks enhances the performance of each task separately. We found 76% more earthquakes using CCFE than the manual catalog across 15 days of continuous data from the Huoshan region. In regions with low seismicity, CCFE can aid in enhancing earthquake monitoring capacity.