Reviewerly/exHarmony huggingface dataset:
Image Privacy Prediction Using Deep Neural Networks

Images today are increasingly shared online on social networking sites such as Facebook, Flickr, and Instagram. Image sharing occurs not only within a group of friends but also more and more outside a user’s social circles for purposes of social discovery. Despite that current social networking sites allow users to change their privacy preferences, this is often a cumbersome task for the vast majority of users on the Web, who face difficulties in assigning and managing privacy settings. When these privacy settings are used inappropriately, online image sharing can potentially lead to unwanted disclosures and privacy violations. Thus, automatically predicting images’ privacy to warn users about private or sensitive content before uploading these images on social networking sites has become a necessity in our current interconnected world. In this article, we explore learning models to automatically predict appropriate images’ privacy as private or public using carefully identified image-specific features. We study deep visual semantic features that are derived from various layers of Convolutional Neural Networks (CNNs) as well as textual features such as user tags and deep tags generated from deep CNNs. Particularly, we extract deep (visual and tag) features from four pre-trained CNN architectures for object recognition, i.e., AlexNet, GoogLeNet, VGG-16, and ResNet, and compare their performance for image privacy prediction. The results of our experiments obtained on a Flickr dataset of 32,000 images show that ResNet yeilds the best results for this task among all four networks. We also fine-tune the pre-trained CNN architectures on our privacy dataset and compare their performance with the models trained on pre-trained features. The results show that even though the overall performance obtained using the fine-tuned networks is comparable to that of pre-trained networks, the fine-tuned networks provide an improved performance for the private class. The results also show that the learning models trained on features extracted from ResNet outperform the state-of-the-art models for image privacy prediction. We further investigate the combination of user tags and deep tags derived from CNN architectures using two settings: (1) Support Vector Machines trained on the bag-of-tags features and (2) text-based CNN. We compare these models with the models trained on ResNet visual features and show that, even though the models trained on the visual features perform better than those trained on the tag features, the combination of deep visual features with image tags shows improvements in performance over the individual feature sets. We also compare our models with prior privacy prediction approaches and show that for private class, we achieve an improvement of ≈ 10% over prior CNN-based privacy prediction approaches. Our code, features, and the dataset used in experiments are available at https://github.com/ashwinitonge/deepprivate.git.
Coarse Alignment of Topic and Sentiment: A Unified Model for Cross-Lingual Sentiment Classification

Cross-lingual sentiment classification (CLSC) aims to leverage rich-labeled resources in the source language to improve prediction models of a resource-scarce domain in the target language. Existing feature representation learning-based approaches try to minimize the difference of latent features between different domains by exact alignment, which is achieved by either one-to-one topic alignment or matrix projection. Exact alignment, however, restricts the representation flexibility and further degrades the model performances on CLSC tasks if the distribution difference between two language domains is large. On the other hand, most previous studies proposed document-level models or ignored sentiment polarities of topics that might lead to insufficient learning of latent features. To solve the abovementioned problems, we propose a coarse alignment mechanism to enhance the model's representation by a group-to-group topic alignment into an aspect-level fine-grained model. First, we propose an unsupervised aspect, opinion, and sentiment unification model (AOS), which trimodels aspects, opinions, and sentiments of reviews from different domains and helps capture more accurate latent feature representation by a coarse alignment mechanism. To further boost AOS, we propose ps-AOS, a partial supervised AOS model, in which labeled source language data help minimize the difference of feature representations between two language domains with the help of logistics regression. Finally, an expectation-maximization framework with Gibbs sampling is then proposed to optimize our model. Extensive experiments on various multilingual product review data sets show that ps-AOS significantly outperforms various kinds of state-of-the-art baselines.
Revealing Backdoors, Post-Training, in DNN Classifiers via Novel Inference on Optimized Perturbations Inducing Group Misclassification

Recently, a special type of data poisoning (DP) attack against deep neural network (DNN) classifiers, known as a backdoor, was proposed. These attacks do not seek to degrade classification accuracy, but rather to have the classifier learn to classify to a target class whenever the backdoor pattern is present in a test example. Here, we address the challenging post-training detection of backdoor attacks in DNN image classifiers, wherein the defender does not have access to the poisoned training set, but only to the trained classifier itself, as well as to clean (unpoisoned) examples from the classification domain. We propose a defense against imperceptible backdoor attacks based on perturbation optimization and novel, robust detection inference. Our method detects whether the trained DNN has been backdoor-attacked and infers the source and target classes involved in an attack. It outperforms alternative defenses for several backdoor patterns, data sets, and attack settings.
End-To-End Multi-Talker Overlapping Speech Recognition

In this paper we present an end-to-end speech recognition system that can recognize single-channel speech where multiple talkers can speak at the same time (overlapping speech) by using a neural network model based on Recurrent Neural Network Transducer (RNN-T) architecture. We augment the conventional RNN-T architecture by including a masking model for separation of encoded audio features, and multiple label encoders to encode transcripts from different speakers. We use a masking L2 loss to prevent transcripts to align to wrong speakers' audio, and a speaker embedding loss to facilitate speaker tracking. We show that by using these additional training objectives, the proposed augmented RNN-T model can be trained with simulated overlapping speech data and can achieve a WER of 32% on words in overlapping speech segments from real-life telephone conversations. Our analysis of manual transcription task on the same test set shows that transcribing overlapping speech is hard even for humans who can get a WER of 37% compared to ground-truth.
Optimizing Through Learned Errors for Accurate Sports Field Registration

We propose an optimization-based framework to register sports field templates onto broadcast videos. For accurate registration we go beyond the prevalent feed-forward paradigm. Instead, we propose to train a deep network that regresses the registration error, and then register images by finding the registration parameters that minimize the regressed error. We demonstrate the effectiveness of our method by applying it to real-world sports broadcast videos, outperforming the state of the art. We further apply our method on a synthetic toy example and demonstrate that our method brings significant gains even when the problem is simplified and unlimited training data is available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Improving the Performance of Transformer Based Low Resource Speech Recognition for Indian Languages

The recent success of the Transformer based sequence-to-sequence framework for various Natural Language Processing tasks has motivated its application to Automatic Speech Recognition. In this work, we explore the application of Transformers on low resource Indian languages in a multilingual framework. We explore various methods to incorporate language information into a multilingual Transformer, i.e., (i) at the decoder, (ii) at the encoder. These methods include using language identity tokens or providing language information to the acoustic vectors. Language information to the acoustic vectors can be given in the form of one hot vector or by learning a language embedding. From our experiments, we observed that providing language identity always improved performance. The language embedding learned from our proposed approach, when added to the acoustic feature vector, gave the best result. The proposed approach with retraining gave 6% - 11% relative improvements in character error rates over the monolingual baseline.
Unsupervised Domain Adaptation of High-Resolution Aerial Images via Correlation Alignment and Self Training

Deep learning-based approaches for land cover segmentation rely on supervision with pixel-level ground truth, but may not generalize well to unseen image domains. Due to the tedious and labor-intensive labeling process, transferring the models trained with label-rich source data to nonannotated target data becomes a popular problem in recent years. Because of the domain shift, the difference between the source and target distributions can degrade the accuracy on target data if the training occurs directly in a source domain without proper domain adaptation (DA). In this letter, we propose a U-Net based network for DA in the context of semantic segmentation. The model is trained in the source domain with ground truth and test in the target domain without any annotations. We introduce the layer alignment method and the feature covariance loss function to alleviate the domain shift between different domains. To further enhance the adapted model, we adopt a self-training method to improve segmentation performance. Experimental results on the images from the 2018 IEEE Geoscience and Remote Sensing Society (GRSS) data fusion contest and the International Society for Photogrammetry and Remote Sensing (ISPRS) 2-D semantic labeling contest data set reveal the effectiveness of the proposed model. By reducing the domain distribution difference, our method shows better performance compared with the mainstream unsupervised DA methods.
A DNN-LSTM based Target Tracking Approach using mmWave Radar and Camera Sensor Fusion

A new sensor fusion study for monocular camera and mmWave radar using deep neural network and LSTMs is presented. The proposed study includes a decision framework to produce reliable output when either sensor fails. Experiment results to demonstrate single sensor uncertainty and the proposed method's advantages are also presented.
Toward Mining Capricious Data Streams: A Generative Approach

Learning with streaming data has received extensive attention during the past few years. Existing approaches assume that the feature space is fixed or changes by following explicit regularities, limiting their applicability in real-time applications. For example, in a smart healthcare platform, the feature space of the patient data varies when different medical service providers use nonidentical feature sets to describe the patients' symptoms. To fill the gap, we in this article propose a novel learning paradigm, namely, Generative Learning With Streaming Capricious (GLSC) data, which does not make any assumption on the feature space dynamics. In other words, GLSC handles the data streams with a varying feature space, where each arriving data instance can arbitrarily carry new features and/or stop carrying partial old features. Specifically, GLSC trains a learner on a universal feature space that establishes relationships between old and new features, so that the patterns learned in the old feature space can be used in the new feature space. The universal feature space is constructed by leveraging the relatednesses among features. We propose a generative graphical model to model the construction process, and show that learning from the universal feature space can effectively improve the performance with theoretical guarantees. The experimental results demonstrate that GLSC achieves conspicuous performance on both synthetic and real data sets.
Correlated Parameters to Accurately Measure Uncertainty in Deep Neural Networks

In this article, a novel approach for training deep neural networks using Bayesian techniques is presented.The Bayesian methodology allows for an easy evaluation of model uncertainty and, additionally, is robust to overfitting.These are commonly the two main problems classical, i.e., non-Bayesian architectures have to struggle with.The proposed approach applies variational inference in order to approximate the intractable posterior distribution.In particular, the variational distribution is defined as the product of multiple multivariate normal distributions with tridiagonal covariance matrices.Every single normal distribution belongs either to the weights or to the biases corresponding to one network layer.The layerwise a posteriori variances are defined based on the corresponding expectation values, and furthermore, the correlations are assumed to be identical.Therefore, only a few additional parameters need to be optimized compared with non-Bayesian settings.The performance of the new approach is evaluated and compared with other recently developed Bayesian methods.Basis of the performance evaluations are the popular benchmark data sets MNIST and CIFAR-10.Among the considered approaches, the proposed one shows the best predictive accuracy.Moreover, extensive evaluations of the provided prediction uncertainty information indicate that the new approach often yields more useful uncertainty estimates than the comparison methods.
Analysis and Classification of Crime Tweets

Nowadays social Networking and micro-blogging sites like Twitter are very popular and millions of users are registered on these websites. The users present on these website use these websites as a platform to express their thoughts and opinions. Our analysis of content posted on Twitter shows that users often post crime related information on Twitter. Among these crime related tweets some tweets are the crime messages that need police attention. Detection of such tweets can be beneficial in utilizing pattroling resources. The analysis of the data present on these websites can have an enormous impact. In this paper,the work is done on analyzing Twitter data to identify crime tweet that need police attention. Text mining based approach is used for classification of 369 tweets into crime and not-crime class. Classifiers such as Naive Bayesian, Random Forest, J48 and ZeroR are used. Among all of these four classifiers, Random forest classifier give the best accuracy of 98.1%.
A New Benchmark on American Sign Language Recognition using Convolutional Neural Network

The listening or hearing impaired (deaf/dumb) people use a set of signs, called sign language instead of speech for communication among them. However, it is very challenging for non-sign language speakers to communicate with this community using signs. It is very necessary to develop an application to recognize gestures or actions of sign languages to make easy communication between the normal and the deaf community. The American Sign Language (ASL) is one of the mostly used sign languages in the World, and considering its importance, there are already existing methods for recognition of ASL with limited accuracy. The objective of this study is to propose a novel model to enhance the accuracy of the existing methods for ASL recognition. The study has been performed on the alphabet and numerals of four publicly available ASL datasets. After preprocessing, the images of the alphabet and numerals were fed to a newly proposed convolutional neural network (CNN) model, and the performance of this model was evaluated to recognize the numerals and alphabet of these datasets. The proposed CNN model significantly (9%) improves the recognition accuracy of ASL reported by some existing prominent methods.
Additive Manufactured Polymer-Bonded Isotropic NdFeB Magnets by Stereolithography and Their Comparison to Fused Filament Fabricated and Selective Laser Sintered Magnets

Magnetic isotropic NdFeB powder with a spherical morphology is used to 3D print magnets by stereolithography (SLA). Complex magnets with small feature sizes in a superior surface quality can be printed with SLA. The magnetic properties of the 3D printed bonded magnets are investigated and compared with magnets manufactured by fused filament fabrication (FFF), and selective laser sintering (SLS). All methods use the same hard magnetic isotropic NdFeB powder material. FFF and SLA use a polymer matrix material as binder, SLS sinters the powder directly. SLA can print magnets with a remanence of 388 mT and a coercivity of 0.923 T. A complex magnetic design for speed wheel sensing applications is presented and printed with all methods.
FedNAS: Federated Deep Learning via Neural Architecture Search

Federated Learning (FL) has been proved to be an effective learning framework when data cannot be centralized due to privacy, communication costs, and regulatory restrictions. When training deep learning models under an FL setting, people employ the predefined model architecture discovered in the centralized environment. However, this predefined architecture may not be the optimal choice because it may not fit data with non-identical and independent distribution (non-IID). Thus, we advocate automating federated learning (AutoFL) to improve model accuracy and reduce the manual design effort. We specifically study AutoFL via Neural Architecture Search (NAS), which can automate the design process. We propose a Federated NAS (FedNAS) algorithm to help scattered workers collaboratively searching for a better architecture with higher accuracy. We also build a system based on FedNAS. Our experiments on non-IID dataset show that the architecture searched by FedNAS can outperform the manually predefined architecture.
Fast video crowd counting with a Temporal Aware Network

Crowd counting aims to count the number of instantaneous people in a crowded space, and many promising solutions have been proposed for single image crowd counting. With the ubiquitous video capture devices in public safety field, how to effectively apply the crowd counting technique to video content has become an urgent problem. In this paper, we introduce a novel framework based on temporal aware modeling of the relationship between video frames. The proposed network contains a few dilated residual blocks, and each of them consists of the layers that compute the temporal convolutions of features from the adjacent frames to improve the prediction. To alleviate the expensive computation and satisfy the demand of fast video crowd counting, we also introduce a lightweight network to balance the computational cost with representation ability. We conduct experiments on the crowd counting benchmarks and demonstrate its superiority in terms of effectiveness and efficiency over previous video-based approaches.
Towards Better Detection and Analysis of Massive Spatiotemporal Co-Occurrence Patterns

With the rapid development of sensing technologies, massive spatiotemporal data have been acquired from the urban space with respect to different domains, such as transportation and environment. Numerous co-occurrence patterns (e.g., traffic speed <; 10km/h, weather = foggy, and air quality = unhealthy) between the transportation data and other types of data can be obtained with given spatiotemporal constraints (e.g., within 3 kilometers and lasting for 2 hours) from these heterogeneous data sources. Such patterns present valuable implications for many urban applications, such as traffic management, pollution diagnosis, and transportation planning. However, extracting and understanding these patterns is beyond manual capability because of the scale, diversity, and heterogeneity of the data. To address this issue, a novel visual analytics system called CorVizor is proposed to identify and interpret these co-occurrence patterns. CorVizor comprises two major components. The first component is a co-occurrence mining framework involving three steps, namely, spatiotemporal indexing, co-occurring instance generation, and pattern mining. The second component is a visualization technique called CorView that implements a level-of-detail mechanism by integrating tailored visualizations to depict the extracted spatiotemporal co-occurrence patterns. The case studies and expert interviews are conducted to demonstrate the effectiveness of CorVizor.
Feature engineering with clinical expert knowledge: A case study assessment of machine learning model complexity and performance

Incorporating expert knowledge at the time machine learning models are trained holds promise for producing models that are easier to interpret. The main objectives of this study were to use a feature engineering approach to incorporate clinical expert knowledge prior to applying machine learning techniques, and to assess the impact of the approach on model complexity and performance. Four machine learning models were trained to predict mortality with a severe asthma case study. Experiments to select fewer input features based on a discriminative score showed low to moderate precision for discovering clinically meaningful triplets, indicating that discriminative score alone cannot replace clinical input. When compared to baseline machine learning models, we found a decrease in model complexity with use of fewer features informed by discriminative score and filtering of laboratory features with clinical input. We also found a small difference in performance for the mortality prediction task when comparing baseline ML models to models that used filtered features. Encoding demographic and triplet information in ML models with filtered features appeared to show performance improvements from the baseline. These findings indicated that the use of filtered features may reduce model complexity, and with little impact on performance.
Social Media Sentiment Analysis On Twitter Datasets

In the era of growing world, Social Sites is one of the platform where lots of people interacted with it. Every individual directly or indirectly connected with the social sites. As we can see, People are used to about taking reviews about anything before doing it-as taking reviews of movies, restaurants, online product shopping and many more. Taking reviews means knowing opinions about things. In a single manner it can state as Sentiment Analysis, or even it can be called as Opinion Mining or Data Mining. Here, the present work is come with the idea of Twitter Sentiment Analysis, in which process is designed to know the person thought about any specific tweet done by them. After knowing the people opinions about any issue, any individual can come up with the conclusion. To do this types of analysis, Present work has taken Natural Language Processing (NLP) in use. So this research paper describe the types of techniques used and also the procedure performed throughout the process.