Reviewerly/exHarmony huggingface dataset:
Tracking Translation Invariance in CNNs

Although Convolutional Neural Networks (CNNs) are widely used, their translation invariance (ability to deal with translated inputs) is still subject to some controversy. We explore this question using translation-sensitivity maps to quantify how sensitive a standard CNN is to a translated input. We propose the use of cosine similarity as sensitivity metric over Euclidean distance, and discuss the importance of restricting the dimensionality of either of these metrics when comparing architectures. Our main focus is to investigate the effect of different architectural components of a standard CNN on that network’s sensitivity to translation. By varying convolutional kernel sizes and amounts of zero padding, we control the size of the feature maps produced, allowing us to quantify the extent to which these elements influence translation invariance. We also measure translation invariance at different locations within the CNN to determine the extent to which convolutional and fully connected layers, respectively, contribute to the translation invariance of a CNN as a whole. Our analysis indicates that both convolutional kernel size and feature map size have a systematic influence on translation invariance. We also see that convolutional layers contribute less than expected to translation invariance, when not specifically forced to do so.
Caption Generation From Road Images for Traffic Scene Modeling

In this traffic-scene-modeling study, we propose an image-captioning network which incorporates element attention into an encoder-decoder mechanism to generate more reasonable scene captions. A visual-relationship-detecting network is also developed to detect the relative positions of object pairs. Firstly, the traffic scene elements are detected and segmented according to their clustered locations. Then, the image-captioning network is applied to generate the corresponding description of each traffic scene element. The visual-relationship-detecting network is utilized to detect the position relations of all object pairs in the subregion. The static and dynamic traffic elements are appropriately selected and organized to construct a 3D model according to the captions and the position relations. The reconstructed 3D traffic scenes can be utilized for the offline test of unmanned vehicles. The evaluations and comparisons based on the TSD-max, KITTI and Microsoft’s COCO datasets demonstrate the effectiveness of the proposed framework.
Multimodal Event-Aware Network for Sentiment Analysis in Tourism

Considering the application of a sentiment analysis in decision-making and personalized advertising, we adopt it in tourism. Specifically, we perform a sentiment analysis on the posted Weibos about the passengers’ experience in civil aviation travel. Different travel events could influence passengers’ sentiment, e.g., flight delay may cause negative sentiment. Inspired by this observation, we propose a novel multimodal event-aware network to analyze sentiment from Weibos that contain multiple modalities, i.e., text and images. We first extract features from each modality and, then, model the cross-modal associations to obtain more discriminative representations, based on which we simultaneously perceive the event and sentiment in a multitask framework. Extensive experiments demonstrate that the proposed method outperforms the existing state-of-the-art approaches.
Privacy Enhanced Energy Prediction in Smart Building using Federated Learning

Prediction of energy consumption is useful in energy budgeting of smart grids, expenditure budgeting by consumers, and comfort management of smart buildings. Essentially, building management systems of smart buildings need to manage energy, efficiently. In order to do this, energy prediction plays an important role. The energy consumption, in general, is predicted using machine learning algorithms. However, machine learning algorithms demand massive amounts of data for performing well. Acquisition of this data from data owners can lead to privacy breaches. Federated learning is a framework of distributed systems which can mitigate privacy breaches to certain extent. Federated learning has as such been developed for mobile edge devices like vehicles, phones etc. In this paper, a novel application of federated learning framework focused on the smart building energy prediction scenario is presented. The architectural details on how the federated learning framework is applied is presented. Also, the performance of this prediction model as compared to centralized method of machine learning is discussed. Deep neural networks are used with the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) dataset to realize and evaluate this architecture.
Single image deblurring with cross-layer feature fusion and consecutive attention

Single image deblurring aims to restore the single blurry image to its sharp counterpart and remains an active topic of enduring interest. Recently, deep Convolutional Neural Network (CNN) based methods have achieved promising performance. However, two primary limitations mainly exist on those CNNs-based image deblurring methods: most of them simply focus on increasing the complexity of the network, and rarely make full use of features extracted by encoder. Meanwhile, most of the methods perform the deblurred image reconstruction immediately after the decoder, and the roles of the decoded features are always underestimated. To address these issues, we propose a single image deblurring method, in which two modules to fuse multiple features learned in encoder (the Cross-layer Feature Fusion (CFF) module) and manipulate the features after decoder (the Consecutive Attention Module (CAM)) are specially designed, respectively. The CFF module is to concatenate different layers of features from encoder to enhance rich structural information to decoder, and the CAM module is able to generate more important and correlated textures to the reconstructed sharp image. Besides, the ranking content loss is employed to further restore more realistic details in the deblurred images. Comprehensive experiments demonstrate that our proposed method can generate less blur and more textures in deblurred image on both synthetic datasets and real-world image examples.
SFPN: Semantic Feature Pyramid Network for Object Detection

Feature Pyramid Network(FPN) employs a top-down path to enhance low level feature by utilizing high level feature. However, further improvement of detector is greatly hindered by the inner defect of FPN. The dilution issue in FPN is analyzed in this paper, and a new architecture named Semantic Feature Pyramid Network(SFPN) is introduced to address the information imbalance problem caused by information dilution. The proposed method consists of two simple and effective components: Semantic Pyramid Module(SPM) and Semantic Feature Fusion Module(SFFM). To compensate for the weaknesses of FPN, the semantic segmentation result is utilized as an extra information source in our architecture. By constructing a semantic pyramid based on the segmentation result and fusing it with FPN, feature maps at each level can obtain the necessary information without suffering from the dilution issue. The proposed architecture could be applied on many detectors, and non-negligible improvement could be achieved. Although this method is designed for object detection, other tasks such as instance segmentation can also largely benefit from it. The proposed method brings Faster R-CNN and Mask R-CNN with ResNet-50 as backbone both 1.8 AP improvements respectively. Furthermore, SFPN improves Cascade R-CNN with backbone ResNet-101 from 42.4 AP to 43.5 AP.
Light3DPose: Real-time Multi-Person 3D Pose Estimation from Multiple Views

We present an approach to perform 3D pose estimation of multiple people from a few calibrated camera views. Our architecture, leveraging the recently proposed unprojection layer, aggregates feature-maps from a 2D pose estimator backbone into a comprehensive representation of the 3D scene. Such intermediate representation is then elaborated by a fully-convolutional volumetric network and a decoding stage to extract 3D skeletons with sub-voxel accuracy. Our method achieves state of the art MPJPE on the CMU Panoptic dataset using a few unseen views and obtains competitive results even with a single input view. We also assess the transfer learning capabilities of the model by testing it against the publicly available Shelf dataset obtaining good performance metrics. The proposed method is inherently efficient: as a pure bottom-up approach, it is computationally independent of the number of people in the scene. Furthermore, even though the computational burden of the 2D part scales linearly with the number of input views, the overall architecture is able to exploit a very lightweight 2D backbone which is orders of magnitude faster than the volumetric counterpart, resulting in fast inference time. The system can run at 6 FPS, processing up to 10 camera views on a single 1080Ti GPU.
User recommendation based on Hybrid filtering in Telegram messenger

Over the past decade, social networks and messengers have found a special place in the creation and development of businesses. User recommendation is a very important feature in social networks that has attracted the attention of many users to these environments. Using this system in an instant messenger environment is very useful. Telegram is a cloud-based messenger with more than 400 million monthly active users. Telegram is used as a social network in Iran, but does not offer the most widely used features of social networks, such as recommending users. This feature is important for marketers to find target audience. This paper presents a hybrid filtering-based algorithm to recommend Telegram users. This method combines the membership graph of users with the profile of groups. The membership graph, models users based on their membership in groups. Also, the profile of each group includes the name and description of the group. We have created a bag of words for each group based on natural language processing methods to combine it with the membership graph. After combination process, users are recommended based on the list of groups obtained. The data used in this study is the information of more than 120 million users and 900,000 supergroups in Telegram. This data is obtained through Telegram API by Idekav system. The evaluation of the proposed method has been done separately on two categories of specialized supergroups. Each category includes 25 specialized supergroups in Telegram. Selected supergroups for evaluation have between 2,000 and 10,000 members. Experimental results show the integrity of the model and error reduction in RMSE.
Painting Portals: connecting homes through live paintings

Feeling co-present and connected over distance is a challenge. To create a sense of sharing a space over distance and being together in the periphery of daily life, we designed and implemented a pair of connected picture frames that display paintings generated from live camera streams. These paintings are abstracted based on the distance of the user from the device, and are sent between two portals that can be placed anywhere in the world. We deployed the Painting Portals during a week-long field study investigating how a distance-separated mother and daughter experienced the use of the Painting Portals. We generated recommendations for future design and research, including: (1) stylizing the paintings in a way that better captures emotion/sentiment, (2) placing multiple Painting Portals in different common areas throughout a home, (3) incorporating a video call feature into the Painting Portals, and (4) adding the option to turn off one's camera for more privacy.
Self-supervised Learning for Astronomical Image Classification

In Astronomy, a huge amount of image data is generated daily by photometric surveys, which scan the sky to collect data from stars, galaxies and other celestial objects. In this paper, we propose a technique to leverage unlabeled astronomical images to pre-train deep convolutional neural networks, in order to learn a domain-specific feature extractor which improves the results of machine learning techniques in setups with small amounts of labeled data available. We show that our technique produces results which are in many cases better than using ImageNet pre-training.
Sub Contractor Life Cycle Management In Enterprise System Using Blockchain Technology

Sub Contractor Life Cycle Management is one of the critical jobs for various industries, there are many flaws and issues in the existing operational management of SubContracting through enterprise systems. Existing issues in Sub Contractor Management System (Life Cycle Management System on-boarding, extending and releasing to be precise), causes a delay in on-boarding and extension of sub-contractors which directly impacts the revenue of an organization. Typically IT Organization suffers an annual revenue loss of 3-5%. The Operations Management (for Sub Contractor Onboarding / Project Extension) involves the interference of various Sub Systems in an organization like Talent Management System, Human Resource Management System, Finance Management System, Time Sheet Management System, Project Management Systems, Vendor Management System. Synchronizing of data and audit process across such different (maybe heterogeneous) system, (heterogeneous because each system might have a different mechanism in storing data) is also a big challenge. There is a lack of visibility and control for each actor involved in the sub-contractor management life cycle. Authors in this literature have proposed a blockchain-based solution to counter the above issue(s) and the system promises to bring down the revenue loss w.r.t Subcontractor Life Cycle management system to 0.1 % - 0.5 %, by leveraging the feature of blockchain to provide better visibility, trust, real-time monitoring of information/data related to Sub-Contractor by integrating the end-points (Blockchain as a Service) where each subsystem is added as a node in the blockchain network, each event (transaction in blockchain world) happening in any subsystem will be updated on the blockchain ledger using Smart Contract or Chaincode, which will provide single source of truth, transparency, end to end visibility in the entire digital operations.
The Impact of Arabic Part of Speech Tagging on Sentiment Analysis: A New Corpus and Deep Learning Approach

Sentiment Analysis is achieved by using Natural Language Processing (NLP) techniques and finds wide applications in analyzing social media content to determine people's opinions, attitudes, and emotions toward entities, individuals, issues, events, or topics. The accuracy of sentiment analysis depends on automatic Part-of-Speech (PoS) tagging which is required to label words according to grammatical categories. The challenge of analyzing the Arabic language has found considerable research interest, but now the challenge is amplified with the addition of social media dialects. While numerous morphological analyzers and PoS taggers were proposed for Modern Standard Arabic (MSA), we are now witnessing an increased interest in applying those techniques to the Arabic dialect that is prominent in social media. Indeed, social media texts (e.g. posts, comments, and replies) differ significantly from MSA texts in terms of vocabulary and grammatical structure. Such differences call for reviewing the PoS tagging methods to adapt social media texts. Furthermore, the lack of sufficiently large and diverse social media text corpora constitutes one of the reasons that automatic PoS tagging of social media content has been rarely studied. In this paper, we address those limitations by proposing a novel Arabic social media text corpus that is enriched with complete PoS information, including tags, lemmas, and synonyms. The proposed corpus constitutes the largest manually annotated Arabic corpus to date, with more than 5 million tokens, 238,600 MSA texts, and words from Arabic social media dialect, collected from 65,000 online users' accounts. Furthermore, our proposed corpus was used to train a custom Long Short-Term Memory deep learning model and showed excellent performance in terms of sentiment classification accuracy and F1-score. The obtained results demonstrate that the use of a diverse corpus that is enriched with PoS information significantly enhances the performance of social media analysis techniques and opens the door for advanced features such as opinion mining and emotion intelligence.
Variational Dialogue Generation with Normalizing Flows

Conditional variational autoencoder (cVAE) has shown promising performance in dialogue generation. However, there still exists two issues in dialog cVAE model. The first issue is the Kullback-Leiblier (KL) vanishing problem which results in degenerating cVAE into a simple recurrent neural network. The second issue is the assumption of isotropic Gaussian prior for latent variable which is too simple to assure diversity of the generated responses. To handle these issues, a simple distribution should be transformed into a complex distribution and simultaneously the value of KL divergence should be preserved. This paper presents the dialogue flow VAE (DF-VAE) for variational dialogue generation. In particular, KL vanishing is tackled by a new normalizing flow. An inverse autoregressive flow is proposed to transform isotropic Gaussian prior to a rich distribution. In the experiments, the proposed DF-VAE is significantly better than the other methods in terms of different evaluation metrics. The diversity of generated dialogue responses is enhanced. Ablation study is conducted to illustrate the merit of the proposed flow models.
Server Allocation for Massively Multiplayer Online Cloud Games Using Evolutionary Optimization

In recent years, Massively Multiplayer Online Games (MMOGs) are becoming popular, partially due to their sophisticated graphics and broad virtual world, and cloud gaming is demanded more than ever especially when entertaining with light and portable devices. This article considers the problem of server allocation for running MMOG on cloud, aiming to reduce the cost on cloud gaming service and meanwhile enhance the quality of service. The problem is formulated into minimizing an objective function involving the cost of server rental, the cost of data transfer and the network latency during the gaming time. A genetic algorithm is developed to solve the minimization problem for processing simultaneous server allocation for the players who log into the system at the same time while many existing players are playing the same game. Extensive experiments based on the player behavior in “World of Warcraft” are conducted to evaluate the proposed method and compare with the state-of-the-art as well. The experimental results show that the method gives a lower cost and a shorter network latency in most of the time.
Gold dispersion in tropical weathering profiles at the Belikombone gold anomaly (Bétaré Oya gold district), east Cameroon

The distribution of gold in the weathering blanket at the Belikombone gold prospect in east Cameroon provides insights into gold mobility in the secondary environment and in tropical terrains worldwide. Both gridline-controlled sampling of topsoil (surface samples) and sampling of various layers in pits are used and the gold assay for each sample determined by NiS fire assay with ICP-AES finish. One hundred and thirty-two (132) surface samples and a total of 206 samples from 19 exploration pits were analyzed. The results from the topsoil samples show an anomaly with the highest Au concentration at 5.9 mgkg−1. The mineralization corridor follows a NE-SW trend. The horizons within the pits range from sap rock at the base, through saprolite, rubble layer rich in relict quartz material to a ferruginous loose layer at the top although some horizons are missing in some pits. All the layers contain gold and the highest concentration in the sap rock horizon is 3.4 mgkg−1 while the rubble layer has a gold high of 6.1 mgkg−1. The individual soil horizons show no systematic gold trends and given the presence of gold in all layers, the patterns point towards supergene dissolution and redistribution of gold. Gold enrichment within the upper horizons in the weathering blanket is attributed to sequestration by Fe oxides of chemically remobilized gold. However, the high gold content within the sap rock and saprolite layers suggests that migration of gold in the particulate form supersedes chemical gold redistribution. Particulate gold obtained by panning samples from the pits varies in shape from euhedral, elongated to irregular. Electron microprobe analysis on the grains record high contents of gold in the rim zones (90.0 to 99.8 wt%). The cores are relatively rich in Ag (12.6 to 14.2 wt%) while the rims are poor in Ag. The low Ag content in the rims is attributed to the preferential leaching of Ag. The soil pH value in this area varies between 3.6 and 7.3. Under such acidic to near neutral conditions, bisulfide and thiosulfate ions can dissolve and transport Au and Ag to be precipitated under surficial conditions creating authigenic Au haloes especially in the saprolite and sap rock layers. Such pH values together with oxidizing Eh conditions explain the solubility of gold in the area. These results are important for geochemical exploration of gold in tropical terrains, and confirm previous studies.
Non-Singular Adversarial Robustness of Neural Networks

Adversarial robustness has become an emerging challenge for neural network owing to its over-sensitivity to small input perturbations. While being critical, we argue that solving this singular issue alone fails to provide a comprehensive robustness assessment. Even worse, the conclusions drawn from singular robustness may give a false sense of overall model robustness. Specifically, our findings show that adversarially trained models that are robust to input perturbations are still (or even more) vulnerable to weight perturbations when compared to standard models. In this paper, we formalize the notion of non-singular adversarial robustness for neural networks through the lens of joint perturbations to data inputs as well as model weights. To our best knowledge, this study is the first work considering simultaneous input-weight adversarial perturbations. Based on a multi-layer feed-forward neural network model with ReLU activation functions and standard classification loss, we establish error analysis for quantifying the loss sensitivity subject to ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> -norm bounded perturbations on data inputs and model weights. Based on the error analysis, we propose novel regularization functions for robust training and demonstrate improved non-singular robustness against joint input-weight adversarial perturbations.