Reviewerly/exHarmony huggingface dataset:
Learning Cross-Aligned Latent Embeddings for Zero-Shot Cross-Modal Retrieval

Zero-Shot Cross-Modal Retrieval (ZS-CMR) is an emerging research hotspot that aims to retrieve data of new classes across different modality data. It is challenging for not only the heterogeneous distributions across different modalities, but also the inconsistent semantics across seen and unseen classes. A handful of recently proposed methods typically borrow the idea from zero-shot learning, i.e., exploiting word embeddings of class labels (i.e., class-embeddings) as common semantic space, and using generative adversarial network (GAN) to capture the underlying multimodal data structures, as well as strengthen relations between input data and semantic space to generalize across seen and unseen classes. In this paper, we propose a novel method termed Learning Cross-Aligned Latent Embeddings (LCALE) as an alternative to these GAN based methods for ZS-CMR. Unlike using the class-embeddings as the semantic space, our method seeks for a shared low-dimensional latent space of input multimodal features and class-embeddings by modality-specific variational autoencoders. Notably, we align the distributions learned from multimodal input features and from class-embeddings to construct latent embeddings that contain the essential cross-modal correlation associated with unseen classes. Effective cross-reconstruction and cross-alignment criterions are further developed to preserve class-discriminative information in latent space, which benefits the efficiency for retrieval and enable the knowledge transfer to unseen classes. We evaluate our model using four benchmark datasets on image-text retrieval tasks and one large-scale dataset on image-sketch retrieval tasks. The experimental results show that our method establishes the new state-of-the-art performance for both tasks on all datasets.
Intrinsic motivation and episodic memories for robot exploration of high-dimensional sensory spaces

This work presents an architecture that generates curiosity-driven goal-directed exploration behaviours for an image sensor of a microfarming robot. A combination of deep neural networks for offline unsupervised learning of low-dimensional features from images, and of online learning of shallow neural networks representing the inverse and forward kinematics of the system have been used. The artificial curiosity system assigns interest values to a set of pre-defined goals, and drives the exploration towards those that are expected to maximise the learning progress. We propose the integration of an episodic memory in intrinsic motivation systems to face catastrophic forgetting issues, typically experienced when performing online updates of artificial neural networks. Our results show that adopting an episodic memory system not only prevents the computational models from quickly forgetting knowledge that has been previously acquired, but also provides new avenues for modulating the balance between plasticity and stability of the models.
Solar Energy Estimations in India Using Remote Sensing Technologies and Validation with Sun Photometers in Urban Areas

Solar radiation ground data is available in poor spatial resolution, which provides an opportunity and demonstrates the necessity to consider solar irradiance modeling based on satellite data. For the first time, solar energy monitoring in near real-time has been performed for India. This study focused on the assessment of solar irradiance from the Indian Solar Irradiance Operational System (INSIOS) using operational cloud and aerosol data from INSAT-3D and Copernicus Atmosphere Monitoring Service (CAMS)-Monitoring Atmospheric Composition Climate (MACC), respectively. Simulations of the global horizontal irradiance (GHI) and direct normal irradiance (DNI) were evaluated for 1 year for India at four Baseline Surface Radiation Network (BSRN) stations located in urban regions. The INSIOS system outputs as per radiative transfer model results presented high accuracy under clear-sky and cloudy conditions for GHI and DNI. DNI was very sensitive to the presence of cloud and aerosols, where even with small optical depths the DNI became zero, and thus it affected the accuracy of simulations under realistic atmospheric conditions. The median BSRN and INSIOS difference was found to vary from −93 to −49 W/m2 for GHI and −103 to −76 W/m2 for DNI under high solar energy potential conditions. Clouds were able to cause an underestimation of 40%, whereas for various aerosol inputs to the model, the overall accuracy was high for both irradiances, with the coefficient of determination being 0.99, whereas the penetration of photovoltaic installation, which exploits GHI, into urban environments (e.g., rooftop) could be effectively supported by the presented methodology, as estimations were reliable during high solar energy potential conditions. The results showed substantially high errors for monsoon season due to increase in cloud coverage that was not well-predicted at satellite and model resolutions.
Edge Computing with Embedded AI

With the rise of the IoT, there has been a growing demand for people counting and occupancy estimation in Intelligent buildings for adapting their heating, ventilation and cooling systems. This can have a significant impact on energy consumption at a global scale as such systems consume about 40% of electricity and create about 36% of the CO2 emissions in Europe. Previous approaches to occupancy estimation either utilize methods that do not ensure people's privacy when obtaining high accuracy estimations, such as RGB cameras, or utilize thermal or radar sensors with lower accuracy. Thermal vision for people detection has several advantages. It protects people's privacy while being less affected by changes in the environment. In addition, most of the previous image processing approaches rely on streaming the data to the cloud to be analyzed. However, with the development of the more distributed network paradigms edge and fog computing, there has been a trend in moving computation towards the edge of the network. This process of embedding intelligence into end-devices enables more efficient energy consumption and network load distribution. In this work, we present an embedded algorithm for room occupancy estimation based on a thermal sensor with accuracy over the state-of-the-art. We study the performance of a variety of deep learning models on different embedded processors. We achieve a prediction accuracy of 98.9% for people counting estimation with minimal 2 KB RAM utilization. Furthermore, the proposed algorithm has very low latency achieving execution times under 14 ms.
A rigorous framework for the mean field limit of multilayer neural networks

We develop a mathematically rigorous framework for multilayer neural networks in the mean field regime. As the network's widths increase, the network's learning trajectory is shown to be well captured by a meaningful and dynamically nonlinear limit (the \textit{mean field} limit), which is characterized by a system of ODEs. Our framework applies to a broad range of network architectures, learning dynamics and network initializations. Central to the framework is the new idea of a \textit{neuronal embedding}, which comprises of a non-evolving probability space that allows to embed neural networks of arbitrary widths. Using our framework, we prove several properties of large-width multilayer neural networks. Firstly we show that independent and identically distributed initializations cause strong degeneracy effects on the network's learning trajectory when the network's depth is at least four. Secondly we obtain several global convergence guarantees for feedforward multilayer networks under a number of different setups. These include two-layer and three-layer networks with independent and identically distributed initializations, and multilayer networks of arbitrary depths with a special type of correlated initializations that is motivated by the new concept of \textit{bidirectional diversity}. Unlike previous works that rely on convexity, our results admit non-convex losses and hinge on a certain universal approximation property, which is a distinctive feature of infinite-width neural networks and is shown to hold throughout the training process. Aside from being the first known results for global convergence of multilayer networks in the mean field regime, they demonstrate flexibility of our framework and incorporate several new ideas and insights that depart from the conventional convex optimization wisdom.
Multi-Scale Dynamic Feature Encoding Network for Image Demoiréing

The prevalence of digital sensors, such as digital cameras and mobile phones, simplifies the acquisition of photos. Digital sensors, however, suffer from producing Moire when photographing objects having complex textures, which deteriorates the quality of photos. Moire spreads across various frequency bands of images and is a dynamic texture with varying colors and shapes, which pose two main challenges in demoireing - an important task in image restoration. In this paper, towards addressing the first challenge, we design a multi-scale network to process images at different spatial resolutions, obtaining features in different frequency bands, and thus our method can jointly remove moire in different frequency bands. Towards solving the second challenge, we propose a dynamic feature encoding module (DFE), embedded in each scale, for dynamic texture. Moire pattern can be eliminated more effectively via DFE.Our proposed method, termed Multi-scale convolutional network with Dynamic feature encoding for image DeMoireing (MDDM), can outperform the state of the arts in fidelity as well as perceptual on benchmarks.
Introduction to Google Dialogflow

Dialogflow is a natural language understanding platform from Google that helps developers to design conversational interfaces and integrate them into their applications. Dialogflow can be integrated with other cognitive services such as sentiment analysis, knowledge base services, etc. Dialogflow can also leverage cognitive, AI, and other services available from other cloud providers through API-based integration.
Prognostic analysis of histopathological images using pre-trained convolutional neural networks: application to hepatocellular carcinoma

Histopathological images contain rich phenotypic descriptions of the molecular processes underlying disease progression. Convolutional neural networks, state-of-the-art image analysis techniques in computer vision, automatically learn representative features from such images which can be useful for disease diagnosis, prognosis, and subtyping. Hepatocellular carcinoma (HCC) is the sixth most common type of primary liver malignancy. Despite the high mortality rate of HCC, little previous work has made use of CNN models to explore the use of histopathological images for prognosis and clinical survival prediction of HCC. We applied three pre-trained CNN models—VGG 16, Inception V3 and ResNet 50—to extract features from HCC histopathological images. Sample visualization and classification analyses based on these features showed a very clear separation between cancer and normal samples. In a univariate Cox regression analysis, 21.4% and 16% of image features on average were significantly associated with overall survival (OS) and disease-free survival (DFS), respectively. We also observed significant correlations between these features and integrated biological pathways derived from gene expression and copy number variation. Using an elastic net regularized Cox Proportional Hazards model of OS constructed from Inception image features, we obtained a concordance index (C-index) of 0.789 and a significant log-rank test ( p = 7.6E−18). We also performed unsupervised classification to identify HCC subgroups from image features. The optimal two subgroups discovered using Inception model image features showed significant differences in both overall (C-index = 0.628 and p = 7.39E−07) and DFS (C-index = 0.558 and p = 0.012). Our work demonstrates the utility of extracting image features using pre-trained models by using them to build accurate prognostic models of HCC as well as highlight significant correlations between these features, clinical survival, and relevant biological pathways. Image features extracted from HCC histopathological images using the pre-trained CNN models VGG 16, Inception V3 and ResNet 50 can accurately distinguish normal and cancer samples. Furthermore, these image features are significantly correlated with survival and relevant biological pathways.
Adaptive fault-tolerant model for improving cloud computing performance using artificial neural network

Cloud computing is a paradigm that has greatly contributed to the emergence of new applications of IT services. However, the design of dependable and efficient cloud architectures requires a high knowledge of the recurring failures causes which compromise the efficient functioning of the system. Nowadays, the number of datasets recording the real state of cloud systems still rarely available to the public. This article uses failure data published by Backblaze which is one of the largest data storage providers. This dataset consists of the operating status of heterogeneous servers that were collected during the period between January 2015 and December 2018. Then, the data has been filtered and preprocessed to keep a total of 2,878,440 records including 128,820 failures. We investigate the correlation between hard drive parameters and failure ​​by exploiting the attributes of Self-Monitoring, Analysis and Reporting Technology (SMART). Then, we investigate the predictive capabilities of five machine learning models including naïve Bayes and artificial neural networks (ANN) to define a failure prediction module for the cloud architecture. The experimental results demonstrate that the artificial neural network (ANN) model offers the best prediction accuracy.
Comparison of Various Learning Rate Scheduling Techniques on Convolutional Neural Network

The learning rate is a hyperparameter which determines how much the model should change concerning the error each time the model parameters are updated. It is important to tune the learning rate properly because if it is set too low, our model will converge very slowly and if set too high, our model may diverge from the optimal error point. Some conventional learning rate tuning techniques include constant learning rate, step decay, cyclical learning rate and many more. In this paper, we have implemented some of these techniques and compared the model performances gained using these techniques.
Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis

In this paper we propose Flowtron: an autoregressive flow-based generative network for text-to-speech synthesis with control over speech variation and style transfer. Flowtron borrows insights from IAF and revamps Tacotron in order to provide high-quality and expressive mel-spectrogram synthesis. Flowtron is optimized by maximizing the likelihood of the training data, which makes training simple and stable. Flowtron learns an invertible mapping of data to a latent space that can be manipulated to control many aspects of speech synthesis (pitch, tone, speech rate, cadence, accent). Our mean opinion scores (MOS) show that Flowtron matches state-of-the-art TTS models in terms of speech quality. In addition, we provide results on control of speech variation, interpolation between samples and style transfer between speakers seen and unseen during training. Code and pre-trained models will be made publicly available at https://github.com/NVIDIA/flowtron
Analyzing hack subnetworks in the bitcoin transaction graph

Abstract Hacks are one of the most damaging types of cryptocurrency related crime, accounting for billions of dollars in stolen funds since 2009. Professional investigators at Chainalysis have traced these stolen funds from the initial breach on an exchange to off-ramps, i.e. services where criminals are able to convert the stolen funds into fiat or other cryptocurrencies. We analyzed six hack subnetworks of bitcoin transactions known to belong to two prominent hacking groups. We analyze each hack according to eight network features, both static and temporal, and successfully classify each hack to its respective hacking group through our newly proposed method. We find that the static features, such as node balance, in degree, and out degree are not as useful in classifying the hacks into hacking groups as temporal features related to how quickly the criminals cash out. We validate our operating hypothesis that the key distinction between the two hacking groups is the acceleration with which the funds exit through terminal nodes in the subnetworks.
Integrated navigation of GPS/INS based on fusion of recursive maximum likelihood IMM and Square-root Cubature Kalman filter

Information fusion of the GPS/INS integrated system is always related to characteristics of the inertial system and the sensor feature, yet prior knowledge is still difficult to obtain in real applications. To deal with the uncertainty of error covariance and state noise in vehicle navigation, this paper presents a novel approach, wherein the integration of Square-root Cubature Kalman Filters (SCKF) and Interacting Multiple Model (IMM) are also introduced. In the framework of IMM, the SCKFs with different covariance are designed to reflect various vehicle dynamics. Besides, since the IMM-SCKF can switch flexibly among the filters, the transition probability matrix is computed with maximum likelihood method to adapt to different noise characteristics. The performance of the proposed algorithm is guaranteed by theoretical analyses, and a series of vehicular experiments with different maneuvers are carried out in an urban environment. The results indicate that, in comparison with the CKF and the IMM-CKF, the accuracy of velocity and attitude are increased by the proposed strategy.
Exploiting non-taxonomic relations for measuring semantic similarity and relatedness in WordNet

Various applications in the areas of computational linguistics and artificial intelligence employ semantic similarity to solve challenging tasks, such as word sense disambiguation, text classification, information retrieval, machine translation, and document clustering. Previous work on semantic similarity followed a mono-relational approach using mostly the taxonomic relation "ISA". This paper explores the benefits of using all types of non-taxonomic relations in large linked data, such as WordNet knowledge graph, to enhance existing semantic similarity and relatedness measures. We propose a holistic poly-relational approach based on a new relation-based information content and non-taxonomic-based weighted paths to devise a comprehensive semantic similarity and relatedness measure. To demonstrate the benefits of exploiting non-taxonomic relations in a knowledge graph, we used three strategies to deploy non-taxonomic relations at different granularity levels. We conducted experiments on four well-known gold standard datasets, and the results demonstrated the robustness and scalability of the proposed semantic similarity and relatedness measure, which significantly improves existing similarity measures.
Incrementally updating the high average-utility patterns with pre-large concept

Abstract High-utility itemset mining (HUIM) is considered as an emerging approach to detect the high-utility patterns from databases. Most existing algorithms of HUIM only consider the itemset utility regardless of the length. This limitation raises the utility as a result of a growing itemset size. High average-utility itemset mining (HAUIM) considers the size of the itemset, thus providing a more balanced scale to measure the average-utility for decision-making. Several algorithms were presented to efficiently mine the set of high average-utility itemsets (HAUIs) but most of them focus on handling static databases. In the past, a fast-updated (FUP)-based algorithm was developed to efficiently handle the incremental problem but it still has to re-scan the database when the itemset in the original database is small but there is a high average-utility upper-bound itemset (HAUUBI) in the newly inserted transactions. In this paper, an efficient framework called PRE-HAUIMI for transaction insertion in dynamic databases is developed, which relies on the average-utility-list (AUL) structures. Moreover, we apply the pre-large concept on HAUIM. A pre-large concept is used to speed up the mining performance, which can ensure that if the total utility in the newly inserted transaction is within the safety bound, the small itemsets in the original database could not be the large ones after the database is updated. This, in turn, reduces the recurring database scans and obtains the correct HAUIs. Experiments demonstrate that the PRE-HAUIMI outperforms the state-of-the-art batch mode HAUI-Miner, and the state-of-the-art incremental IHAUPM and FUP-based algorithms in terms of runtime, memory, number of assessed patterns and scalability.
Hybrid Data Hiding Based on AMBTC Using Enhanced Hamming Code

The image-based data hiding method is a technology used to transmit confidential information secretly. Since images (e.g., grayscale images) usually have sufficient redundancy information, they are a very suitable medium for hiding data. Absolute Moment Block Truncation Coding (AMBTC) is one of several compression methods and is appropriate for embedding data due to its very low complexity and acceptable distortion. However, since there is not enough redundant data compared to grayscale images, the research to embed data in the compressed image is a very challenging topic. That is the motivation and challenge of this research. Meanwhile, the Hamming codes are used to embed secret bits, as well as a block code that can detect up to two simultaneous bit errors and correct single bit errors. In this paper, we propose an effective data hiding method for two quantization levels of each block of AMBTC using Hamming codes. Bai and Chang introduced a method of applying Hamming (7,4) to two quantization levels; however, the scheme is ineffective, and the image distortion error is relatively large. To solve the problem with the image distortion errors, this paper introduces a way of optimizing codewords and reducing pixel distortion by utilizing Hamming (7,4) and lookup tables. In the experiments, when concealing 150,000 bits in the Lena image, the averages of the Normalized Cross-Correlation (NCC) and Mean-Squared Error (MSE) of our proposed method were 0.9952 and 37.9460, respectively, which were the highest. The sufficient experiments confirmed that the performance of the proposed method is satisfactory in terms of image embedding capacity and quality.