Reviewerly/exHarmony huggingface dataset:
Deep Neural Network Through an InP SOA-Based Photonic Integrated Cross-Connect

Photonic neuromorphic computing is raising a growing interest as it promises to provide massive parallelism and low power consumption. In this paper, we demonstrate for the first time a feed-forward neural network via an 8 × 8 Indium Phosphide cross-connect chip, where up to 8 on-chip weighted addition circuits are co-integrated, based on semiconductor optical amplifier technology. We perform the weight calibration per neuron, resulting in a normalized root mean square error smaller than 0.08 and a best case dynamic range of 27 dB. The 4 input to 1 output weighted addition operation is executed on-chip and is part of a neuron, whose non-linear function is implemented via software. A three feedback loop optimization procedure is demonstrated to enable an output neuron accuracy improvement of up to 55%. The exploitation of this technology as neural network is evaluated by implementing a trained 3-layer photonic deep neural network to solve the Iris flower classification problem. Prediction accuracy of 85.8% is achieved, with respect to the 95% accuracy obtained via a computer. A comprehensive analysis of the error evolution in our system reveals that the electrical/optical conversions dominate the error contribution, which suggests that an all optical approach is preferable for future neuromorphic computing hardware design.
OPU: An FPGA-Based Overlay Processor for Convolutional Neural Networks

Field-programmable gate array (FPGA) provides rich parallel computing resources with high energy efficiency, making it ideal for deep convolutional neural network (CNN) acceleration. In recent years, automatic compilers have been developed to generate network-specific FPGA accelerators. However, with more cascading deep CNN algorithms adapted by various complicated tasks, reconfiguration of FPGA devices during runtime becomes unavoidable when network-specific accelerators are employed. Such reconfiguration can be difficult for edge devices. Moreover, network-specific accelerator means regeneration of RTL code and physical implementation whenever the network is updated. This is not easy for CNN end users. In this article, we propose a domain-specific FPGA overlay processor, named OPU to accelerate CNN networks. It offers software-like programmability for CNN end users, as CNN algorithms are automatically compiled into executable codes, which are loaded and executed by OPU without reconfiguration of FPGA for switch or update of CNN networks. Our OPU instructions have complicated functions with variable runtimes but a uniform length. The granularity of instruction is optimized to provide good performance and sufficient flexibility, while reducing complexity to develop microarchitecture and compiler. Experiments show that OPU can achieve an average of 91% runtime multiplication and accumulation unit (MAC) efficiency (RME) among nine different networks. Moreover, for VGG and YOLO networks, OPU outperforms automatically compiled network-specific accelerators in the literature. In addition, OPU shows 5.35× better power efficiency compared with Titan Xp. For a real-time cascaded CNN networks scenario, OPU is 2.9× faster compared with edge computing GPU Jetson Tx2, which has a similar amount of computing resources.
State-of-the-art speaker recognition with neural network embeddings in NIST SRE18 and Speakers in the Wild evaluations

We present a thorough analysis of the systems developed by the JHU-MIT consortium in the context of NIST speaker recognition evaluation 2018. In the previous NIST evaluation, in 2016, i-vectors were the speaker recognition state-of-the-art. However now, neural network embeddings (a.k.a. x-vectors) rise as the best performing approach. We show that in some conditions, x-vectors’ detection error reduces by 2 w.r.t. i-vectors. In this work, we experimented on the Speakers In The Wild evaluation (SITW), NIST SRE18 VAST (Video Annotation for Speech Technology), and SRE18 CMN2 (Call My Net 2, telephone Tunisian Arabic) to compare network architectures, pooling layers, training objectives, back-end adaptation methods, and calibration techniques. x-Vectors based on factorized and extended TDNN networks achieved performance without parallel on SITW and CMN2 data. However for VAST, performance was significantly worse than for SITW. We noted that the VAST audio quality was severely degraded compared to the SITW, even though they both consist of Internet videos. This degradation caused strong domain mismatch between training and VAST data. Due to this mismatch, large networks performed just slightly better than smaller networks. This also complicated VAST calibration. However, we managed to calibrate VAST by adapting SITW scores distribution to VAST, using a small amount of in-domain development data. Regarding pooling methods, learnable dictionary encoder performed the best. This suggests that representations learned by x-vector encoders are multi-modal. Maximum margin losses were better than cross-entropy for in-domain data but not in VAST mismatched data. We also analyzed back-end adaptation methods in CMN2. PLDA semi-supervised adaptation and adaptive score normalization (AS-Norm) yielded significant improvements. However, results were still worse than in English in-domain conditions like SITW. We conclude that x-vectors have become the new state-of-the-art in speaker recognition. However, their advantages reduce in cases of strong domain mismatch. We need to investigate domain adaptation and domain invariant training approaches to improve performance in all conditions. Also, speech enhancement techniques with a focus on improving the speaker recognition performance could be of great help.
The Microsoft Academic Knowledge Graph: A Linked Data Source with 8 Billion Triples of Scholarly Data

In this paper, we present the Microsoft Academic Knowledge Graph (MAKG), a large RDF data set with over eight billion triples with information about scientific publications and related entities, such as authors, institutions, journals, and fields of study. The data set is licensed under the Open Data Commons Attribution License (ODC-By). By providing the data as RDF dump files as well as a data source in the Linked Open Data cloud with resolvable URIs and links to other data sources, we bring a vast amount of scholarly data to the Web of Data. Furthermore, we provide entity embeddings for all 210 million represented publications. We facilitate a number of use case scenarios, particularly in the field of digital libraries, such as (1) entity-centric exploration of papers, researchers, affiliations, etc.; (2) data integration tasks using RDF as a common data model and links to other data sources; and (3) data analysis and knowledge discovery of scholarly data.
Learning Fragment Self-Attention Embeddings for Image-Text Matching

In image-text matching task, the key to good matching quality is to capture the rich contextual dependencies between fragments of image and text. However, previous works either simply aggregate the similarity of all possible pairs of image regions and words, or take multi-step cross attention to attend to image regions and words with each other as context, which requires exhaustive similarity computation between all image region and word pairs. In this paper, we propose Self-Attention Embeddings (SAEM) to exploit fragment relations in images or texts by self-attention mechanism, and aggregate fragment information into visual and textual embeddings. Specifically, SAEM extracts salient image regions based on bottom-up attention, and takes WordPiece tokens as sentence fragments. The self-attention layers are built to model subtle and fine-grained fragment relation in image and text respectively, which consists of multi-head self-attention sub-layer and position-wise feed-forward network sub-layer. Consequently, the fragment self-attention mechanism can discover the fragment relations and identify the semantically salient regions in images or words in sentences, and capture their interaction more accurately. By simultaneously exploiting the fine-grained fragment relation in both visual and textual modalities, our method produces more semantically consistent embeddings for representing images and texts, and demonstrates promising image-text matching accuracy and high efficiency on Flickr30K and MSCOCO datasets.
Bridging the Domain Gap for Ground-to-Aerial Image Matching

The visual entities in cross-view (e.g. ground and aerial) images exhibit drastic domain changes due to the differences in viewpoints each set of images is captured from. Existing state-of-the-art methods address the problem by learning view-invariant images descriptors. We propose a novel method for solving this task by exploiting the generative powers of conditional GANs to synthesize an aerial representation of a ground-level panorama query and use it to minimize the domain gap between the two views. The synthesized image being from the same view as the reference (target) image, helps the network to preserve important cues in aerial images following our Joint Feature Learning approach. We fuse the complementary features from a synthesized aerial image with the original ground-level panorama features to obtain a robust query representation. In addition, we employ multi-scale feature aggregation in order to preserve image representations at different scales useful for solving this complex task. Experimental results show that our proposed approach performs significantly better than the state-of-the-art methods on the challenging CVUSA dataset in terms of top-1 and top-1% retrieval accuracies. Furthermore, we evaluate the generalization of the proposed method for urban landscapes on our newly collected cross-view localization dataset with geo-reference information.
Integrated computational intelligent paradigm for nonlinear electric circuit models using neural networks, genetic algorithms and sequential quadratic programming

In this paper, a novel application of biologically inspired computing paradigm is presented for solving initial value problem (IVP) of electric circuits based on nonlinear RL model by exploiting the competency of accurate modeling with feed forward artificial neural network (FF-ANN), global search efficacy of genetic algorithms (GA) and rapid local search with sequential quadratic programming (SQP). The fitness function for IVP of associated nonlinear RL circuit is developed by exploiting the approximation theory in mean squared error sense using an approximate FF-ANN model. Training of the networks is conducted by integrated computational heuristic based on GA-aided with SQP, i.e., GA-SQP. The designed methodology is evaluated to variants of nonlinear RL systems based on both AC and DC excitations for number of scenarios with different voltages, resistances and inductance parameters. The comparative studies of the proposed results with Adam’s numerical solutions in terms of various performance measures verify the accuracy of the scheme. Results of statistics based on Monte-Carlo simulations validate the accuracy, convergence, stability and robustness of the designed scheme for solving problem in nonlinear circuit theory.
Deep Learning-Based Classification of Liver Cancer Histopathology Images Using Only Global Labels

Liver cancer is a leading cause of cancer deaths worldwide due to its high morbidity and mortality. Histopathological image analysis (HIA) is a crucial step in the early diagnosis of liver cancer and is routinely performed manually. However, this process is time-consuming, error-prone, and easily affected by the expertise of pathologists. Recently, computer-aided methods have been widely applied to medical image analysis; however, the current medical image analysis studies have not yet focused on the histopathological morphology of liver cancer due to its complex features and the insufficiency of training images with detailed annotations. This paper proposes a deep learning method for liver cancer histopathological image classification using only global labels. To compensate for the lack of detailed cancer region annotations in those images, patch features are extracted and fully utilized. Transfer learning is used to obtain the patch-level features and then combined with multiple-instance learning to acquire the image-level features for classification. The method proposed here solves the processing of large-scale images and training sample insufficiency in liver cancer histopathological images for image classification. The proposed method can distinguish and classify liver histopathological images as abnormal or normal with high accuracy, thus providing support for the early diagnosis of liver cancer.
Convolutional Sequence Generation for Skeleton-Based Action Synthesis

In this work, we aim to generate long actions represented as sequences of skeletons. The generated sequences must demonstrate continuous, meaningful human actions, while maintaining coherence among body parts. Instead of generating skeletons sequentially following an autoregressive model, we propose a framework that generates the entire sequence altogether by transforming from a sequence of latent vectors sampled from a Gaussian process (GP). This framework, named Convolutional Sequence Generation Network (CSGN), jointly models structures in temporal and spatial dimensions. It captures the temporal structure at multiple scales through the GP prior and the temporal convolutions; and establishes the spatial connection between the latent vectors and the skeleton graphs via a novel graph refining scheme. It is noteworthy that CSGN allows bidirectional transforms between the latent and the observed spaces, thus enabling semantic manipulation of the action sequences in various forms. We conducted empirical studies on multiple datasets, including a set of high-quality dancing sequences collected by us. The results show that our framework can produce long action sequences that are coherent across time steps and among body parts.
Cost-Effective Video Summarization Using Deep CNN With Hierarchical Weighted Fusion for IoT Surveillance Networks

Video summarization (VS) has attracted intense attention recently due to its enormous applications in various computer vision domains, such as video retrieval, indexing, and browsing. Traditional VS researches mostly target at the effectiveness of the VS algorithms by introducing the high quality of features and clusters for selecting representative visual elements. Due to the increased density of vision sensors network, there is a tradeoff between the processing time of the VS methods with reasonable and representative quality of the generated summaries. It is a challenging task to generate a video summary of significant importance while fulfilling the needs of Internet of Things (IoT) surveillance networks with constrained resources. This article addresses this problem by proposing a new computationally effective solution through designing a deep CNN framework with hierarchical weighted fusion for the summarization of surveillance videos captured in IoT settings. The first stage of our framework designs discriminative rich features extracted from deep CNNs for shot segmentation. Then, we employ image memorability predicted from a fine-tuned CNN model in the framework, along with aesthetic and entropy features to maintain the interestingness and diversity of the summary. Third, a hierarchical weighted fusion mechanism is proposed to produce an aggregated score for the effective computation of the extracted features. Finally, an attention curve is constituted using the aggregated score for deciding outstanding keyframes for the final video summary. Experiments are conducted using benchmark data sets for validating the importance and effectiveness of our framework, which outperforms the other state-of-the-art schemes.
Spatial-temporal data-driven service recommendation with privacy-preservation

The ever-increasing popularity of web service sharing communities have produced a considerable amount of web services that share similar functionalities but vary in Quality of Services (QoS) performances. To alleviate the heavy service selection burden on users, lightweight recommendation ideas, e.g., Collaborative Filtering (CF) have been developed to aid users to select their preferred services. However, existing CF methods often face two challenges. First, service QoS is often context-aware and hence depends on the spatial and temporal information of service invocations heavily. While it requires challenging efforts to integrate both spatial and temporal information into service recommendation decision-making process simultaneously. Second, the location-aware and time-aware QoS data often contain partial sensitive information of users, which raise an emergent privacy-preservation requirement when performing service recommendations. In view of above two challenges, in this paper, we integrate the spatial-temporal information of QoS data and Locality-Sensitive Hashing (LSH) into recommendation domain and bring forth a location-aware and time-aware recommendation approach considering privacy concerns. At last, a set of experiments conducted on well-known WS-DREAM dataset show the feasibility of our approach.
Fundamental Concepts of Convolutional Neural Network

Convolutional neural network (or CNN) is a special type of multilayer neural network or deep learning architecture inspired by the visual system of living beings. The CNN is very much suitable for different fields of computer vision and natural language processing. The main focus of this chapter is an elaborate discussion of all the basic components of CNN. It also gives a general view of foundation of CNN, recent advancements of CNN and some major application areas.
Kernel density estimation based sampling for imbalanced class distribution

Imbalanced response variable distribution is a common occurrence in data science. In fields such as fraud detection, medical diagnostics, system intrusion detection and many others where abnormal behavior is rarely observed the data under study often features disproportionate target class distribution. One common way to combat class imbalance is through resampling the minority class to achieve a more balanced distribution. In this paper, we investigate the performance of the sampling method based on kernel density estimation (KDE). We believe that KDE offers a more natural way of generating new instances of minority class that is less prone to overfitting than other standard sampling techniques. It is based on a well established theory of nonparametric statistical estimation. Numerical experiments show that KDE can outperform other sampling techniques on a range of real life datasets as measured by F1-score and G-mean. The results remain consistent across a number of classification algorithms used in the experiments. Furthermore, the proposed method outperforms the benchmark methods irregardless of the class distribution ratio. We conclude, based on the solid theoretical foundation and strong experimental results, that the proposed method would be a valuable tool in problems involving imbalanced class distribution.
A New Algorithm on Application of Blockchain Technology in Live Stream Video Transmissions and Telecommunications

This article develops and defines Blockchain technology in its classic format. New suggested proposed algorithms are then analyzed in order to introduce new and modified versions of Blockchain technology. After that, fundamental infrastructure is presented in order to represent its application in new generation of telecommunications. In addition, this article interrogates these algorithms and their efficiency to make secure connections that transfer data packets in any format (boxes or packets of information) in a secure and encrypted method at which sender and receiver of information remain anonymous. Then, this research describes applications of the novel approach in new format of making live stream technology in real world communications. Moreover, according to this new approach, new concepts can be predicted in the new generation of social media based on live communications. Meanwhile, the compatibility is justified for consistency, reliability and flexibility of this new proposed technology with other existing and defined format of technology in today's world. At last, conclusions of this new emerging technology and its superiority compared to other designed technologies in the field of live streaming and telecommunications are discussed.
IoT Public Fog Nodes Reputation System: A Decentralized Solution Using Ethereum Blockchain

Public fog nodes extend cloud services for the Internet of Things (IoT) clients and smart devices to provide additional computation capabilities, storage space, and reduce latency and response time.The openness and pervasiveness of public fog nodes leads to the requirement of using trust models to ensure reliability, security, privacy, and meet the service-level agreements (SLAs).Conventional trust models for public fog nodes are centrally configured, deployed, and maintained considering security, privacy, and SLA requirements.However, these trust models enforce centralized governance policies across the system which leads towards the single-point-of-failure and single-point-of-compromise over IoT devices' and users' personal data.This paper proposes a decentralized trust model in order to maintain the reputation of publicly available fog nodes.The reputation is maintained considering users' opinions about their past interactions with the public fog nodes.The proposed trust model is designed using public Ethereum blockchain and smart contract technologies in order to enable decentralized trustworthy service provisioning between IoT devices and public fog nodes.The proposed approach is tested and evaluated in terms of security, performance, and cost.The results show that using blockchain for decentralized reputation management could become more advantageous when compared to the existing centralized trust models.
Implementing healthcare services on a large scale: Challenges and remedies based on blockchain technology

The accessibility of electronic healthcare data is necessary for effective treatment, policy decisions, and healthcare information exchange. Due to the intangibility of digital data, healthcare information is also prone to privacy-breach and security attacks. Further, the importance of immutability and privacy of healthcare data becomes colossal when a nationwide healthcare and wellness scheme is planned to be implemented. Providing quality healthcare services to such an enormous population size is challenging and requires proper technological infrastructure. The cooperation from the society is equally important to lay such a copious architecture on which the healthcare services should seamlessly run. To assess the social and technical challenges that lie ahead in implementing large-scale comprehensive healthcare services and suggest a technology-intervening solution to serve the society at large. This study considers India's National Health Policy (2017) initiatives. The social and technical hurdles in implementation of the schemes are discussed, and AarogyaChain, a Blockchain technology-based solution is proposed to eliminate the health policy implementation hiccups. We find that the scalability is a primary concern in implementing healthcare services on blockchain at such a large scale. We experimented by creating a blockchain and found that the system throughput is a function of the number of special nodes called ordering nodes, and a trade-off is required to balance between time-to-commit and system's fault tolerance. Blockchain provides a secure and transparent system of integrated healthcare services that keeps patients at the center and provides for corruption intolerant and efficient implementation of nationwide health-insurance programs.