Reviewerly/exHarmony huggingface dataset:
Deep into visual saliency for immersive VR environments rendered in real-time

As virtual reality (VR) headsets with head-mounted-displays (HMDs) are becoming more and more prevalent, new research questions are arising. One of the emergent questions is how best to employ visual saliency prediction in VR applications using current line of advanced HMDs. Due to the complex nature of human visual attention mechanism, the problem needs to be investigated from different points of view using different approaches. Having such an outlook, this work extends the previous effort on exploring a set of well-studied visual saliency cues and saliency prediction methods making use of these cues with the aim of assessing how applicable they are for estimating visual saliency in immersive VR environments that are rendered in real-time and experienced with consumer HMDs. To that end, a new user study was conducted with a larger sample and reveals the effects of experiencing dynamic computer-generated scenes with reduced navigation speeds on visual saliency. Using these scenes that have varying visual experiences in terms of contents and range of depth-of-field, the study also compares VR viewing to 2D desktop viewing with an expanded set of results. The presented evaluation offers the most in-depth view of visual saliency in immersive, real-time rendered VR to date. The analysis encompassing the results of both studies indicate that decreasing navigation speed reduces the contribution of depth-cue to visual saliency and has a boosting effect for cues based on 2D image features only. While there are content-dependent variances among their scores, it is seen that the saliency prediction methods based on boundary connectivity and surroundedness work best in general for the given settings.
Generalized pseudo Bayesian algorithms for tracking of multiple model underwater maneuvering target

The strength of Generalized Pseudo Bayesian (GPB) algorithms is exploited in the presented study to enhance the target tracking precision, effective model approximation and rapid convergence of multimodel maneuvering object tracking. The GPB methods are considered to be suitable for approximating systems whose dynamics follow discrete-time and fixed state Markov process. Underwater maneuvering target tracking problems are usually solved with nonlinear Bayesian algorithms, in which kinetics of object are associated with passive bearings using state-space modeling. Here accuracy and convergence of GPB methods based on Interacting Multiple Model Extended Kalman Filter (IMMEKF), Interacting Multiple Model Extended Kalman Smoother (IMMEKS), Interacting Multiple Model Unscented Kalman Filter (IMMUKF) and Interacting Multiple Model Unscented Kalman Smoother (IMMUKS) are efficiently analyzed for tracking of multimodel maneuvering target in complex ocean environment. Application of these algorithms is systematically presented for estimating the real-time state of a maneuvering object that follows a coordinated turn trajectory. Performance analysis of IMM Kalman filters and smoothers is done with variations in the standard deviation of white Gaussian measurement noise by following Bearings Only Tracking (BOT) phenomena. Least Mean Square Error (MSE) between approximated and the real position of maneuvering target in rectangular coordinates is calculated for analyzing the performance of filtering and smoothing techniques. Simulation results of the Monte Carlo runs validate the effectiveness of IMMEKS and IMMUKS over IMMEKF and IMMUKF for scenario of given framework.
<i>Retracted:</i> Design of multimedia blockchain privacy protection system based on distributed trusted communication

Abstract Blockchain is a hash chain with chain structure formed by data blocks in chronological order. With the promotion of bitcoin, the research and application of blockchain, the underlying core technology of bitcoin, is growing rapidly, which is considered to be the fifth generation of Internet disruptive technology after mobile Internet. At present, there are various identity management applications and platforms based on blockchain, but while blockchain has gradually become the cornerstone of the value Internet, it also brings great challenges to the privacy protection needs of users. This article analyzes the privacy protection mechanism of blockchain network layer, transaction layer and application layer, including malicious node detection and access restriction technology in network layer, mixed currency technology and encryption technology in blockchain transaction layer. In this article, distributed communicable network technology and multimedia data security technology are used to build a blockchain security system. We designed the efficient system to accurately analyze the data and the information protection. The experimental results show that the proposed method has better data security guarantee ability.
Robust pruning for efficient CNNs

Deep convolutional neural network (CNN) with considerable number of parameters is one of the promising methods for image recognition. There, however, is generally difficult in applying deep CNNs to resource constrained devices due to the heavy computational burden. For reducing computational cost of CNNs while retaining the classification performance, it is effective to apply pruning methods that remove from CNNs redundant parameters less contributing to classification. The contribution of parameters can be estimated by the empirical classification loss computed over training samples to which ground-truth labels are assigned. The empirical classification loss, however, might be vulnerable to the outlier samples and/or the hard ones that are difficult to classify, and thus the pruning would accordingly be degraded. In this paper, we propose a pruning method based on a novel criterion to measure the redundancy of the parameters in CNNs through empirical classification loss. We start with the Taylor expansion of the loss function and then derive the mathematical formulation of the pruning criterion so as to be robust against some sort of outlier samples. The proposed pruning criterion can also provide stable metric for parameters and evaluate layers of various depth fairly without biases toward shallower or deeper layers. In addition, we present an effective method to normalize the criterion scores for further improving performance. In the experiments on image classification, our method exhibits favorable performance compared with the other methods.
Fixed Smooth Convolutional Layer for Avoiding Checkerboard Artifacts in CNNS

In this paper, we propose a fixed convolutional layer with an order of smoothness not only for avoiding checkerboard artifacts in convolutional neural networks (CNNs) but also for enhancing the performance of CNNs, where the smoothness of its filter kernel can be controlled by a parameter. It is well-known that a number of CNNs generate checkerboard artifacts in both of two process: forward-propagation of upsampling layers and backward-propagation of strided convolutional layers. The proposed layer can perfectly prevent checkerboard artifacts caused by strided convolutional layers or upsampling layers including transposed convolutional layers. In an image-classification experiment with four CNNs: a simple CNN, VGG8, ResNet-18, and ResNet-101, applying the fixed layers to these CNNs is shown to improve the classification performance of all CNNs. In addition, the fixed layer are applied to generative adversarial networks (GANs), for the first time. From image-generation results, a smoother fixed convolutional layer is demonstrated to enable us to improve the quality of images generated with GANs.
In-Domain and Out-of-Domain Data Augmentation to Improve Children’s Speaker Verification System in Limited Data Scenario

In this paper, we present our efforts towards developing a robust automatic speaker verification (ASV) system for children when the domain-specific data is limited. For that purpose, we have studied the effect of in-domain and out-of-domain data augmentation. Several different combinations of data augmentation are studied in this work. Speed and pitch perturbation of children's speech are employed for synthetically creating in-domain data to be used for augmentation. For out-of-domain data augmentation, on the other hand, adults' speech is pooled together with children's speech. At the same time, voice conversion (VC) is also applied on adults' speech to alter the acoustic attributes. VC of adults' speech makes it perceptually similar to that of children's speech. The converted adults' data is then used for augmentation. The ASV systems developed in this study employ x-vectors derived using a time-delay deep neural network. In addition to that, probabilistic linear discriminant analysis is used for scoring the performance. The explored methods of data augmentation are noted to reduce the equal error rate as well as minimum decision cost function by a large margin.
Novel Image Watermarking Approach against Noise and RST Attacks

Digital watermarking is used for copyright protection of intellectual properties and hence it is important to protect and recover data from attacks. Existing systems work on and recover the image from single attacks. This paper aims at proposing a technique for retrieval of cover image and watermarked data from dual attacks like rotation and noise attacks. The proposed system takes as input, Color images and then embeds data using alpha blending method. And also tries to recover from dual attacks using median filtering and pseudo Zernike moment. Proposed approach selects LH band from the 4 sub bands image divided into using DWT and for reducing noise from the image SVD is used. Proposed approach is giving better results in terms of PSNR and MSE as compared to other existing systems using QWT.
Where to buy face masks? Survey of applications using Taiwan’s open data in the time of coronavirus disease 2019

Background: The coronavirus disease 2019 (COVID-19) had spread rapidly since late December 2019. Personal protective equipment was essential to prevent transmission. Owing to shortage of face masks, Taiwan government began to implement quasi rationing on February 6, 2020, by allowing each resident to purchase two masks in seven days. Taiwan National Health Insurance Administration offered online data with real-time updates on face mask availability in all contracted pharmacies and selected local health centers. Based on the open data, numerous software applications quickly emerged to assist the public in finding sales locations efficiently. Methods: Up until March 15, 2020, the Public Digital Innovation Space of Taiwan government had recorded 134 software applications of face mask availability, and 24 software applications were excluded due to defect, duplicate, and unavailability. These applications were analyzed according to platform, developer type, and display mode. Results: Of the 110 valid software applications, 67 (60.9%) applications were deployed on websites, followed by 21 (19.1%) on social networking sites, 19 (17.3%) as mobile applications, and 3 (2.7%) in other modes. Nearly two thirds (n = 70) of applications were developed by individuals, one third (n = 37) by commercial companies, only two applications by central and local governments, and one by a nongovernmental organization. With respect to the display mode, 47 (42.7%) applications adopted map-view only, 41 (37.3%) adopted table-view only, and 19 (17.3%) adopted both modes. Of the remaining three applications, two offered voice user interfaces and one used augmented reality. Conclusion: Taiwan’s open data strategy facilitated rapid development of software applications for information dissemination to the public during the COVID-19 crisis. The transparency of real-time data could help alleviate the panic of the public. The collaborative contributions from the grassroots in disasters were priceless treasures.
Fuzzy Neural Network for Pattern Classification

This research work proposes a fuzzy neural network (FNN) for pattern classification. The proposed network is the modified version of the Radial basis function neural network (RBFNN). FNN uses supervised fuzzy clustering and pruning algorithm to determine the precise number of clusters with proper centroid and width to form the processing nodes in the hidden layer. These clusters represent fuzzy set hyperspheres (FSHs), which are defined by the fuzzy membership function. The training between the hidden layer to output layer which is done by using the LMS algorithm in RBFNN is avoided, and the output is determined by using the fuzzy union operation. The fuzzy membership function shields the clustered patterns resulting in 100% accuracy for the data set used during training. Unlike other clustering algorithms used to construct the hidden layer of RBFNN, the proposed clustering algorithm is independent of tuning parameters and is fast in training and retrieval. Thus FNN reduces the computation time, guarantees 100% accuracy for any training set, and provides superior and comparable recognition accuracy for the datasets with the precise number of FSHs in the hidden layer. Hence the proposed FNN can be used for pattern classification.
“StoryWeb”: A storytelling‐based knowledge‐sharing application among multiple stakeholders

The research objective of this paper is to develop a storytelling‐based knowledge‐sharing application that enables users to co‐create their own stories for both individuals and groups. To address this, a design science research methodology was applied for elucidating users' requirements. As empirical evidence, a case study was conducted on the children's book industry to synthesize a knowledge‐sharing design application named “ StoryWeb ”. Usability tests were conducted to reconfigure users' feedback and suggestions after two StoryWeb prototypes were developed. This study makes three main contributions. First, it empirically tests individual's or group's creativity and co‐creation by a view of knowledge sharing. Second, it methodologically applies a design thinking approach into a knowledge‐sharing study. Third, it also practically suggests feasible guidelines for the creativity and innovation research community on which features of storytelling‐based applications can be configured.
SF-Sketch: A Two-Stage Sketch for Data Streams

Sketches are probabilistic data structures designed for recording frequencies of items in a multi-set. They are widely used in various fields, especially for gathering Internet statistics from distributed data streams in network measurements. In a distributed streaming application with high data rates, a sketch in each monitoring node “fills up” very quickly and then its content is transferred to a remote collector responsible for answering queries. Thus, the size of the contents transferred must be kept as small as possible while meeting the desired accuracy requirement. To obtain significantly higher accuracy while keeping the same update and query speed as the best prior sketches, in this article, we propose a new sketch - the Slim-Fat (SF) sketch. The key idea behind the SF-sketch is to maintain two separate sketches: a larger sketch, the Fat-subsketch, and a smaller sketch, the Slim-subsketch. The Fat-subsketch is used for updating and periodically producing the Slim-subsketch, which is then transferred to the remote collector for answering queries quickly and accurately. We also present the error bound as well as an accurate model of the correct rate of the SF-sketch, and verify their correctness through experiments. We implemented and extensively evaluated the SF-sketch along with several prior sketches. Our results show that when the size of our Slim-subsketch and of the widely used Count-Min (CM) sketch are kept the same, our SF-sketch outperforms the CM-sketch by up to 33.1 times in terms of accuracy (when the ratio of the sizes of the Fat-subsketch and the Slim-subsketch is 16:1). We have made all source codes publicly available at Github [“Source code of SF sketches,” [Online]. Available: https://github.com/paper2017/SF-sketch].
Relaxation of the Radio-Frequency Linewidth for Coherent-Optical Orthogonal Frequency-Division Multiplexing Schemes by Employing the Improved Extreme Learning Machine

A coherent optical (CO) orthogonal frequency division multiplexing (OFDM) scheme gives a scalable and flexible solution for increasing the transmission rate, being extremely robust to chromatic dispersion as well as polarization mode dispersion. Nevertheless, as any coherent-detection OFDM system, the overall system performance is limited by laser phase noises. On the other hand, extreme learning machines (ELMs) have gained a lot of attention from the machine learning community owing to good generalization performance, negligible learning speed, and minimum human intervention. In this manuscript, a phase-error mitigation method based on the single-hidden layer feedforward network prone to the improved ELM algorithm for CO-OFDM systems is introduced for the first time. In the training step, two steps are distinguished. Firstly, pilots are used, which is very common in OFDM-based systems, to diminish laser phase noises as well as to correct frequency-selective impairments and, therefore, the bandwidth efficiency can be maximized. Secondly, the regularization parameter is included in the ELM to balance the empirical and structural risks, namely to minimize the root mean square error in the test stage and, consequently, the bit error rate (BER) metric. The operational principle of the real-complex (RC) ELM is analytically explained, and then, its sub-parameters (number of hidden neurons, regularization parameter, and activation function) are numerically found in order to enhance the system performance. For binary and quadrature phase-shift keying modulations, the RC-ELM outperforms the benchmark pilot-assisted equalizer as well as the fully-real ELM, and almost matches the common phase error (CPE) compensation and the ELM defined in the complex domain (C-ELM) in terms of the BER over an additive white Gaussian noise channel and different laser oscillators. However, both techniques are characterized by the following disadvantages: the CPE compensator reduces the transmission rate since an additional preamble is mandatory for channel estimation purposes, while the C-ELM requires a bounded and differentiable activation function in the complex domain and can not follow semi-supervised training. In the same context, the novel ELM algorithm can not compete with the CPE compensator and C-ELM for the 16-ary quadrature amplitude modulation. On the other hand, the novel ELM exposes a negligible computational cost with respect to the C-ELM and PAE methods.
Missing the Forest for the Trees: End-to-End AI Application Performance in Edge Data Centers

Artificial intelligence and machine learning are experiencing widespread adoption in the industry, academia, and even public consciousness. This has been driven by the rapid advances in the applications and accuracy of AI through increasingly complex algorithms and models; this, in turn, has spurred research into developing specialized hardware AI accelerators. The rapid pace of the advances makes it easy to miss the forest for the trees: they are often developed and evaluated in a vacuum without considering the full application environment in which they must eventually operate. In this paper, we deploy and characterize Face Recognition, an AI-centric edge video analytics application built using open source and widely adopted infrastructure and ML tools. We evaluate its holistic, end-to-end behavior in a production-size edge data center and reveal the "AI tax" for all the processing that is involved. Even though the application is built around state-of-the-art AI and ML algorithms, it relies heavily on pre-and post-processing code which must be executed on a general-purpose CPU. As AI-centric applications start to reap the acceleration promised by so many accelerators, we find they impose stresses on the underlying software infrastructure and the data center's capabilities: storage and network bandwidth become major bottlenecks with increasing AI acceleration. By not having to serve a wide variety of applications, we show that a purpose-built edge data center can be designed to accommodate the stresses of accelerated AI at 15% lower TCO than one derived from homogeneous servers and infrastructure. We also discuss how our conclusions generalize beyond Face Recognition as many AI-centric applications at the edge rely upon the same underlying software and hardware infrastructure.
A Blockchain-Assisted Trust Access Authentication System for Solid

The Solid (Social Linked Data) project focuses on data sharing and privacy security and aims to build a decentralized ecosystem that radically changes the way web applications work today. Our goal is to introduce a “trust access authentication system” to achieve secure authentication and fine-grained access control, thereby promoting the implementation of Solid. Blockchain, equipped with multiple security properties and authentication functions, is a crucial technology. In this paper, we present a blockchain-assisted system for secure authentication in Solid and for implementation of fine-grained access control policies. Specifically, we explore to integrate threshold RSA signatures in a permissioned blockchain system to enable a fault-tolerant distributed signature scheme, thereby enhancing the resilience and robustness of authentication system. Moreover, we utilize smart contract to control transaction flows and manage access control policies automatically. Experimental results show that our proposed trust access authentication system enhances security, scales well, and is efficient and economically feasible.
IRONHIDE: A Secure Multicore that Efficiently Mitigates Microarchitecture State Attacks for Interactive Applications

Microprocessors enable aggressive hardware virtualization by means of which multiple processes temporally execute on the system. These security-critical and ordinary processes interact with each other to assure application progress. However, temporal sharing of hardware resources exposes the processor to various microarchitecture state attacks. State-of-the-art secure processors, such as MI6 adopt Intel's SGX enclave execution model. MI6 architects strong isolation by statically isolating shared memory state, and purging the microarchitecture state of private core, cache, and TLB resources on every enclave entry and exit. The purging overhead significantly impacts performance as the interactivity across the secure and insecure processes increases. This paper proposes IRONHIDE that implements strong isolation in the context of multicores to form spatially isolated secure and insecure clusters of cores. For an interactive application comprising of secure and insecure processes, IRONHIDE pins the secure process(es) to the secure cluster, where they execute and interact with the insecure process(es) without incurring the microarchitecture state purging overheads on every interaction event. IRONHIDE improves performance by 2.1x over the MI6 baseline for a set of user and OS interactive applications. Moreover, IRONHIDE improves performance by 20% over an SGX-like baseline, while also ensuring strong isolation guarantees against microarchitecture state attacks.