Reviewerly/exHarmony huggingface dataset:
DepthLab: Real-time 3D Interaction with Depth Maps for Mobile Augmented Reality

Mobile devices with passive depth sensing capabilities are ubiquitous, and recently active depth sensors have become available on some tablets and AR/VR devices. Although real-time depth data is accessible, its rich value to mainstream AR applications has been sorely under-explored. Adoption of depth-based UX has been impeded by the complexity of performing even simple operations with raw depth data, such as detecting intersections or constructing meshes. In this paper, we introduce DepthLab, a software library that encapsulates a variety of depth-based UI/UX paradigms, including geometry-aware rendering (occlusion, shadows), surface interaction behaviors (physics-based collisions, avatar path planning), and visual effects (relighting, 3D-anchored focus and aperture effects). We break down the usage of depth into localized depth, surface depth, and dense depth, and describe our real-time algorithms for interaction and rendering tasks. We present the design process, system, and components of DepthLab to streamline and centralize the development of interactive depth features. We have open-sourced our software at https://github.com/googlesamples/arcore-depth-lab to external developers, conducted performance evaluation, and discussed how DepthLab can accelerate the workflow of mobile AR designers and developers. With DepthLab we aim to help mobile developers to effortlessly integrate depth into their AR experiences and amplify the expression of their creative vision.
Decentralized Finance: On Blockchain- and Smart Contract-based Financial Markets

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI
AN OTSU image segmentation based on fruitfly optimization algorithm

Despite its simplicity and high accuracy, the real-time efficiency of OTSU segmentation is not high. In order to promote the real-timeliness of image segmentation, this paper introduces the fruitfly optimization algorithm (FOA) to OTSU segmentation, creating an FOA-OTSU segmentation algorithm. In the proposed algorithm, the optimal threshold for segmentation is searched for by the FOA. The classic Lena picture, Flower picture and Cameraman picture were used for simulation experiment, and the results were evaluated by signal-to-noise ratio (SNR) and peak signal-to-noise ratio (PSNR). The results show that the segmentation time is reduced by about 50.0% on the premise of the PSNR and SNR evaluation criteria and the segmentation effect basically unchanged. The simulations indicate that our method converges faster and consumes fewer time than traditional OTSU algorithm, without sacrificing the segmentation accuracy. The research provides a desirable tool with high real-time performance of rapid image segmentation.
DNA and Plaintext Dependent Chaotic Visual Selective Image Encryption

Visual selective image encryption can both improve the efficiency of the image encryption algorithm and reduce the frequency and severity of attacks against data.In this article, a new form of encryption is proposed based on keys derived from Deoxyribonucleic Acid (DNA) and plaintext image.The proposed scheme results in chaotic visual selective encryption of image data.In order to make and ensure that this new scheme is robust and secure against various kinds of attacks, the initial conditions of the chaotic maps utilized are generated from a random DNA sequence as well as plaintext image via an SHA-512 hash function.To increase the key space, three different single dimension chaotic maps are used.In the proposed scheme, these maps introduce diffusion in a plain image by selecting a block that have greater correlation and then it is bitwise XORed with the random matrix.The other two chaotic maps break the correlation among adjacent pixels via confusion (row and column shuffling).Once the ciphertext image has been divided into the respective units of Most Significant Bits (MSBs) and Least Significant Bit (LSBs), the host image is passed through lifting wavelet transformation, which replaces the low-frequency blocks of the host image (i.e., HL and HH) with the aforementioned MSBs and LSBs of ciphertext.This produces a final visual selective encrypted image and all security measures proves the robustness of the proposed scheme.
Tackling mode collapse in multi-generator GANs with orthogonal vectors

Generative Adversarial Networks (GANs) have been widely used to generate realistic-looking instances. However, training robust GAN is a non-trivial task due to the problem of mode collapse. Although many GAN variants are proposed to overcome this problem, they have limitations. Those existing studies either generate identical instances or result in negative gradients during training. In this paper, we propose a new approach to training GAN to overcome mode collapse by employing a set of generators, an encoder and a discriminator. A new minimax formula is proposed to simultaneously train all components in a similar spirit to vanilla GAN. The orthogonal vector strategy is employed to guide multiple generators to learn different information in a complementary manner. In this way, we term our approach Multi-Generator Orthogonal GAN (MGO-GAN). Specifically, the synthetic data produced by those generators are fed into the encoder to obtain feature vectors. The orthogonal value is calculated between any two feature vectors, which loyally reflects the correlation between vectors. Such a correlation indicates how different information has been learnt by generators. The lower the orthogonal value is, the more different information the generators learn. We minimize the orthogonal value along with minimizing the generator loss through back-propagation in the training of GAN. The orthogonal value is integrated with the original generator loss to jointly update the corresponding generator's parameters. We conduct extensive experiments utilizing MNIST, CIFAR10 and CelebA datasets to demonstrate the significant performance improvement of MGO-GAN in terms of generated data quality and diversity at different resolutions.
A tutorial on distance metric learning: Mathematical foundations, algorithms, experimental analysis, prospects and challenges

Distance metric learning is a branch of machine learning that aims to learn distances from the data, which enhances the performance of similarity-based algorithms. This tutorial provides a theoretical background and foundations on this topic and a comprehensive experimental analysis of the most-known algorithms. We start by describing the distance metric learning problem and its main mathematical foundations, divided into three main blocks: convex analysis, matrix analysis and information theory. Then, we will describe a representative set of the most popular distance metric learning methods used in classification. All the algorithms studied in this paper will be evaluated with exhaustive testing in order to analyze their capabilities in standard classification problems, particularly considering dimensionality reduction and kernelization. The results, verified by Bayesian statistical tests, highlight a set of outstanding algorithms. Finally, we will discuss several potential future prospects and challenges in this field. This tutorial will serve as a starting point in the domain of distance metric learning from both a theoretical and practical perspective.
Momentum search algorithm: a new meta-heuristic optimization algorithm inspired by momentum conservation law

A novel optimization methodology, Momentum Search Algorithm (MSA) is presented based on Newton's laws: the law of conservation of momentum. It includes a set of masses in a closed system considering the conservation of momentum and kinetic energy of bodies. The possible solutions are presented by system bodies' positions in an n-dimensional space. The mass of bodies is proportional to their fitness function. Larger masses represent the better solutions. At each iteration, an external body collides separately with all solution bodies and moves them toward the optimum solution. The direction of the collision depends on the position of solution bodies and the position of the body with the best fitness function. As the better solutions have heavier bodies, the external body has less effect on their positions. On the other hand, the worse solutions are lighter and moved easily by the external body toward the better positions. The best position is achieved by allowing the external body to move the solution bodies toward better positions. The numerical results obtained from several standard benchmark test functions indicate the superiority of the proposed method over many other optimization techniques such as Genetic Algorithm, Particle Swarm Optimization, Gravitational Search Algorithm, Teaching–Learning-Based Optimization, Grey Wolf Optimizer, Grasshopper Optimization Algorithm, Spotted Hyena Optimizer, and Emperor Penguin Optimizer.
Optimal path planning approach based on Q-learning algorithm for mobile robots

In fact, optimizing path within short computation time still remains a major challenge for mobile robotics applications. In path planning and obstacles avoidance, Q-Learning (QL) algorithm has been widely used as a computational method of learning through environment interaction. However, less emphasis is placed on path optimization using QL because of its slow and weak convergence toward optimal solutions. Therefore, this paper proposes an Efficient Q-Learning (EQL) algorithm to overcome these limitations and ensure an optimal collision-free path in less possible time. In the QL algorithm, successful learning is closely dependent on the design of an effective reward function and an efficient selection strategy for an optimal action that ensures exploration and exploitation. In this regard, a new reward function is proposed to initialize the Q-table and provide the robot with prior knowledge of the environment, followed by a new efficient selection strategy proposal to accelerate the learning process through search space reduction while ensuring a rapid convergence toward an optimized solution. The main idea is to intensify research at each learning stage, around the straight-line segment linking the current position of the robot to Target (optimal path in terms of length). During the learning process, the proposed strategy favors promising actions that not only lead to an optimized path but also accelerate the convergence of the learning process. The proposed EQL algorithm is first validated using benchmarks from the literature, followed by a comparison with other existing QL-based algorithms. The achieved results showed that the proposed EQL gained good learning proficiency; besides, the training performance is significantly improved compared to the state-of-the-art. Concluded, EQL improves the quality of the paths in terms of length, computation time and robot safety, furthermore outperforms other optimization algorithms.
ST-DeepHAR: Deep Learning Model for Human Activity Recognition in IoHT Applications

Human activity recognition (HAR) has been regarded as an indispensable part of many smart home systems and smart healthcare applications. Specifically, HAR is of great importance in the Internet of Healthcare Things (IoHT), owing to the rapid proliferation of Internet of Things (IoT) technologies embedded in various smart appliances and wearable devices (such as smartphones and smartwatches) that have a pervasive impact on an individual's life. The inertial sensors of smartphones generate massive amounts of multidimensional time-series data, which can be exploited effectively for HAR purposes. Unlike traditional approaches, deep learning techniques are the most suitable choice for such multivariate streams. In this study, we introduce a supervised dual-channel model that comprises long short-term memory (LSTM), followed by an attention mechanism for the temporal fusion of inertial sensor data concurrent with a convolutional residual network for the spatial fusion of sensor data. We also introduce an adaptive channel-squeezing operation to fine-tune convolutional a neural network feature extraction capability by exploiting multichannel dependency. Finally, two widely available and public HAR data sets are used in experiments to evaluate the performance of our model. The results demonstrate that our proposed approach can overcome state-of-the-art methods.
Generate Neural Template Explanations for Recommendation

Personalized recommender systems are important to assist user decision-making in the era of information overload. Meanwhile, explanations of the recommendations further help users to better understand the recommended items so as to make informed choices, which gives rise to the importance of explainable recommendation research. Textual sentence-based explanation has been an important form of explanations for recommender systems due to its advantage in communicating rich information to users. However, current approaches to generating sentence explanations are either limited to predefined sentence templates, which restricts the sentence expressiveness, or opt for free-style sentence generation, which makes it difficult for sentence quality control. In an attempt to benefit both sentence expressiveness and quality, we propose a Neural Template (NETE) explanation generation framework, which brings the best of both worlds by learning sentence templates from data and generating template-controlled sentences that comment about specific features. Experimental results on real-world datasets show that NETE consistently outperforms state-of-the-art explanation generation approaches in terms of sentence quality and expressiveness. Further analysis on case study also shows the advantages of NETE on generating diverse and controllable explanations.
eThor: Practical and Provably Sound Static Analysis of Ethereum Smart Contracts

Ethereum has emerged as the most popular smart contract platform, with hundreds of thousands of contracts stored on the blockchain and covering diverse application scenarios, such as auctions, trading platforms, or elections. Given the financial nature of smart contracts, security vulnerabilities may lead to catastrophic consequences and, even worse, can hardly be fixed as data stored on the blockchain, including the smart contract code itself, are immutable. An automated security analysis of these contracts is thus of utmost interest, but at the same time technically challenging. This is as e.g., Ethereum's transaction-oriented programming mechanisms feature a subtle semantics, and since the blockchain data at execution time, including the code of callers and callees, are not statically known. In this work, we present eThor, the first sound and automated static analyzer for EVM bytecode, which is based on an abstraction of the EVM bytecode semantics based on Horn clauses. In particular, our static analysis supports reachability properties, which we show to be sufficient for capturing interesting security properties for smart contracts (e.g., single-entrancy) as well as contract-specific functional properties. Our analysis is proven sound against a complete semantics of EVM bytecode, and a large-scale experimental evaluation on real-world contracts demonstrates that eThor is practical and outperforms the state-of-the-art static analyzers: specifically, eThor is the only one to provide soundness guarantees, terminates on 94% of a representative set of real-world contracts, and achieves an F-measure (which combines sensitivity and specificity) of 89%.
Efficient Spatio-Temporal Recurrent Neural Network for Video Deblurring

Real-time video deblurring still remains a challenging task due to the complexity of spatially and temporally varying blur itself and the requirement of low computational cost. To improve the network efficiency, we adopt residual dense blocks into RNN cells, so as to efficiently extract the spatial features of the current frame. Furthermore, a global spatio-temporal attention module is proposed to fuse the effective hierarchical features from past and future frames to help better deblur the current frame. For evaluation, we also collect a novel dataset with paired blurry/sharp video clips by using a co-axis beam splitter system. Through experiments on synthetic and realistic datasets, we show that our proposed method can achieve better deblurring performance both quantitatively and qualitatively with less computational cost against state-of-the-art video deblurring methods.
Domain Adaptive Relational Reasoning for 3D Multi-organ Segmentation

In this paper, we present a novel unsupervised domain adaptation (UDA) method, named Domain Adaptive Relational Reasoning (DARR), to generalize 3D multi-organ segmentation models to medical data collected from different scanners and/or protocols (domains). Our method is inspired by the fact that the spatial relationship between internal structures in medical images is relatively fixed, e.g., a spleen is always located at the tail of a pancreas, which serves as a latent variable to transfer the knowledge shared across multiple domains. We formulate the spatial relationship by solving a jigsaw puzzle task, i.e., recovering a CT scan from its shuffled patches, and jointly train it with the organ segmentation task. To guarantee the transferability of the learned spatial relationship to multiple domains, we additionally introduce two schemes: 1) Employing a super-resolution network also jointly trained with the segmentation model to standardize medical images from different domain to a certain spatial resolution; 2) Adapting the spatial relationship for a test image by test-time jigsaw puzzle training. Experimental results show that our method improves the performance by \(29.60\%\) DSC on target datasets on average without using any data from the target domain during training.
Solar Irradiance Forecasting Based on Deep Learning Methodologies and Multi-Site Data

The ever-growing interest in and requirement for green energy have led to an increased focus on research related to forecasting solar irradiance recently. This study aims to develop forecast models based on deep learning (DL) methodologies and multiple-site data to predict the daily solar irradiance in two locations of India based on the daily solar radiation data obtained from NASA’s POWER project repository over 36 years (1983–2019). The forecast modeling of solar irradiance data is performed for extracting and learning the symmetry latent in data patterns and relationships by the machine learning models and utilizing it to predict future solar data. The goodness of fit and model performance are compared with rolling window evaluation using mean squared error, root-mean-square error and coefficient of determination (R2) for evaluation. The contributions of this study can be summarized as follows: (i) time series models based on deep learning methodologies were implemented to forecast the daily solar irradiance of two locations in India in consideration of the historical data collected by NASA; (ii) the models were developed on the basis of single-location univariate data as well as multiple-location data; (iii) the accuracy, performance and reliability of the model were investigated on the basis of standard performance evaluation metrics and rolling window evaluation; (iv) the feature importance of the nearby locations with respect to forecasting target location solar irradiance was analyzed and compared based on the solar irradiance data obtained from NASA over 36 years. The results indicate that the bidirectional long short-term memory (LSTM) and attention-based LSTM models can be used for forecasting daily solar irradiance data. According to the findings, the multiple-site data with solar irradiance historical data improve upon the forecast performance of single-location univariate solar data.
Artificial intelligence-based pathology for gastrointestinal and hepatobiliary cancers

Artificial intelligence (AI) can extract complex information from visual data. Histopathology images of gastrointestinal (GI) and liver cancer contain a very high amount of information which human observers can only partially make sense of. Complementing human observers, AI allows an in-depth analysis of digitised histological slides of GI and liver cancer and offers a wide range of clinically relevant applications. First, AI can automatically detect tumour tissue, easing the exponentially increasing workload on pathologists. In addition, and possibly exceeding pathologist's capacities, AI can capture prognostically relevant tissue features and thus predict clinical outcome across GI and liver cancer types. Finally, AI has demonstrated its capacity to infer molecular and genetic alterations of cancer tissues from histological digital slides. These are likely only the first of many AI applications that will have important clinical implications. Thus, pathologists and clinicians alike should be aware of the principles of AI-based pathology and its ability to solve clinically relevant problems, along with its limitations and biases.
3D Face Reconstruction From A Single Image Assisted by 2D Face Images in the Wild

3D face reconstruction from a single image is an important task in many multimedia applications. Recent works typically learn a CNN-based 3D face model that regresses coefficients of a 3D Morphable Model (3DMM) from 2D images to perform 3D face reconstruction. However, the shortage of training data with 3D annotations considerably limits performance of these methods. To alleviate this issue, we propose a novel 2D-Assisted Learning (2DAL) method that can effectively use “in the wild” 2D face images with noisy landmark information to substantially improve 3D face model learning. Specifically, taking the sparse 2D facial landmark heatmaps as additional information, 2DAL introduces four novel self-supervision schemes that view the 2D landmark and 3D landmark prediction as a self-mapping process, including the landmark self-prediction consistency for 2D and 3D faces respectively, cycle-consistency over the 2D landmark prediction and self-critic over the predicted 3DMM coefficients based on landmark prediction. Using these four self-supervision schemes, 2DAL significantly relieves the demands for the the conventional paired 2D-to-3D annotations and gives much higher-quality 3D face models without requiring any additional 3D annotations. Experiments on AFLW2000-3D, AFLW-LFPA and Florence benchmarks show that our method outperforms state-of-the-arts for both 3D face reconstruction and dense face alignment by a large margin.