Reviewerly/exHarmony huggingface dataset:
User Experience Research, Experience Design, and Evaluation Methods for Museum Mixed Reality Experience

Wearable Mixed Reality (MR) technology is a tool that gives people a new enhanced experience that they have not encountered before. This study shows the process of designing new museum experiences while considering how this technology changes previous museum experiences, what those experiences are, and what people should feel through these experiences. This process was systematically conducted according to the UX design process of analysis, synthesis, and evaluation. In the analysis step, six types of museum artifact viewing experiences were defined: knowing, restoring, exploring, expanded scale, encountering, and sharing experience through research and user surveys related to the museum experience. In addition, through research analysis related to MR technology, presence, flow, and natural interaction were defined as three essential factors that users should feel in the MR experience. In the synthesis stage, optimized wearable MR experiences were designed and implemented by applying the necessary experience types and essential factors according to the characteristics of each artifact. In the evaluation stage, user experience evaluations such as user experience tests for essential factors in the MR experience, User Experience Questionnaire (UEQ) tests for interaction products, and the Visual Aesthetics of Websites Inventory (VisAWI) test for visual experiences from various perspectives were conducted on the developed results. Through these evaluations, users gave positive scores to the design results based on the experience types and essential factors defined in this study. When applying new media technologies such as wearable MR technology, improved technology implementation is important, but an understanding of the applied field must first be obtained, and user analysis must first be thoroughly conducted. This study will be a guide to the systematic development process to be followed when applying wearable MR technology to other fields.
A new estimate of the n-gram language model

Modeling a natural language aims to build systems that are able to reproduce, correct and more or less predict the structure of a given language; moreover it summarizes the general knowledge related to that language. In this context, we have suggested a new language model which efficiently estimate the n-gram language model. This new model has made it possible to remedy the shortcomings of the n-gram language model. By using an automatic spelling error application, the evaluation of this model has been done, and after the comparison of the results obtained; we see that this new model give quick and precise results. These results show that this new proposed model better models the succession of words within sentences, and can therefore replace the n-gram model in Natural Language Processing (NLP) applications.
Mixed Deep Gaussian Mixture Model: a clustering model for mixed datasets

Clustering mixed data presents numerous challenges inherent to the very heterogeneous nature of the variables. A clustering algorithm should be able, despite of this heterogeneity, to extract discriminant pieces of information from the variables in order to design groups. In this work we introduce a multilayer architecture model-based clustering method called Mixed Deep Gaussian Mixture Model (MDGMM) that can be viewed as an automatic way to merge the clustering performed separately on continuous and non-continuous data. This architecture is flexible and can be adapted to mixed as well as to continuous or non-continuous data. In this sense we generalize Generalized Linear Latent Variable Models and Deep Gaussian Mixture Models. We also design a new initialisation strategy and a data driven method that selects the best specification of the model and the optimal number of clusters for a given dataset "on the fly". Besides, our model provides continuous low-dimensional representations of the data which can be a useful tool to visualize mixed datasets. Finally, we validate the performance of our approach comparing its results with state-of-the-art mixed data clustering models over several commonly used datasets.
FASTBLOCK: Accelerating Blockchains via Hardware Transactional Memory

The efficiency of block lifecycle determines the performance of blockchain, which is critically affected by the execution, mining and validation steps in blockchain lifecycle. To accelerate blockchains, many works focus on optimizing the mining step while ignoring other steps. In this paper, we propose a novel blockchain framework-FastBlock to speed up the execution and validation steps by introducing efficient concurrency. To efficiently prevent the potential concurrency violations, FastBlock utilizes symbolic execution to identify minimal atomic sections in each transaction and guarantees the atomicity of these sections in execution step via an efficient concurrency control mechanism-hardware transactional memory (HTM). To enable a deterministic validation step, FastBlock concurrently re-executes transactions based on a happen-before graph without increasing block size. Finally, we implement FastBlock and evaluate it in terms of conflicting transactions rate, number of transactions per block, and varying thread number. Our results indicate that FastBlock is efficient: the execution step and validation step speed up to 3.0x and 2.3x on average over the original serial model respectively with eight concurrent threads.
Which emphasis technique to use? Perception of emphasis techniques with varying distractors, backgrounds, and visualization types

Emphasis effects are visual changes that make data elements distinct from their surroundings. Designers may use computational saliency models to predict how a viewer's attention will be guided by a specific effect; however, although saliency models provide a foundational understanding of emphasis perception, they only cover specific visual effects in abstract conditions. To address these limitations, we carried out crowdsourced studies that evaluate emphasis perception in a wider range of conditions than previously studied. We varied effect magnitude, distractor number and type, background, and visualization type, and measured the perceived emphasis of 12 visual effects. Our results show that there are perceptual commonalities of emphasis across a wide range of environments, but also that there are limitations on perceptibility for some effects, dependent on a visualization's background or type. We developed a model of emphasis predictability based on simple scatterplots that can be extended to other viewing conditions. Our studies provide designers with new understanding of how viewers experience emphasis in realistic visualization settings.
Quantized Adam with Error Feedback

In this article, we present a distributed variant of an adaptive stochastic gradient method for training deep neural networks in the parameter-server model. To reduce the communication cost among the workers and server, we incorporate two types of quantization schemes, i.e., gradient quantization and weight quantization, into the proposed distributed Adam. In addition, to reduce the bias introduced by quantization operations, we propose an error-feedback technique to compensate for the quantized gradient. Theoretically, in the stochastic nonconvex setting, we show that the distributed adaptive gradient method with gradient quantization and error feedback converges to the first-order stationary point, and that the distributed adaptive gradient method with weight quantization and error feedback converges to the point related to the quantized level under both the single-worker and multi-worker modes. Last, we apply the proposed distributed adaptive gradient methods to train deep neural networks. Experimental results demonstrate the efficacy of our methods.
IoT Nudge: IoT Data-driven Nudging for Health Behavior Change

This paper presents an Internet of Things (IoT)-based concept called “IoT Nudge”, which aims to induce continuous healthful behavior changes through interactions between human users and IoT devices, including smart sensors and actuators. More specifically, IoT Nudge systems aim to establish self-reflection loops in which our daily behaviors are reflected in the IoT devices around us. The changes in the presentations of those IoT devices promote self-awareness and provide us with opportunities to improve our lifestyle habits. To reconfigure poor behavior into good behavior, the IoT Nudge provides three functions: (1) automatic tracking of a user’s lifestyle habits using IoT sensors, (2) playfully reframing the user’s daily activities into other more healthful representations based on analogies, and (3) delivering real-time ambient feedback through reframed representations from IoT actuators that encourage self-reflection. Herein, we describe our conceptual model of IoT Nudge and introduce three prototype systems aimed at promoting health improvements: fit2plant, eat2pic, and brush2music.
Structured Outdoor Architecture Reconstruction by Exploration and Classification

This paper presents an explore-and-classify framework for structured architectural reconstruction from an aerial image. Starting from a potentially imperfect building reconstruction by an existing algorithm, our approach 1) explores the space of building models by modifying the reconstruction via heuristic actions; 2) learns to classify the correctness of building models while generating classification labels based on the ground-truth; and 3) repeat. At test time, we iterate exploration and classification, seeking for a result with the best classification score. We evaluate the approach using initial reconstructions by two baselines and two state-of-the-art reconstruction algorithms. Qualitative and quantitative evaluations demonstrate that our approach consistently improves the reconstruction quality from every initial reconstruction.
TumorCP: A Simple but Effective Object-Level Data Augmentation for Tumor Segmentation

Deep learning models are notoriously data-hungry. Thus, there is an urging need for data-efficient techniques in medical image analysis, where well-annotated data are costly and time consuming to collect. Motivated by the recently revived “Copy-Paste” augmentation, we propose TumorCP, a simple but effective object-level data augmentation method tailored for tumor segmentation. TumorCP is online and stochastic, providing unlimited augmentation possibilities for tumors’ subjects, locations, appearances, as well as morphologies. Experiments on kidney tumor segmentation task demonstrate that TumorCP surpasses the strong baseline by a remarkable margin of 7.12% on tumor Dice. Moreover, together with image-level data augmentation, it beats the current state-of-the-art by 2.32% on tumor Dice. Comprehensive ablation studies are performed to validate the effectiveness of TumorCP. Meanwhile, we show that TumorCP can lead to striking improvements in extremely low-data regimes. Evaluated with only 10% labeled data, TumorCP significantly boosts tumor Dice by 21.87%. To the best of our knowledge, this is the very first work exploring and extending the “Copy-Paste” design in medical imaging domain. Code is available at: https://github.com/YaoZhang93/TumorCP.
Knowledge Distillation Classifier Generation Network for Zero-Shot Learning

In this article, we present a conceptually simple but effective framework called knowledge distillation classifier generation network (KDCGN) for zero-shot learning (ZSL), where the learning agent requires recognizing unseen classes that have no visual data for training. Different from the existing generative approaches that synthesize visual features for unseen classifiers' learning, the proposed framework directly generates classifiers for unseen classes conditioned on the corresponding class-level semantics. To ensure the generated classifiers to be discriminative to the visual features, we borrow the knowledge distillation idea to both supervise the classifier generation and distill the knowledge with, respectively, the visual classifiers and soft targets trained from a traditional classification network. Under this framework, we develop two, respectively, strategies, i.e., class augmentation and semantics guidance, to facilitate the supervision process from the perspectives of improving visual classifiers. Specifically, the class augmentation strategy incorporates some additional categories to train the visual classifiers, which regularizes the visual classifier weights to be compact, under supervision of which the generated classifiers will be more discriminative. The semantics-guidance strategy encodes the class semantics into the visual classifiers, which would facilitate the supervision process by minimizing the differences between the generated and the real-visual classifiers. To evaluate the effectiveness of the proposed framework, we have conducted extensive experiments on five datasets in image classification, i.e., AwA1, AwA2, CUB, FLO, and APY. Experimental results show that the proposed approach performs best in the traditional ZSL task and achieves a significant performance improvement on four out of the five datasets in the generalized ZSL task.
Multi-Agent Path Finding with heterogeneous edges and roundtrips

Multi-Agent Path Finding (MAPF) aims to find a set of conflict-free paths for multiple agents on a given graph from parking locations to specified goal locations while optimizing related costs. Currently, existing MAPF studies often consider the simplified problem setup where each agent does not return to its parking location after completing its task on the underlying graph with uniform edge costs. Nevertheless, within some real-word scenarios such as the Unmanned Aircraft System (UAS), agents are situated in the underlying graph with non-uniform edge costs. These agents are required to travel from their respective parking locations to complete tasks, and then return without conflicts. Therefore, this paper explores a new version of MAPF, formally called Multi-Agent Path Finding with Heterogeneous edges and Roundtrips (MAPF-HR). In this version, all agents are engaged in completing tasks by navigating their respective conflict-free paths with roundtrips on the graph with heterogeneous edges. This paper investigates a novel algorithm for this problem, called Improved Conflict-Based Search (CBS) with Helpful Bypass (ICBS-HB), which improves the CBS framework by utilizing the scheme of bypassing conflicts during path finding execution. With extensive experiments on MAPF benchmark maps, it shows that ICBS-HB outperforms the state-of-the-art algorithms for dealing with MAPF-HR.
Cross-Modality Transfer Learning for Image-Text Information Management

In the past decades, information from all kinds of data has been on a rapid increase. With state-of-the-art performance, machine learning algorithms have been beneficial for information management. However, insufficient supervised training data is still an adversity in many real-world applications. Therefore, transfer learning (TF) was proposed to address this issue. This article studies a not well investigated but important TL problem termed cross-modality transfer learning (CMTL). This topic is closely related to distant domain transfer learning (DDTL) and negative transfer. In general, conventional TL disciplines assume that the source domain and the target domain are in the same modality. DDTL aims to make efficient transfers even when the domains or the tasks are entirely different. As an extension of DDTL, CMTL aims to make efficient transfers between two different data modalities, such as from image to text. As the main focus of this study, we aim to improve the performance of image classification by transferring knowledge from text data. Previously, a few CMTL algorithms were proposed to deal with image classification problems. However, most existing algorithms are very task specific, and they are unstable on convergence. There are four main contributions in this study. First, we propose a novel heterogeneous CMTL algorithm, which requires only a tiny set of unlabeled target data and labeled source data with associate text tags. Second, we introduce a latent semantic information extraction method to connect the information learned from the image data and the text data. Third, the proposed method can effectively handle the information transfer across different modalities (text-image). Fourth, we examined our algorithm on a public dataset, Office-31. It has achieved up to 5% higher classification accuracy than “non-transfer” algorithms and up to 9% higher than existing CMTL algorithms.
Mixup Without Hesitation

Mixup linearly interpolates pairs of examples to form new samples, which has been shown to be effective in image classification tasks. However, there are two drawbacks in mixup: one is that more training epochs are needed to obtain a well-trained model; the other is that mixup requires tuning a hyper-parameter to gain appropriate capacity. In this paper, we find that mixup constantly explores the representation space, and inspired by the exploration-exploitation dilemma, we propose mixup Without hesitation (mWh), a concise and effective training algorithm. We show that mWh strikes a good balance between exploration and exploitation by gradually replacing mixup with basic data augmentation. It can achieve a strong baseline with less training time than original mixup and without searching for optimal hyper-parameter, i.e., mWh acts as mixup without hesitation.
Compute with Light: Architectures, Technologies and Training Models for Neuromorphic Photonic Circuits

We discuss recent advances in the field of neuromorphic photonics, presenting our recent work and perspective towards optimizing the architecture, the enabling technology and the Deep Learning training models through a hardware/software co-design and co-development framework.
How Valuable Is Your Data? Optimizing Client Recruitment in Federated Learning

Federated learning allows distributed clients to train a shared machine learning model while preserving user privacy. In this framework, an operator recruits user devices (i.e., clients) to occasionally perform local iterations of the learning algorithm on their data. We propose the first work to theoretically analyze the resulting performance tradeoffs in deciding which clients to recruit for federated learning, complementing other works on the selection of recruited clients in each iteration. Specifically, we define and optimize the tradeoffs between both accuracy (training and testing) and efficiency (completion time and cost) metrics. We provide efficient solutions to this NP-Hard optimization problem, and verify the value of client recruitment in experiments on synthetic and real-world data. The results of this work can serve as guidelines for the real-world deployment of federated learning and an initial investigation of the client recruitment problem.
An Ensemble Methods for Medical Insurance Costs Prediction Task

The paper reports three new ensembles of supervised learning predictors for managing medical insurance costs. The open dataset is used for data analysis methods development. The usage of artificial intelligence in the management of financial risks will facilitate economic wear time and money and protect patients’ health. Machine learning is associated with many expectations, but its quality is determined by choosing a good algorithm and the proper steps to plan, develop, and implement the model. The paper aims to develop three new ensembles for individual insurance costs prediction to provide high prediction accuracy. Pierson coefficient and Boruta algorithm are used for feature selection. The boosting, stacking, and bagging ensembles are built. A comparison with existing machine learning algorithms is given. Boosting modes based on regression tree and stochastic gradient descent is built. Bagged CART and Random Forest algorithms are proposed. The boosting and stacking ensembles shown better accuracy than bagging. The tuning parameters for boosting do not allow to decrease the RMSE too. So, bagging shows its weakness in generalizing the prediction. The stacking is developed using K Nearest Neighbors (KNN), Support Vector Machine (SVM), Regression Tree, Linear Regression, Stochastic Gradient Boosting. The random forest (RF) algorithm is used to combine the predictions. One hundred trees are built for RF. Root Mean Square Error (RMSE) has lifted the to 3173.213 in comparison with other predictors. The quality of the developed ensemble for Root Mean Squared Error metric is 1.47 better than for the best weak predictor (SVR).
Fair Peer-to-Peer Content Delivery via Blockchain

In comparison with conventional content delivery networks, peer-to-peer (p2p) content delivery is promising to save cost and handle high peak-demand, and can also complement the decentralized storage networks such as Filecoin. However, reliable p2p delivery requires proper enforcement of delivery fairness, i.e., the deliverers should be rewarded according to their in-time delivery. Unfortunately, most existing studies on delivery fairness are based on non-cooperative game-theoretic assumptions that are arguably unrealistic in the ad-hoc p2p setting.We for the first time put forth an expressive yet still minimalist security notion for desired fair p2p content delivery, and give two efficient solutions \(\mathsf {FairDownload}\) and \(\mathsf {FairStream}\) via the blockchain for p2p downloading and p2p streaming scenarios, respectively. Our designs not only guarantee delivery fairness to ensure deliverers be paid (nearly) proportional to their in-time delivery but also ensure the content consumers and content providers are fairly treated. The fairness of each party can be guaranteed when the other two parties collude to arbitrarily misbehave. Moreover, the systems are efficient in the sense of attaining nearly asymptotically optimal on-chain costs and deliverer communication.We implement the protocols and build the prototype systems atop the Ethereum Ropsten network. Extensive experiments done in LAN and WAN settings showcase their high practicality.
DDLVis: Real-time Visual Query of Spatiotemporal Data Distribution via Density Dictionary Learning

Visual query of spatiotemporal data is becoming an increasingly important function in visual analytics applications. Various works have been presented for querying large spatiotemporal data in real time. However, the real-time query of spatiotemporal data distribution is still an open challenge. As spatiotemporal data become larger, methods of aggregation, storage and querying become critical. We propose a new visual query system that creates a low-memory storage component and provides real-time visual interactions of spatiotemporal data. We first present a peak-based kernel density estimation method to produce the data distribution for the spatiotemporal data. Then a novel density dictionary learning approach is proposed to compress temporal density maps and accelerate the query calculation. Moreover, various intuitive query interactions are presented to interactively gain patterns. The experimental results obtained on three datasets demonstrate that the presented system offers an effective query for visual analytics of spatiotemporal data.