Reviewerly/exHarmony huggingface dataset:
Real-time Virtual Object Insertion for Moving 360° Videos

We propose an approach for real-time insertion of virtual objects into pre-recorded moving-camera 360° video. First, we reconstruct camera motion and sparse scene content via structure from motion on stitched equirectangular video. Then, to plausibly reproduce real-world lighting conditions for virtual objects, we use inverse tone mapping to recover high dynamic range environment maps which vary spatially along the camera path. We implement our approach into the Unity rendering engine for real-time virtual object insertion via differential rendering, with dynamic lighting, image-based shadowing, and user interaction. This expands the use and flexibility of 360° video for interactive computer graphics and visual effects applications.
Multimodal joint learning for personal knowledge base construction from Twitter-based lifelogs

People are used to log their life on social media platforms. In this paper, we aim to extract life events by leveraging both visual and textual information shared on Twitter and construct personal knowledge bases of individuals. The issues to be tackled include (1) not all text descriptions are related to life events, (2) life events in a text description can be expressed explicitly or implicitly, (3) the predicates in the implicit life events are often absent, and (4) the mapping from natural language predicates to knowledge base relations may be ambiguous. A multimodal joint learning approach trained on both text and images from social media posts shared on Twitter is proposed to detect life events in tweets and extract event components including subjects, predicates, objects, and time expressions. Finally, the extracted information is transformed to knowledge base facts. The evaluation is performed on a collection of lifelogs from 18 Twitter users. Experimental results show our proposed system is effective in life event extraction, and the constructed personal knowledge bases are expected to be useful to memory recall applications.
A Case for Performance-Aware Deployment of Containers

Cloud-native applications are increasingly adopting microservices architectures that support the development agility required by modern software. These applications deploy their components in containers that enable microservices to be deployed across different platforms, supporting the independent scaling of the different components. However, the operational complexity of microservices presents significant challenges in maintaining the performance of such applications, especially in clouds with performance variability and unpredictability. While virtual machine based deployment of applications has been well studied---with sophisticated orchestrators in the literature and practice---there has been little such studies on the opportunity in improving application performance using performance-aware deployment strategies for containers. In this paper, we consider both the runtime and initialization time performance of containerized applications and show that default placement strategies provided by orchestrators are often insufficient. Our experiments on multiple services show that a performance-aware approach is able to outperform the default placement strategy by up to factor of 2x and 2.21x for the 50th and 99th percentiles.
Harvesting railway passenger opinions on multi themes by using social graph clustering

The largest service provider in public transportation like railways has a lot of challenges with changing dynamics. Identification of heterogeneity in perceptions of service quality among groups of railway passengers to make indicators has a lot of importance in service oriented organizations like railways. Railway passengers' opinions have also been given significant importance for monitoring, enhancing the existing services, and understanding the current needs of the people. The existing practice involves people for collecting the data, along with responding to the complaints and forwarding these to the concerned heads. Now the research is centered on dynamic passenger behaviour analysis by using social data, such as twitter, the facebook etc. In this paper, the importance of dynamic social sentiment analysis and affective computing in railways, issues associated with data collection and processing are elaborated. The proposed work offers a framework for social sentiment analysis on railway passenger tweets. Firstly, it identifies the theme wise data and then provides a sentiment score which expresses about multiple themes. A novel social graph clustering approach is utilized to separate the data according to theme and perform the sentiment analysis on each cluster to anticipate the passenger opinions. This work will give the indicators and assessment of railway services quality with an understanding of the passengers' opinions. It acts as an expert feedback, service indicator, and is also helpful to enhance the services through a thorough analysis of Indian railways passengers' opinions.
MedTruth

Knowledge Graph (KG) contains entities and the relations between entities. Due to its representation ability, KG has been successfully applied to support many medical/healthcare tasks. However, in the medical domain, knowledge holds under certain conditions. Such conditions for medical knowledge are crucial for decision-making in various medical applications, which is missing in existing medical KGs. In this paper, we aim to discovery medical knowledge conditions from texts to enrich KGs. Electronic Medical Records (EMRs) are systematized collection of clinical data and contain detailed information about patients, thus EMRs can be a good resource to discover medical knowledge conditions. Unfortunately, the amount of available EMRs is limited due to reasons such as regularization. Meanwhile, a large amount of medical question answering (QA) data is available, which can greatly help the studied task. However, the quality of medical QA data is quite diverse, which may degrade the quality of the discovered medical knowledge conditions. In the light of these challenges, we propose a new truth discovery method, MedTruth, for medical knowledge condition discovery, which incorporates prior source quality information into the source reliability estimation procedure, and also utilizes the knowledge triple information for trustworthy information computation. We conduct series of experiments on real-world medical datasets to demonstrate that the proposed method can discover meaningful and accurate conditions for medical knowledge by leveraging both EMR and QA data. Further, the proposed method is tested on synthetic datasets to validate its effectiveness under various scenarios.
Obtaining Hyperspectral Signatures for Seafloor Massive Sulphide Exploration

Seafloor massive sulphide (SMS) deposits are hosts to a wide range of economic minerals, and may become an important resource in the future. The exploitation of these resources is associated with considerable expenses, and a return on investment may depend on the availability of multiple deposits. Therefore, efficient exploration methodologies for base metal deposits are important for future deep sea mining endeavours. Underwater hyperspectral imaging (UHI) has been demonstrated to be able to differentiate between different types of materials on the seafloor. The identification of possible end-members from field data requires prior information in the form of representative signatures for distinct materials. This work presents hyperspectral imaging applied to a selection of materials from the Loki’s Castle active hydrothermal vent site in a laboratory setting. A methodology for compensating for systematic effects and producing the reflectance spectra is detailed, and applied to recover the spectral signatures from the samples. The materials investigated were found to be distinguishable using unsupervised dimensionality reduction methods, and may be used as a reference for future field application.
Fourth-Wave HCI Meets the 21st Century Manifesto

We take up Bødker's [9] challenge to ‘identify’ a fourth wave HCI, building on the work of Blevis et al. [8] and others to shore up a new vision that places ‘politics and values and ethics’ at the forefront without abandoning the strengths of previous waves. We insist that a fourth wave must push harder, beyond measured criticism for actual (e.g. institutional) change. We present two studies performed at CHI’19, where we used our MANIFESTO! game to: 1) take the temperature of colleagues on adopting an activist stance, 2) test manifesto writing as a key activity in pushing HCI forward into the fourth wave, and 3) test our game for subsequent iterations, and as a probe for inspiring new digital tools. With the enthusiastic response received to gameplay, facilitated in part through a novel method using tableau vivant, we argue for taking political activism from the margins into mainstream HCI.
A Weighted Late Fusion Framework for Recognizing Human Activity from Wearable Sensors

Following the technological advancement and the constantly emerging assisted living applications, sensor-based activity recognition research receives great attention. Until recently, the majority of relevant research involved extracting knowledge out of single modalities, however, when individual sensors performances are not satisfactory, combining information from multiple sensors can be of use and improve the activity recognition rate. Early and late fusion classifier strategies are usually employed to successfully merge multiple sensors. This paper proposes a novel framework for combining accelerometers and gyroscopes at decision level, in order to recognize human activity. More specifically, we propose a weighted late fusion framework that utilizes the detection rate of a classifier. Furthermore, we propose the modification of an already existing class-based weighted late fusion framework. Experimental results on a publicly available and widely used dataset demonstrated that the combination of accelerometer and gyroscope under the proposed frameworks improves the classification performance.
EyeDescribe: Combining Eye Gaze and Speech to Automatically Create Accessible Touch Screen Artwork

Many images on the Web, including photographs and artistic images, feature spatial relationships between objects that are inaccessible to someone who is blind or visually impaired even when a text description is provided. While some tools exist to manually create accessible image descriptions, this work is time consuming and requires specialized tools. We introduce an approach that automatically creates spatially registered image labels based on how a sighted person naturally interacts with the image. Our system collects behavioral data from sighted viewers of an image, specifically eye gaze data and spoken descriptions, and uses them to generate a spatially indexed accessible image that can then be explored using an audio-based touch screen application. We describe our approach to assigning text labels to locations in an image based on eye gaze. We then report on two formative studies with blind users testing EyeDescribe. Our approach resulted in correct labels for all objects in our image set. Participants were able to better recall the location of objects when given both object labels and spatial locations. This approach provides a new method for creating accessible images with minimum required effort.
Target-less approach of vibration measurement with virtual points constructed with cross ratios

For structures with distinguished edge features, a target-less approach of vibration measurement method based on stereo vision system is proposed in this paper. The edge is precisely identified through skeleton thinning, and then the virtual points are constructed and tracked. A sub-pixel algorithm to minimize continuous curvature change is used, which is based on the interpolation in the normal direction of the line edge skeleton. Virtual points are constructed according to different cross ratios. Based on the invariance of cross ratio in affine transformation, the image coordinates of the same virtual point can be found accurately. The edge displacement is measured through matching and tracking of these virtual homologous points. A 3 KW wind turbine blade is experimented. The measured frequencies are compared with those from target approach and hammer impact testing. The reliability and efficiency of the proposed target-less approach is verified by the experimental results.
Content-guided deep residual network for single image super-resolution

We propose a content-guided deep residual network for single image super-resolution. Firstly, the relationship between the sparsity of images and the difficulty of reconstruction through a convolution network is studied. On this basis, a guided residual block is built, which can compensate for part of the high-frequency information and improve the convergence of the network. On the other hand, local loss and curvature loss values are integrated into the loss function, enhancing the network's ability to perceive the advanced texture information of the image. Finally, the optimized loss function is used to the network composed of several guided residual blocks. Experimental results demonstrate that the network achieves superior results over some existing deep residual networks in PSNR and SSIM evaluation.
Collective Representation Learning on Spatiotemporal Heterogeneous Information Networks

Representation learning is a technique that is used to capture the underlying latent features of complex data. Representation learning on networks has been widely implemented for learning network structure and embedding it in a low dimensional vector space. In recent years, network embedding using representation learning has attracted increasing attention, and many deep architectures have been widely proposed. However, existing network embedding techniques ignore the multi-class spatial and temporal relationships that crucially reflect the complex nature among vertices and links in spatiotemporal heterogeneous information networks(SHINs).
Answer Acquisition for Knowledge Base Question Answering Systems Based on Dynamic Memory Network

In recent years, with the rapid growth of Artificial Intelligence (AI) and the Internet of Things (IoT), the question answering systems for human-machine interaction based on deep learning have become a research hotspot of the IoT. Different from the structured query method in traditional Knowledge Base Question Answering (KBQA) systems based on templates or rules, representation learning is one of the most promising approaches to solving the problems of data sparsity and semantic gaps. In this paper, an answer acquisition method for KBQA systems based on a dynamic memory network is proposed, in which representation learning is employed to represent the natural language questions that are raised by users and the knowledge base subgraphs of the related entities. These representations are taken as inputs of the dynamic memory network. The correct answers are obtained by utilizing the memory and inferential capabilities. The experimental results demonstrate the effectiveness of the proposed approach.
Graph Spectral Domain Feature Learning With Application to in-Air Hand-Drawn Number and Shape Recognition

This paper addresses the problem of recognition of dynamic shapes by representing the structure in a shape as a graph and learning the graph spectral domain features. Our proposed method includes pre-processing for converting the dynamic shapes into a fully connected graph, followed by analysis of the eigenvectors of the normalized Laplacian of the graph adjacency matrix for forming the feature vectors. The method proposes to use the eigenvector corresponding to the lowest eigenvalue for formulating the feature vectors as it captures the details of the structure of the graph. The use of the proposed graph spectral domain representation has been demonstrated in an in-air hand-drawn number and symbol recognition applications. It has achieved average accuracy rates of 99.56% and 99.44%, for numbers and symbols, respectively, outperforming the existing methods for all datasets used. It also has the added benefits of fast real-time operation and invariance to rotation and flipping, making the recognition system robust to different writing and drawing variations.
Improving Human Action Recognition Using Hierarchical Features And Multiple Classifier Ensembles

Abstract This paper presents a simple, fast and efficacious system to promote the human action classification outcome using the depth action sequences. Firstly, the motion history images (MHIs) and static history images (SHIs) are created from the front (XOY), side (YOZ) and top (XOZ) projected scenes of each depth sequence in a 3D Euclidean space through engaging the 3D Motion Trail Model (3DMTM). Then, the Local Binary Patterns (LBPs) algorithm is operated on the MHIs and SHIs to learn motion and static hierarchical features to represent the action sequence. The motion and static hierarchical feature vectors are then fed into a classifier ensemble to classify action classes, where the ensemble comprises of two classifiers. Thus, each ensemble includes a pair of Kernel-based Extreme Learning Machine (KELM) or ${\mathrm{l}}_{\mathrm{2}}$-regularized Collaborative Representation Classifier (${\mathrm{l}}_{\mathrm{2}}$-CRC) or Multi-class Support Vector Machine. To extensively assess the framework, we perform experiments on a couple of standard available datasets such as MSR-Action3D, UTD-MHAD and DHA. Experimental consequences demonstrate that the proposed approach gains a state-of-the-art recognition performance in comparison with other available approaches. Several statistical measurements on recognition results also indicate that the method achieves superiority when the hierarchical features are adopted with the KELM ensemble. In addition, to ensure real-time processing capability of the algorithm, the running time of major components is investigated. Based on machine dependency of the running time, the computational complexity of the system is also shown and compared with other methods. Experimental results and evaluation of the computational time and complexity reflect real-time compatibility and feasibility of the proposed system.
HP-Mapper

Docker containers are widely deployed to provide lightweight virtualization, and they have many desirable features such as ease of deployment and near bare-metal performance. However, both the performance and cache efficiency of containers are still limited by their storage drivers due to the coarse-grained copy-on-write operations, and the large amount of redundancy in both I/O requests and page cache. To improve I/O performance and cache efficiency of containers, we develop HP-Mapper, a high performance storage driver for Docker containers. HP-Mapper provides a two-level mapping strategy to support fine-grained copy-on-write with low overhead, and an efficient interception method to reduce redundant I/Os. Furthermore, it uses a novel cache management mechanism to reduce duplicate cached data. Experiment results with our prototype system show that HP-Mapper significantly reduces copy-on-write latency due to its finer-grained copy-on-write scheme. Moreover, HP-Mapper can also reduce 65.4% cache usage on average due to elimination of duplicated data. As a result, HP-Mapper improves the throughput of real-world workloads by up to 39.4%, and improves the startup speed of containers by 2.0x.
Can Virtual Observers Affect Our Behavior? Social Facilitation in Virtual Environments: A Mini-Review

The social facilitation effect describes the change in the performance of the task under the influence of the presence of observers. The effect itself consists of two components: social facilitation in simple tasks and social inhibition in complex tasks. In the context of the dynamic development of new technologies, the question of the possible influence on human behavior by virtual characters gains importance. We attempted to critically describe and summarize current research on social facilitation in order to answer the question of whether it occurs in virtual environments. We found 13 relevant studies, 3 of which demonstrated social facilitation, 4 social inhibition and 1 demonstrated the whole effect. The conclusions drawn from the analysis are ambiguous. Firstly, we identified that 12 out of 13 analyzed studies failed to show the whole effect. Secondly, we encountered several shortcomings of the summarized research that further complicated its interpretation. The shortcomings: presence of the researcher, unclear usage of “agent” and “avatar”, evaluation of activation, no pilot tests of observers and no description of how their characteristics are generated, among others, are discussed. Furthermore, we investigated the effect sizes and their variability. The average effect size for social facilitation was g = 0.18, CI [-0.28; 0.64] and for social inhibition g = -0.18, CI [-0.40; 0.04]. In social facilitation, a substantial level of heterogeneity was detected. Finally, we conclude that it is still too early to provide a definite answer to the question of whether social facilitation exists in Virtual Environments. We recommend limiting evaluation activation to the lowest possible level, conducting pilot tests prior to the experiment, avoiding the presence of the researcher in the experimental room and a clear distinction of “agent” and “avatar”, as measures to achieve a better quality in future research.
Project Activities of University Students by Means of Digital Technologies

The goal is to analyze possibilities of the project method using digital technologies in the process of teaching students. Relevance of the research lies in the wide application of the project method in the process of teaching university students, on the one hand, on the other hand, the potential of using this method in combination with digital technology is not fully disclosed.Methods—general scientific methods were used in the course of research: analysis, including, domestic and foreign literature, synthesis, analogy; special: pedagogical experiment, method of studying products of students’ activity, etc.The main results and novelty of the research—the analysis of digital technologies in the project activity of university students was carried out. It was revealed that digital technologies take on several functions: tool for project creation and presentation, information environment of project operation, information space of project presentation and storage.Practical significance of research—demonstrated the use of various digital tools in the development of business plans, organization of network project activities using different Internet tools. The impact of the project method using digital technologies as a means of implementing the activity method on the student in the process, interaction with social and virtual environments. Examples of the impact of project activities on the formation of personal and professional qualities of future graduates are presented.
Domain Differential Adaptation for Neural Machine Translation

Neural networks are known to be data hungry and domain sensitive, but it is nearly impossible to obtain large quantities of labeled data for every domain we are interested in.This necessitates the use of domain adaptation strategies.One common strategy encourages generalization by aligning the global distribution statistics between source and target domains, but one drawback is that the statistics of different domains or tasks are inherently divergent, and smoothing over these differences can lead to sub-optimal performance.In this paper, we propose the framework of Domain Differential Adaptation (DDA), where instead of smoothing over these differences we embrace them, directly modeling the difference between domains using models in a related task.We then use these learned domain differentials to adapt models for the target task accordingly.Experimental results on domain adaptation for neural machine translation demonstrate the effectiveness of this strategy, achieving consistent improvements over other alternative adaptation strategies in multiple experimental settings. 1