Reviewerly/exHarmony huggingface dataset:
Interactive and Multimodal-based Augmented Reality for Remote Assistance using a Digital Surgical Microscope

We present an interactive and multimodal-based augmented reality system for computer-assisted surgery in the context of ear, nose and throat (ENT) treatment. The proposed processing pipeline uses fully digital stereoscopic imaging devices, which support multispectral and white light imaging to generate high resolution image data, and consists of five modules. Input/output data handling, a hybrid multimodal image analysis and a bi-directional interactive augmented reality (AR) and mixed reality (MR) interface for local and remote surgical assistance are of high relevance for the complete framework. The hybrid multimodal 3D scene analysis module uses different wavelengths to classify tissue structures and combines this spectral data with metric 3D information. Additionally, we propose a zoom-independent intraoperative tool for virtual ossicular prosthesis insertion (e.g. stapedectomy) guaranteeing very high metric accuracy in sub-millimeter range (1/10 mm). A bi-directional interactive AR/MR communication module guarantees low latency, while consisting surgical information and avoiding informational overload. Display agnostic AR/MR visualization can show our analyzed data synchronized inside the digital binocular, the 3D display or any connected head-mounted-display (HMD). In addition, the analyzed data can be enriched with annotations by involving external clinical experts using AR/MR and furthermore an accurate registration of preoperative data. The benefits of such a collaborative surgical system are manifold and will lead to a highly improved patient outcome through an easier tissue classification and reduced surgery risk.
Model-Based Fail-Safe Module for Autonomous Multirotor UAVs with Parachute Systems

This paper presents development of model-based fail-safe modules for autonomous multirotor Unmanned Aerial Vehicles (UAVs) with safety parachute systems. The module is based on the adaptive eXogenous Kalman filter for actuator fault diagnosis. We assume all states can be measured, such that the primary goal of the filter is not the state estimation from the measurements, but the accurate reconstruction of the multirotor dynamics in real-time. Numerical simulations show the proposed diagnostic filter can be used to estimate the magnitude of the actuator faults accurately. Furthermore, based on simulated and real data recorded during a hexacopter UAV flight when its actuators experiencing complete failure, the experiment results demonstrate the effectiveness of the approach.
DSNet: Joint Learning for Scene Segmentation and Disparity Estimation

Recently, research works have attempted the joint prediction of scene semantics and optical flow estimation, which demonstrate the mutual improvement between both tasks. Besides, the depth information is also indispensable for the scene understanding, and disparity estimation is necessary for outputting dense depth maps. Such task shares a great similarity with the optical flow estimation since they can all be cast into a problem of capturing the difference at a location of two image frames. However, as far as we know, currently there are few networks for the joint learning of semantic and disparity. Moreover, since deep semantic information and disparity feature maps can learn from each other, we find it unnecessary with two independent encoding modules to separately extract semantic and disparity features. Therefore, we propose a unified multi-tasking architecture DSNet, for the simultaneous estimation of semantic and disparity information. In our model, semantic features, extracted by the encoding module ResNet from the left and right images, are used to obtain the deep disparity features via a novel matching module which performs pixel-to-pixel matching. In addition, we also use the disparity map to perform warp operation on deep features of the right image to deal with the problem of lacking of semantic labels. The effectiveness of our method is demonstrated by extensive experiments.
EventAction

People use recommender systems to improve their decisions; for example, item recommender systems help them find films to watch or books to buy. Despite the ubiquity of item recommender systems, they can be improved by giving users greater transparency and control. This article develops and assesses interactive strategies for transparency and control, as applied to event sequence recommender systems, which provide guidance in critical life choices such as medical treatments, careers decisions, and educational course selections. This article’s main contribution is the use of both record attributes and temporal event information as features to identify similar records and provide appropriate recommendations. While traditional item recommendations are based on choices by people with similar attributes, such as those who looked at this product or watched this movie, our event sequence recommendation approach allows users to select records that share similar attribute values and start with a similar event sequence. Then users see how different choices of actions and the orders and times between them might lead to users’ desired outcomes. This paper applies a visual analytics approach to present and explain recommendations of event sequences. It presents a workflow for event sequence recommendation that is implemented in EventAction and reports on three case studies in two domains to illustrate the use of generating event sequence recommendations based on personal histories. It also offers design guidelines for the construction of user interfaces for event sequence recommendation and discusses ethical issues in dealing with personal histories. A demo video of EventAction is available at https://hcil.umd.edu/eventaction.
Zero-Shot Multilingual Sentiment Analysis using Hierarchical Attentive Network and BERT

Sentiment analysis is considered an important downstream task in language modelling. We propose Hierarchical Attentive Network using BERT for document sentiment classification. We further showed that importing representation from Multiplicative LSTM model in our architecture results in faster convergence. We then propose a method to build a sentiment classifier for a language in which we have no labelled sentiment data. We exploit the possible semantic invariance across languages in the context of sentiment to achieve this.
Using 3D modelling and game engine technologies for interactive exploration of cultural heritage: An evaluation of four game engines in relation to roman archaeological heritage

Developments in information technology have challenged the traditional model of museums, libraries and similar venues acting as relatively passive ‘learning spaces’ for the public to access ‘knowledge’ as an exchange between tutor and learner, or in this context curator and visitor enabling them to offer more immersive and interactive modes of transfer. This article examines the development of a 3D model built from plans of a Roman edifice and its transfer into four game engines as vehicles for independent navigation around the ‘virtual building’. The game engines are evaluated in respect of their ability to enhance visitors’ experience by using an on-site facility when visiting a museum constructed over the physical remains. Cost and licensing override technical factors such as audiovisual and functional fidelity or composability and installing the system on a PC is preferable to more specialist game control devices if a broad user base is targeted.
Urban streets and urban social sustainability: a case study on Bagdat street in Kadikoy, Istanbul

This paper focuses on the social function of historical public space in Turkey that has been transformed with rise of modernism. Before that, it functioned as a suburban recreational area. The increasing demand for urban lands has been led to its transformation and its function as an urban component. The historical pattern of urban space can be conserved to protect and strengthen social interactions as the key issue of urban social sustainability. With a focus on the urban design through literature review, the aim is to identify the role of social interactions on social sustainability of historical urban street in Turkey. A field survey study was conducted to observe and in-depth interviews to locally evaluate the socio-spatial design principles in Bagdat street, as a research case. Fuzzy Cognitive Map (FCM) and Cognitive Spatial Map (CSM) were used to investigate the spatial experiences of the respondents. Results indicated that the historical continuity of socio-spatial activities through the time in respect to the socio-cultural values and user satisfaction has led to enhance the sense of belongings for citizens. As a conclusion, all data were integrated to propose a socio-spatial framework for designing and developing of Turkish public street, to enhance the urban social sustainability in future communities.
A Fault Diagnosis and Reconfiguration Strategy for Self-Validating Hydrogen Sensor Array Based on MWPCA and ELM

The flammable and explosive property of hydrogen is the main danger in its safe use, storage and transportation. In this paper, a novel hydrogen monitoring system is designed based on the principle of semiconductor, catalytic combustion and heat-conducting gas sensors. Also, the gas sensor will inevitably fail due to the nature of gas sensitive materials in the long-time monitoring process. To ensure the accuracy and reliability of hydrogen concentration measurement, a novel fault diagnosis and reconfiguration strategy for hydrogen sensor array based on moving window principle component analysis and extreme learning machine (MWPCA-ELM) is proposed. Firstly, online multiple faults detection is carried out by using MWPCA. Once one or multiple faults are detected, the measured values of other fault-free sensors will be used to recover the faulty data in real-time by using ELM predictor according to the relevancy among the hydrogen sensors. Secondly, the hydrogen concentration is reconfigured seamlessly and accurately based on ELM under the condition of small calibration data sample. Finally, fault diagnosis is conducted by MWPCA feature extraction coupled with ELM multi-classifier. In order to illustrate the effectiveness and feasibility of the proposed fault diagnosis and reconfiguration strategy, a hydrogen concentration monitoring experimental system was established. The average relative error (ARE) of hydrogen concentration estimation is declined from 1.18% to 0.82% compared with the traditional regression methods. Particularly, the proposed fault reconfiguration model can recover the fault data even if the concentration is changed, and the accuracy of fault diagnosis is 100% within 250 samples.
Performance of concentrating solar power plants in a whole-of-grid context

Concentrating Solar Power (CSP) plants can make a significant contribution to renewable energy supply in many regions of the world. Several bottom-up engineering performance models for CSP plants have been developed. However, they require the capacity of the plant to be known. In whole-of-grid optimization models, the capacity of the plants is an optimization variable and therefore not known in advance. In this case, researchers have taken a simplified modelling approach that overestimated CSP performance, especially for low-DNI conditions. In this paper, we offer a novel density-based engineering modelling approach in which CSP performance is determined independently of the capacity of the power plant, and which can be used in whole-of-grid optimization models. We investigate a case study of the site of the new Aurora CSP project in Port Augusta, South Australia, to be one of the world's largest solar thermal power plants by 2020. Using a Geographical Information System (GIS)-grid representation of Australia, we compare CSP performance resulting from a simplified model with our new approach.
A Case Study of the Augmentation and Evaluation of Training Data for Deep Learning

Deep learning has been widely used for extracting values from big data. As many other machine learning algorithms, deep learning requires significant training data. Experiments have shown both the volume and the quality of training data can significantly impact the effectiveness of the value extraction. In some cases, the volume of training data is not sufficiently large for effectively training a deep learning model. In other cases, the quality of training data is not high enough to achieve the optimal performance. Many approaches have been proposed for augmenting training data to mitigate the deficiency. However, whether the augmented data are “fit for purpose” of deep learning is still a question. A framework for comprehensively evaluating the effectiveness of the augmented data for deep learning is still not available. In this article, we first discuss a data augmentation approach for deep learning. The approach includes two components: the first one is to remove noisy data in a dataset using a machine learning based classification to improve its quality, and the second one is to increase the volume of the dataset for effectively training a deep learning model. To evaluate the quality of the augmented data in fidelity, variety, and veracity, a data quality evaluation framework is proposed. We demonstrated the effectiveness of the data augmentation approach and the data quality evaluation framework through studying an automated classification of biology cell images using deep learning. The experimental results clearly demonstrated the impact of the volume and quality of training data to the performance of deep learning and the importance of the data quality evaluation. The data augmentation approach and the data quality evaluation framework can be straightforwardly adapted for deep learning study in other domains.
Behavioral Program Logic

We present Behavioral Program Logic (BPL), a dynamic logic for trace properties that incorporates concepts from behavioral types and allows reasoning about non-functional properties within a sequent calculus. BPL uses behavioral modalities , to verify statements against behavioral specifications $$\tau $$ . Behavioral specifications generalize postconditions and behavioral types. They can be used to specify other static analyses, e.g., data flow analyses. This enables deductive reasoning about the results of multiple analyses on the same program, potentially implemented in different formalisms. Our calculus for BPL verifies the behavioral specification gradually, as common for behavioral types. This vastly simplifies specification, calculus and composition of local results. We present a sequent calculus for object-oriented actors with futures that integrates a pointer analysis and bridges the gap between behavioral types and deductive verification.
Fractional-Order Modeling and Fuzzy Clustering of Improved Artificial Bee Colony Algorithms

This article proposes an improved artificial bee colony algorithm (IABC) for maintaining a balance between the exploratory and exploitative abilities of the algorithm. Therefore, unmodeled dynamic characteristics can be effectively estimated, which can improve the estimation accuracy of the system and create conditions for later system modeling and control. The proposed IABC algorithm uses two new search expressions to generate new alternative solutions and the global optimal solution is considered in the search process. Exploiting the simple structure and fast global convergence of artificial bee colony algorithm, an improved clustering algorithm combining IABC and Kernel fuzzy c-means (KFCM) iteration is also proposed. KFCM algorithm is sensitive to the initial clustering center and can easily fall into local optimum; and the improved clustering algorithm solves this issue. Compared with other algorithms, IABC algorithm can converge faster and more accurately for a given parameter structure and number of cycles. Three sets of benchmark test functions and six sets of UC Irvine standard datasets were used in the simulation experiments conducted. Experimental results show that KFCM-IABC can aggregate datasets faster than generalized fuzzy c-means (GFCM)-IABC because of the use of IABC. The proposed algorithm improves class validity index from 1 to 4%, thereby exhibiting the advantages of strong robustness and high clustering accuracy.
Wiera: Policy-Driven Multi-Tiered Geo-Distributed Cloud Storage System

Multi-tiered geo-distributed cloud storage systems must tame complexity at many levels: uniform APIs for storage access, supporting flexible storage policies that meet a wide array of application metrics, determining an optimal data placement, handling uncertain network dynamics and access dynamism, and operating across many levels of heterogeneity both within and across data-centers (DCs). In this paper, we present an integrated solution called Wiera. Wiera enables the specification of data management policies both within a local DC and across DCs. Such policies enable the user to optimize for cost, performance, reliability, durability, and consistency, and to express their tradeoffs. In addition, Wiera determines an optimal data placement for the user to meet their desired tradeoffs easily in such an environment. A key aspect of Wiera is first-class support for dynamism due to network, workload, and access patterns changes. As far as we know, Wiera is the first geo-distributed cloud storage system which handles dynamism actively at run-time. Wiera allows <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unmodified applications</i> to reap the benefits of flexible data/storage policies by externalizing the policy specification. We show how Wiera enables a rich specification of dynamic policies using a concise notation and describe the design and implementation of the system. We have implemented a Wiera prototype on multiple cloud environments, AWS and Azure, that illustrates potential benefits from managing dynamics and in using multiple cloud storage tiers both within and across DCs.
Deep Fuzzy Tree for Large-Scale Hierarchical Visual Classification

Deep learning models often use a flat softmax layer to classify samples after feature extraction in visual classification tasks. However, it is hard to make a single decision of finding the true label from massive classes. In this scenario, hierarchical classification is proved to be an effective solution and can be utilized to replace the softmax layer. A key issue of hierarchical classification is to construct a good label structure, which is very significant for classification performance. Several works have been proposed to address the issue, but they have some limitations and are almost designed heuristically. In this article, inspired by fuzzy rough set theory, we propose a deep fuzzy tree model which learns a better tree structure and classifiers for hierarchical classification with theory guarantee. Experimental results show the effectiveness and efficiency of the proposed model in various visual classification datasets.
Deep Co-Saliency Detection via Stacked Autoencoder-Enabled Fusion and Self-Trained CNNs

Image co-saliency detection via fusion-based or learning-based methods faces cross-cutting issues. Fusion-based methods often combine saliency proposals using a majority voting rule. Their performance hence highly depends on the quality and coherence of individual proposals. Learning-based methods typically require ground-truth annotations for training, which are not available for co-saliency detection. In this work, we present a two-stage approach to address these issues jointly. At the first stage, an unsupervised deep learning model with stacked autoencoder (SAE) is proposed to evaluate the quality of saliency proposals. It employs latent representations for image foregrounds, and auto-encodes foreground consistency and foreground-background distinctiveness in a discriminative way. The resultant model, SAE-enabled fusion ( <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SAEF</monospace> ), can combine multiple saliency proposals to yield a more reliable saliency map. At the second stage, motivated by the fact that fusion often leads to over-smoothed saliency maps, we develop self-trained convolutional neural networks ( <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">STCNN</monospace> ) to alleviate this negative effect. <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">STCNN</monospace> takes the saliency maps produced by <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SAEF</monospace> as inputs. It propagates information from regions of high confidence to those of low confidence. During propagation, feature representations are distilled, resulting in sharper and better co-saliency maps. Our approach is comprehensively evaluated on three benchmarks, including MSRC, iCoseg, and Cosal2015, and performs favorably against the state-of-the-arts. In addition, we demonstrate that our method can be applied to object co-segmentation and object co-localization, achieving the state-of-the-art performance in both applications.
Challenges and Methods of Violence Detection in Surveillance Video: A Survey

This article presents a survey of the latest methods of violence detection in video sequences. Although many studies have described the approaches taken to detect violence, there are few surveys providing exhaustive review of the available methods. We expose the main challenges in this area and we classify the methods into five broad categories. We discuss each category and present the main techniques that proposed improvements as well as some performance measures using public datasets to evaluate the different existing techniques of violence detection.
Reversible data hiding in homomorphically encrypted image using interpolation technique

In this paper, a reversible data hiding scheme based on interpolation technique for encrypted images by using homomorphic and probabilistic properties of Paillier cryptosystem is presented. At first, the image owner generates a location map by using an interpolation technique to estimate the Most Significant Bits (MSBs) of pixel to find whether a pixel can be used for embedding or not. Next, with the help of the location map, the original image is preprocessed to create some spare space for data embedding. Meanwhile, the location map is compressed losslessly and the information of compressed location map is substituted with Least Significant Bits (LSBs) of border pixels. Furthermore, the preprocessed image is encrypted by Paillier cryptosystem and sent to the data hider along with location map and original LSBs of border pixels. At the data hiding phase (note that original image cannot be accessed), the additional data and LSBs of border pixels are embedded into homomorphic encrypted image using location map. At the receiver side, the embedded additional data and original image are recovered losslessly in non-separable manner (i.e., from directly decrypted image). Experimental results demonstrate feasibility and efficiency of the proposed scheme, particularly in embedding, image recovery and performance on different security key size.