Reviewerly/exHarmony huggingface dataset:
Physics-Based Deep Neural Network for Augmented Reality During Liver Surgery

In this paper we present an approach combining a finite element method and a deep neural network to learn complex elastic deformations with the objective of providing augmented reality during hepatic surgery. Derived from the U-Net architecture, our network is built entirely from physically-based simulations of a preoperative segmentation of the organ. These simulations are performed using an immersed-boundary method, which offers several numerical and practical benefits, such as not requiring boundary-conforming volume elements. We perform a quantitative assessment of the method using synthetic and ex vivo patient data. Results show that the network is capable of solving the deformed state of the organ using only a sparse partial surface displacement data and achieve similar accuracy as a FEM solution, while being about 100 $$\times $$ faster. When applied to an ex vivo liver example, we achieve the registration in only 3 ms with a mean target registration error (TRE) of 2.9 mm.
Simulated local climatic impacts of large-scale photovoltaics over the barren area of Qinghai, China

Solar energy has developed rapidly in recent years, and its impacts on climate have been revealed mainly through model simulations and limited field observations. However, the representation of solar plants in these models is highly simplistic and does not account for dynamical effects, which may pose risks under the simulations for tracking photovoltaics (PV). To make a more comprehensive assessment of the PV-induced climatic impacts, it is relevant to further develop a more sophisticated PV module based on the observed PV-induced impacts from specific cases. To remedy the deficiencies in the existing PV energy models, the newly sophisticated PV module established in this paper includes both the land surface radiation balance, sensible heat balance and the surface physical dray process over the locations of PV plants. It is then used to simulate the local climatic impacts of PV plants through three parallel sensitivity experiments. Comparison analysis between control run and two PV installation scenarios showed obvious changes in the 10-m wind speed, land surface temperature, and 2-m specific humidity were found locally over the locations of PV plants during both the warm and cold periods. The magnitudes of these changes are positively related to the installed capacity of the PV plants, but are much smaller than the natural climate variations. It is expected that the observation-based PV module established in this paper is a meaningful attempt for the model simulation on this field.
Serving deep neural networks at the cloud edge for vision applications on mobile platforms

The proliferation of high resolution cameras on embedded devices along with the growing maturity of deep neural networks (DNNs) has spawned powerful mobile vision applications. To enable applications on mobile devices, the offloading approach processes live video streams using DNNs on server-class GPU accelerators. However, their use in latency constrained applications is particularly challenging because of the large and unpredictable round-trip latency from mobile devices to the cloud computing resources. As a consequence, system designers routinely look for ways to offload to local servers at the cloud edge, known as the cloudlet. This paper explores the potential of serving multiple DNNs using the cloudlet model to implement complex vision applications on mobile devices. We present DeepQuery, a new mobile offloading system that is capable to serve DNNs with different structures for a wide range of tasks including object detection and tracking, scene graph detection, and video description. DeepQuery provides application programming interfaces to offload applications programed as Directed Acyclic Graphs of DNN queries, and employs data parallelization and input batching techniques to reduce processing delays. To improve GPU utilization, it co-locates real-time and delay-tolerant tasks on shared GPUs, and exploits a predictive and plan-ahead approach to alleviate resource contention caused by co-locating. We evaluate DeepQuery and demonstrate its effectiveness using several real world applications.
Differential Privacy and the US Census

Differential privacy is a mathematically rigorous definition of privacy tailored to statistical analysis of large datasets. Differentially private systems simultaneously provide useful statistics to the well-intentioned data analyst and strong protection against arbitrarily powerful adversarial system users -- without needing to distinguish between the two. Differentially private systems "don't care'' what the adversary knows, now or in the future. Finally, differentially private systems can rigorously bound and control the cumulative privacy loss that accrues over many interactions with the confidential data. These unique properties, together with the abundance of auxiliary data sources and the ease with which they can be deployed by a privacy adversary, led the US Census Bureau to adopt differential privacy as the disclosure avoidance methodology of the 2020 decennial census. This talk will motivate the definition of differential privacy, reflect on the theory-meets-practice experiences of the decennial census, and highlight a few pressing challenges in the field.
Movie scene segmentation using object detection and set theory

Movie data has a prominent role in the exponential growth of multimedia data over the Internet, and its analysis has become a hot topic with computer vision. The initial step towards movie analysis is scene segmentation. In this article, we investigated this problem through a novel intelligent Convolutional Neural Network (CNN) based three folded framework. The first fold segments the input movie into shots, the second fold detects objects in the segmented shots and the third fold performs object-based shots matching for detecting scene boundaries. Texture and shape features are fused for shots segmentation, and each shot is represented by a set of detected objects acquired from a light-weight CNN model. Finally, we apply set theory with the sliding window–based approach to integrate the same shots to decide scene boundaries. The experimental evaluation indicates that our proposed approach outran the existing movie scene segmentation approaches.
Multimodal Deep Network Embedding With Integrated Structure and Attribute Information

Network embedding is the process of learning low-dimensional representations for nodes in a network while preserving node features. Existing studies only leverage network structure information and emphasize the preservation of structural features. However, nodes in real-world networks often have a rich set of attributes providing extra semantic information. It has been demonstrated that both structural and attribute features are important for network analysis tasks. To preserve both features, we investigate the problem of integrating structure and attribute information to perform network embedding and propose a multimodal deep network embedding (MDNE) method. MDNE captures the non-linear network structures and the complex interactions among structures and attributes using a deep model consisting of multiple layers of non-linear functions. Since structures and attributes are two different types of information, a multimodal learning method is adopted to pre-process them and help the model to better capture the correlations between node structure and attribute information. We define the loss function employing structural and attribute proximities to preserve the respective features, and the representations are obtained by minimizing the loss function. Results of extensive experiments on four real-world data sets show that the proposed method performs significantly better than baselines on a variety of tasks, which demonstrates the effectiveness and generality of our method.
PI-Bully: Personalized Cyberbullying Detection with Peer Influence

Cyberbullying has become one of the most pressing online risks for adolescents and has raised serious concerns in society. Recent years have witnessed a surge in research aimed at developing principled learning models to detect cyberbullying behaviors. These efforts have primarily focused on building a single generic classification model to differentiate bullying content from normal (non-bullying) content among all users. These models treat users equally and overlook idiosyncratic information about users that might facilitate the accurate detection of cyberbullying. In this paper, we propose a personalized cyberbullying detection framework, PI-Bully, that draws on empirical findings from psychology highlighting unique characteristics of victims and bullies and peer influence from like-minded users as predictors of cyberbullying behaviors. Our framework is novel in its ability to model peer influence in a collaborative environment and tailor cyberbullying prediction for each individual user. Extensive experimental evaluations on real-world datasets corroborate the effectiveness of the proposed framework.
Learning to Generate Grounded Visual Captions Without Localization Supervision

When automatically generating a sentence description for an image or video, it often remains unclear how well the generated caption is grounded, that is whether the model uses the correct image regions to output particular words, or if the model is hallucinating based on priors in the dataset and/or the language model. The most common way of relating image regions with words in caption models is through an attention mechanism over the regions that are used as input to predict the next word. The model must therefore learn to predict the attentional weights without knowing the word it should localize. This is difficult to train without grounding supervision since recurrent models can propagate past information and there is no explicit signal to force the captioning model to properly ground the individual decoded words. In this work, we help the model to achieve this via a novel cyclical training regimen that forces the model to localize each word in the image after the sentence decoder generates it, and then reconstruct the sentence from the localized image region(s) to match the ground-truth. Our proposed framework only requires learning one extra fully-connected layer (the localizer), a layer that can be removed at test time. We show that our model significantly improves grounding accuracy without relying on grounding supervision or introducing extra computation during inference, for both image and video captioning tasks. Code is available at https://github.com/chihyaoma/cyclical-visual-captioning .
Stack Development and Industrial Scale-Up

Sunfire commercially distributes products for various markets (SOFC, SOEC and Co-SOEC) with its stack and system technology. To achieve an ideal tradeoff between cost, performance, and degradation for its customers, Sunfire's stack design is continuously being improved. In this publication, latest results that resulted in a decrease of the combined life cycle costs (€ kW-1 h-1) of approx. 40% and cost projections for large scale production will be presented. Furthermore, recent changes in automation and industrial scale-up that lead to a current annual production volume of >10MW and next steps to further increase this value will be shown. Last but not least, updates on the recent development for Sunfire's products Home, Remote and Hylink/Synlink will be given.
Robustness of Adversarial Attacks in Sound Event Classification

An adversarial attack is a method to generate perturbations to the input of a machine learning model in order to make the output of the model incorrect.The perturbed inputs are known as adversarial examples.In this paper, we investigate the robustness of adversarial examples to simple input transformations such as mp3 compression, resampling, white noise and reverb in the task of sound event classification.By performing this analysis, we aim to provide insights on strengths and weaknesses in current adversarial attack algorithms as well as provide a baseline for defenses against adversarial attacks.Our work shows that adversarial attacks are not robust to simple input transformations.White noise is the most consistent method to defend against adversarial attacks with a success rate of 73.72% averaged across all models and attack algorithms.
DepthCalorieCam: A Mobile Application for Volume-Based FoodCalorie Estimation using Depth Cameras

Some recent smartphones such as iPhone Xs have pair of cameras which can be used as stereo cameras on their backside. Regarding iPhone with iOS11 or more, the official API provides the function to estimate depth information from two backside cameras in the real-time way. By taking advantage of this function, we have developed an iOS app, "DepthCalorieCam", which estimates the amount of food calories based on food volumes. In the proposed app, it takes a RGB-D image of a dish, estimate categories and volumes of foods on the dish, and calculate the amount of their calories using the pre-registered calorie density of each food category. We have achieved very accurate calorie estimation by using depth information. The error of estimated calories was reduced greatly compared with the existing size-based systems.
Dynamic Knowledge Graph Construction for Zero-shot Commonsense Question Answering

Understanding narratives requires dynamically reasoning about the implicitcauses, effects, and states of the situations described in text, which in turnrequires understanding rich background knowledge about how the social andphysical world works. At the core of this challenge is how to accesscontextually relevant knowledge on demand and reason over it. In this paper, we present initial studies toward zero-shot commonsense QA byformulating the task as probabilistic inference over dynamically generatedcommonsense knowledge graphs. In contrast to previous studies for knowledgeintegration that rely on retrieval of existing knowledge from static knowledgegraphs, our study requires commonsense knowledge integration where contextuallyrelevant knowledge is often not present in existing knowledge bases. Therefore,we present a novel approach that generates contextually relevant knowledge ondemand using generative neural commonsense knowledge models. Empirical results on the SocialIQa and StoryCommonsense datasets in azero-shot setting demonstrate that using commonsense knowledge models todynamically construct and reason over knowledge graphs achieves performanceboosts over pre-trained language models and using knowledge models to directlyevaluate answers.
Deep Learning and Explainable AI in Healthcare Using EHR

With the evolving time, Artificial Intelligence (AI) has proved to be of great assistance in the medical field. Rapid advancements led to the availability of technology which could predict many different diseases risks. Patients Electronic Health Records (EHR) contains all different kinds of medical data for each patient, for each medical visit. Now there are many predictive models like random forests, boosted trees which provide high accuracy but not end-to-end interpretability while the ones such as Naive-Bayes, logistic regression and single decision trees are intelligible enough but less accurate. These models are interpretable but they lack to see the temporal relationships in the characteristic attributes present in the EHR data. Eventually, the model accuracy is compromised. Interpretability of a model is essential in critical healthcare applications. Interpretability helps the medical personnel with explanations that build trust towards machine learning systems. This chapter contains the design and implementation of an Explainable Deep Learning System for Healthcare using EHR. In this chapter, use of an attention mechanism and Recurrent Neural Network(RNN) on EHR data has been discussed, for predicting heart failure of patients and providing insight into the key diagnoses that have led to the prediction. The patient’s medical history is given as a sequential input to the RNN which predicts the heart failure risk and provides explainability along with it. This represents an ante-hoc explainability model. A neural network having two levels and attention model is trained for detecting those visits of the patient in his history that could be influential and significant to understand the reasons behind any prediction done on the medical history of the patient data. Thus, considering the last visit first proves to be beneficial. When a prediction is made, the visit-level contribution is prioritized i.e. which visit contributes the most to the final prediction where each visit consists of multiple codes. This model can be helpful to medical persons for predicting the heart failure risks of patients with diseases they have been diagnosed with based on EHR. This model is then worked upon by local interpretable model-agnostic explanations (LIME) which provide the different features that positively and negatively contribute to heart failure risk.
Asymptotics of Wide Networks from Feynman Diagrams

Understanding the asymptotic behavior of wide networks is of considerable interest. In this work, we present a general method for analyzing this large width behavior. The method is an adaptation of Feynman diagrams, a standard tool for computing multivariate Gaussian integrals. We apply our method to study training dynamics, improving existing bounds and deriving new results on wide network evolution during stochastic gradient descent. Going beyond the strict large width limit, we present closed-form expressions for higher-order terms governing wide network training, and test these predictions empirically.
Increasing the robustness of DNNs against image corruptions by playing the Game of Noise

The human visual system is remarkably robust against a wide range of naturally occurring variations and corruptions like rain or snow. In contrast, the performance of modern image recognition models strongly degrades when evaluated on previously unseen corruptions. Here, we demonstrate that a simple but properly tuned training with additive Gaussian and Speckle noise generalizes surprisingly well to unseen corruptions, easily reaching the previous state of the art on the corruption benchmark ImageNet-C (with ResNet50) and on MNIST-C. We build on top of these strong baseline results and show that an adversarial training of the recognition model against uncorrelated worst-case noise distributions leads to an additional increase in performance. This regularization can be combined with previously proposed defense methods for further improvement.
BYOD Policy Compliance: Risks and Strategies in Organizations

The proliferation of mobile devices has brought the Bring Your Own Device (BYOD) trend in organizations, along with significant challenges when employees fail to comply with security policies. Previous reviews in this research area had focused solely on the technical issues surrounding BYOD implementation while leaving out the human behavior in complying with security policies which is a major contributing factor to security vulnerabilities. In this paper, we systematically review the literature to gather evidences related to security risks, challenges posed by employees’ security policy noncompliance behavior and mitigation strategies to address them. The risks are mapped according to the People, Process and Technology (PPT) Model. The review reports that security policy compliance in a BYOD environment remains scarce which makes this review a novel contribution to research on human security behavior in Information System (IS). In addition, open research problems and future research directions are presented in the paper.
Federating recommendations using differentially private prototypes

Machine learning methods exploit similarities in users’ activity patterns to provide recommendations in applications across a wide range of fields including entertainment, dating, and commerce. However, in domains that demand protection of personally sensitive data, such as medicine or banking, how can we learn recommendation models without accessing the sensitive data and without inadvertently leaking private information? Many situations in the medical field prohibit centralizing the data from different hospitals and thus require learning from information kept in separate databases. We propose a new federated approach to learning global and local private models for recommendation without collecting raw data, user statistics, or information about personal preferences. Our method produces a set of locally learned prototypes that allow us to infer global behavioral patterns while providing differential privacy guarantees for users in any database of the system. By requiring only two rounds of communication, we both reduce the communication costs and avoid excessive privacy loss associated with typical federated learning iterative procedures. We test our framework on synthetic data, real federated medical data, and a federated version of Movielens ratings. We show that local adaptation of the global model allows the proposed method to outperform centralized matrix-factorization-based recommender system models, both in terms of the accuracy of matrix reconstruction and in terms of the relevance of recommendations, while maintaining provable privacy guarantees. We also show that our method is more robust and has smaller variance than individual models learned by independent entities.
Opinion Mining on E-Commerce Data Using Sentiment Analysis and K-Medoid Clustering

This study aimed to analyze sentiment opinions to find out the opinions of users on an e-commerce Web. The method used was through analyzing text reviews obtained from customers on an e-commerce website. The algorithm used was k-medoid clustering.
Video Summarization Using Highlight Detection and Pairwise Deep Ranking Model

With mobile phones and camera enabled devices becoming pervasive and user-friendly, a large number of videos are being shot every day and uploaded to social media and video streaming websites. This makes them an important information dispensing tool. Searching and analysing such large amount of videos is an extremely tedious task. Thus, automatic video summarization is used to produce a short, informative summary of a long video, which is useful in the indexing and the classification of such videos in the video database. Video summarization is a very challenging task. This work aims to generate a video summary, by modelling a two stream architecture consisting of deep convolutional neural network in each stream for extracting both spatial and temporal information of a video. Two dimensional Convolutional Neural Network (2D CNN) is used to exploit spatial information whereas a three dimensional Convolutional Neural Network (3D CNN) is used to exploit temporal information to generate highlight scores for segments of the video. The segment scores from each stream are fused to detect highlight segments from the video. Furthermore, as the highlight result depicts only a relative level of interest of a user in a video, the DCNN in each stream is trained with a pairwise deep ranking model. The goal is making the highlight score of highlight segment higher than non-highlight segment by optimizing the model. The obtained highlight segments are then used to summarize a video.
A Comprehensive Framework for Saturation Theorem Proving

We present a framework for formal refutational completeness proofs of abstract provers that implement saturation calculi, such as ordered resolution or superposition. The framework relies on modular extensions of lifted redundancy criteria. It allows us to extend redundancy criteria so that they cover subsumption, and also to model entire prover architectures in such a way that the static refutational completeness of a calculus immediately implies the dynamic refutational completeness of a prover implementing the calculus, for instance within an Otter or loop. Our framework is mechanized in Isabelle/ .