Reviewerly/exHarmony huggingface dataset:
Model-Inspired Deep Learning for Light-Field Microscopy with Application to Neuron Localization

Light-field microscopes are able to capture spatial and angular information of incident light rays. This allows reconstructing 3D locations of neurons from a single snap-shot. In this work, we propose a model-inspired deep learning approach to perform fast and robust 3D localization of sources using light-field microscopy images. This is achieved by developing a deep network that efficiently solves a convolutional sparse coding (CSC) problem to map Epipolar Plane Images (EPI) to corresponding sparse codes. The network architecture is designed systematically by unrolling the convolutional Iterative Shrinkage and Thresholding Algorithm (ISTA) while the network parameters are learned from a training dataset. Such principled design enables the deep network to leverage both domain knowledge implied in the model, as well as new parameters learned from the data, thereby combining advantages of model-based and learning-based methods. Practical experiments on localization of mammalian neurons from light-fields show that the proposed approach simultaneously provides enhanced performance, interpretability and efficiency.
En-De-Cap: An Encoder Decoder model for Image Captioning

Image captioning is the technique for generating descriptive text of a given image. Modern image captioning techniques are multi-model techniques that uses both Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) to achieve the task of describing the image . The area of image captioning is considered one of the crucial step in advancement in AI. After achieving high success rate in object detection, the next difficult task is to describe the relation between objects in the image. This paper proposes an image captioning framework which incorporates best in class CNN network with attention based Gated Recurrent Unit (GRU) network to generate better descriptive text for a given image.
Inverse reinforcement learning in contextual MDPs

Abstract We consider the task of Inverse Reinforcement Learning in Contextual Markov Decision Processes (MDPs). In this setting, contexts, which define the reward and transition kernel, are sampled from a distribution. In addition, although the reward is a function of the context, it is not provided to the agent. Instead, the agent observes demonstrations from an optimal policy. The goal is to learn the reward mapping, such that the agent will act optimally even when encountering previously unseen contexts, also known as zero-shot transfer. We formulate this problem as a non-differential convex optimization problem and propose a novel algorithm to compute its subgradients. Based on this scheme, we analyze several methods both theoretically, where we compare the sample complexity and scalability, and empirically. Most importantly, we show both theoretically and empirically that our algorithms perform zero-shot transfer (generalize to new and unseen contexts). Specifically, we present empirical experiments in a dynamic treatment regime, where the goal is to learn a reward function which explains the behavior of expert physicians based on recorded data of them treating patients diagnosed with sepsis.
IBM Watson: Redefining Artificial Intelligence Through Cognitive Computing

IBM Watson the so-called genius supercomputer that can supposedly assist doctors in treatments of their patients, too hyped or a groundbreaking technology? The following project analysis of IBM Watson, more specifically Watson for heath, has been described using materials from both IBM documents itself and reviews by third parties. Emphasis on both early and recent failures and inaccuracies have been mentioned, and solutions with regard to the context of the mentioned issues have been proposed. Going further, facts and even hypothetical situations will be discussed in regard to the above proposed question, what is IBM Watson? How exactly does it work? Is it really as good as IBM claims it to be? Is it really going to help completely revolutionize the AI world? Finally comes the comparative analysis of IBM Watson’s competitors to take a look at how they differ and which one is better. With technology there is always room for growth and enhancement. Toward the end will Watson prove itself to be worth all the public and mainstream media attraction? Or will it turn out to be another overhyped AI technology?
Immersion, Flow and Usability in video games

Context: Nowadays, video games look and feel increasingly realistic, and provide engaging and immersive experiences to players. Understanding what makes games so appealing and immersive is important to grasp in order to provide better experiences.
Technical Leverage in a Software Ecosystem: Development Opportunities and Security Risks

Assurance and certification in secure Multiparty Open Software and Services (Assure-MOSS) No single company does master its own national, in-house software.Software is mostly assembled from "the internet" and more than half come from Open Source Software repositories (some in Europe, most elsewhere).Security & privacy assurance, verification and certification techniques designed for large, slow and controlled updates, must now cope with small, continuous changes in weeks, happening in sub-components and decided by third party developers one did not even know they existed.AssureMOSS proposes to switch from processbased to artefact-based security evaluation by supporting all phases of the continuous software lifecycle (Design, Develop, Deploy, Evaluate and back) and their artefacts (Models, Source code, Container images, Services).The key idea is to support mechanisms for lightweigth and scalable screenings applicable automatically to the entire population of software components by Machine intelligent identification of security issues, Sound analysis and verification of changes, Business insight by risk analysis and security evaluation.This approach supports fast-paced development of better software by a new notion: continuous (re)certification.The project will generate also benchmark datasets with thousands of vulnerabilities.
How Trustworthy are Performance Evaluations for Basic Vision Tasks?

This article examines performance evaluation criteria for basic vision tasks involving sets of objects namely, object detection, instance-level segmentation and multi-object tracking. The rankings of algorithms by a criterion can fluctuate with different choices of parameters, e.g. Intersection over Union (IoU) threshold, making their evaluations unreliable. More importantly, there is no means to verify whether we can trust the evaluations of a criterion. This work suggests a notion of trustworthiness for performance criteria, which requires (i) robustness to parameters for reliability, (ii) contextual meaningfulness in sanity tests, and (iii) consistency with mathematical requirements such as the metric properties. We observe that these requirements were overlooked by many widely-used criteria, and explore alternative criteria using metrics for sets of shapes. We also assess all these criteria based on the suggested requirements for trustworthiness.
Speech Bert Embedding for Improving Prosody in Neural TTS

This paper presents a speech BERT model to extract embedded prosody information in speech segments for improving the prosody of synthesized speech in neural text-to-speech (TTS). As a pre-trained model, it can learn prosody attributes from a large amount of speech data, which can utilize more data than the original training data used by the target TTS. The embedding is extracted from the previous segment of a fixed length in the proposed BERT. The extracted embedding is then used together with the mel-spectrogram to predict the following segment in the TTS decoder. Experimental results obtained by the Transformer TTS show that the proposed BERT can extract fine-grained, segment-level prosody, which is complementary to utterance-level prosody to improve the final prosody of the TTS speech. The objective distortions measured on a single speaker TTS are reduced between the generated speech and original recordings. Subjective listening tests also show that the proposed approach is favorably preferred over the TTS without the BERT prosody embedding module, for both in-domain and out-of-domain applications. For Microsoft professional, single/multiple speakers and the LJ Speaker in the public database, subjective preference is similarly confirmed with the new BERT prosody embedding. TTS demo audio samples are in https://judy44chen.github.io/TTSSpeechBERT/.
Cost-Effective Adversarial Attacks against Scene Text Recognition

Scene text recognition is a challenging task due to the diversity in text appearance and complexity of natural scenes. Thanks to the development of deep learning and the large volume of training data, scene text recognition has made impressive progress in recent years. However, recent research on adversarial examples has shown that deep learning models are vulnerable to adversarial input with often imperceptible changes. As one of the most practical tasks in computer vision, scene text recognition is also facing huge security risks. To our best knowledge, there has been no work on adversarial attacks against scene text recognition. To investigate its effects on scene text recognition, we make the first attempt to attack the state-of-the-art scene text recognizers, i.e., attention-based recognizers. To that end, we first adjust the objective function designed for non-sequential tasks, such as image classification, semantic segmentation and image retrieval, to the sequential form. We then propose a novel and effective objective function to further reduce the amount of perturbation while achieving a higher attack success rate. Comprehensive experiments on several standard benchmarks clearly demonstrate effective adversarial effects on scene text recognition by the proposed attacks. It is note worthy that while this is harmful for a recognition system, it is highly advantageous for a text-based CAPTCHA system.
Research on Foreign Object Debris Detection in Airport Runway Based on Semantic Segmentation

Semantic segmentation based on deep learning has very broad application scenarios in the field of computer vision. DeepLab v3+ is currently a widely used semantic segmentation framework. This paper uses the DeepLab v3+ model in TensorFlow to semantically segment foreign objects on the airport runway. Experimental results show that the semantic segmentation model based on DeepLab v3+ can accurately classify foreign objects at the pixel level, which effectively improves the accuracy of foreign object detection and reduces the risk of accidental aircraft take-off and landing.
The Availability-Accountability Dilemma and Its Resolution via Accountability Gadgets

For applications of Byzantine fault tolerant (BFT) consensus protocols where the participants are economic agents, recent works highlighted the importance of accountability: the ability to identify participants who provably violate the protocol. At the same time, being able to reach consensus under dynamic levels of participation is desirable for censorship resistance. We identify an availability-accountability dilemma: in an environment with dynamic participation, no protocol can simultaneously be accountably-safe and live. We provide a resolution to this dilemma by constructing a provably secure optimally-resilient accountability gadget to checkpoint a longest chain protocol, such that the full ledger is live under dynamic participation and the checkpointed prefix ledger is accountable. Our accountability gadget construction is black-box and can use any BFT protocol which is accountable under static participation. Using HotStuff as the black box, we implemented our construction as a protocol for the Ethereum 2.0 beacon chain, and our Internet-scale experiments with more than 4, 000 nodes show that the protocol achieves the required scalability and has better latency than the current solution Gasper, which was shown insecure by recent attacks.
Independent effects of collocation strength and contextual predictability on eye movements in reading

Collocations are commonly co-occurring word pairs, such as "black coffee". Previous research has demonstrated a processing advantage for collocations compared to novel phrases, suggesting that readers are sensitive to the frequency that words co-occur in phrases. However, a further question concerns whether this processing advantage for collocations occurs independently from effects of contextual predictability. We examined this issue in an eye movement experiment using adjective–noun pairs that are strong collocations (e.g. "black coffee") or weak collocations (e.g. "bitter coffee"), based on co-occurrence statistics. These were presented in sentences where the shared concept they expressed (e.g. coffee) was predictable or unpredictable from the prior sentence context. We observed clear effects of collocation strength, with shorter reading times for strong compared to weak collocations. Moreover, these effects occurred independently of effects of contextual predictability. The findings therefore provide novel evidence that a processing advantage for collocations is not driven by contextual expectations.
Cam: Context-Aware Masking for Robust Speaker Verification

Performance degradation caused by noise has been a long-standing challenge for speaker verification. Previous methods usually involve applying a denoising transformation to speaker embeddings or enhancing input features. Nevertheless, these methods are lossy and inefficient for speaker embedding. In this paper, we propose context- aware masking (CAM), a novel method to extract robust speaker embedding. CAM enables the speaker embedding network to "focus" on the speaker of interest and "blur" unrelated noise. The threshold of masking is dynamically controlled by an auxiliary context embedding that captures speaker and noise characteristics. Moreover, models adopting CAM can be trained in an end-to-end manner without using synthesized noisy-clean speech pairs. Our results show that CAM improves speaker verification performance in the wild by a large margin, compared to the baselines.
Designing for Narrative Influence:

Health and safety concerns have led to policies that put individuals under lockdown, but such restrictions lose effectiveness in the long-term due to inherent human needs of connection and physical action. People maintain prosocial behaviors long-term only if they make decisions themselves intrinsically as opposed to forced restrictions. To build systems for effecting positive social purpose in pandemic and environmental concerns, we apply speculative design to create story structures and interactions that promote behaviors for social good. We designed stories and interactions using both plot-based narrative frameworks and character-based machine-learning-generated dialogues for effecting cooperation. We then ran a series of workshops investigating how designers negotiate and collaborate to tell stories for social purpose using a "finish each other's stories" approach. This work illustrates the application of design fiction to promote sustainable behavioral patterns that value societal good.
Cybersafety Approach to Cybersecurity Analysis and Mitigation for Mobility-as-a-Service and Internet of Vehicles

Urban mobility is in the midst of a revolution, driven by the convergence of technologies such as artificial intelligence, on-demand ride services, and Internet-connected and self-driving vehicles. Technological advancements often lead to new hazards. Coupled with the increased levels of automation and connectivity in the new generation of autonomous vehicles, cybersecurity is emerging as a key threat affecting these vehicles. Traditional hazard analysis methods treat safety and security in isolation and are limited in their ability to account for interactions among organizational, sociotechnical, human, and technical components. In response to these challenges, the cybersafety method, based on System Theoretic Process Analysis (STPA and STPA-Sec), was developed to meet the growing need to holistically analyze complex sociotechnical systems. We applied cybersafety to coanalyze safety and security hazards, as well as identify mitigation requirements. The results were compared with another promising method known as Combined Harm Analysis of Safety and Security for Information Systems (CHASSIS). Both methods were applied to the Mobility-as-a-Service (MaaS) and Internet of Vehicles (IoV) use cases, focusing on over-the-air software updates feature. Overall, cybersafety identified additional hazards and more effective requirements compared to CHASSIS. In particular, cybersafety demonstrated the ability to identify hazards due to unsafe/unsecure interactions among sociotechnical components. This research also suggested using CHASSIS methods for information lifecycle analysis to complement and generate additional considerations for cybersafety. Finally, results from both methods were backtested against a past cyber hack on a vehicular system, and we found that recommendations from cybersafety were likely to mitigate the risks of the incident.
Dicom_wsi: A Python Implementation for Converting Whole-Slide Images to Digital Imaging and Communications in Medicine Compliant Files

Adoption of the Digital Imaging and Communications in Medicine (DICOM) standard for whole slide images (WSIs) has been slow, despite significant time and effort by standards curators. One reason for the lack of adoption is that there are few tools which exist that can meet the requirements of WSIs, given an evolving ecosystem of best practices for implementation. Eventually, vendors will conform to the specification to ensure enterprise interoperability, but what about archived slides? Millions of slides have been scanned in various proprietary formats, many with examples of rare histologies. Our hypothesis is that if users and developers had access to easy to use tools for migrating proprietary formats to the open DICOM standard, then more tools would be developed as DICOM first implementations.The technology we present here is dicom_wsi, a Python based toolkit for converting any slide capable of being read by the OpenSlide library into DICOM conformant and validated implementations. Moreover, additional postprocessing such as background removal, digital transformations (e.g., ink removal), and annotation storage are also described. dicom_wsi is a free and open source implementation that anyone can use or modify to meet their specific purposes.We compare the output of dicom_wsi to two other existing implementations of WSI to DICOM converters and also validate the images using DICOM capable image viewers.dicom_wsi represents the first step in a long process of DICOM adoption for WSI. It is the first open source implementation released in the developer friendly Python programming language and can be freely downloaded at .
Metric-learning-assisted domain adaptation

Domain alignment (DA) has been widely used in unsupervised domain adaptation. Many existing DA methods assume that a low source risk, together with the alignment of distributions of source and target, means a low target risk. In this paper, we show that this does not always hold. We thus propose a novel metric-learning-assisted domain adaptation (MLA-DA) method, which employs a novel triplet loss for helping better feature alignment. We explore the relationship between the second largest probability of a target sample’s prediction and its distance to the decision boundary. Based on the relationship, we propose a novel mechanism to adaptively adjust the margin in the triplet loss according to target predictions. Experimental results show that the use of proposed triplet loss can achieve clearly better results. We also demonstrate the performance improvement of MLA-DA on all four standard benchmarks compared with the state-of-the-art unsupervised domain adaptation methods. Furthermore, MLA-DA shows stable performance in robust experiments.
Towards Practical Compressed Video Action Recognition: A Temporal Enhanced Multi-Stream Network

Current compressed video action recognition methods are mainly based on complete data. However, in a real transmission scenario, the compressed video packets are usually disorderly received and even lost due to network jitters or congestion. To recognize actions in early phases with limited packets, e.g. for quickly forecasting possible potential risks, in this paper, we propose a Temporal Enhanced Multi-Stream Network (TEMSN) towards practical compressed video action recognition. First, we make use of three modalities in the compressed domain as complementary cues and build a multi-stream network to capture rich information from compressed video packets. Second, we design a temporal enhanced module based on an Encoder-Decoder structure, which is applied to each stream to infer missing packets, generating more accurate action dynamics. Thanks to the multiple modalities and their temporal enhancement, our approach better models actions with partial available compressed video packets. Experiments on the HMDB-51 and UCF-101 datasets validate its effectiveness and efficiency.
Online Object Recognition Using CNN-based Algorithm on High-speed Camera Imaging: Framework for fast and robust high-speed camera object recognition based on population data cleansing and data ensemble

High-speed camera imaging (e.g., 1,000 fps) is effective to detect and recognize objects moving at high speeds because temporally dense images obtained by a high-speed camera can usually capture the best moment for object detection and recognition. However, the latest recognition algorithms, with their high complexity, are difficult to utilize in real-time applications involving high-speed cameras because a vast number of images need to be processed with no latency. To tackle this problem, we propose a novel framework for real-time object recognition with high-speed camera imaging. The proposed framework has the key processes of population data cleansing and data ensemble. Population data cleansing improves the recognition accuracy by quantifying the recognizability and by excluding part of the images prior to the recognition process, while data ensemble improves the robustness of object recognition by merging the class probabilities with multiple images from the object tracking sequence. Experimental results with a real dataset show that our framework is more effective than existing methods.
Requirements for Location-Based Games for Social Interaction

Social interaction is part of the fabric of society and is essential to challenge many types of social barriers. Location-based games (LBGs) provide a means to foster such interaction in local communities. The design of such games is currently based primarily on designer experience and on the literature on game design in general, and not on an understanding of user requirements. This article explores the preferences and desires of adolescents in neighborhoods of Rotterdam South to socially interact with others and engage with their own neighborhood via LBGs. Adolescents are informants in the exploration of gaming activities for social interaction, which, when subjected to expert review with the mechanics-dynamics-aesthetics framework, produce gameplay requirements for the desired purpose: social interaction in public space. Such requirements provide researchers and game designers insights on the game dynamics best suited to foster location-based social interaction.