Reviewerly/exHarmony huggingface dataset:
Structured Two-Stream Attention Network for Video Question Answering

To date, visual question answering (VQA) (i.e., image QA and video QA) is still a holy grail in vision and language understanding, especially for video QA. Compared with image QA that focuses primarily on understanding the associations between image region-level details and corresponding questions, video QA requires a model to jointly reason across both spatial and long-range temporal structures of a video as well as text to provide an accurate answer. In this paper, we specifically tackle the problem of video QA by proposing a Structured Two-stream Attention network, namely STA, to answer a free-form or open-ended natural language question about the content of a given video. First, we infer rich longrange temporal structures in videos using our structured segment component and encode text features. Then, our structured two-stream attention component simultaneously localizes important visual instance, reduces the influence of background video and focuses on the relevant text. Finally, the structured two-stream fusion component incorporates different segments of query and video aware context representation and infers the answers. Experiments on the large-scale video QA dataset TGIF-QA show that our proposed method significantly surpasses the best counterpart (i.e., with one representation for the video input) by 13.0%, 13.5%, 11.0% and 0.3 for Action, Trans., TrameQA and Count tasks. It also outperforms the best competitor (i.e., with two representations) on the Action, Trans., TrameQA tasks by 4.1%, 4.7%, and 5.1%.
Distributed programming using role-parametric session types in go: statically-typed endpoint APIs for dynamically-instantiated communication structures

This paper presents a framework for the static specification and safe programming of message passing protocols where the number and kinds of participants are dynamically instantiated. We develop the first theory of distributed multiparty session types (MPST) to support parameterised protocols with indexed roles—our framework statically infers the different kinds of participants induced by a protocol definition as role variants, and produces decoupled endpoint projections of the protocol onto each variant. This enables safe MPST-based programming of the parameterised endpoints in distributed settings: each endpoint can be implemented separately by different programmers, using different techniques (or languages). We prove the decidability of role variant inference and well-formedness checking, and the correctness of projection. We implement our theory as a toolchain for programming such role-parametric MPST protocols in Go. Our approach is to generate API families of lightweight, protocol- and variant-specific type wrappers for I/O. The APIs ensure a well-typed Go endpoint program (by native Go type checking) will perform only compliant I/O actions w.r.t. the source protocol. We leverage the abstractions of MPST to support the specification and implementation of Go applications involving multiple channels, possibly over mixed transports (e.g., Go channels, TCP), and channel passing via a unified programming interface. We evaluate the applicability and run-time performance of our generated APIs using microbenchmarks and real-world applications.
Azure SQL Database Always Encrypted

This paper presents Always Encrypted, a recently released feature of Microsoft SQL Server that uses column granularity encryption to provide cryptographic data protection guarantees. Always Encrypted can be used to outsource database administration while keeping the data confidential from an administrator, including cloud operators. The first version of Always Encrypted was released in Azure SQL Database and as part of SQL Server 2016, and supported equality operations over deterministically encrypted columns. The second version, released as part of SQL Server 2019, uses an enclave running within a trusted execution environment to provide richer functionality that includes comparison and string pattern matching for an IND-CPA-secure (randomized) encryption scheme. We present the security, functionality, and design of Always Encrypted, and provide a performance evaluation using the TPC-C benchmark.
Preserving Edge Knowledge Sharing Among IoT Services: A Blockchain-Based Approach

Edge computational intelligence, integrating artificial intelligence (AI) and edge computing into Internet of Things (IoT), will generate many scattered knowledge. To enable auditable and delay-sensitive IoT services, these knowledge will be shared among decentralized intelligent network edges (DINEs), end users, and supervisors frequently. Blockchain has a promising ability to provide a traceable, privacy-preserving and tamper-resistant ledger for sharing edge knowledge. However, due to the complicated environments of network edges, knowledge sharing among DINEs still faces many challenges. Firstly, the resource limitation and mobility of DINEs impede the applicability of existing consensus tricks (e.g., Poof of Work, Proof of Stake, and Paxos) of blockchain. Secondly, the adversaries may eavesdrop the content of edge knowledge or entice the blockchain to forks using some attacking models (like man-in-the-middle attack, denial of services, etc.). In this article, an user-centric blockchain (UCB) framework is proposed for preserving edge knowledge sharing in IoT. Significant superiorities of UCB benefit from the proof of popularity (PoP) consensus mechanism, which is more energy-efficient and fast. Security analysis and experiments based on Raspberry Pi 3 Model B demonstrate its feasibility with low block generating delay and complexity.
A study on the prevalence of human values in software engineering publications, 2015 -- 2018

Failure to account for human values in software (e.g., equality and fairness) can result in user dissatisfaction and negative socio-economic impact. Engineering these values in software, however, requires technical and methodological support throughout the development life cycle. This paper investigates to what extent top Software Engineering (SE) conferences and journals have included research on human values in SE. We investigate the prevalence of human values in recent (2015 -- 2018) publications in these top venues. We classify these publications, based on their relevance to different values, against a widely used value structure adopted from the social sciences. Our results show that: (a) only a small proportion of the publications directly consider values, classified as directly relevant publications; (b) for the majority of the values, very few or no directly relevant publications were found; and (c) the prevalence of directly relevant publications was higher in SE conferences compared to SE journals. This paper shares these and other insights that may motivate future research on human values in software engineering.
Virtually Real or Really Virtual: Towards a Heritage Metaverse

The hype surrounding the impending mainstreaming of Virtual Reality can seem to prioritize the digital above the critical. With the development of VR said to be at a pivotal point, there is an important opportunity to consider the emergence of virtual heritage and its potential futures. This paper argues that there is a disjunction between the present reality of virtual heritage and virtual reality, and discusses the twin challenges of presence and realism within virtual reality. In particular, it highlights a paradox inherent in virtual heritage and virtual reality and proposes the use of ‘loose-realism’ as a solution. Ultimately, the challenge is to address the claims that virtual reality represents a new class of information system, or metaverse, in order that virtual heritage fully engages with enquiry about the past.
100,000 Podcasts: A Spoken English Document Corpus

Podcasts are a large and growing repository of spoken audio.As an audio format, podcasts are more varied in style and production type than broadcast news, contain more genres than typically studied in video data, and are more varied in style and format than previous corpora of conversations.When transcribed with automatic speech recognition they represent a noisy but fascinating collection of documents which can be studied through the lens of natural language processing, information retrieval, and linguistics.Paired with the audio files, they are also a resource for speech processing and the study of paralinguistic, sociolinguistic, and acoustic aspects of the domain.We introduce the Spotify Podcast Dataset, a new corpus of 100,000 podcasts.We demonstrate the complexity of the domain with a case study of two tasks: (1) passage search and (2) summarization.This is orders of magnitude larger than previous speech corpora used for search and summarization.Our results show that the size and variability of this corpus opens up new avenues for research.
CT-wasm: type-driven secure cryptography for the web ecosystem

A significant amount of both client and server-side cryptography is implemented in JavaScript. Despite widespread concerns about its security, no other language has been able to match the convenience that comes from its ubiquitous support on the "web ecosystem" - the wide variety of technologies that collectively underpins the modern World Wide Web. With the new introduction of the WebAssembly bytecode language (Wasm) into the web ecosystem, we have a unique opportunity to advance a principled alternative to existing JavaScript cryptography use cases which does not compromise this convenience. We present Constant-Time WebAssembly (CT-Wasm), a type-driven strict extension to WebAssembly which facilitates the verifiably secure implementation of cryptographic algorithms. CT-Wasm's type system ensures that code written in CT-Wasm is both information flow secure and resistant to timing side channel attacks; like base Wasm, these guarantees are verifiable in linear time. Building on an existing Wasm mechanization, we mechanize the full CT-Wasm specification, prove soundness of the extended type system, implement a verified type checker, and give several proofs of the language's security properties. We provide two implementations of CT-Wasm: an OCaml reference interpreter and a native implementation for Node.js and Chromium that extends Google's V8 engine. We also implement a CT-Wasm to Wasm rewrite tool that allows developers to reap the benefits of CT-Wasm's type system today, while developing cryptographic algorithms for base Wasm environments. We evaluate the language, our implementations, and supporting tools by porting several cryptographic primitives - Salsa20, SHA-256, and TEA - and the full TweetNaCl library. We find that CT-Wasm is fast, expressive, and generates code that we experimentally measure to be constant-time.
COVID-19 in the Field of Education: State of the Art

COVID-19 has produced a transformation in society that has, in turn, influenced the field of education. The purpose of this study was to analyze the influence of COVID-19 on education using the Web of Science database. A methodology based on bibliometrics was used. Specifically, a scientific mapping of the literature was carried out, as well as a co-word analysis on the state of the question. The analysis included 940 publications. The results show that the institution with the highest volume of production in this field is the University of London. Among the journals, the Journal of Chemical Education stands out. Furthermore, in the analysis of the structural and thematic development of co-words, a high percentage of keyword matching was observed. In 2020, the motor themes were mental-health, organic-chemistry, general-public, first-year-undergraduate, and upper-division-undergraduate, while in 2021, they were autism-spectrum-disorder, adoption, internet, and intervention. It can be concluded that investigation into COVID-19 in the educational field is in its initial process. The research is currently mainly oriented to pedagogical methods, especially e-learning or collaborative learning, although they are not the only trends in this field. Research is also focusing on mental health, students with various disorders, and higher education. In the near future, research on COVID-19 in the field of education will probably be oriented to the application of effective pedagogical methods to train students at various educational stages.
Total solar irradiance's effect on the performance of empirical models for estimating global solar radiation: An empirical-based review

Many Total Solar Irradiance (TSI) and Solar Constant (SC) values are given and used in the literature, sometimes leading to confusion. The TSI value is relevant and has great importance in engineering and scientific research in energy. In this study, a review has been done to study the TSI and SC concepts. A reevaluation of the TSI and SC is undertaken to consider state of the art. The effect of three different TSI values concerning different locations is studied for estimating global radiation with the Ångström-Prescott (A-P) formulation over Spain. New seasonal A-P coefficient sets are developed for Spain. The good performance for all the 29 stations and seasons and the slight differences observed for the different TSI values employed ensure global solar radiation (GSR) accuracy with the A-P model. The results highlight the local and seasonal A-P coefficient calibration relevance instead of using a single general pair of calibration coefficients. Based on the TSI study, we recommended calculations of solar radiation models to adopt the TSI value equal to 1361 W m−2. The revised TSI is lower than the previous values proposed. Therefore, it is highly recommended to review the value and concept of TSI.
A deep-learning prediction model for imbalanced time series data forecasting

Time series forecasting has attracted wide attention in recent decades. However, some time series are imbalanced and show different patterns between special and normal periods, leading to the prediction accuracy degradation of special periods. In this paper, we aim to develop a unified model to alleviate the imbalance and thus improving the prediction accuracy for special periods. This task is challenging because of two reasons: (1) the temporal dependency of series, and (2) the tradeoff between mining similar patterns and distinguishing different distributions between different periods. To tackle these issues, we propose a self-attention-based time-varying prediction model with a two-stage training strategy. First, we use an encoder-decoder module with the multi-head self-attention mechanism to extract common patterns of time series. Then, we propose a time-varying optimization module to optimize the results of special periods and eliminate the imbalance. Moreover, we propose reverse distance attention in place of traditional dot attention to highlight the importance of similar historical values to forecast results. Finally, extensive experiments show that our model performs better than other baselines in terms of mean absolute error and mean absolute percentage error.
Measuring Human Trust in a Virtual Assistant using Physiological Sensing in Virtual Reality

With the advancement of Artificial Intelligence technology to make smart devices, understanding how humans develop trust in virtual agents is emerging as a critical research field. Through our research, we report on a novel methodology to investigate user’s trust in auditory assistance in a Virtual Reality (VR) based search task, under both high and low cognitive load and under varying levels of agent accuracy. We collected physiological sensor data such as electroencephalography (EEG), galvanic skin response (GSR), and heart-rate variability (HRV), subjective data through questionnaire such as System Trust Scale (STS), Subjective Mental Effort Questionnaire (SMEQ) and NASA-TLX. We also collected a behavioral measure of trust (congruency of users’ head motion in response to valid/ invalid verbal advice from the agent). Our results indicate that our custom VR environment enables researchers to measure and understand human trust in virtual agents using the matrices, and both cognitive load and agent accuracy play an important role in trust formation. We discuss the implications of the research and directions for future work.
Hands-on reservoir computing: a tutorial for practical implementation

Abstract This manuscript serves a specific purpose: to give readers from fields such as material science, chemistry, or electronics an overview of implementing a reservoir computing (RC) experiment with her/his material system. Introductory literature on the topic is rare and the vast majority of reviews puts forth the basics of RC taking for granted concepts that may be nontrivial to someone unfamiliar with the machine learning field (see for example reference Lukoševičius (2012 Neural Networks: Tricks of the Trade (Berlin: Springer) pp 659–686). This is unfortunate considering the large pool of material systems that show nonlinear behavior and short-term memory that may be harnessed to design novel computational paradigms. RC offers a framework for computing with material systems that circumvents typical problems that arise when implementing traditional, fully fledged feedforward neural networks on hardware, such as minimal device-to-device variability and control over each unit/neuron and connection. Instead, one can use a random, untrained reservoir where only the output layer is optimized, for example, with linear regression. In the following, we will highlight the potential of RC for hardware-based neural networks, the advantages over more traditional approaches, and the obstacles to overcome for their implementation. Preparing a high-dimensional nonlinear system as a well-performing reservoir for a specific task is not as easy as it seems at first sight. We hope this tutorial will lower the barrier for scientists attempting to exploit their nonlinear systems for computational tasks typically carried out in the fields of machine learning and artificial intelligence. A simulation tool to accompany this paper is available online 7 7 https://github.com/stevenabreu7/handson_reservoir . .
Fast and Low-GPU-memory abdomen CT organ segmentation: The FLARE challenge

Automatic segmentation of abdominal organs in CT scans plays an important role in clinical practice. However, most existing benchmarks and datasets only focus on segmentation accuracy, while the model efficiency and its accuracy on the testing cases from different medical centers have not been evaluated. To comprehensively benchmark abdominal organ segmentation methods, we organized the first Fast and Low GPU memory Abdominal oRgan sEgmentation (FLARE) challenge, where the segmentation methods were encouraged to achieve high accuracy on the testing cases from different medical centers, fast inference speed, and low GPU memory consumption, simultaneously. The winning method surpassed the existing state-of-the-art method, achieving a 19× faster inference speed and reducing the GPU memory consumption by 60% with comparable accuracy. We provide a summary of the top methods, make their code and Docker containers publicly available, and give practical suggestions on building accurate and efficient abdominal organ segmentation models. The FLARE challenge remains open for future submissions through a live platform for benchmarking further methodology developments at https://flare.grand-challenge.org/.
Deliberate Attention Networks for Image Captioning

In daily life, deliberation is a common behavior for human to improve or refine their work (e.g., writing, reading and drawing). To date, encoder-decoder framework with attention mechanisms has achieved great progress for image captioning. However, such framework is in essential an one-pass forward process while encoding to hidden states and attending to visual features, but lacks of the deliberation action. The learned hidden states and visual attention are directly used to predict the final captions without further polishing. In this paper, we present a novel Deliberate Residual Attention Network, namely DA, for image captioning. The first-pass residual-based attention layer prepares the hidden states and visual attention for generating a preliminary version of the captions, while the second-pass deliberate residual-based attention layer refines them. Since the second-pass is based on the rough global features captured by the hidden layer and visual attention in the first-pass, our DA has the potential to generate better sentences. We further equip our DA with discriminative loss and reinforcement learning to disambiguate image/caption pairs and reduce exposure bias. Our model improves the state-of-the-arts on the MSCOCO dataset and reaches 37.5% BELU-4, 28.5% METEOR and 125.6% CIDEr. It also outperforms the-state-ofthe-arts from 25.1% BLEU-4, 20.4% METEOR and 53.1% CIDEr to 29.4% BLEU-4, 23.0% METEOR and 66.6% on the Flickr30K dataset.
Loss-Based Attention for Deep Multiple Instance Learning

Although attention mechanisms have been widely used in deep learning for many tasks, they are rarely utilized to solve multiple instance learning (MIL) problems, where only a general category label is given for multiple instances contained in one bag. Additionally, previous deep MIL methods firstly utilize the attention mechanism to learn instance weights and then employ a fully connected layer to predict the bag label, so that the bag prediction is largely determined by the effectiveness of learned instance weights. To alleviate this issue, in this paper, we propose a novel loss based attention mechanism, which simultaneously learns instance weights and predictions, and bag predictions for deep multiple instance learning. Specifically, it calculates instance weights based on the loss function, e.g. softmax+cross-entropy, and shares the parameters with the fully connected layer, which is to predict instance and bag predictions. Additionally, a regularization term consisting of learned weights and cross-entropy functions is utilized to boost the recall of instances, and a consistency cost is used to smooth the training process of neural networks for boosting the model generalization performance. Extensive experiments on multiple types of benchmark databases demonstrate that the proposed attention mechanism is a general, effective and efficient framework, which can achieve superior bag and image classification performance over other state-of-the-art MIL methods, with obtaining higher instance precision and recall than previous attention mechanisms. Source codes are available on https://github.com/xsshi2015/Loss-Attention.
A Two-Step Approach for Implicit Event Argument Detection

In this work, we explore the implicit event argument detection task, which studies event arguments beyond sentence boundaries.The addition of cross-sentence argument candidates imposes great challenges for modeling.To reduce the number of candidates, we adopt a two-step approach, decomposing the problem into two sub-problems: argument head-word detection and head-to-span expansion.Evaluated on the recent RAMS dataset (Ebner et al., 2020), our model achieves overall better performance than a strong sequence labeling baseline.We further provide detailed error analysis, presenting where the model mainly makes errors and indicating directions for future improvements.It remains a challenge to detect implicit arguments, calling for more future work of document-level modeling for this task.