Reviewerly/exHarmony huggingface dataset:
Proposal-free Temporal Moment Localization of a Natural-Language Query in Video using Guided Attention

This paper studies the problem of temporal moment localization in a long untrimmed video using natural language as the query.Given an untrimmed video and a query sentence, the goal is to determine the start and end of the relevant visual moment in the video that corresponds to the query sentence.While most previous works have tackled this by a propose-and-rank approach, we introduce a more efficient, end-to-end trainable, and proposal-free approach that is built upon three key components: a dynamic filter which adaptively transfers language information to visual domain attention map, a new loss function to guide the model to attend the most relevant part of the video, and soft labels to cope with annotation uncertainties.Our method is evaluated on three standard benchmark datasets, Charades-STA, TACoS and ActivityNet-Captions. Experimental results show our method outperforms state-of-theart methods on these datasets, confirming the effectiveness of the method.We believe the proposed dynamic filter-based guided attention mechanism will prove valuable for other vision and language tasks as well.
Simulation of performance differences between offshore and land‐based photovoltaic systems

Abstract The purpose of this study is to model, simulate, and compare the performance of a photovoltaics system on land and at sea. To be able to have a fair comparison the effect of sea waves, wind speed and relative humidity are considered in this model. The sea waves are modeled in the frequency domain, using a wave spectrum. The irradiation on a tilted surface for a floating system is calculated considering the tilt angle that is affected by the sea waves. Moreover, the temperature is estimated based on heat transfer theory and the natural cooling system for both floating and land‐based photovoltaic systems. Actual measured weather data from two different locations, one located at Utrecht University campus and the other one on the North Sea, are used to simulate the systems, thus making the comparison possible. Energy yield is calculated for these weather conditions. The results show that the relative annual average output energy is about 12.96% higher at sea compared with land. However, in some months, this relative output energy increases up to 18% higher energy yield at sea.
The smart city as time-space cartographer in COVID-19 control: the South Korean strategy and democratic control of surveillance technology

While the US, UK, France, Italy, and many other countries ended up implementing complete lockdown after tens of thousands of deaths from COVID-19, South Korea kept factories and offices running, flattened the curve, and maintained a low mortality rate. Extensive media coverage has focused on South Korea's testing capacity as the primary reason, but there has been little discussion of the vital role of the smart city. In this paper, we describe how smart city technologies form a crucial part of disease control in South Korea, explain the social conditions for the extensive use of smart city technology, and offer critical insights into contemporary discussions on the issue of smart cities and surveillance.
Security and Trust in Blockchains: Architecture, Key Technologies, and Open Issues

As a new promising distributed technology, blockchains have been widely applied since its inception. Its decentralization feature reduces the reliance on the trusted authorities and third parties. It can well solve the problem of data being tampered and increase data sharing. However, a blockchain system faces various security and trust issues, such as attacks against consensus mechanisms and propagation processes, which may make it store malicious information or delay data propagation. The work discusses the basic architecture of blockchains as well as its potential security and trust issues at data, network, consensus, smart contract, and application layers. Then, the related literature work is analyzed in terms of the issues at these layers. Some open issues are presented and discussed.
Easing Into Open Science: A Guide for Graduate Students and Their Advisors

This article provides a roadmap to assist graduate students and their advisors to engage in open science practices. We suggest eight open science practices that novice graduate students could begin adopting today. The topics we cover include journal clubs, project workflow, preprints, reproducible code, data sharing, transparent writing, preregistration, and registered reports. To address concerns about not knowing how to engage in open science practices, we provide a difficulty rating of each behavior (easy, medium, difficult), present them in order of suggested adoption, and follow the format of what, why, how, and worries. We give graduate students ideas on how to approach conversations with their advisors/collaborators, ideas on how to integrate open science practices within the graduate school framework, and specific resources on how to engage with each behavior. We emphasize that engaging in open science behaviors need not be an all or nothing approach, but rather graduate students can engage with any number of the behaviors outlined.
Cybercrime in America amid COVID-19: the Initial Results from a Natural Experiment

The COVID-19 pandemic has radically altered life, killing hundreds of thousands of people and leading many countries to issue "stay-at-home" orders to contain the virus's spread. Based on insights from routine activity theory (Cohen & Felson 1979), it is likely that COVID-19 will influence victimization rates as people alter their routines and spend more time at home and less time in public. Yet, the pandemic may affect victimization differently depending on the type of crime as street crimes appear to be decreasing while domestic crimes may be increasing. We consider a third type of crime: cybercrime. Treating the pandemic as a natural experiment, we investigate how the pandemic has affected rates of cybervictimization. We compare pre-pandemic rates of victimization with post-pandemic rates of victimization using datasets designed to track cybercrime. After considering how the pandemic may alter routines and affect cybervictimization, we find that the pandemic has not radically altered cyberroutines nor changed cybervictimization rates. However, a model using routine activity theory to predict cybervictimization offers clear support for the theory's efficacy both before and after the pandemic. We conclude by considering plausible explanations for our findings.
Hierarchical Human Parsing With Typed Part-Relation Reasoning

Human parsing is for pixel-wise human semantic understanding. As human bodies are underlying hierarchically structured, how to model human structures is the central theme in this task. Focusing on this, we seek to simultaneously exploit the representational capacity of deep graph networks and the hierarchical human structures. In particular, we provide following two contributions. First, three kinds of part relations, i.e., decomposition, composition, and dependency, are, for the first time, completely and precisely described by three distinct relation networks. This is in stark contrast to previous parsers, which only focus on a portion of the relations and adopt a type-agnostic relation modeling strategy. More expressive relation information can be captured by explicitly imposing the parameters in the relation networks to satisfy the specific characteristics of different relations. Second, previous parsers largely ignore the need for an approximation algorithm over the loopy human hierarchy, while we instead address an iterative reasoning process, by assimilating generic message-passing networks with their edge-typed, convolutional counterparts. With these efforts, our parser lays the foundation for more sophisticated and flexible human relation patterns of reasoning. Comprehensive experiments on five datasets demonstrate that our parser sets a new state-of-the-art on each.
SketchyCOCO: Image Generation From Freehand Scene Sketches

We introduce the first method for automatic image generation from scene-level freehand sketches. Our model allows for controllable image generation by specifying the synthesis goal via freehand sketches. The key contribution is an attribute vector bridged Generative Adversarial Network called EdgeGAN, which supports high visual-quality object-level image content generation without using freehand sketches as training data. We have built a large-scale composite dataset called SketchyCOCO to support and evaluate the solution. We validate our approach on the tasks of both object-level and scene-level image generation on SketchyCOCO. Through quantitative, qualitative results, human evaluation and ablation studies, we demonstrate the method's capacity to generate realistic complex scene-level images from various freehand sketches.
FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal Retrieval

In this paper, we address the text and image matching in cross-modal retrieval of the fashion industry. Different from the matching in the general domain, the fashion matching is required to pay much more attention to the fine-grained information in the fashion images and texts. Pioneer approaches detect the region of interests (i.e., RoIs) from images and use the RoI embeddings as image representations. In general, RoIs tend to represent the "object-level" information in the fashion images, while fashion texts are prone to describe more detailed information, e.g. styles, attributes. RoIs are thus not fine-grained enough for fashion text and image matching. To this end, we propose FashionBERT, which leverages patches as image features. With the pre-trained BERT model as the backbone network, FashionBERT learns high level representations of texts and images. Meanwhile, we propose an adaptive loss to trade off multitask learning in the FashionBERT modeling. Two tasks (i.e., text and image matching and cross-modal retrieval) are incorporated to evaluate FashionBERT. On the public dataset, experiments demonstrate FashionBERT achieves significant improvements in performances than the baseline and state-of-the-art approaches. In practice, FashionBERT is applied in a concrete cross-modal retrieval application. We provide the detailed matching performance and inference efficiency analysis.
HDR-GAN: HDR Image Reconstruction From Multi-Exposed LDR Images With Large Motions

Synthesizing high dynamic range (HDR) images from multiple low-dynamic range (LDR) exposures in dynamic scenes is challenging. There are two major problems caused by the large motions of foreground objects. One is the severe misalignment among the LDR images. The other is the missing content due to the over-/under-saturated regions caused by the moving objects, which may not be easily compensated for by the multiple LDR exposures. Thus, it requires the HDR generation model to be able to properly fuse the LDR images and restore the missing details without introducing artifacts. To address these two problems, we propose in this paper a novel GAN-based model, HDR-GAN, for synthesizing HDR images from multi-exposed LDR images. To our best knowledge, this work is the first GAN-based approach for fusing multi-exposed LDR images for HDR reconstruction. By incorporating adversarial learning, our method is able to produce faithful information in the regions with missing content. In addition, we also propose a novel generator network, with a reference-based residual merging block for aligning large object motions in the feature domain, and a deep HDR supervision scheme for eliminating artifacts of the reconstructed HDR images. Experimental results demonstrate that our model achieves state-of-the-art reconstruction performance over the prior HDR methods on diverse scenes.
The Visual Social Distancing Problem

One of the main and most effective measures to contain the recent viral outbreak is the maintenance of the so-called Social Distancing (SD). To comply with this constraint, governments are adopting restrictions over the minimum inter-personal distance between people. Given this actual scenario, it is crucial to massively measure the compliance to such physical constraint in our life, in order to figure out the reasons of the possible breaks of such distance limitations, and understand if this implies a potential threat. To this end, we introduce the Visual Social Distancing (VSD) problem, defined as the automatic estimation of the inter-personal distance from an image, and the characterization of related people aggregations. VSD is pivotal for a non-invasive analysis to whether people comply with the SD restriction, and to provide statistics about the level of safety of specific areas whenever this constraint is violated. We first point out that measuring VSD is not only a geometrical problem, but it also implies a deeper understanding of the social behaviour in the scene. The aim is to truly detect potentially dangerous situations while avoiding false alarms (e.g., a family with children or relatives, an elder with their caregivers), all of this by complying with current privacy policies. We then discuss how VSD relates with previous literature in Social Signal Processing and indicate a path to research new Computer Vision methods that can possibly provide a solution to such problem. We conclude with future challenges related to the effectiveness of VSD systems, ethical implications and future application scenarios.
Blockchain-Based Sharing and Tamper-Proof Framework of Big Data Networking

With the development of information technology, big data has had a positive impact on the development of various industries. However, the continued growth of data has led to difficulties in data sharing. Blockchain, with its information sharing characteristics and decentralized management principles, is expected to become the new engine for building a future data sharing platform. Blockchain-based data sharing is useful for recording user transactions and sharing of multi-platform data. However, this technology not only has challenges in information storage, but also risks in third-party transactions. In this article, we propose a big data sharing and transaction framework based on blockchain. This framework uses the blockchain decentralization and openness idea to establish a big data transaction platform for multiple users. For the problem of distributed secure storage of big data in the blockchain, we propose a data-tamperproof mechanism that introduces a cryptographic algorithm to prevent transaction data from being tampered with during user storage. The purpose of this article is to ensure transaction security and data reliability when trading in the blockchain.
Automatic Detection of White Blood Cancer From Bone Marrow Microscopic Images Using Convolutional Neural Networks

Leukocytes, produced in the bone marrow, make up around one percent of all blood cells. Uncontrolled growth of these white blood cells leads to the birth of blood cancer. Out of the three different types of cancers, the proposed study provides a robust mechanism for the classification of Acute Lymphoblastic Leukemia (ALL) and Multiple Myeloma (MM) using the SN-AM dataset. Acute lymphoblastic leukemia (ALL) is a type of cancer where the bone marrow forms too many lymphocytes. On the other hand, Multiple myeloma (MM), a different kind of cancer, causes cancer cells to accumulate in the bone marrow rather than releasing them into the bloodstream. Therefore, they crowd out and prevent the production of healthy blood cells. Conventionally, the process was carried out manually by a skilled professional in a considerable amount of time. The proposed model eradicates the probability of errors in the manual process by employing deep learning techniques, namely convolutional neural networks. The model, trained on cells’ images, first pre-processes the images and extracts the best features. This is followed by training the model with the optimized Dense Convolutional neural network framework (termed DCNN here) and finally predicting the type of cancer present in the cells. The model was able to reproduce all the measurements correctly while it recollected the samples exactly 94 times out of 100. The overall accuracy was recorded to be 97.2%, which is better than the conventional machine learning methods like Support Vector Machine (SVMs), Decision Trees, Random Forests, Naive Bayes, etc. This study indicates that the DCNN model’s performance is close to that of the established CNN architectures with far fewer parameters and computation time tested on the retrieved dataset. Thus, the model can be used effectively as a tool for determining the type of cancer in the bone marrow.
A divide-and-conquer algorithm for quantum state preparation

Advantages in several fields of research and industry are expected with the rise of quantum computers. However, the computational cost to load classical data in quantum computers can impose restrictions on possible quantum speedups. Known algorithms to create arbitrary quantum states require quantum circuits with depth O(N) to load an N-dimensional vector. Here, we show that it is possible to load an N-dimensional vector with a quantum circuit with polylogarithmic depth and entangled information in ancillary qubits. Results show that we can efficiently load data in quantum devices using a divide-and-conquer strategy to exchange computational time for space. We demonstrate a proof of concept on a real quantum device and present two applications for quantum machine learning. We expect that this new loading strategy allows the quantum speedup of tasks that require to load a significant volume of information to quantum devices.
EnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networks

Graph neural networks (GNNs) emerge as a powerful approach to process non-euclidean data structures and have been proved powerful in various application domains such as social networks and e-commerce. While such graph data maintained in real-world systems can be extremely large and sparse, thus employing GNNs to deal with them requires substantial computational and memory overhead, which induces considerable energy and resource cost on CPUs and GPUs. In this article, we present a specialized accelerator architecture, EnGN, to enable high-throughput and energy-efficient processing of large-scale GNNs. The proposed EnGN is designed to accelerate the three key stages of GNN propagation, which is abstracted as common computing patterns shared by typical GNNs. To support the key stages simultaneously, we propose the ring-edge-reduce(RER) dataflow that tames the poor locality of sparsely-and-randomly connected vertices, and the RER PE-array to practice RER dataflow. In addition, we utilize a graph tiling strategy to fit large graphs into EnGN and make good use of the hierarchical on-chip buffers through adaptive computation reordering and tile scheduling. Overall, EnGN achieves performance speedup by 1802.9X, 19.75X, and 2.97X and energy efficiency by 1326.35X, 304.43X, and 6.2X on average compared to CPU, GPU, and a state-of-the-art GCN accelerator HyGCN, respectively.
Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms

In several scientific and industrial applications, it is desirable to build compact, interpretable learning models where the output depends on a small number of input features. Recent work has shown that such best-subset selection-type problems can be solved with modern mixed integer optimization solvers. Despite their promise, such solvers often come at a steep computational price when compared with open-source, efficient specialized solvers based on convex optimization and greedy heuristics. In “Fast Best-Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms,” Hussein Hazimeh and Rahul Mazumder push the frontiers of computation for best-subset-type problems. Their algorithms deliver near-optimal solutions for problems with up to a million features—in times comparable with the fast convex solvers. Their work suggests that principled optimization methods play a key role in devising tools central to interpretable machine learning, which can help in gaining a deeper understanding of their statistical properties.
Foundations of population-based SHM, Part III: Heterogeneous populations – Mapping and transfer

This is the third and final paper in a series laying foundations for a theory/methodology of Population-Based Structural Health Monitoring (PBSHM). PBSHM involves utilising knowledge from one set of structures in a population and applying it to a different set, such that predictions about the health states of each member in the population can be performed and improved. Central ideas behind PBSHM are those of knowledge transfer and mapping. In the context of PBSHM, knowledge transfer involves using information from a source domain structure, where labels are known for given feature sets, and mapping these onto the unlabelled feature space of a different, target domain structure. This mapping means a classifier trained on the transformed source domain data will generalise to the unlabelled target domain data; i.e. a classifier built on one structure will generalise to another, making Structural Heath Monitoring (SHM) cost-effective and applicable to a wide range of challenging industrial scenarios. This process of mapping features and labels across source and target domains is defined here via domain adaptation, a subcategory of transfer learning. A mathematical underpinning for when domain adaptation is possible in a structural dynamics context is provided, with reference to topology within a graphical representation of structures. Subsequently, a novel procedure for performing domain adaptation on topologically different structures is outlined.
Identifying normal mammograms in a large screening population using artificial intelligence

To evaluate the potential of artificial intelligence (AI) to identify normal mammograms in a screening population.In this retrospective study, 9581 double-read mammography screening exams including 68 screen-detected cancers and 187 false positives, a subcohort of the prospective population-based Malmö Breast Tomosynthesis Screening Trial, were analysed with a deep learning-based AI system. The AI system categorises mammograms with a cancer risk score increasing from 1 to 10. The effect on cancer detection and false positives of excluding mammograms below different AI risk thresholds from reading by radiologists was investigated. A panel of three breast radiologists assessed the radiographic appearance, type, and visibility of screen-detected cancers assigned low-risk scores (≤ 5). The reduction of normal exams, cancers, and false positives for the different thresholds was presented with 95% confidence intervals (CI).If mammograms scored 1 and 2 were excluded from screen-reading, 1829 (19.1%; 95% CI 18.3-19.9) exams could be removed, including 10 (5.3%; 95% CI 2.1-8.6) false positives but no cancers. In total, 5082 (53.0%; 95% CI 52.0-54.0) exams, including 7 (10.3%; 95% CI 3.1-17.5) cancers and 52 (27.8%; 95% CI 21.4-34.2) false positives, had low-risk scores. All, except one, of the seven screen-detected cancers with low-risk scores were judged to be clearly visible.The evaluated AI system can correctly identify a proportion of a screening population as cancer-free and also reduce false positives. Thus, AI has the potential to improve mammography screening efficiency.• Retrospective study showed that AI can identify a proportion of mammograms as normal in a screening population. • Excluding normal exams from screening using AI can reduce false positives.