Reviewerly/exHarmony huggingface dataset:
Shonan Rotation Averaging: Global Optimality by Surfing $$SO(p)^n$$

Shonan Rotation Averaging is a fast, simple, and elegant rotation averaging algorithm that is guaranteed to recover globally optimal solutions under mild assumptions on the measurement noise. Our method employs semidefinite relaxation in order to recover provably globally optimal solutions of the rotation averaging problem. In contrast to prior work, we show how to solve large-scale instances of these relaxations using manifold minimization on (only slightly) higher-dimensional rotation manifolds, re-using existing high-performance (but local) structure-from-motion pipelines. Our method thus preserves the speed and scalability of current SFM methods, while recovering globally optimal solutions.
Validation of cluster analysis results on validation data: A systematic framework

Abstract Cluster analysis refers to a wide range of data analytic techniques for class discovery and is popular in many application fields. To assess the quality of a clustering result, different cluster validation procedures have been proposed in the literature. While there is extensive work on classical validation techniques, such as internal and external validation, less attention has been given to validating and replicating a clustering result using a validation dataset. Such a dataset may be part of the original dataset, which is separated before analysis begins, or it could be an independently collected dataset. We present a systematic, structured review of the existing literature about this topic. For this purpose, we outline a formal framework that covers most existing approaches for validating clustering results on validation data. In particular, we review classical validation techniques such as internal and external validation, stability analysis, and visual validation, and show how they can be interpreted in terms of our framework. We define and formalize different types of validation of clustering results on a validation dataset, and give examples of how clustering studies from the applied literature that used a validation dataset can be seen as instances of our framework. This article is categorized under: Technologies &gt; Structure Discovery and Clustering Algorithmic Development &gt; Statistics Technologies &gt; Machine Learning
Review of the role of big data and digital technologies in controlling COVID-19 in Asia: Public health interest vs. privacy

Asia has been at the forefront of leveraging big data and digital technologies to strengthen measures against SARS-CoV-2 spread. Understanding strengths and challenges of these new approaches is important to inform improvements and implementation. In this review, we aimed to explore how these tools were utilized in four countries in Asia to facilitate COVID-19 preventative control measures.We conducted a pragmatic review of English-language literature and web-based information in Pubmed, MedRxiv, national and international public health institution websites and media sources between 1st January-3rd August 2020 to identify examples of big data and digital technologies to facilitate COVID-19 preventative control measures in Taiwan, South Korea, Hong Kong, and Singapore. Results were summarized narratively by common technological themes, and examples of integration highlighted.Digital tools implemented included real-time epidemiological dashboards, interactive maps of case location, mobile apps for tracing patients' contacts and geofencing to monitor quarantine compliance. Examples of integration of tools included linkage of national health and immigration databases to identify high-risk individuals in Taiwan, and the use of multiple digital surveillance sources to map patients' movements in South Korea. Challenges in balancing privacy and public good were identified.Digital technologies have facilitated and strengthened traditional public health measures for prevention of SARS-CoV-2 spread in Asia. Resolving issues around privacy concerns would improve future preparedness, implementation speed and uptake of digital measures. The significant technological advances and lessons learned can be adopted or adapted by other countries to ensure public health preparedness for future waves of COVID-19 and other pandemics.
HAIR: Hierarchical Visual-Semantic Relational Reasoning for Video Question Answering

Relational reasoning is at the heart of video question answering. However, existing approaches suffer from several common limitations: (1) they only focus on either object-level or frame-level relational reasoning, and fail to integrate the both; and (2) they neglect to leverage semantic knowledge for relational reasoning. In this work, we propose a Hierarchical VisuAl-Semantic RelatIonal Reasoning (HAIR) framework to address these limitations. Specifically, we present a novel graph memory mechanism to perform relational reasoning, and further develop two types of graph memory: a) visual graph memory that leverages visual information of video for relational reasoning; b) semantic graph memory that is specifically designed to explicitly leverage semantic knowledge contained in the classes and attributes of video objects, and perform relational reasoning in the semantic space. Taking advantage of both graph memory mechanisms, we build a hierarchical framework to enable visual-semantic relational reasoning from object level to frame level. Experiments on four challenging benchmark datasets show that the proposed framework leads to state-of-the-art performance, with fewer parameters and faster inference speed. Besides, our approach also shows superior performance on other video+language task.
Secure Cloud Infrastructure: A Survey on Issues, Current Solutions, and Open Challenges

Cloud computing is currently becoming a well-known buzzword in which business titans, such as Microsoft, Amazon, and Google, among others, are at the forefront in developing and providing sophisticated cloud computing systems to their users in a cost-effective manner. Security is the biggest concern for cloud computing and is a major obstacle to users adopting cloud computing systems. Maintaining the security of cloud computing is important, especially for the infrastructure. Several research works have been conducted in the cloud infrastructure security area; however, some gaps have not been completely addressed, while new challenges continue to arise. This paper presents a comprehensive survey of the security issues at different cloud infrastructure levels (e.g., application, network, host, and data). It investigates the most prominent issues that may affect the cloud computing business model with regard to infrastructure. It further discusses the current solutions proposed in the literature to mitigate the different security issues at each level. To assist in solving the issues, the challenges that are still unsolved are summarized. Based on the exploration of the current challenges, some cloud features such as flexibility, elasticity and the multi-tenancy are found to pose new challenges at each infrastructure level. More specifically, the multi-tenancy is found to have the most impact at all infrastructure levels, as it can lead to several security problems such as unavailability, abuse, data loss and privacy breach. This survey concludes by giving some recommendations for future research.
Multi-Modal Sarcasm Detection with Interactive In-Modal and Cross-Modal Graphs

Sarcasm is a peculiar form and sophisticated linguistic act to express the incongruity of someone's implied sentiment expression, which is a pervasive phenomenon in social media platforms. Compared with sarcasm detection purely on texts, multi-modal sarcasm detection is more adapted to the rapidly growing social media platforms, where people are interested in creating multi-modal messages. When focusing on the multi-modal sarcasm detection for tweets consisting of texts and images on Twitter, the significant clue of improving the performance of multi-modal sarcasm detection evolves into how to determine the incongruity relations between texts and images. In this paper, we investigate multi-modal sarcasm detection from a novel perspective, so as to determine the sentiment inconsistencies within a certain modality and across different modalities by constructing heterogeneous in-modal and cross-modal graphs (InCrossMGs) for each multi-modal example. Based on it, we explore an interactive graph convolution network (GCN) structure to jointly and interactively learn the incongruity relations of in-modal and cross-modal graphs for determining the significant clues in sarcasm detection. Experimental results demonstrate that our proposed model achieves state-of-the-art performance in multi-modal sarcasm detection.
Heterogeneously integrated, superconducting silicon-photonic platform for measurement-device-independent quantum key distribution

Integrated photonics provides a route to both miniaturization of quantum key distribution (QKD) devices and enhancing their performance. A key element for achieving discrete-variable QKD is a single-photon detector. It is highly desirable to integrate detectors onto a photonic chip to enable the realization of practical and scalable quantum networks. We realize a heterogeneously integrated, superconducting silicon-photonic chip. Harnessing the unique high-speed feature of our optical waveguide-integrated superconducting detector, we perform the first optimal Bell-state measurement (BSM) of time-bin encoded qubits generated from two independent lasers. The optimal BSM enables an increased key rate of measurement-device-independent QKD (MDI-QKD), which is immune to all attacks against the detection system and hence provides the basis for a QKD network with untrusted relays. Together with the time-multiplexed technique, we have enhanced the sifted key rate by almost one order of magnitude. With a 125-MHz clock rate, we obtain a secure key rate of 6.166 kbps over 24.0 dB loss, which is comparable to the state-of-the-art MDI-QKD experimental results with a GHz clock rate. Combined with integrated QKD transmitters, a scalable, chip-based, and cost-effective QKD network should become realizable in the near future.
PillarNet: Real-Time and High-Performance Pillar-Based 3D Object Detection

Real-time and high-performance 3D object detection is of critical importance for autonomous driving. Recent top-performing 3D object detectors mainly rely on point-based or 3D voxel-based convolutions, which are both computationally inefficient for onboard deployment. In contrast, pillar-based methods use solely 2D convolutions, which consume less computation resources, but they lag far behind their voxel-based counterparts in detection accuracy. In this paper, by examining the primary performance gap between pillar- and voxel-based detectors, we develop a real-time and high-performance pillar-based detector, dubbed PillarNet. The proposed PillarNet consists of a powerful encoder network for effective pillar feature learning, a neck network for spatial-semantic feature fusion and the commonly used detect head. Using only 2D convolutions, PillarNet is flexible to an optional pillar size and compatible with classical 2D CNN backbones, such as VGGNet and ResNet. Additionally, PillarNet benefits from our designed orientation-decoupled IoU regression loss along with the IoU-aware prediction branch. Extensive experimental results on the large-scale nuScenes Dataset and Waymo Open Dataset demonstrate that the proposed PillarNet performs well over state-of-the-art 3D detectors in terms of effectiveness and efficiency. Code is available at https://github.com/VISION-SJTU/PillarNet .
ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic

Recent text-to-image matching models apply contrastive learning to large corpora of uncurated pairs of images and sentences. While such models can provide a powerful score for matching and subsequent zero-shot tasks, they are not capable of generating caption given an image. In this work, we repurpose such models to generate a descriptive text given an image at inference time, without any further training or tuning step. This is done by combining the visual-semantic model with a large language model, benefiting from the knowledge in both web-scale models. The resulting captions are much less restrictive than those obtained by supervised captioning methods. Moreover, as a zero-shot learning method, it is extremely flexible and we demonstrate its ability to perform image arithmetic in which the inputs can be either images or text and the output is a sentence. This enables novel high-level vision capabilities such as comparing two images or solving visual analogy tests. Our code is available at: https://github.com/YoadTew/zero-shot-image-to-text.
Superresolution beyond the diffraction limit using phase spatial light modulator between incoherently illuminated objects and the entrance of an imaging system

We present a superresolution technique for imaging objects beyond the diffraction limit imposed by the limited numerical aperture (NA) of a general optical system. A coded phase mask (CPM) displayed on a spatial light modulator is introduced between the object and the input aperture of an ordinary lens-based imaging system. Consequently, the effective NA is increased beyond the inherent NA of the optical imaging system. Unlike conventional systems, the imaging in our proposed method is not direct from an object to a sensor, and the system requires a one-time calibration. In the calibration mode, a point object is mounted in the object plane, and the point spread intensity pattern is recorded. Following the calibration, the system is ready for imaging an arbitrary number of 2D objects. The intensity pattern from any object placed at the same axial location of the point object, and modulated by the same CPM, is recorded once by a digital camera. The superresolved image of the object is reconstructed by a nonlinear cross-correlation between the abovementioned two intensity patterns. The effective NA and the new resolution limit can be tuned by changing the scattering degree of the CPM.
Correlation Coefficients and Semantic Textual Similarity

Vitalii Zhelezniak, Aleksandar Savkov, April Shen, Nils Hammerla. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
Investigation and analysis of research support services in academic libraries

Purpose This paper aims at understanding the current situation of research support services offered by academic libraries in world-leading universities and providing useful implications and insights for other academic libraries. Design/methodology/approach Of the top 100 universities listed in the QS World University Rankings in 2017, 76 libraries were selected as samples and a website investigation was conducted to explore the research support services. The statistical method and visualization software was used to generalize the key services, and the text analysis and case analysis were applied to reveal the corresponding implementation. Findings Research support service has become one of the significant services of academic libraries in the context of e-research and data-intensive research. The research support services can be generally divided into seven aspects, as follows: research data management (62, 81.58 per cent), open access (64, 84.21 per cent), scholarly publishing (59, 77.63 per cent), research impact measurement (32, 42.11 per cent), research guides (47, 61.84 per cent), research consultation (59, 77.63 per cent) and research tools recommendation (38, 50.00 per cent). Originality/value This paper makes a comprehensive investigation of research support services in academic libraries of top-ranking universities worldwide. The findings will help academic libraries improve research support services; thus, advancing the work of researchers and promoting scientific discovery.
Research on Data Encryption Standard Based on AES Algorithm in Internet of Things Environment

The Internet of Things makes it possible to exchange information between people, things and things, people and things, and makes information security more and more important. However, there are few standards in the field of information security of the Internet of Things. Therefore, based on the characteristics and application requirements of the Internet of Things environment, this paper optimizes the Advanced Encryption Standard (AES) and builds the data encryption standard DESI(Data Encryption Standard in IoT) in the Internet of Things environment.. The analysis shows that DESI has better security and is therefore suitable for encrypting data in the IoT environment. Keywords: Internet of Things, encryption standard, AES, DESI, information security.
Mixed Supervised Object Detection with Robust Objectness Transfer

In this paper, we consider the problem of leveraging existing fully labeled categories to improve the weakly supervised detection (WSD) of new object categories, which we refer to as mixed supervised detection (MSD). Different from previous MSD methods that directly transfer the pre-trained object detectors from existing categories to new categories, we propose a more reasonable and robust objectness transfer approach for MSD. In our framework, we first learn domain-invariant objectness knowledge from the existing fully labeled categories. The knowledge is modeled based on invariant features that are robust to the distribution discrepancy between the existing categories and new categories; therefore the resulting knowledge would generalize well to new categories and could assist detection models to reject distractors (e.g., object parts) in weakly labeled images of new categories. Under the guidance of learned objectness knowledge, we utilize multiple instance learning (MIL) to model the concepts of both objects and distractors and to further improve the ability of rejecting distractors in weakly labeled images. Our robust objectness transfer approach outperforms the existing MSD methods, and achieves state-of-the-art results on the challenging ILSVRC2013 detection dataset and the PASCAL VOC datasets.
Invert to Learn to Invert

Iterative learning to infer approaches have become popular solvers for inverse problems. However, their memory requirements during training grow linearly with model depth, limiting in practice model expressiveness. In this work, we propose an iterative inverse model with constant memory that relies on invertible networks to avoid storing intermediate activations. As a result, the proposed approach allows us to train models with 400 layers on 3D volumes in an MRI image reconstruction task. In experiments on a public data set, we demonstrate that these deeper, and thus more expressive, networks perform state-of-the-art image reconstruction.
Making AI Forget You: Data Deletion in Machine Learning

Intense recent discussions have focused on how to provide individuals with control over when their data can and cannot be used --- the EU’s Right To Be Forgotten regulation is an example of this effort. In this paper we initiate a framework studying what to do when it is no longer permissible to deploy models derivative from specific user data. In particular, we formulate the problem of efficiently deleting individual data points from trained machine learning models. For many standard ML models, the only way to completely remove an individual's data is to retrain the whole model from scratch on the remaining data, which is often not computationally practical. We investigate algorithmic principles that enable efficient data deletion in ML. For the specific setting of $k$-means clustering, we propose two provably deletion efficient algorithms which achieve an average of over $100\times$ improvement in deletion efficiency across 6 datasets, while producing clusters of comparable statistical quality to a canonical $k$-means++ baseline.
A Deep Architecture for Multimodal Summarization of Soccer Games

The massive growth of sports videos, specially in soccer, has resulted in a need for the automatic generation of summaries, where the objective is not only to show the most important actions of the match but also to elicit as much emotion as the ones bring upon by human editors. State-of-the-art methods on video summarization mostly rely on video processing, however this is not an optimal approach for long videos such as soccer matches. In this paper we propose a multimodal approach to automatically generate summaries of soccer match videos that consider both event and audio features. The event features get a shorter and better representation of the match, and the audio helps detect the excitement generated by the game. Our method consists of three consecutive stages: Proposals, Summarization and Content Refinement. The first one generates summary proposals, using Multiple Instance Learning to deal with the similarity between the events inside the summary and the rest of the match. The Summarization stage uses event and audio features as input of a hierarchical Recurrent Neural Network to decide which proposals should indeed be in the summary. And the last stage, takes advantage of the visual content to create the final summary. The results show that our approach outperforms by a large margin not only the video processing methods but also methods that use event and audio features.
BoTorch: Programmable Bayesian Optimization in PyTorch.

Bayesian optimization provides sample-efficient global optimization for a broad range of applications, including automatic machine learning, engineering, physics, and experimental design. We introduce BoTorch, a modern programming framework for Bayesian optimization that combines Monte-Carlo (MC) acquisition functions, a novel sample average approximation optimization approach, auto-differentiation, and variance reduction techniques. BoTorch's modular design facilitates flexible specification and optimization of probabilistic models written in PyTorch, simplifying implementation of new acquisition functions. Our approach is backed by novel theoretical convergence results and made practical by a distinctive algorithmic foundation that leverages fast predictive distributions, hardware acceleration, and deterministic optimization. In experiments, we demonstrate the improved sample efficiency of BoTorch relative to other popular libraries.
HawKEY: Efficient and Versatile Text Entry for Virtual Reality

Text entry is still a challenging task in modern Virtual Reality (VR) systems. The lack of efficient text entry methods limits the applications that can be used productively in VR. Previous work has addressed this issue through virtual keyboards or showing the physical keyboard in VR. While physical keyboards afford faster text entry, they usually require a seated user and an instrumented environment. We introduce a new keyboard, worn on a hawker’s tray in front of the user, which affords a compact, simple, flexible, and efficient text entry solution for VR, without restricting physical movement. In our new video condition, we also show the keyboard only when the user is looking down at it. To evaluate our novel solution and to identify good keyboard visualizations, we ran a user study where we asked participants to enter both lowercase sentences as well as complex text while standing. The results show that text entry rates are affected negatively by simplistic keyboard visualization conditions and that our solution affords desktop text entry rates, even when standing.