Reviewerly/exHarmony huggingface dataset:
S4Net: Single stage salient-instance segmentation

Abstract In this paper, we consider salient instance segmentation. As well as producing bounding boxes, our network also outputs high-quality instance-level segments as initial selections to indicate the regions of interest. Taking into account the category-independent property of each target, we design a single stage salient instance segmentation framework, with a novel segmentation branch. Our new branch regards not only local context inside each detection window but also the surrounding context, enabling us to distinguish instances in the same scope even with partial occlusion. Our network is end-to-end trainable and is fast (running at 40 fps for images with resolution 320 × 320). We evaluate our approach on a publicly available benchmark and show that it outperforms alternative solutions. We also provide a thorough analysis of our design choices to help readers better understand the function of each part of our network. Source code can be found at https://github.com/RuochenFan/S4Net.
Smart Contract Definition for Land Registry in Blockchain

Land Registry System in India is a very time-consuming process involving many middlemen, thus increasing the fraudulent cases. These problems can be eliminated by the usage of Blockchain Technology for Land Registry Management in India. This paper highlights issues, such as minimal transparency, accountability, incoherent data sets with different Government Departments relating to the same portion of land and delays in the current Land registry management process. The paper describes the current process of land records maintenance and land registration in the country. It emphasizes on the importance of smart contract for land registry applying the Blockchain.
Learners' approaches, motivation and patterns of problem-solving on lines and angles in geometry using augmented reality

Abstract There are several concepts in 2D Geometry that require understanding their application in the real practical world. However, in classrooms, such concepts are often taught without the analysis of the learners’ realization and interpretation of the existing concepts around them, in their surroundings. For this purpose, an Augmented Reality (AR) based module for the 7th and 8th grade syllabus has been designed to encourage the active participation of the learners in the classroom while learning the concept of Lines and Angles. It comprises three AR learning activities that enable the participants to recall, visualize, and identify the type of angle and then mark it by drawing on the augmented 3D house. Before conducting the main studies, a pilot study was conducted with 6 students of 8th grade. This helped in validating the data instruments, timing, and execution of the research study. The first study was conducted with 21 students of 8th grade where 12 participants performed the AR learning activities in dyads and 9 participants performed individually. Their perspectives, approaches, and motivation in performing the AR learning activities have been reported. Findings from the study showed that the majority i.e. 90.4% participants preferred to perform the AR learning activities in dyads than individually. Though the usability score was higher for the participants who performed the AR learning activities individually (M = 70.28) as compared to dyads (M = 65.23), there was no significant difference in the motivation scores between the participants of the two groups. In the second study, 28 students of 7th grade were divided into dyads and their behavior patterns of performing the AR learning activities have been reported. Using Lag Sequential Analysis, significant sequences were obtained based on the behaviors belonging to three categories of peer involvement, teacher prompts and AR interactions. It was found that the designed AR learning activities encouraged the participants to discuss the concepts with peers, enhanced their immersive experience as they together moved around and inside the house to find and identify the angles.
Extractive Snippet Generation for Arguments

Snippets are used in web search to help users assess the relevance of retrieved results to their query. Recently, specialized search engines have arisen that retrieve pro and con arguments on controversial issues. We argue that standard snippet generation is insufficient to represent the core reasoning of an argument. In this paper, we introduce the task of generating a snippet that represents the main claim and reason of an argument. We propose a query-independent extractive summarization approach to this task that uses a variant of PageRank to assess the importance of sentences based on their context and argumentativeness. In both automatic and manual evaluation, our approach outperforms strong baselines.
From Two Rolling Shutters to One Global Shutter

Most consumer cameras are equipped with electronic rolling shutter, leading to image distortions when the camera moves during image capture. We explore a surprisingly simple camera configuration that makes it possible to undo the rolling shutter distortion: two cameras mounted to have different rolling shutter directions. Such a setup is easy and cheap to build and it possesses the geometric constraints needed to correct rolling shutter distortion using only a sparse set of point correspondences between the two images. We derive equations that describe the underlying geometry for general and special motions and present an efficient method for finding their solutions. Our synthetic and real experiments demonstrate that our approach is able to remove large rolling shutter distortions of all types without relying on any specific scene structure.
Vec2Face: Unveil Human Faces From Their Blackbox Features in Face Recognition

Unveiling face images of a subject given his/her high-level representations extracted from a blackbox Face Recognition engine is extremely challenging. It is because the limitations of accessible information from that engine including its structure and uninterpretable extracted features. This paper presents a novel generative structure with Bijective Metric Learning, namely Bijective Generative Adversarial Networks in a Distillation framework (DiBiGAN), for synthesizing faces of an identity given that person's features. In order to effectively address this problem, this work firstly introduces a bijective metric so that the distance measurement and metric learning process can be directly adopted in image domain for an image reconstruction task. Secondly, a distillation process is introduced to maximize the information exploited from the blackbox face recognition engine. Then a Feature-Conditional Generator Structure with Exponential Weighting Strategy is presented for a more robust generator that can synthesize realistic faces with ID preservation. Results on several benchmarking datasets including CelebA, LFW, AgeDB, CFP-FP against matching engines have demonstrated the effectiveness of DiBiGAN on both image realism and ID preservation properties.
ReDA:Reinforced Differentiable Attribute for 3D Face Reconstruction

The key challenge for 3D face shape reconstruction is to build the correct dense face correspondence between the deformable mesh and the single input image. Given the ill-posed nature, previous works heavily rely on prior knowledge (such as 3DMM [2]) to reduce depth ambiguity. Although impressive result has been made recently [42, 14, 8], there is still a large room to improve the correspondence so that projected face shape better aligns with the silhouette of each face region (i.e, eye, mouth, nose, cheek, etc.) on the image. To further reduce the ambiguities, we present a novel framework called “Reinforced Differentiable Attributes” (“ReDA”) which is more general and effective than previous Differentiable Rendering (“DR”). Specifically, we first extend from color to more broad attributes, including the depth and the face parsing mask. Secondly, unlike the previous Z-buffer rendering, we make the rendering to be more differentiable through a set of convolution operations with multi-scale kernel sizes. In the meanwhile, to make “ReDA” to be more successful for 3D face recon-struction, we further introduce a new free-form deformation layer that sits on top of 3DMM to enjoy both the prior knowledge and out-of-space modeling. Both techniques can be easily integrated into existing 3D face reconstruction pipeline. Extensive experiments on both RGB and RGB-D datasets show that our approach outperforms prior arts.
A novel image retrieval method based on multi-features fusion

Image retrieval is an important research topic in pattern recognition and computer vision. We propose a novel content-based image retrieval method. At the image description stage, our method first modifies the traditional micro-structure descriptor (MSD) to capture the direct relationship between the shape and texture features and that between the color and texture features. Our method then extracts the uniform local binary patterns (LBP) histogram of the image to capture the color difference information. At the image comparison stage, our method first compares the descriptors of the images to compute their similarities. The similarity between each pair of images is then updated by also considering the similarities to comparable images within the dataset. Accordingly, our method obtains the final similarities of the images. Experimental results demonstrate the effectiveness of our method for image retrieval.
Multi-Context And Enhanced Reconstruction Network For Single Image Super Resolution

Most existing single image super-resolution (SISR) methods continually increase the depth or width of networks, without adequately exploring contextual features which are essential for reconstruction. Moreover, such existing methods pay little attention to the final high-resolution(HR) image reconstruction step and therefore hinder the desired SR performance. In this paper, we propose a multi-context and enhanced reconstruction network (MCERN) for SISR. Specifically, a novel model named Multi-Context Block (MCB) which extracts more image contextual features with multibranch dilated convolution. Applying multiple MCBs with residual and dense connections, we can effectively extract contextual and hierarchical features for obtaining the coarse super-resolution result. Then an enhanced reconstruction block (ERB) is followed to extract essential spatial features on the high-resolution image to refine the coarse result to a better result. Extensive benchmark evaluations demonstrate the efficacy of our proposed MCERN in terms of metric accuracy and visual effects.
Footprints and Free Space From a Single Color Image

Understanding the shape of a scene from a single color image is a formidable computer vision task. However, most methods aim to predict the geometry of surfaces that are visible to the camera, which is of limited use when planning paths for robots or augmented reality agents. Such agents can only move when grounded on a traversable surface, which we define as the set of classes which humans can also walk over, such as grass, footpaths and pavement. Models which predict beyond the line of sight often parameterize the scene with voxels or meshes, which can be expensive to use in machine learning frameworks. We introduce a model to predict the geometry of both visible and occluded traversable surfaces, given a single RGB image as input. We learn from stereo video sequences, using camera poses, per-frame depth and semantic segmentation to form training data, which is used to supervise an image-to-image network. We train models from the KITTI driving dataset, the indoor Matterport dataset, and from our own casually captured stereo footage. We find that a surprisingly low bar for spatial coverage of training scenes is required. We validate our algorithm against a range of strong baselines, and include an assessment of our predictions for a path-planning task.
Least squares binary quantization of neural networks

Quantizing weights and activations of deep neural networks results in significant improvement in inference efficiency at the cost of lower accuracy. A source of the accuracy gap between full precision and quantized models is the quantization error. In this work, we focus on the binary quantization, in which values are mapped to -1 and 1. We provide a unified framework to analyze different scaling strategies. Inspired by the pareto-optimality of 2-bits versus 1-bit quantization, we introduce a novel 2-bits quantization with provably least squares error. Our quantization algorithms can be implemented efficiently on the hardware using bitwise operations. We present proofs to show that our proposed methods are optimal, and also provide empirical error analysis. We conduct experiments on the ImageNet dataset and show a reduced accuracy gap when using the proposed least squares quantization algorithms.
Institutional Isomorphism in Organizational Cybersecurity: A Text Analytics Approach

This research examines how organizational cybersecurity responses become isomorphic over time. Drawing on institutional theory, this study theorizes that mimetic pressures, normative pressures, and coercive pressures impact cybersecurity responses. Using the textual data gathered from the annual 10-K reports published by 87 large organizations and their competitors, topic modeling was employed to assess the cybersecurity responses and institutional pressures. Linear regression was applied to the resultant topic weights. Findings show that mimetic pressures were significant over time while coercive pressures were significant in the near-term and normative pressures were significant in the long-term. Implications for research and practice are discussed.
Constant factor approximations to edit distance on far input pairs in nearly linear time

For any T ≥ 1, there are constants R=R(T) ≥ 1 and ζ=ζ(T)>0 and a randomized algorithm that takes as input an integer n and two strings x,y of length at most n, and runs in time O(n 1+1/T ) and outputs an upper bound U on the edit distance of edit(x,y) that with high probability, satisfies U ≤ R(edit(x,y)+n 1−ζ). In particular, on any input with edit(x,y) ≥ n 1−ζ the algorithm outputs a constant factor approximation with high probability. A similar result has been proven independently by Brakensiek and Rubinstein (this proceedings).
Semi-supervised 3D Face Representation Learning from Unconstrained Photo Collections

Recovering 3D geometry shape, albedo, and lighting from a single image is a typical ill-posed problem. To address this challenging problem, we propose to utilize the joint constraints from unconstrained photo collections of one person to recover his or her identity shape and albedo. Unconstrained photo collections include one's photos captured under different times, backgrounds, and expressions, e.g., photos posted on Instagram. We train our model in a semi-supervised manner with adversarial loss to exploit large amounts of unconstrained facial images. A novel center loss is introduced to make sure that facial images from the same subject have the same identity shape and albedo. Besides, our proposed model disentangles identity, expression, pose, and lighting representations, which improves the overall reconstruction performance and facilitates facial editing applications, e.g., expression transfer. Comprehensive experiments demonstrate that our model produces high-quality reconstruction compared to state-of-the-art methods and is robust to various expression, pose, and lighting conditions.
Seeing the World in a Bag of Chips

We address the dual problems of novel view synthesis and environment reconstruction from hand-held RGBD sensors. Our contributions include 1) modeling highly specular objects, 2) modeling inter-reflections and Fresnel effects, and 3) enabling surface light field reconstruction with the same input needed to reconstruct shape alone. In cases where scene surface has a strong mirror-like material component, we generate highly detailed environment images, revealing room composition, objects, people, buildings, and trees visible through windows. Our approach yields state of the art view synthesis techniques, operates on low dynamic range imagery, and is robust to geometric and calibration errors.
Provable Tradeoffs in Adversarially Robust Classification

It is well known that machine learning methods can be vulnerable to adversarially-chosen perturbations of their inputs. Despite significant progress in the area, foundational open problems remain. In this paper, we address several key questions. We derive exact and approximate Bayes-optimal robust classifiers for the important setting of two- and three-class Gaussian classification problems with arbitrary imbalance, for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{\infty} $ </tex-math></inline-formula> adversaries. In contrast to classical Bayes-optimal classifiers, determining the optimal decisions here cannot be made pointwise and new theoretical approaches are needed. We develop and leverage new tools, including recent breakthroughs from probability theory on robust isoperimetry, which, to our knowledge, have not yet been used in the area. Our results reveal fundamental tradeoffs between standard and robust accuracy that grow when data is imbalanced. We also show further results, including an analysis of classification calibration for convex losses in certain models, and finite sample rates for the robust risk.
Accelerating Proximal Policy Optimization on CPU-FPGA Heterogeneous Platforms

Reinforcement Learning (RL) is a technique that enables an agent to learn to behave optimally by repeatedly interacting with the environment and receiving the rewards. RL is widely used in domains such as robotics, game playing and finance. Proximal Policy Optimization (PPO) is the state-of-the-art policy optimization algorithm which achieves superior overall performance on various RL benchmarks. PPO iteratively optimizes its policy - a function which chooses optimal actions, with each iteration consisting of two computationally intensive phases: Inference phase - agents infer actions to interact with the environment and collect data, and Training phase - agents train the policy using the collected data. In this work, we develop the first high-throughput PPO accelerator on CPU-FPGA heterogeneous platform, targeting both phases of the algorithm for acceleration. We implement a systolic-array based architecture coupled with a novel memory-blocked data layout that enables streaming data access in both forward and backward propagations to achieve high-throughput performance. Additionally, we develop a novel systolic array compute sharing technique to mitigate the potential load imbalance in the training of two networks. We develop an accurate performance model of our design, based on which we perform design space exploration to obtain optimal design points. Our design is evaluated on widely used robotics benchmarks, achieving $2.1 \times - 30.5 \times$ and $2 \times - 27.5 \times$ improvements in throughput against state-of-the-art CPU and CPU-GPU implementations, respectively.
Fashion Compatibility Modeling through a Multi-modal Try-on-guided Scheme

Recent years have witnessed a growing trend of fashion compatibility modeling, which scores the matching degree of the given outfit and then provides people with some dressing advice. Existing methods have primarily solved this problem by analyzing the discrete interaction among multiple complementary items. However, the fashion items would present certain occlusion and deformation when they are worn on the body. Therefore, the discrete item interaction cannot capture the fashion compatibility in a combined manner due to the neglect of a crucial factor: the overall try-on appearance. In light of this, we propose a multi-modal try-on-guided compatibility modeling scheme to jointly characterize the discrete interaction and try-on appearance of the outfit. In particular, we first propose a multi-modal try-on template generator to automatically generate a try-on template from the visual and textual information of the outfit, depicting the overall look of its composing fashion items. Then, we introduce a new compatibility modeling scheme which integrates the outfit try-on appearance into the traditional discrete item interaction modeling. To fulfill the proposal, we construct a large-scale real-world dataset from SSENSE, named FOTOS, consisting of 11,000 well-matched outfits and their corresponding realistic try-on images. Extensive experiments have demonstrated its superiority to state-of-the-arts.
Joint Learning of Blind Video Denoising and Optical Flow Estimation

Many deep-learning-based image/video denoising models have been developed, and recently, several approaches for training a denoising neural network without using clean images have been proposed. However, Noise2Noise method requires paired noisy data, and obtaining them is occasionally difficult, whereas other existing models trained using unpaired noisy data deliver limited performance. Obtaining an accurate optical flow from noisy videos is also a difficult task because conventional optical flow estimation methods are primarily focused on estimating the optical flow using clean videos. This study proposes a new framework to fine-tune video denoising and optical flow estimation networks using unpaired noisy videos. These two networks are jointly trained to realize synergy; an improvement in the denoising performance increases the accuracy of the flow estimation, and an improvement in the flow-estimation performance enhances the quality of the training data for the denoiser. Our experimental results reveal that proposed approach outperforms the existing training schemes in video denoising and also provides accurate optical flows even when the videos contain a considerable amount of noise.
Internet of things assisted public security management platform for urban transportation using hybridised cryptographic‐integrated steganography

In cloud computing, data sharing and security are considered as the significant challenging issues in the present era. Until recently, considering secure data sharing in the cloud, several problems have been taken into consideration, such as data security, privacy, data integrity, confidentiality and resource scheduling. In this study, the hybridised cryptographic-integrated steganography (HCIS) algorithm has been used with auxiliary data inputs for secured data sharing in an internet of things assisted cloud environment for the urban transportation system. Further, the cryptography or cryptology generates code that provides a high level of privacy to the group or individual who can access cloud data. Furthermore, cryptography converts data into a secure format that is readable by an authorised user and steganography helps to transmit secret data to avoid the detection of information. The confidential data is extracted at its destination, and the encryption key has been utilised for hiding or effectively protecting data. The experimental results show that efficient and secure data sharing with multi-owners in cloud computing has been achieved using the HCIS method.