OpenRaiser/Intern-Atlas huggingface dataset:
Curiosity-driven Intuitive Physics Learning
Venue: arXiv cs.AI 2021
Year: 2021

Biological infants are naturally curious and try to comprehend their physical surroundings by interacting, in myriad multisensory ways, with different objects - primarily macroscopic solid objects - around them. Through their various interactions, they build hypotheses and predictions, and eventually learn, infer and understand the nature of the physical characteristics and behavior of these objects. Inspired thus, we propose a model for curiosity-driven learning and inference for real-world AI agents. This model is based on the arousal of curiosity, deriving from observations along discontinuities in the fundamental macroscopic solid-body physics parameters, i.e., shape constancy, spatial-temporal continuity, and object permanence. We use the term body-budget to represent the perceived fundamental properties of solid objects. The model aims to support the emulation of learning from scratch followed by substantiation through experience, irrespective of domain, in real-world AI agents.
Zero Aware Configurable Data Encoding by Skipping Transfer for Error Resilient Applications
Venue: arXiv cs.AR 2021
Year: 2021

In this paper, we propose Zero Aware Configurable Data Encoding by Skipping Transfer (ZAC-DEST), a data encoding scheme to reduce the energy consumption of DRAM channels, specifically targeted towards approximate computing and error resilient applications. ZAC-DEST exploits the similarity between recent data transfers across channels and information about the error resilience behavior of applications to reduce on-die termination and switching energy by reducing the number of 1's transmitted over the channels. ZAC-DEST also provides a number of knobs for trading off the application's accuracy for energy savings, and vice versa, and can be applied to both training and inference. We apply ZAC-DEST to five machine learning applications. On average, across all applications and configurations, we observed a reduction of $40$% in termination energy and $37$% in switching energy as compared to the state of the art data encoding technique BD-Coder with an average output quality loss of $10$%. We show that if both training and testing are done assuming the presence of ZAC-DEST, the output quality of the applications can be improved upto 9 times as compared to when ZAC-DEST is only applied during testing leading to energy savings during training and inference with increased output quality.
Uncertainty in Minimum Cost Multicuts for Image and Motion Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

The minimum cost lifted multicut approach has proven practically good performance in a wide range of applications such as image decomposition, mesh segmentation, multiple object tracking, and motion segmentation. It addresses such problems in a graph-based model, where real-valued costs are assigned to the edges between entities such that the minimum cut decomposes the graph into an optimal number of segments. Driven by a probabilistic formulation of minimum cost multicuts, we provide a measure for the uncertainties of the decisions made during the optimization. We argue that access to such uncertainties is crucial for many practical applications and conduct an evaluation by means of sparsifications on three different, widely used datasets in the context of image decomposition (BSDS-500) and motion segmentation (DAVIS2016 and FBMS59) in terms of variation of information (VI) and Rand index (RI).
Left Dihedral Codes over Finite Chain Rings
Venue: arXiv cs.IT 2021
Year: 2021

Let $R$ be a finite commutative chain ring, $D_{2n}$ be the dihedral group of size $2n$ and $R[D_{2n}]$ be the dihedral group ring. In this paper, we completely characterize left ideals of $R[D_{2n}]$ (called left $D_{2n}$-codes) when ${\rm gcd}(char(R),n)=1$. In this way, we explore the structure of some skew-cyclic codes of length 2 over $R$ and also over $R\times S$, where $S$ is an isomorphic copy of $R$. As a particular result, we give the structure of cyclic codes of length 2 over $R$. In the case where $R=\F_{p^m}$ is a Galois field, we give a classification for left $D_{2N}$-codes over $\F_{p^m}$, for any positive integer $N$. In both cases we determine dual codes and identify self-dual ones.
Doc2Dict: Information Extraction as Text Generation
Venue: arXiv cs.CL 2021
Year: 2021

Typically, information extraction (IE) requires a pipeline approach: first, a sequence labeling model is trained on manually annotated documents to extract relevant spans; then, when a new document arrives, a model predicts spans which are then post-processed and standardized to convert the information into a database entry. We replace this labor-intensive workflow with a transformer language model trained on existing database records to directly generate structured JSON. Our solution removes the workload associated with producing token-level annotations and takes advantage of a data source which is generally quite plentiful (e.g. database records). As long documents are common in information extraction tasks, we use gradient checkpointing and chunked encoding to apply our method to sequences of up to 32,000 tokens on a single GPU. Our Doc2Dict approach is competitive with more complex, hand-engineered pipelines and offers a simple but effective baseline for document-level information extraction. We release our Doc2Dict model and code to reproduce our experiments and facilitate future work.
Retrofitting FSO Systems in Existing RF Infrastructure: A Non-Zero Sum Game Technology
Venue: arXiv cs.ET 2021
Year: 2021

Progress in optical wireless communication (OWC) has unleashed the potential to transmit data in an ultra-fast manner without incurring large investments and bulk infrastructure. OWC includes wireless data transmissions in three optical sub-bands; ultraviolet, visible, and infrared. This paper discusses installing infrared OWC, known as free space optics (FSO), systems on top of installed radio frequency (RF) networks for outdoor applications to benefit from the reliability of RF links and the unlicensed broad optical spectrum, and the large data rates carried by laser beams propagating in free space. We equally review commercially available solutions and the hardware requirements for RF and FSO technology co-existence. The potential of hybrid RF/FSO for space communication is further discussed. Finally, open problems and future research directions are presented.
Sound Event Detection with Adaptive Frequency Selection
Venue: arXiv cs.SD 2021
Year: 2021

In this work, we present HIDACT, a novel network architecture for adaptive computation for efficiently recognizing acoustic events. We evaluate the model on a sound event detection task where we train it to adaptively process frequency bands. The model learns to adapt to the input without requesting all frequency sub-bands provided. It can make confident predictions within fewer processing steps, hence reducing the amount of computation. Experimental results show that HIDACT has comparable performance to baseline models with more parameters and higher computational complexity. Furthermore, the model can adjust the amount of computation based on the data and computational budget.
Towards Unsupervised Domain Adaptation for Deep Face Recognition under Privacy Constraints via Federated Learning
Venue: arXiv cs.CV 2021
Year: 2021

Unsupervised domain adaptation has been widely adopted to generalize models for unlabeled data in a target domain, given labeled data in a source domain, whose data distributions differ from the target domain. However, existing works are inapplicable to face recognition under privacy constraints because they require sharing sensitive face images between two domains. To address this problem, we propose a novel unsupervised federated face recognition approach (FedFR). FedFR improves the performance in the target domain by iteratively aggregating knowledge from the source domain through federated learning. It protects data privacy by transferring models instead of raw data between domains. Besides, we propose a new domain constraint loss (DCL) to regularize source domain training. DCL suppresses the data volume dominance of the source domain. We also enhance a hierarchical clustering algorithm to predict pseudo labels for the unlabeled target domain accurately. To this end, FedFR forms an end-to-end training pipeline: (1) pre-train in the source domain; (2) predict pseudo labels by clustering in the target domain; (3) conduct domain-constrained federated learning across two domains. Extensive experiments and analysis on two newly constructed benchmarks demonstrate the effectiveness of FedFR. It outperforms the baseline and classic methods in the target domain by over 4% on the more realistic benchmark. We believe that FedFR will shed light on applying federated learning to more computer vision tasks under privacy constraints.
Global Wheat Head Dataset 2021: more diversity to improve the benchmarking of wheat head localization methods
Venue: arXiv cs.CV 2021
Year: 2021

The Global Wheat Head Detection (GWHD) dataset was created in 2020 and has assembled 193,634 labelled wheat heads from 4,700 RGB images acquired from various acquisition platforms and 7 countries/institutions. With an associated competition hosted in Kaggle, GWHD has successfully attracted attention from both the computer vision and agricultural science communities. From this first experience in 2020, a few avenues for improvements have been identified, especially from the perspective of data size, head diversity and label reliability. To address these issues, the 2020 dataset has been reexamined, relabeled, and augmented by adding 1,722 images from 5 additional countries, allowing for 81,553 additional wheat heads to be added. We now release a new version of the Global Wheat Head Detection (GWHD) dataset in 2021, which is bigger, more diverse, and less noisy than the 2020 version. The GWHD 2021 is now publicly available at http://www.global-wheat.com/ and a new data challenge has been organized on AIcrowd to make use of this updated dataset.
EA-Net: Edge-Aware Network for Flow-based Video Frame Interpolation
Venue: arXiv cs.CV 2021
Year: 2021

Video frame interpolation can up-convert the frame rate and enhance the video quality. In recent years, although the interpolation performance has achieved great success, image blur usually occurs at the object boundaries owing to the large motion. It has been a long-standing problem, and has not been addressed yet. In this paper, we propose to reduce the image blur and get the clear shape of objects by preserving the edges in the interpolated frames. To this end, the proposed Edge-Aware Network (EA-Net) integrates the edge information into the frame interpolation task. It follows an end-to-end architecture and can be separated into two stages, \emph{i.e.}, edge-guided flow estimation and edge-protected frame synthesis. Specifically, in the flow estimation stage, three edge-aware mechanisms are developed to emphasize the frame edges in estimating flow maps, so that the edge-maps are taken as the auxiliary information to provide more guidance to boost the flow accuracy. In the frame synthesis stage, the flow refinement module is designed to refine the flow map, and the attention module is carried out to adaptively focus on the bidirectional flow maps when synthesizing the intermediate frames. Furthermore, the frame and edge discriminators are adopted to conduct the adversarial training strategy, so as to enhance the reality and clarity of synthesized frames. Experiments on three benchmarks, including Vimeo90k, UCF101 for single-frame interpolation and Adobe240-fps for multi-frame interpolation, have demonstrated the superiority of the proposed EA-Net for the video frame interpolation task.
A Fusion-Denoising Attack on InstaHide with Data Augmentation
Venue: arXiv cs.CR 2021
Year: 2021

InstaHide is a state-of-the-art mechanism for protecting private training images, by mixing multiple private images and modifying them such that their visual features are indistinguishable to the naked eye. In recent work, however, Carlini et al. show that it is possible to reconstruct private images from the encrypted dataset generated by InstaHide. Nevertheless, we demonstrate that Carlini et al.'s attack can be easily defeated by incorporating data augmentation into InstaHide. This leads to a natural question: is InstaHide with data augmentation secure? In this paper, we provide a negative answer to this question, by devising an attack for recovering private images from the outputs of InstaHide even when data augmentation is present. The basic idea is to use a comparative network to identify encrypted images that are likely to correspond to the same private image, and then employ a fusion-denoising network for restoring the private image from the encrypted ones, taking into account the effects of data augmentation. Extensive experiments demonstrate the effectiveness of the proposed attack in comparison to Carlini et al.'s attack.
A Cloud-based Deep Learning Framework for Remote Detection of Diabetic Foot Ulcers
Venue: arXiv cs.LG 2021
Year: 2021

This research proposes a mobile and cloud-based framework for the automatic detection of diabetic foot ulcers and conducts an investigation of its performance. The system uses a cross-platform mobile framework which enables the deployment of mobile apps to multiple platforms using a single TypeScript code base. A deep convolutional neural network was deployed to a cloud-based platform where the mobile app could send photographs of patient's feet for inference to detect the presence of diabetic foot ulcers. The functionality and usability of the system were tested in two clinical settings: Salford Royal NHS Foundation Trust and Lancashire Teaching Hospitals NHS Foundation Trust. The benefits of the system, such as the potential use of the app by patients to identify and monitor their condition are discussed.
Deep Learning Models in Software Requirements Engineering
Venue: arXiv cs.SE 2021
Year: 2021

Requirements elicitation is an important phase of any software project: the errors in requirements are more expensive to fix than the errors introduced at later stages of software life cycle. Nevertheless, many projects do not devote sufficient time to requirements. Automated requirements generation can improve the quality of software projects. In this article we have accomplished the first step of the research on this topic: we have applied the vanilla sentence autoencoder to the sentence generation task and evaluated its performance. The generated sentences are not plausible English and contain only a few meaningful words. We believe that applying the model to a larger dataset may produce significantly better results. Further research is needed to improve the quality of generated data.
Be Causal: De-biasing Social Network Confounding in Recommendation
Venue: arXiv cs.LG 2021
Year: 2021

In recommendation systems, the existence of the missing-not-at-random (MNAR) problem results in the selection bias issue, degrading the recommendation performance ultimately. A common practice to address MNAR is to treat missing entries from the so-called "exposure" perspective, i.e., modeling how an item is exposed (provided) to a user. Most of the existing approaches use heuristic models or re-weighting strategy on observed ratings to mimic the missing-at-random setting. However, little research has been done to reveal how the ratings are missing from a causal perspective. To bridge the gap, we propose an unbiased and robust method called DENC (De-bias Network Confounding in Recommendation) inspired by confounder analysis in causal inference. In general, DENC provides a causal analysis on MNAR from both the inherent factors (e.g., latent user or item factors) and auxiliary network's perspective. Particularly, the proposed exposure model in DENC can control the social network confounder meanwhile preserves the observed exposure information. We also develop a deconfounding model through the balanced representation learning to retain the primary user and item features, which enables DENC generalize well on the rating prediction. Extensive experiments on three datasets validate that our proposed model outperforms the state-of-the-art baselines.
Efficient yield optimization with limited gradient information
Venue: arXiv cs.CE 2021
Year: 2021

In this work an efficient strategy for yield optimization with uncertain and deterministic optimization variables is presented. The gradient based adaptive Newton-Monte Carlo method is modified, such that it can handle variables with (uncertain parameters) and without (deterministic parameters) analytical gradient information. This mixed strategy is numerically compared to derivative free approaches.
Shared data granularity: A latent dimension of privacy scoring over online social networks
Venue: arXiv cs.CR 2021
Year: 2021

Privacy scoring aims at measuring the privacy violation risk of a user over an online social network (OSN). Existing work in the field rely on possibly biased or emotional survey data and focus only on personel purpose OSNs like Facebook. In contrast to existing work, in this thesis, we work with real-world OSN data collected from LinkedIn, the most popular professional-purpose OSN (ProOSN). Towards this end, we developed an extensive crawler to collect all relevant profile data of 5,389 LinkedIn users, modelled these data using both relational and graph databases and quantitatively analyzed all privacy risk scoring methods in the literature. Additionally, we propose a novel scoring method that consider the granularity of data an OSN user shares on her profile page. Extensive experimental evaluation of existing and proposed scoring methods indicates the effectiveness of the proposed solution.
Cybernetics and the Future of Work
Venue: arXiv cs.CY 2021
Year: 2021

The disruption caused by the pandemic has called into question industrial norms and created an opportunity to reimagine the future of work. We discuss how this period of opportunity may be leveraged to bring about a future in which the workforce thrives rather than survives. Any coherent plan of such breadth must address the interaction of multiple technological, social, economic, and environmental systems. A shared language that facilitates communication across disciplinary boundaries can bring together stakeholders and facilitate a considered response. The origin story of cybernetics and the ideas posed therein serve to illustrate how we may better understand present complex challenges, to create a future of work that places human values at its core.
A Two-Sided Matching Model for Data Stream Processing in the Cloud-Fog Continuum
Venue: arXiv cs.DC 2021
Year: 2021

Latency-sensitive and bandwidth-intensive stream processing applications are dominant traffic generators over the Internet network. A stream consists of a continuous sequence of data elements, which require processing in nearly real-time. To improve communication latency and reduce the network congestion, Fog computing complements the Cloud services by moving the computation towards the edge of the network. Unfortunately, the heterogeneity of the new Cloud-Fog continuum raises important challenges related to deploying and executing data stream applications. We explore in this work a two-sided stable matching model called Cloud-Fog to data stream application matching (CODA) for deploying a distributed application represented as a workflow of stream processing microservices on heterogeneous Cloud-Fog computing resources. In CODA, the application microservices rank the continuum resources based on their microservice stream processing time, while resources rank the stream processing microservices based on their residual bandwidth. A stable many-to-one matching algorithm assigns microservices to resources based on their mutual preferences, aiming to optimize the complete stream processing time on the application side, and the total streaming traffic on the resource side. We evaluate the CODA algorithm using simulated and real-world Cloud-Fog scenarios. We achieved 11 to 45% lower stream processing time and 1.3 to 20% lower streaming traffic compared to related state-of-the-art approaches.
RoSmEEry: Robotic Simulated Environment for Evaluation and Benchmarking of Semantic Mapping Algorithms
Venue: arXiv cs.RO 2021
Year: 2021

Human-robot interaction requires a common understanding of the operational environment, which can be provided by a representation that blends geometric and symbolic knowledge: a semantic map. Through a semantic map the robot can interpret user commands by grounding them to its sensory observations. Semantic mapping is the process that builds such a representation. Despite being fundamental to enable cognition and high-level reasoning in robotics, semantic mapping is a challenging task due to generalization to different scenarios and sensory data types. In fact, it is difficult to obtain a rich and accurate semantic map of the environment and of the objects therein. Moreover, to date, there are no frameworks that allow for a comparison of the performance in building semantic maps for a given environment. To tackle these issues we design RoSmEEry, a novel framework based on the Gazebo simulator, where we introduce an accessible and ready-to-use methodology for a systematic evaluation of semantic mapping algorithms. We release our framework, as an open-source package, with multiple simulation environments with the aim to provide a general set-up to quantitatively measure the performances in acquiring semantic knowledge about the environment.
Unknown-box Approximation to Improve Optical Character Recognition Performance
Venue: arXiv cs.CV 2021
Year: 2021

Optical character recognition (OCR) is a widely used pattern recognition application in numerous domains. There are several feature-rich, general-purpose OCR solutions available for consumers, which can provide moderate to excellent accuracy levels. However, accuracy can diminish with difficult and uncommon document domains. Preprocessing of document images can be used to minimize the effect of domain shift. In this paper, a novel approach is presented for creating a customized preprocessor for a given OCR engine. Unlike the previous OCR agnostic preprocessing techniques, the proposed approach approximates the gradient of a particular OCR engine to train a preprocessor module. Experiments with two datasets and two OCR engines show that the presented preprocessor is able to improve the accuracy of the OCR up to 46% from the baseline by applying pixel-level manipulations to the document image. The implementation of the proposed method and the enhanced public datasets are available for download.
Stage-wise Fine-tuning for Graph-to-Text Generation
Venue: arXiv cs.CL 2021
Year: 2021

Graph-to-text generation has benefited from pre-trained language models (PLMs) in achieving better performance than structured graph encoders. However, they fail to fully utilize the structure information of the input graph. In this paper, we aim to further improve the performance of the pre-trained language model by proposing a structured graph-to-text model with a two-step fine-tuning mechanism which first fine-tunes the model on Wikipedia before adapting to the graph-to-text generation. In addition to using the traditional token and position embeddings to encode the knowledge graph (KG), we propose a novel tree-level embedding method to capture the inter-dependency structures of the input graph. This new approach has significantly improved the performance of all text generation metrics for the English WebNLG 2017 dataset.