OpenRaiser/Intern-Atlas huggingface dataset:
Natural interaction with traffic control cameras through multimodal interfaces
Venue: arXiv cs.HC 2021
Year: 2021

Human-Computer Interfaces have always played a fundamental role in usability and commands' interpretability of the modern software systems. With the explosion of the Artificial Intelligence concept, such interfaces have begun to fill the gap between the user and the system itself, further evolving in Adaptive User Interfaces (AUI). Meta Interfaces are a further step towards the user, and they aim at supporting the human activities in an ambient interactive space; in such a way, the user can control the surrounding space and interact with it. This work aims at proposing a meta user interface that exploits the Put That There paradigm to enable the user to fast interaction by employing natural language and gestures. The application scenario is a video surveillance control room, in which the speed of actions and reactions is fundamental for urban safety and driver and pedestrian security. The interaction is oriented towards three environments: the first is the control room itself, in which the operator can organize the views of the monitors related to the cameras on site by vocal commands and gestures, as well as conveying the audio on the headset or in the speakers of the room. The second one is related to the control of the video, in order to go back and forth to a particular scene showing specific events, or zoom in/out a particular camera; the third allows the operator to send rescue vehicle in a particular street, in case of need. The gestures data are acquired through a Microsoft Kinect 2 which captures pointing and gestures allowing the user to interact multimodally thus increasing the naturalness of the interaction; the related module maps the movement information to a particular instruction, also supported by vocal commands which enable its execution. (cont...)
A Pose-only Solution to Visual Reconstruction and Navigation
Venue: arXiv cs.CV 2021
Year: 2021

Visual navigation and three-dimensional (3D) scene reconstruction are essential for robotics to interact with the surrounding environment. Large-scale scenes and critical camera motions are great challenges facing the research community to achieve this goal. We raised a pose-only imaging geometry framework and algorithms that can help solve these challenges. The representation is a linear function of camera global translations, which allows for efficient and robust camera motion estimation. As a result, the spatial feature coordinates can be analytically reconstructed and do not require nonlinear optimization. Experiments demonstrate that the computational efficiency of recovering the scene and associated camera poses is significantly improved by 2-4 orders of magnitude. This solution might be promising to unlock real-time 3D visual computing in many forefront applications.
Inter-class Discrepancy Alignment for Face Recognition
Venue: arXiv cs.CV 2021
Year: 2021

The field of face recognition (FR) has witnessed great progress with the surge of deep learning. Existing methods mainly focus on extracting discriminative features, and directly compute the cosine or L2 distance by the point-to-point way without considering the context information. In this study, we make a key observation that the local con-text represented by the similarities between the instance and its inter-class neighbors1plays an important role forFR. Specifically, we attempt to incorporate the local in-formation in the feature space into the metric, and pro-pose a unified framework calledInter-class DiscrepancyAlignment(IDA), with two dedicated modules, Discrepancy Alignment Operator(IDA-DAO) andSupport Set Estimation(IDA-SSE). IDA-DAO is used to align the similarity scores considering the discrepancy between the images and its neighbors, which is defined by adaptive support sets on the hypersphere. For practical inference, it is difficult to acquire support set during online inference. IDA-SSE can provide convincing inter-class neighbors by introducing virtual candidate images generated with GAN. Further-more, we propose the learnable IDA-SSE, which can implicitly give estimation without the need of any other images in the evaluation process. The proposed IDA can be incorporated into existing FR systems seamlessly and efficiently. Extensive experiments demonstrate that this frame-work can 1) significantly improve the accuracy, and 2) make the model robust to the face images of various distributions.Without bells and whistles, our method achieves state-of-the-art performance on multiple standard FR benchmarks.
Performance Analysis of OTFS Modulation with Receive Antenna Selection
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we analyze the performance of orthogonal time frequency space (OTFS) modulation with antenna selection at the receiver, where $n_s$ out of $n_r$ receive antennas with maximum channel Frobenius norms in the delay-Doppler (DD) domain are selected. Single-input multiple-output OTFS (SIMO-OTFS), multiple-input multiple-output OTFS (MIMO-OTFS), and space-time coded OTFS (STC-OTFS) systems with receive antenna selection (RAS) are considered. We consider these systems without and with phase rotation. Our diversity analysis results show that, with no phase rotation, SIMO-OTFS and MIMO-OTFS systems with RAS are rank deficient, and therefore they do not extract the full receive diversity as well as the diversity present in the DD domain. Also, Alamouti coded STC-OTFS system with RAS and no phase rotation extracts the full transmit diversity, but it fails to extract the DD diversity. On the other hand, SIMO-OTFS and STC-OTFS systems with RAS become full-ranked when phase rotation is used, because of which they extract the full spatial as well as the DD diversity present in the system. Also, when phase rotation is used, MIMO-OTFS systems with RAS extract the full DD diversity, but they do not extract the full receive diversity because of rank deficiency. Simulation results are shown to validate the analytically predicted diversity performance.
The Capacity Region of Distributed Multi-User Secret Sharing
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we study the problem of distributed multi-user secret sharing, including a trusted master node, $N\in \mathbb{N}$ storage nodes, and $K$ users, where each user has access to the contents of a subset of storage nodes. Each user has an independent secret message with certain rate, defined as the size of the message normalized by the size of a storage node. Having access to the secret messages, the trusted master node places encoded shares in the storage nodes, such that (i) each user can recover its own message from the content of the storage nodes that it has access to, (ii) each user cannot gain any information about the message of any other user. We characterize the capacity region of the distributed multi-user secret sharing, defined as the set of all achievable rate tuples, subject to the correctness and privacy constraints. In the achievable scheme, for each user, the master node forms a polynomial with the degree equal to the number of its accessible storage nodes minus one, where the value of this polynomial at certain points are stored as the encoded shares. The message of that user is embedded in some of the coefficients of the polynomial. The remaining coefficients are determined such that the content of each storage node serves as the encoded shares for all users that have access to that storage node.
DeepCert: Verification of Contextually Relevant Robustness for Neural Network Image Classifiers
Venue: arXiv cs.LG 2021
Year: 2021

We introduce DeepCert, a tool-supported method for verifying the robustness of deep neural network (DNN) image classifiers to contextually relevant perturbations such as blur, haze, and changes in image contrast. While the robustness of DNN classifiers has been the subject of intense research in recent years, the solutions delivered by this research focus on verifying DNN robustness to small perturbations in the images being classified, with perturbation magnitude measured using established Lp norms. This is useful for identifying potential adversarial attacks on DNN image classifiers, but cannot verify DNN robustness to contextually relevant image perturbations, which are typically not small when expressed with Lp norms. DeepCert addresses this underexplored verification problem by supporting:(1) the encoding of real-world image perturbations; (2) the systematic evaluation of contextually relevant DNN robustness, using both testing and formal verification; (3) the generation of contextually relevant counterexamples; and, through these, (4) the selection of DNN image classifiers suitable for the operational context (i)envisaged when a potentially safety-critical system is designed, or (ii)observed by a deployed system. We demonstrate the effectiveness of DeepCert by showing how it can be used to verify the robustness of DNN image classifiers build for two benchmark datasets (`German Traffic Sign' and `CIFAR-10') to multiple contextually relevant perturbations.
Sparse Training Theory for Scalable and Efficient Agents
Venue: arXiv cs.AI 2021
Year: 2021

A fundamental task for artificial intelligence is learning. Deep Neural Networks have proven to cope perfectly with all learning paradigms, i.e. supervised, unsupervised, and reinforcement learning. Nevertheless, traditional deep learning approaches make use of cloud computing facilities and do not scale well to autonomous agents with low computational resources. Even in the cloud, they suffer from computational and memory limitations, and they cannot be used to model adequately large physical worlds for agents which assume networks with billions of neurons. These issues are addressed in the last few years by the emerging topic of sparse training, which trains sparse networks from scratch. This paper discusses sparse training state-of-the-art, its challenges and limitations while introducing a couple of new theoretical research directions which has the potential of alleviating sparse training limitations to push deep learning scalability well beyond its current boundaries. Nevertheless, the theoretical advancements impact in complex multi-agents settings is discussed from a real-world perspective, using the smart grid case study.
A Novel CNN-LSTM-based Approach to Predict Urban Expansion
Venue: arXiv cs.CV 2021
Year: 2021

Time-series remote sensing data offer a rich source of information that can be used in a wide range of applications, from monitoring changes in land cover to surveilling crops, coastal changes, flood risk assessment, and urban sprawl. This paper addresses the challenge of using time-series satellite images to predict urban expansion. Building upon previous work, we propose a novel two-step approach based on semantic image segmentation in order to predict urban expansion. The first step aims to extract information about urban regions at different time scales and prepare them for use in the training step. The second step combines Convolutional Neural Networks (CNN) with Long Short Term Memory (LSTM) methods in order to learn temporal features and thus predict urban expansion. In this paper, experimental results are conducted using several multi-date satellite images representing the three largest cities in Saudi Arabia, namely: Riyadh, Jeddah, and Dammam. We empirically evaluated our proposed technique, and examined its results by comparing them with state-of-the-art approaches. Following this evaluation, we determined that our results reveal improved performance for the new-coupled CNN-LSTM approach, particularly in terms of assessments based on Mean Square Error, Root Mean Square Error, Peak Signal to Noise Ratio, Structural Similarity Index, and overall classification accuracy.
Does Interaction Improve Bayesian Reasoning with Visualization?
Venue: arXiv cs.HC 2021
Year: 2021

Interaction enables users to navigate large amounts of data effectively, supports cognitive processing, and increases data representation methods. However, there have been few attempts to empirically demonstrate whether adding interaction to a static visualization improves its function beyond popular beliefs. In this paper, we address this gap. We use a classic Bayesian reasoning task as a testbed for evaluating whether allowing users to interact with a static visualization can improve their reasoning. Through two crowdsourced studies, we show that adding interaction to a static Bayesian reasoning visualization does not improve participants' accuracy on a Bayesian reasoning task. In some cases, it can significantly detract from it. Moreover, we demonstrate that underlying visualization design modulates performance and that people with high versus low spatial ability respond differently to different interaction techniques and underlying base visualizations. Our work suggests that interaction is not as unambiguously good as we often believe; a well designed static visualization can be as, if not more, effective than an interactive one.
Transportation Density Reduction Caused by City Lockdowns Across the World during the COVID-19 Epidemic: From the View of High-resolution Remote Sensing Imagery
Venue: arXiv cs.CV 2021
Year: 2021

As the COVID-19 epidemic began to worsen in the first months of 2020, stringent lockdown policies were implemented in numerous cities throughout the world to control human transmission and mitigate its spread. Although transportation density reduction inside the city was felt subjectively, there has thus far been no objective and quantitative study of its variation to reflect the intracity population flows and their corresponding relationship with lockdown policy stringency from the view of remote sensing images with the high resolution under 1m. Accordingly, we here provide a quantitative investigation of the transportation density reduction before and after lockdown was implemented in six epicenter cities (Wuhan, Milan, Madrid, Paris, New York, and London) around the world during the COVID-19 epidemic, which is accomplished by extracting vehicles from the multi-temporal high-resolution remote sensing images. A novel vehicle detection model combining unsupervised vehicle candidate extraction and deep learning identification was specifically proposed for the images with the resolution of 0.5m. Our results indicate that transportation densities were reduced by an average of approximately 50% (and as much as 75.96%) in these six cities following lockdown. The influences on transportation density reduction rates are also highly correlated with policy stringency, with an R^2 value exceeding 0.83. Even within a specific city, the transportation density changes differed and tended to be distributed in accordance with the city's land-use patterns. Considering that public transportation was mostly reduced or even forbidden, our results indicate that city lockdown policies are effective at limiting human transmission within cities.
Safepaths: Vaccine Diary Protocol and Decentralized Vaccine Coordination System using a Privacy Preserving User Centric Experience
Venue: arXiv cs.CR 2021
Year: 2021

In this early draft, we present an end-to-end decentralized protocol for the secure and privacy preserving workflow of vaccination, vaccination status verification, and adverse reactions or symptoms reporting. The proposed system improves the efficiency, privacy, equity, and effectiveness of the existing manual system while remaining interoperable with its capabilities. We also discuss various security concerns and alternate methodologies based on the proposed protocols.
Determinants of ICT Adoption Among Small Scale Agribusiness Enterprises In Somalia
Venue: arXiv cs.CY 2021
Year: 2021

The use of Information and Communication Technology (ICT) can advance the Agricultural Business sector, particularly in a country seeking opportunities to explore the sector. There is evidence that ICT has made significant contributions to agribusiness because it allows enterprises to manage their operations, and it has major impacts on the business. However, the critical factors that motivate the adoption of new innovative technology by agribusiness enterprises are underexplored. The literature has indicated ICT adoption among small-scale agribusiness enterprises in Somalia is not fully understood. Nevertheless, this study addresses this gap by investigating the adoption of ICT among small-scale agribusiness enterprises in Somalia. The paper reports the use of the Technology, Organization, Environment (TOE) framework. An online survey has been conducted with random sampling for data collection, with 107 respondents. The respondents are from agribusiness staff and farmers from various agricultural companies in Somalia. After quantitative data analysis, the results indicated that relative advantage, complexity, top management support, and competitive pressure factors are significant contributors to ICT adoption in Somalian agribusiness enterprises, while ICT costs and vendor support are not significantly related to the adoption of ICT in agricultural business. This study concludes that ICT adoption in Somalia is inspired by insight and motivation rather than financial and external support. Understanding these factors leads to a better understanding of ICT adoption in Somalia. Additionally, it enriches the literature about the agriculture business on the African continent Keywords: Determinants, ICT Adoption, Agribusiness, Small Scale, TOE framework
Modelling mobility and visualizing people's flow patterns in rural areas for future infrastructure development as a good transnational land-governance practice
Venue: arXiv cs.CY 2021
Year: 2021

This paper summarizes a cross-border mobility study, origin-destination mobility modelling and visualization, conducted in support of the infrastructure development efforts of local authorities and NGOs on the area over the Kayanga-Geba River, at the border between Senegal and Guinea Bissau. It builds on the data collected through participatory mapping for the elaboration of the Cross-Border Land Management and Development Plans (Plans PAGET) aiming to harmonize the different national territorial management tools into a unique transnational tool through the consideration of border areas as a territorial unity. Despite a small amount of available mobility data, we were able to build a mobility model for the considered area, and implemented it in the Traffic Simulation Framework, which was later used to calculate origin-destination matrices for the studied regions in two cases: with and without a cross-border mobility. We analyzed the differences in the mobility patterns and visualized the mobility flows, deliberating on what may be the potential impacts of building a bridge in the study area. Our methodology is general and can be applied in similar studies on different areas. However, the quality of results may depend on the available data.
A Data-Centric Framework for Composable NLP Workflows
Venue: arXiv cs.CL 2021
Year: 2021

Empirical natural language processing (NLP) systems in application domains (e.g., healthcare, finance, education) involve interoperation among multiple components, ranging from data ingestion, human annotation, to text retrieval, analysis, generation, and visualization. We establish a unified open-source framework to support fast development of such sophisticated NLP workflows in a composable manner. The framework introduces a uniform data representation to encode heterogeneous results by a wide range of NLP tasks. It offers a large repository of processors for NLP tasks, visualization, and annotation, which can be easily assembled with full interoperability under the unified representation. The highly extensible framework allows plugging in custom processors from external off-the-shelf NLP and deep learning libraries. The whole framework is delivered through two modularized yet integratable open-source projects, namely Forte (for workflow infrastructure and NLP function processors) and Stave (for user interaction, visualization, and annotation).
HED-UNet: Combined Segmentation and Edge Detection for Monitoring the Antarctic Coastline
Venue: arXiv cs.CV 2021
Year: 2021

Deep learning-based coastline detection algorithms have begun to outshine traditional statistical methods in recent years. However, they are usually trained only as single-purpose models to either segment land and water or delineate the coastline. In contrast to this, a human annotator will usually keep a mental map of both segmentation and delineation when performing manual coastline detection. To take into account this task duality, we therefore devise a new model to unite these two approaches in a deep learning model. By taking inspiration from the main building blocks of a semantic segmentation framework (UNet) and an edge detection framework (HED), both tasks are combined in a natural way. Training is made efficient by employing deep supervision on side predictions at multiple resolutions. Finally, a hierarchical attention mechanism is introduced to adaptively merge these multiscale predictions into the final model output. The advantages of this approach over other traditional and deep learning-based methods for coastline detection are demonstrated on a dataset of Sentinel-1 imagery covering parts of the Antarctic coast, where coastline detection is notoriously difficult. An implementation of our method is available at \url{https://github.com/khdlr/HED-UNet}.
Dual Reinforcement-Based Specification Generation for Image De-Rendering
Venue: arXiv cs.CL 2021
Year: 2021

Advances in deep learning have led to promising progress in inferring graphics programs by de-rendering computer-generated images. However, current methods do not explore which decoding methods lead to better inductive bias for inferring graphics programs. In our work, we first explore the effectiveness of LSTM-RNN versus Transformer networks as decoders for order-independent graphics programs. Since these are sequence models, we must choose an ordering of the objects in the graphics programs for likelihood training. We found that the LSTM performance was highly sensitive to the sequence ordering (random order vs. pattern-based order), while Transformer performance was roughly independent of the sequence ordering. Further, we present a policy gradient based reinforcement learning approach for better inductive bias in the decoder via multiple diverse rewards based both on the graphics program specification and the rendered image. We also explore the combination of these complementary rewards. We achieve state-of-the-art results on two graphics program generation datasets.