OpenRaiser/Intern-Atlas huggingface dataset:
Packaging research artefacts with RO-Crate
Venue: arXiv cs.DL 2021
Year: 2021

An increasing number of researchers support reproducibility by including pointers to and descriptions of datasets, software and methods in their publications. However, scientific articles may be ambiguous, incomplete and difficult to process by automated systems. In this paper we introduce RO-Crate, an open, community-driven, and lightweight approach to packaging research artefacts along with their metadata in a machine readable manner. RO-Crate is based on Schema$.$org annotations in JSON-LD, aiming to establish best practices to formally describe metadata in an accessible and practical way for their use in a wide variety of situations. An RO-Crate is a structured archive of all the items that contributed to a research outcome, including their identifiers, provenance, relations and annotations. As a general purpose packaging approach for data and their metadata, RO-Crate is used across multiple areas, including bioinformatics, digital humanities and regulatory sciences. By applying "just enough" Linked Data standards, RO-Crate simplifies the process of making research outputs FAIR while also enhancing research reproducibility. An RO-Crate for this article is available at https://w3id.org/ro/doi/10.5281/zenodo.5146227
Evolution Toward 6G Wireless Networks: A Resource Management Perspective
Venue: arXiv cs.NI 2021
Year: 2021

In this article, we first present the vision, key performance indicators, key enabling techniques (KETs), and services of 6G wireless networks. Then, we highlight a series of general resource management (RM) challenges as well as unique RM challenges corresponding to each KET. The unique RM challenges in 6G necessitate the transformation of existing optimization-based solutions to artificial intelligence/machine learning-empowered solutions. In the sequel, we formulate a joint network selection and subchannel allocation problem for 6G multi-band network that provides both further enhanced mobile broadband (FeMBB) and extreme ultra reliable low latency communication (eURLLC) services to the terrestrial and aerial users. Our solution highlights the efficacy of multi-band network and demonstrates the robustness of dueling deep Q-learning in obtaining efficient RM solution with faster convergence rate compared to deep-Q network and double deep Q-network algorithms.
SEIGuard: An Authentication-simplified and Deceptive Scheme to Protect Server-side Social Engineering Information Against Brute-force Attacks
Venue: arXiv cs.CR 2021
Year: 2021

This paper proposes an authentication-simplified and deceptive scheme (SEIGuard) to protect server-side social engineering information (SEI) and other information against brute-force attacks. In SEIGuard, the password check in authentication is omitted and this design is further combined with the SEI encryption design using honey encryption. The login password merely serves as a temporary key to encrypt SEI and there is no password plaintext or ciphertext stored in the database. During the login, the server doesn't check the login passwords, correct passwords decrypt ciphertexts to be correct plaintexts; incorrect passwords decrypt ciphertexts to be phony but plausible-looking plaintexts (sampled from the same distribution). And these two situations share the same undifferentiated backend procedures. This scheme eliminates the anchor that both online and offline brute-force attacks depending on. Furthermore, this paper presents four SEIGuard scheme designs and algorithms for 4 typical social engineering information objects (mobile phone number, identification number, email address, personal name), which represent 4 different types of message space, i.e. 1) limited and uniformly distributed, 2) limited, complex and uniformly distributed, 3) unlimited and uniformly distributed, 4) unlimited and non-uniformly distributed message space. Specially, we propose multiple small mapping files strategies, binary search algorithms, two-part HE (DTE) design and incremental mapping files solutions for the applications of SEIGuard scheme. Finally, this paper develops the SEIGuard system based on the proposed schemes, designs and algorithms. Experiment result shows that the SEIGuard scheme can effectively protect server-side SEI against brute-force attacks, and SEIGuard also has an impressive real-time response performance that is better than conventional PBE server scheme and HE encryption/decryption.
MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding
Venue: arXiv cs.CV 2021
Year: 2021

We present MMOCR-an open-source toolbox which provides a comprehensive pipeline for text detection and recognition, as well as their downstream tasks such as named entity recognition and key information extraction. MMOCR implements 14 state-of-the-art algorithms, which is significantly more than all the existing open-source OCR projects we are aware of to date. To facilitate future research and industrial applications of text recognition-related problems, we also provide a large number of trained models and detailed benchmarks to give insights into the performance of text detection, recognition and understanding. MMOCR is publicly released at https://github.com/open-mmlab/mmocr.
Findings of the LoResMT 2021 Shared Task on COVID and Sign Language for Low-resource Languages
Venue: arXiv cs.CL 2021
Year: 2021

We present the findings of the LoResMT 2021 shared task which focuses on machine translation (MT) of COVID-19 data for both low-resource spoken and sign languages. The organization of this task was conducted as part of the fourth workshop on technologies for machine translation of low resource languages (LoResMT). Parallel corpora is presented and publicly available which includes the following directions: English$\leftrightarrow$Irish, English$\leftrightarrow$Marathi, and Taiwanese Sign language$\leftrightarrow$Traditional Chinese. Training data consists of 8112, 20933 and 128608 segments, respectively. There are additional monolingual data sets for Marathi and English that consist of 21901 segments. The results presented here are based on entries from a total of eight teams. Three teams submitted systems for English$\leftrightarrow$Irish while five teams submitted systems for English$\leftrightarrow$Marathi. Unfortunately, there were no systems submissions for the Taiwanese Sign language$\leftrightarrow$Traditional Chinese task. Maximum system performance was computed using BLEU and follow as 36.0 for English--Irish, 34.6 for Irish--English, 24.2 for English--Marathi, and 31.3 for Marathi--English.
Real-Time Multi-Modal Semantic Fusion on Unmanned Aerial Vehicles
Venue: arXiv cs.CV 2021
Year: 2021

Unmanned aerial vehicles (UAVs) equipped with multiple complementary sensors have tremendous potential for fast autonomous or remote-controlled semantic scene analysis, e.g., for disaster examination. In this work, we propose a UAV system for real-time semantic inference and fusion of multiple sensor modalities. Semantic segmentation of LiDAR scans and RGB images, as well as object detection on RGB and thermal images, run online onboard the UAV computer using lightweight CNN architectures and embedded inference accelerators. We follow a late fusion approach where semantic information from multiple modalities augments 3D point clouds and image segmentation masks while also generating an allocentric semantic map. Our system provides augmented semantic images and point clouds with $\approx\,$9$\,$Hz. We evaluate the integrated system in real-world experiments in an urban environment.
A Sparse Coding Interpretation of Neural Networks and Theoretical Implications
Venue: arXiv cs.CV 2021
Year: 2021

Neural networks, specifically deep convolutional neural networks, have achieved unprecedented performance in various computer vision tasks, but the rationale for the computations and structures of successful neural networks is not fully understood. Theories abound for the aptitude of convolutional neural networks for image classification, but less is understood about why such models would be capable of complex visual tasks such as inference and anomaly identification. Here, we propose a sparse coding interpretation of neural networks that have ReLU activation and of convolutional neural networks in particular. In sparse coding, when the model's basis functions are assumed to be orthogonal, the optimal coefficients are given by the soft-threshold function of the basis functions projected onto the input image. In a non-negative variant of sparse coding, the soft-threshold function becomes a ReLU. Here, we derive these solutions via sparse coding with orthogonal-assumed basis functions, then we derive the convolutional neural network forward transformation from a modified non-negative orthogonal sparse coding model with an exponential prior parameter for each sparse coding coefficient. Next, we derive a complete convolutional neural network without normalization and pooling by adding logistic regression to a hierarchical sparse coding model. Finally we motivate potentially more robust forward transformations by maintaining sparse priors in convolutional neural networks as well performing a stronger nonlinear transformation.
LayerPipe: Accelerating Deep Neural Network Training by Intra-Layer and Inter-Layer Gradient Pipelining and Multiprocessor Scheduling
Venue: arXiv cs.DC 2021
Year: 2021

The time required for training the neural networks increases with size, complexity, and depth. Training model parameters by backpropagation inherently creates feedback loops. These loops hinder efficient pipelining and scheduling of the tasks within the layer and between consecutive layers. Prior approaches, such as PipeDream, have exploited the use of delayed gradient to achieve inter-layer pipelining. However, these approaches treat the entire backpropagation as a single task; this leads to an increase in computation time and processor underutilization. This paper presents novel optimization approaches where the gradient computations with respect to the weights and the activation functions are considered independently; therefore, these can be computed in parallel. This is referred to as intra-layer optimization. Additionally, the gradient computation with respect to the activation function is further divided into two parts and distributed to two consecutive layers. This leads to balanced scheduling where the computation time of each layer is the same. This is referred to as inter-layer optimization. The proposed system, referred to as LayerPipe, reduces the number of clock cycles required for training while maximizing processor utilization with minimal inter-processor communication overhead. LayerPipe achieves an average speedup of 25% and upwards of 80% with 7 to 9 processors with less communication overhead when compared to PipeDream.
SAPPHIRE: Approaches for Enhanced Concept-to-Text Generation
Venue: arXiv cs.CL 2021
Year: 2021

We motivate and propose a suite of simple but effective improvements for concept-to-text generation called SAPPHIRE: Set Augmentation and Post-hoc PHrase Infilling and REcombination. We demonstrate their effectiveness on generative commonsense reasoning, a.k.a. the CommonGen task, through experiments using both BART and T5 models. Through extensive automatic and human evaluation, we show that SAPPHIRE noticeably improves model performance. An in-depth qualitative analysis illustrates that SAPPHIRE effectively addresses many issues of the baseline model generations, including lack of commonsense, insufficient specificity, and poor fluency.
On Multi-Modal Learning of Editing Source Code
Venue: arXiv cs.SE 2021
Year: 2021

In recent years, Neural Machine Translator (NMT) has shown promise in automatically editing source code. Typical NMT based code editor only considers the code that needs to be changed as input and suggests developers with a ranked list of patched code to choose from - where the correct one may not always be at the top of the list. While NMT based code editing systems generate a broad spectrum of plausible patches, the correct one depends on the developers' requirement and often on the context where the patch is applied. Thus, if developers provide some hints, using natural language, or providing patch context, NMT models can benefit from them. As a proof of concept, in this research, we leverage three modalities of information: edit location, edit code context, commit messages (as a proxy of developers' hint in natural language) to automatically generate edits with NMT models. To that end, we build MODIT, a multi-modal NMT based code editing engine. With in-depth investigation and analysis, we show that developers' hint as an input modality can narrow the search space for patches and outperform state-of-the-art models to generate correctly patched code in top-1 position.
Joint Dynamic Passive Beamforming and Resource Allocation for IRS-Aided Full-Duplex WPCN
Venue: arXiv cs.IT 2021
Year: 2021

This paper studies intelligent reflecting surface (IRS)-aided full-duplex (FD) wireless-powered communication network (WPCN), where a hybrid access point (HAP) broadcasts energy signals to multiple devices for their energy harvesting in the downlink (DL) and meanwhile receives information signals in the uplink (UL) with the help of IRS. Particularly, we propose three types of IRS beamforming configurations to strike a balance between the system performance and signaling overhead as well as implementation complexity. We first propose the fully dynamic IRS beamforming, where the IRS phase-shift vectors vary with each time slot for both DL wireless energy transfer (WET) and UL wireless information transmission (WIT). To further reduce signaling overhead and implementation complexity, we then study two special cases, namely, partially dynamic IRS beamforming and static IRS beamforming. For the former case, two different phase-shift vectors can be exploited for the DL WET and the UL WIT, respectively, whereas for the latter case, the same phase-shift vector needs to be applied for both DL and UL transmissions. We aim to maximize the system throughput by jointly optimizing the time allocation, HAP transmit power, and IRS phase shifts for the above three cases. Two efficient algorithms based on alternating optimization and penalty-based algorithms are respectively proposed for both perfect self-interference cancellation (SIC) case and imperfect SIC case by applying successive convex approximation and difference-of-convex optimization techniques. Simulation results demonstrate the benefits of IRS for enhancing the performance of FD-WPCN, and also show that the IRS-aided FD-WPCN is able to achieve significantly performance gain compared to its counterpart with half-duplex when the self-interference (SI) is properly suppressed.
Deep Geospatial Interpolation Networks
Venue: arXiv cs.LG 2021
Year: 2021

Interpolation in Spatio-temporal data has applications in various domains such as climate, transportation, and mining. Spatio-Temporal interpolation is highly challenging due to the complex spatial and temporal relationships. However, traditional techniques such as Kriging suffer from high running time and poor performance on data that exhibit high variance across space and time dimensions. To this end, we propose a novel deep neural network called as Deep Geospatial Interpolation Network(DGIN), which incorporates both spatial and temporal relationships and has significantly lower training time. DGIN consists of three major components: Spatial Encoder to capture the spatial dependencies, Sequential module to incorporate the temporal dynamics, and an Attention block to learn the importance of the temporal neighborhood around the gap. We evaluate DGIN on the MODIS reflectance dataset from two different regions. Our experimental results indicate that DGIN has two advantages: (a) it outperforms alternative approaches (has lower MSE with p-value < 0.01) and, (b) it has significantly low execution time than Kriging.
Windowed Decoding for Delayed Bit-Interleaved Coded Modulation
Venue: arXiv cs.IT 2021
Year: 2021

Delayed bit-interleaved coded modulation (DBICM) generalizes bit-interleaved coded modulation (BICM) by modulating differently delayed sub-blocks of codewords onto the same signals. DBICM improves transmission reliability over BICM due to its capability of detecting undelayed sub-blocks with the extrinsic information of the decoded delayed sub-blocks. In this work, we propose a novel windowed decoding algorithm for DBICM, which uses the extrinsic information of both the decoded delayed and undelayed sub-blocks, to improve the detection on all sub-blocks. Numerical results show that the proposed windowed decoding significantly outperforms the original decoding.
Reference Service Model for Federated Identity Management
Venue: arXiv cs.CR 2021
Year: 2021

With the pandemic of COVID-19, people around the world increasingly work from home. Each natural person typically has several digital identities with different associated information. During the last years, various identity and access management approaches have gained attraction, helping for example to access other organization's services within trust boundaries. The resulting heterogeneity creates a high complexity to differentiate between these approaches and scenarios as participating entity; combining them is even harder. Last but not least, various actors have a different understanding or perspective of the terms, like 'service', in this context. Our paper describes a reference service with standard components in generic federated identity management. This is utilized with modern Enterprise Architecture using the framework ArchiMate. The proposed universal federated identity management service model (FIMSM) is applied to describe various federated identity management scenarios in a generic service-oriented way. The presented reference design is approved in multiple aspects and is easily applicable in numerous scenarios.
Learning Dynamical System for Grasping Motion
Venue: arXiv cs.RO 2021
Year: 2021

Dynamical System has been widely used for encoding trajectories from human demonstration, which has the inherent adaptability to dynamically changing environments and robustness to perturbations. In this paper we propose a framework to learn a dynamical system that couples position and orientation based on a diffeomorphism. Different from other methods, it can realise the synchronization between positon and orientation during the whole trajectory. Online grasping experiments are carried out to prove its effectiveness and online adaptability.
Towards Visual Explainable Active Learning for Zero-Shot Classification
Venue: arXiv cs.HC 2021
Year: 2021

Zero-shot classification is a promising paradigm to solve an applicable problem when the training classes and test classes are disjoint. Achieving this usually needs experts to externalize their domain knowledge by manually specifying a class-attribute matrix to define which classes have which attributes. Designing a suitable class-attribute matrix is the key to the subsequent procedure, but this design process is tedious and trial-and-error with no guidance. This paper proposes a visual explainable active learning approach with its design and implementation called semantic navigator to solve the above problems. This approach promotes human-AI teaming with four actions (ask, explain, recommend, respond) in each interaction loop. The machine asks contrastive questions to guide humans in the thinking process of attributes. A novel visualization called semantic map explains the current status of the machine. Therefore analysts can better understand why the machine misclassifies objects. Moreover, the machine recommends the labels of classes for each attribute to ease the labeling burden. Finally, humans can steer the model by modifying the labels interactively, and the machine adjusts its recommendations. The visual explainable active learning approach improves humans' efficiency of building zero-shot classification models interactively, compared with the method without guidance. We justify our results with user studies using the standard benchmarks for zero-shot classification.
Development of the InBan_CIDO Ontology by Reusing the Concepts along with Detecting Overlapping Information
Venue: arXiv cs.AI 2021
Year: 2021

The covid19 pandemic is a global emergency that badly impacted the economies of various countries. Covid19 hit India when the growth rate of the country was at the lowest in the last 10 years. To semantically analyze the impact of this pandemic on the economy, it is curial to have an ontology. CIDO ontology is a well standardized ontology that is specially designed to assess the impact of coronavirus disease and utilize its results for future decision forecasting for the government, industry experts, and professionals in the field of various domains like research, medical advancement, technical innovative adoptions, and so on. However, this ontology does not analyze the impact of the Covid19 pandemic on the Indian banking sector. On the other side, Covid19IBO ontology has been developed to analyze the impact of the Covid19 pandemic on the Indian banking sector but this ontology does not reflect complete information of Covid19 data. Resultantly, users cannot get all the relevant information about Covid19 and its impact on the Indian economy. This article aims to extend the CIDO ontology to show the impact of Covid19 on the Indian economy sector by reusing the concepts from other data sources. We also provide a simplified schema matching approach that detects the overlapping information among the ontologies. The experimental analysis proves that the proposed approach has reasonable results.
Learning Open-World Object Proposals without Learning to Classify
Venue: arXiv cs.CV 2021
Year: 2021

Object proposals have become an integral preprocessing steps of many vision pipelines including object detection, weakly supervised detection, object discovery, tracking, etc. Compared to the learning-free methods, learning-based proposals have become popular recently due to the growing interest in object detection. The common paradigm is to learn object proposals from data labeled with a set of object regions and their corresponding categories. However, this approach often struggles with novel objects in the open world that are absent in the training set. In this paper, we identify that the problem is that the binary classifiers in existing proposal methods tend to overfit to the training categories. Therefore, we propose a classification-free Object Localization Network (OLN) which estimates the objectness of each region purely by how well the location and shape of a region overlap with any ground-truth object (e.g., centerness and IoU). This simple strategy learns generalizable objectness and outperforms existing proposals on cross-category generalization on COCO, as well as cross-dataset evaluation on RoboNet, Object365, and EpicKitchens. Finally, we demonstrate the merit of OLN for long-tail object detection on large vocabulary dataset, LVIS, where we notice clear improvement in rare and common categories.
The Marine Debris Dataset for Forward-Looking Sonar Semantic Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Accurate detection and segmentation of marine debris is important for keeping the water bodies clean. This paper presents a novel dataset for marine debris segmentation collected using a Forward Looking Sonar (FLS). The dataset consists of 1868 FLS images captured using ARIS Explorer 3000 sensor. The objects used to produce this dataset contain typical house-hold marine debris and distractor marine objects (tires, hooks, valves,etc), divided in 11 classes plus a background class. Performance of state of the art semantic segmentation architectures with a variety of encoders have been analyzed on this dataset and presented as baseline results. Since the images are grayscale, no pretrained weights have been used. Comparisons are made using Intersection over Union (IoU). The best performing model is Unet with ResNet34 backbone at 0.7481 mIoU. The dataset is available at https://github.com/mvaldenegro/marine-debris-fls-datasets/