OpenRaiser/Intern-Atlas huggingface dataset:
True Few-Shot Learning with Prompts -- A Real-World Perspective
Venue: arXiv cs.CL 2021
Year: 2021

Prompt-based approaches are strong at few-shot learning. However, Perez et al. (2021) have recently cast doubt on their performance because they had difficulty getting good results in a "true" few-shot setting in which prompts and hyperparameters cannot be tuned on a dev set. In view of this, we conduct an extensive study of PET, a method that combines textual instructions with example-based finetuning. We show that, if correctly configured, PET performs strongly in a true few-shot setting, i.e., without a dev set. Crucial for this strong performance is PET's ability to intelligently handle multiple prompts. We then put our findings to a real-world test by running PET on RAFT, a benchmark of tasks taken directly from realistic NLP applications for which no labeled dev or test sets are available. PET achieves a new state of the art on RAFT and performs close to non-expert humans for 7 out of 11 tasks. These results demonstrate that prompt-based learners like PET excel at true few-shot learning and underpin our belief that learning from instructions will play an important role on the path towards human-like few-shot learning capabilities.
How heavy independent sets help to find arborescences with many leaves in DAGs
Venue: arXiv cs.DS 2021
Year: 2021

Trees with many leaves have applications on broadcasting, which is a method in networks for transferring a message to all recipients simultaneously. Internal nodes of a broadcasting tree require more expensive technology, because they have to forward the messages received. We address a problem that captures the main goal, which is to find spanning trees with few internal nodes in a given network. The Maximum Leaf Spanning Arborescence problem consists of, given a directed graph D, finding a spanning arborescence of D, if one exists, with the maximum number of leaves. This problem is known to be NP-hard in general and MaxSNP-hard on the class of rooted directed acyclic graphs. In this paper, we explore a relation between Maximum Leaf Spanning Arborescence in rooted directed acyclic graphs and maximum weight set packing. The latter problem is related to independent sets on particular classes of intersection graphs. Exploiting this relation, we derive a 7/5-approximation for Maximum Leaf Spanning Arborescence on rooted directed acyclic graphs, improving on the previous 3/2-approximation. The approach used might lead to improvements on the best approximation ratios for the weighted k-set packing problem.
Emotion Embedding Spaces for Matching Music to Stories
Venue: arXiv cs.IR 2021
Year: 2021

Content creators often use music to enhance their stories, as it can be a powerful tool to convey emotion. In this paper, our goal is to help creators find music to match the emotion of their story. We focus on text-based stories that can be auralized (e.g., books), use multiple sentences as input queries, and automatically retrieve matching music. We formalize this task as a cross-modal text-to-music retrieval problem. Both the music and text domains have existing datasets with emotion labels, but mismatched emotion vocabularies prevent us from using mood or emotion annotations directly for matching. To address this challenge, we propose and investigate several emotion embedding spaces, both manually defined (e.g., valence/arousal) and data-driven (e.g., Word2Vec and metric learning) to bridge this gap. Our experiments show that by leveraging these embedding spaces, we are able to successfully bridge the gap between modalities to facilitate cross modal retrieval. We show that our method can leverage the well established valence-arousal space, but that it can also achieve our goal via data-driven embedding spaces. By leveraging data-driven embeddings, our approach has the potential of being generalized to other retrieval tasks that require broader or completely different vocabularies.
State Complexity Investigations on Commutative Languages -- The Upward and Downward Closure, Commutative Aperiodic and Commutative Group Languages
Venue: arXiv cs.FL 2021
Year: 2021

We investigate the state complexity of the upward and downward closure and interior operations on commutative regular languages. Then, we systematically study the state complexity of these operations and of the shuffle operation on commutative group languages and commutative aperiodic (or star-free) languages.
Fabric-SCF: A Blockchain-based Secure Storage and Access Control Scheme for Supply Chain Finance
Venue: arXiv cs.CR 2021
Year: 2021

Supply chain finance(SCF) is committed to providing credit for small and medium-sized enterprises(SMEs) with low credit lines and small financing scales. The resulting financial credit data and related business transaction data are highly confidential and private. However, traditional SCF management schemes mostly use third-party platforms and centralized designs, which cannot achieve highly reliable secure storage and fine-grained access control. To fill this gap, this paper designs and implements Fabric-SCF, a secure storage and access control system based on blockchain and attribute-based access control (\textbf{ABAC}) model. This scheme uses distributed consensus to realize data security, traceability, and immutability. We also use smart contracts to define system processes and access policies to ensure the efficient operation of the system. To verify the performance of Fabric-SCF, we designed two sets of simulation experiments. The results show that Fabric-SCF achieves dynamic and fine-grained access control while maintaining high throughput in a simulated real-world operating scenario.
A Survey of Blockchain Data Management Systems
Venue: arXiv cs.CR 2021
Year: 2021

Blockchain has been widely deployed in various sectors, such as finance, education, and public services. Since blockchain runs as an immutable distributed ledger, it has decentralized mechanisms with persistency, anonymity, and auditability, where transactions are jointly performed through cryptocurrency-based consensus algorithms by worldwide distributed nodes. There have been many survey papers reviewing the blockchain technologies from different perspectives, e.g., digital currencies, consensus algorithms, and smart contracts. However, none of them have focused on the blockchain data management systems. To fill in this gap, we have conducted a comprehensive survey on the data management systems, based on three typical types of blockchain, i.e., standard blockchain, hybrid blockchain, and DAG (Directed Acyclic Graph)-based blockchain. We categorize their data management mechanisms into three layers: blockchain architecture, blockchain data structure, and blockchain storage engine, where block architecture indicates how to record transactions on a distributed ledger, blockchain data structure refers to the internal structure of each block, and blockchain storage engine specifies the storage form of data on the blockchain system. For each layer, the works advancing the state-of-the-art are discussed together with technical challenges. Furthermore, we lay out the future research directions for the blockchain data management systems.
Demystifying Ten Big Ideas and Rules Every Fire Scientist & Engineer Should Know About Blackbox, Whitebox & Causal Artificial Intelligence
Venue: arXiv cs.LG 2021
Year: 2021

Artificial intelligence (AI) is paving the way towards the fourth industrial revolution with the fire domain (Fire 4.0). As a matter of fact, the next few years will be elemental to how this technology will shape our academia, practice, and entrepreneurship. Despite the growing interest between fire research groups, AI remains absent of our curriculum, and we continue to lack a methodical framework to adopt, apply and create AI solutions suitable for our problems. The above is also true for parallel engineering domains (i.e., civil/mechanical engineering), and in order to negate the notion of history repeats itself (e.g., look at the continued debate with regard to modernizing standardized fire testing, etc.), it is the motivation behind this letter to the Editor to demystify some of the big ideas behind AI to jump-start prolific and strategic discussions on the front of AI & Fire. In addition, this letter intends to explain some of the most fundamental concepts and clear common misconceptions specific to the adoption of AI in fire engineering. This short letter is a companion to the Smart Systems in Fire Engineering special issue sponsored by Fire Technology. An in-depth review of AI algorithms [1] and success stories to the proper implementations of such algorithms can be found in the aforenoted special issue and collection of papers. This letter comprises two sections. The first section outlines big ideas pertaining to AI, and answers some of the burning questions with regard to the merit of adopting AI in our domain. The second section presents a set of rules or technical recommendations an AI user may deem helpful to practice whenever AI is used as an investigation methodology. The presented set of rules are complementary to the big ideas.
DSC: Deep Scan Context Descriptor for Large-Scale Place Recognition
Venue: arXiv cs.CV 2021
Year: 2021

LiDAR-based place recognition is an essential and challenging task both in loop closure detection and global relocalization. We propose Deep Scan Context (DSC), a general and discriminative global descriptor that captures the relationship among segments of a point cloud. Unlike previous methods that utilize either semantics or a sequence of adjacent point clouds for better place recognition, we only use raw point clouds to get competitive results. Concretely, we first segment the point cloud egocentrically to acquire centroids and eigenvalues of the segments. Then, we introduce a graph neural network to aggregate these features into an embedding representation. Extensive experiments conducted on the KITTI dataset show that DSC is robust to scene variants and outperforms existing methods.
Achieving Energy-Efficient Uplink URLLC with MIMO-Aided Grant-Free Access
Venue: arXiv cs.IT 2021
Year: 2021

The optimal design of the energy-efficient multiple-input multiple-output (MIMO) aided uplink ultra-reliable low-latency communications (URLLC) system is an important but unsolved problem. For such a system, we propose a novel absorbing-Markov-chain-based analysis framework to shed light on the puzzling relationship between the delay and reliability, as well as to quantify the system energy efficiency. We derive the transition probabilities of the absorbing Markov chain considering the Rayleigh fading, the channel estimation error, the zero-forcing multi-user-detection (ZF-MUD), the grant-free access, the ACK-enabled retransmissions within the delay bound and the interactions among these technical ingredients. Then, the delay-constrained reliability and the system energy efficiency are derived based on the absorbing Markov chain formulated. Finally, we study the optimal number of user equipments (UEs) and the optimal number of receiving antennas that maximize the system energy efficiency, while satisfying the reliability and latency requirements of URLLC simultaneously. Simulation results demonstrate the accuracy of our theoretical analysis and the effectiveness of massive MIMO in supporting large-scale URLLC systems.
Towards Understanding the Impact of Model Size on Differential Private Classification
Venue: arXiv cs.LG 2021
Year: 2021

Differential privacy (DP) is an essential technique for privacy-preserving. It was found that a large model trained for privacy preserving performs worse than a smaller model (e.g. ResNet50 performs worse than ResNet18). To better understand this phenomenon, we study high dimensional DP learning from the viewpoint of generalization. Theoretically, we show that for the simple Gaussian model with even small DP noise, if the dimension is large enough, then the classification error can be as bad as the random guessing. Then we propose a feature selection method to reduce the size of the model, based on a new metric which trades off the classification accuracy and privacy preserving. Experiments on real data support our theoretical results and demonstrate the advantage of the proposed method.
Sparse Subspace Clustering Friendly Deep Dictionary Learning for Hyperspectral Image Classification
Venue: arXiv cs.CV 2021
Year: 2021

Subspace clustering techniques have shown promise in hyperspectral image segmentation. The fundamental assumption in subspace clustering is that the samples belonging to different clusters/segments lie in separable subspaces. What if this condition does not hold? We surmise that even if the condition does not hold in the original space, the data may be nonlinearly transformed to a space where it will be separable into subspaces. In this work, we propose a transformation based on the tenets of deep dictionary learning (DDL). In particular, we incorporate the sparse subspace clustering (SSC) loss in the DDL formulation. Here DDL nonlinearly transforms the data such that the transformed representation (of the data) is separable into subspaces. We show that the proposed formulation improves over the state-of-the-art deep learning techniques in hyperspectral image clustering.
On lossy Compression of Directed Graphs
Venue: arXiv cs.IT 2021
Year: 2021

The method of types presented by Csiszar and Korner is a central tool used to develop and analyze the basic properties and constraints on sequences of data over finite alphabets. A central problem considered using these tools is that of data compression, and specifically lossy data compression. In this work we consider this very problem, however, instead of sequences of data we consider directed graphs. We show that given a more natural distortion measure, fitting the data structure of a directed graph, the method of types cannot be applied. The suggested distortion measure aims to preserves the local structure of a directed graph. We build on the recent work of Barvinok and extend the method of types to the two dimensional setting of directed graphs. We see that the extension is quite natural in many ways. Given this extension we provide a lower and upper bound on the rate-distortion problem of lossy compression given the suggested distortion measure.
Exploring Transformer Based Models to Identify Hate Speech and Offensive Content in English and Indo-Aryan Languages
Venue: arXiv cs.CL 2021
Year: 2021

Hate speech is considered to be one of the major issues currently plaguing online social media. Repeated and repetitive exposure to hate speech has been shown to create physiological effects on the target users. Thus, hate speech, in all its forms, should be addressed on these platforms in order to maintain good health. In this paper, we explored several Transformer based machine learning models for the detection of hate speech and offensive content in English and Indo-Aryan languages at FIRE 2021. We explore several models such as mBERT, XLMR-large, XLMR-base by team name "Super Mario". Our models came 2nd position in Code-Mixed Data set (Macro F1: 0.7107), 2nd position in Hindi two-class classification(Macro F1: 0.7797), 4th in English four-class category (Macro F1: 0.8006) and 12th in English two-class category (Macro F1: 0.6447).
Deep Q-Learning based Reinforcement Learning Approach for Network Intrusion Detection
Venue: arXiv cs.CR 2021
Year: 2021

The rise of the new generation of cyber threats demands more sophisticated and intelligent cyber defense solutions equipped with autonomous agents capable of learning to make decisions without the knowledge of human experts. Several reinforcement learning methods (e.g., Markov) for automated network intrusion tasks have been proposed in recent years. In this paper, we introduce a new generation of network intrusion detection methods that combines a Q-learning-based reinforcement learning with a deep-feed forward neural network method for network intrusion detection. Our proposed Deep Q-Learning (DQL) model provides an ongoing auto-learning capability for a network environment that can detect different types of network intrusions using an automated trial-error approach and continuously enhance its detection capabilities. We provide the details of fine-tuning different hyperparameters involved in the DQL model for more effective self-learning. According to our extensive experimental results based on the NSL-KDD dataset, we confirm that the lower discount factor which is set as 0.001 under 250 episodes of training yields the best performance results. Our experimental results also show that our proposed DQL is highly effective in detecting different intrusion classes and outperforms other similar machine learning approaches.
Multi-modality fusion using canonical correlation analysis methods: Application in breast cancer survival prediction from histology and genomics
Venue: arXiv cs.LG 2021
Year: 2021

The availability of multi-modality datasets provides a unique opportunity to characterize the same object of interest using multiple viewpoints more comprehensively. In this work, we investigate the use of canonical correlation analysis (CCA) and penalized variants of CCA (pCCA) for the fusion of two modalities. We study a simple graphical model for the generation of two-modality data. We analytically show that, with known model parameters, posterior mean estimators that jointly use both modalities outperform arbitrary linear mixing of single modality posterior estimators in latent variable prediction. Penalized extensions of CCA (pCCA) that incorporate domain knowledge can discover correlations with high-dimensional, low-sample data, whereas traditional CCA is inapplicable. To facilitate the generation of multi-dimensional embeddings with pCCA, we propose two matrix deflation schemes that enforce desirable properties exhibited by CCA. We propose a two-stage prediction pipeline using pCCA embeddings generated with deflation for latent variable prediction by combining all the above. On simulated data, our proposed model drastically reduces the mean-squared error in latent variable prediction. When applied to publicly available histopathology data and RNA-sequencing data from The Cancer Genome Atlas (TCGA) breast cancer patients, our model can outperform principal components analysis (PCA) embeddings of the same dimension in survival prediction.
Learning Continuous Environment Fields via Implicit Functions
Venue: arXiv cs.CV 2021
Year: 2021

We propose a novel scene representation that encodes reaching distance -- the distance between any position in the scene to a goal along a feasible trajectory. We demonstrate that this environment field representation can directly guide the dynamic behaviors of agents in 2D mazes or 3D indoor scenes. Our environment field is a continuous representation and learned via a neural implicit function using discretely sampled training data. We showcase its application for agent navigation in 2D mazes, and human trajectory prediction in 3D indoor environments. To produce physically plausible and natural trajectories for humans, we additionally learn a generative model that predicts regions where humans commonly appear, and enforce the environment field to be defined within such regions. Extensive experiments demonstrate that the proposed method can generate both feasible and plausible trajectories efficiently and accurately.
AI-supported Framework of Semi-Automatic Monoplotting for Monocular Oblique Visual Data Analysis
Venue: arXiv cs.CV 2021
Year: 2021

In the last decades, the development of smartphones, drones, aerial patrols, and digital cameras enabled high-quality photographs available to large populations and, thus, provides an opportunity to collect massive data of the nature and society with global coverage. However, the data collected with new photography tools is usually oblique - they are difficult to be georeferenced, and huge amounts of data is often obsolete. Georeferencing oblique imagery data may be solved by a technique called monoplotting, which only requires a single image and Digital Elevation Model (DEM). In traditional monoplotting, a human user has to manually choose a series of ground control point (GCP) pairs in the image and DEM and then determine the extrinsic and intrinsic parameters of the camera to establish a pixel-level correspondence between photos and the DEM to enable the mapping and georeferencing of objects in photos. This traditional method is difficult to scale due to several challenges including the labor-intensive inputs, the need of rich experience to identify well-defined GCPs, and limitations in camera pose estimation. Therefore, existing monoplotting methods are rarely used in analyzing large-scale databases or near-real-time warning systems. In this paper, we propose and demonstrate a novel semi-automatic monoplotting framework that provides pixel-level correspondence between photos and DEMs requiring minimal human interventions. A pipeline of analyses was developed including key point detection in images and DEM rasters, retrieving georeferenced 3D DEM GCPs, regularized gradient-based optimization, pose estimation, ray tracing, and the correspondence identification between image pixels and real world coordinates. Two numerical experiments show that the framework is superior in georeferencing visual data in 3-D coordinates, paving a way toward fully automatic monoplotting methodology.
Bounds and Constructions for Insertion and Deletion Codes
Venue: arXiv cs.IT 2021
Year: 2021

The present paper mainly studies limits and constructions of insertion and deletion (insdel for short) codes. The paper can be divided into two parts. The first part focuses on various bounds, while the second part concentrates on constructions of insdel codes. Although the insdel-metric Singleton bound has been derived before, it is still unknown if there are any nontrivial codes achieving this bound. Our first result shows that any nontrivial insdel codes do not achieve the insdel-metric Singleton bound. The second bound shows that every $[n,k]$ Reed-Solomon code has insdel distance upper bounded by $2n-4k+4$ and it is known in literature that an $[n,k]$ Reed-Solomon code can have insdel distance $2n-4k+4$ as long as the field size is sufficiently large. The third bound shows a trade-off between insdel distance and code alphabet size for codes achieving the Hamming-metric Singleton bound. In the second part of the paper, we first provide a non-explicit construction of nonlinear codes that can approach the insdel-metric Singleton bound arbitrarily when the code alphabet size is sufficiently large. The second construction gives two-dimensional Reed-Solomon codes of length $n$ and insdel distance $2n-4$ with field size $q=O(n^5)$.
Learning Wildfire Model from Incomplete State Observations
Venue: arXiv cs.LG 2021
Year: 2021

As wildfires are expected to become more frequent and severe, improved prediction models are vital to mitigating risk and allocating resources. With remote sensing data, valuable spatiotemporal statistical models can be created and used for resource management practices. In this paper, we create a dynamic model for future wildfire predictions of five locations within the western United States through a deep neural network via historical burned area and climate data. The proposed model has distinct features that address the characteristic need in prediction evaluations, including dynamic online estimation and time-series modeling. Between locations, local fire event triggers are not isolated, and there are confounding factors when local data is analyzed due to incomplete state observations. When compared to existing approaches that do not account for incomplete state observation within wildfire time-series data, on average, we are able to achieve higher prediction performances.