OpenRaiser/Intern-Atlas huggingface dataset:
Cyber-Physical-Systems and Secrecy Outage Probability: Revisited
Venue: arXiv cs.IT 2022
Year: 2022

This paper technically explores the secrecy rate $Λ$ and a maximisation problem over the concave version of the secrecy outage probability (SOP) as $\mathop{{\rm \mathbb{M}ax}}\limits_{Δ} {\rm \; } \mathbb{P}\mathscr{r} \big( Λ\ge λ\big) $. We do this from a generic viewpoint even though we use a traditional Wyner's wiretap channel for our system model $-$ something that can be extended to every kind of secrecy modeling and analysis. We consider a Riemannian mani-fold for it and we mathematically define a volume for it as $\mathbb{V}\mathscr{ol}\big \lbrace Λ\big \rbrace$. Through achieving a new bound for the Riemannian mani-fold and its volume, we subsequently relate it to the number of eigen-values existing in the relative probabilistic closure. We prove in-between some novel lemmas with the aid of some useful inequalities such as the \textit{Finsler's} lemma, the generalised \textit{Young's} inequality, the generalised \textit{Brunn-Minkowski} inequality, the \textit{Talagrand's} concentration inequality. We additionally propose a novel Markov decision process based reinforcement learning algorithm in order to find the optimal policy in relation to the eigenvalue distributions $-$ something that is extended to a possibilisitically semi-Markov decision process for the case of periodic attacks.
Novel Information-theoretic Game-theoretical Insights to Broadcasting in Internet-of-UAVs
Venue: arXiv cs.IT 2022
Year: 2022

For the Internet-of-unmanned aerial vehicles (UAVs) some challenges in broadcasting and from new points of view are explored. In this paper, first, we investigate a single broadcast transceiver. From a control of noisy-channel viewpoint, we consider: (\textit{i}) Alice sends $\mathcal{X}$ to Bob as more \textcolor{black}{efficient} as possible while she wishes Bob not to get access to the private message $\mathcal{S}$ regarding the correlation between $\mathcal{S}$ and $\mathcal{X}$ $-$ i.e., Alice purposefully sends a \textit{turbulent-flow} of the information to Bob; and (\textit{ii}) where $\big (Θ_1;Θ_2 \big)$ is the control-action-pair which actualise a \textit{pursuit-Evasion}. We consider \textit{dissipativity} in our system due to the memory effect relating to the previous states. We thus propose a federated-learning based \textit{Blahut-Arimoto} algorithm while a 2-D \textit{dissipativity}-theoretic continuous-Mean-Field-Game (MFG) is proposed with regard to (w.r.t.) a joint probability-distribution-function (PDF) of the population distribution $-$ relating to a continuous-control-law. We also analyse what if Alice is owed to multiple Bobs in a multi-user scenario which we apply a bankruptcy based $3-$level nested game for.
Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentation
Venue: arXiv cs.CL 2022
Year: 2022

Common designs of model evaluation typically focus on monolingual settings, where different models are compared according to their performance on a single data set that is assumed to be representative of all possible data for the task at hand. While this may be reasonable for a large data set, this assumption is difficult to maintain in low-resource scenarios, where artifacts of the data collection can yield data sets that are outliers, potentially making conclusions about model performance coincidental. To address these concerns, we investigate model generalizability in crosslinguistic low-resource scenarios. Using morphological segmentation as the test case, we compare three broad classes of models with different parameterizations, taking data from 11 languages across 6 language families. In each experimental setting, we evaluate all models on a first data set, then examine their performance consistency when introducing new randomly sampled data sets with the same size and when applying the trained models to unseen test sets of varying sizes. The results demonstrate that the extent of model generalization depends on the characteristics of the data set, and does not necessarily rely heavily on the data set size. Among the characteristics that we studied, the ratio of morpheme overlap and that of the average number of morphemes per word between the training and test sets are the two most prominent factors. Our findings suggest that future work should adopt random sampling to construct data sets with different sizes in order to make more responsible claims about model evaluation.
Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model
Venue: arXiv cs.CL 2022
Year: 2022

Despite the rapid progress of end-to-end (E2E) automatic speech recognition (ASR), it has been shown that incorporating external language models (LMs) into the decoding can further improve the recognition performance of E2E ASR systems. To align with the modeling units adopted in E2E ASR systems, subword-level (e.g., characters, BPE) LMs are usually used to cooperate with current E2E ASR systems. However, the use of subword-level LMs will ignore the word-level information, which may limit the strength of the external LMs in E2E ASR. Although several methods have been proposed to incorporate word-level external LMs in E2E ASR, these methods are mainly designed for languages with clear word boundaries such as English and cannot be directly applied to languages like Mandarin, in which each character sequence can have multiple corresponding word sequences. To this end, we propose a novel decoding algorithm where a word-level lattice is constructed on-the-fly to consider all possible word sequences for each partial hypothesis. Then, the LM score of the hypothesis is obtained by intersecting the generated lattice with an external word N-gram LM. The proposed method is examined on both Attention-based Encoder-Decoder (AED) and Neural Transducer (NT) frameworks. Experiments suggest that our method consistently outperforms subword-level LMs, including N-gram LM and neural network LM. We achieve state-of-the-art results on both Aishell-1 (CER 4.18%) and Aishell-2 (CER 5.06%) datasets and reduce CER by 14.8% relatively on a 21K-hour Mandarin dataset.
Flexible FPGA ECDSA Design with a Field Multiplier Inherently Resistant against HCCA
Venue: arXiv cs.CR 2022
Year: 2022

In this paper we describe our flexible ECDSA design for elliptic curve over binary extended fields GF(2l). We investigated its resistance against Horizontal Collision Correlation Attacks (HCCA). Due to the fact that our design is based on the Montgomery kP algorithm using Lopez-Dahab projective coordinates the scalar k cannot be successful revealed using HCCA, but this kind of attacks can be helpful to divide the measured traces into parts that correspond to processing of a single bit of the scalar k. The most important contribution of this paper is that our flexible field multiplier is resistant against horizontal attacks. This inherent resistance makes it a valuable building block for designing unified field multipliers.
On the Performance of Reed-Muller Codes Over $(d,\infty)$-RLL Input-Constrained BMS Channels
Venue: arXiv cs.IT 2022
Year: 2022

This paper considers the input-constrained binary memoryless symmetric (BMS) channel, without feedback. The channel input sequence respects the $(d,\infty)$-runlength limited (RLL) constraint, which mandates that any pair of successive $1$s be separated by at least $d$ $0$s. We consider the problem of designing explicit codes for such channels. In particular, we work with the Reed-Muller (RM) family of codes, which were shown by Reeves and Pfister (2021) to achieve the capacity of any unconstrained BMS channel, under bit-MAP decoding. We show that it is possible to pick $(d,\infty)$-RLL subcodes of a capacity-achieving (over the unconstrained BMS channel) sequence of RM codes such that the subcodes achieve, under bit-MAP decoding, rates of $C\cdot{2^{-\left \lceil \log_2(d+1)\right \rceil}}$, where $C$ is the capacity of the BMS channel. Finally, we also introduce techniques for upper bounding the rate of any $(1,\infty)$-RLL subcode of a specific capacity-achieving sequence of RM codes.
Traffic Flow Modeling for UAV-Enabled Wireless Networks
Venue: arXiv cs.NI 2022
Year: 2022

This paper investigates traffic flow modeling issue in multi-services oriented unmanned aerial vehicle (UAV)-enabled wireless networks, which is critical for supporting future various applications of such networks. We propose a general traffic flow model for multi-services oriented UAV-enable wireless networks. Under this model, we first classify the network services into three subsets: telemetry, Internet of Things (IoT), and streaming data. Based on the Pareto distribution, we then partition all UAVs into three subgroups with different network usage. We further determine the number of packets for different network services and total data size according to the packet arrival rate for the nine segments, each of which represents one map relationship between a subset of services and a subgroup of UAVs. Simulation results are provided to illustrate that the number of packets and the data size predicted by our traffic model can well match with these under a real scenario.
Deep Learning Based Classification System For Recognizing Local Spinach
Venue: arXiv cs.CV 2022
Year: 2022

A deep learning model gives an incredible result for image processing by studying from the trained dataset. Spinach is a leaf vegetable that contains vitamins and nutrients. In our research, a Deep learning method has been used that can automatically identify spinach and this method has a dataset of a total of five species of spinach that contains 3785 images. Four Convolutional Neural Network (CNN) models were used to classify our spinach. These models give more accurate results for image classification. Before applying these models there is some preprocessing of the image data. For the preprocessing of data, some methods need to happen. Those are RGB conversion, filtering, resize & rescaling, and categorization. After applying these methods image data are pre-processed and ready to be used in the classifier algorithms. The accuracy of these classifiers is in between 98.68% - 99.79%. Among those models, VGG16 achieved the highest accuracy of 99.79%.
Implementing simple spectral denoising for environmental audio recordings
Venue: arXiv cs.SD 2022
Year: 2022

This technical report details changes applied to a noise filter to facilitate its application and improve its results. The filter is applied to denoise natural sounds recorded in the wild and to generate an acoustic index used in soundscape analysis.
Large-scale protein-protein post-translational modification extraction with distant supervision and confidence calibrated BioBERT
Venue: arXiv cs.LG 2022
Year: 2022

Protein-protein interactions (PPIs) are critical to normal cellular function and are related to many disease pathways. However, only 4% of PPIs are annotated with PTMs in biological knowledge databases such as IntAct, mainly performed through manual curation, which is neither time nor cost-effective. We use the IntAct PPI database to create a distant supervised dataset annotated with interacting protein pairs, their corresponding PTM type, and associated abstracts from the PubMed database. We train an ensemble of BioBERT models - dubbed PPI-BioBERT-x10 to improve confidence calibration. We extend the use of ensemble average confidence approach with confidence variation to counteract the effects of class imbalance to extract high confidence predictions. The PPI-BioBERT-x10 model evaluated on the test set resulted in a modest F1-micro 41.3 (P =5 8.1, R = 32.1). However, by combining high confidence and low variation to identify high quality predictions, tuning the predictions for precision, we retained 19% of the test predictions with 100% precision. We evaluated PPI-BioBERT-x10 on 18 million PubMed abstracts and extracted 1.6 million (546507 unique PTM-PPI triplets) PTM-PPI predictions, and filter ~ 5700 (4584 unique) high confidence predictions. Of the 5700, human evaluation on a small randomly sampled subset shows that the precision drops to 33.7% despite confidence calibration and highlights the challenges of generalisability beyond the test set even with confidence calibration. We circumvent the problem by only including predictions associated with multiple papers, improving the precision to 58.8%. In this work, we highlight the benefits and challenges of deep learning-based text mining in practice, and the need for increased emphasis on confidence calibration to facilitate human curation efforts.
Consistent Style Transfer
Venue: arXiv cs.CV 2022
Year: 2022

Recently, attentional arbitrary style transfer methods have been proposed to achieve fine-grained results, which manipulates the point-wise similarity between content and style features for stylization. However, the attention mechanism based on feature points ignores the feature multi-manifold distribution, where each feature manifold corresponds to a semantic region in the image. Consequently, a uniform content semantic region is rendered by highly different patterns from various style semantic regions, producing inconsistent stylization results with visual artifacts. We proposed the progressive attentional manifold alignment (PAMA) to alleviate this problem, which repeatedly applies attention operations and space-aware interpolations. The attention operation rearranges style features dynamically according to the spatial distribution of content features. This makes the content and style manifolds correspond on the feature map. Then the space-aware interpolation adaptively interpolates between the corresponding content and style manifolds to increase their similarity. By gradually aligning the content manifolds to style manifolds, the proposed PAMA achieves state-of-the-art performance while avoiding the inconsistency of semantic regions. Codes are available at https://github.com/computer-vision2022/PAMA.
Budget-aware Few-shot Learning via Graph Convolutional Network
Venue: arXiv cs.CV 2022
Year: 2022

This paper tackles the problem of few-shot learning, which aims to learn new visual concepts from a few examples. A common problem setting in few-shot classification assumes random sampling strategy in acquiring data labels, which is inefficient in practical applications. In this work, we introduce a new budget-aware few-shot learning problem that not only aims to learn novel object categories, but also needs to select informative examples to annotate in order to achieve data efficiency. We develop a meta-learning strategy for our budget-aware few-shot learning task, which jointly learns a novel data selection policy based on a Graph Convolutional Network (GCN) and an example-based few-shot classifier. Our selection policy computes a context-sensitive representation for each unlabeled data by graph message passing, which is then used to predict an informativeness score for sequential selection. We validate our method by extensive experiments on the mini-ImageNet, tiered-ImageNet and Omniglot datasets. The results show our few-shot learning strategy outperforms baselines by a sizable margin, which demonstrates the efficacy of our method.
Multi-Behavior Enhanced Recommendation with Cross-Interaction Collaborative Relation Modeling
Venue: arXiv cs.IR 2022
Year: 2022

Many previous studies aim to augment collaborative filtering with deep neural network techniques, so as to achieve better recommendation performance. However, most existing deep learning-based recommender systems are designed for modeling singular type of user-item interaction behavior, which can hardly distill the heterogeneous relations between user and item. In practical recommendation scenarios, there exist multityped user behaviors, such as browse and purchase. Due to the overlook of user's multi-behavioral patterns over different items, existing recommendation methods are insufficient to capture heterogeneous collaborative signals from user multi-behavior data. Inspired by the strength of graph neural networks for structured data modeling, this work proposes a Graph Neural Multi-Behavior Enhanced Recommendation (GNMR) framework which explicitly models the dependencies between different types of user-item interactions under a graph-based message passing architecture. GNMR devises a relation aggregation network to model interaction heterogeneity, and recursively performs embedding propagation between neighboring nodes over the user-item interaction graph. Experiments on real-world recommendation datasets show that our GNMR consistently outperforms state-of-the-art methods. The source code is available at https://github.com/akaxlh/GNMR.
A Transfer Learning Pipeline for Educational Resource Discovery with Application in Leading Paragraph Generation
Venue: arXiv cs.CL 2022
Year: 2022

Effective human learning depends on a wide selection of educational materials that align with the learner's current understanding of the topic. While the Internet has revolutionized human learning or education, a substantial resource accessibility barrier still exists. Namely, the excess of online information can make it challenging to navigate and discover high-quality learning materials. In this paper, we propose the educational resource discovery (ERD) pipeline that automates web resource discovery for novel domains. The pipeline consists of three main steps: data collection, feature extraction, and resource classification. We start with a known source domain and conduct resource discovery on two unseen target domains via transfer learning. We first collect frequent queries from a set of seed documents and search on the web to obtain candidate resources, such as lecture slides and introductory blog posts. Then we introduce a novel pretrained information retrieval deep neural network model, query-document masked language modeling (QD-MLM), to extract deep features of these candidate resources. We apply a tree-based classifier to decide whether the candidate is a positive learning resource. The pipeline achieves F1 scores of 0.94 and 0.82 when evaluated on two similar but novel target domains. Finally, we demonstrate how this pipeline can benefit an application: leading paragraph generation for surveys. This is the first study that considers various web resources for survey generation, to the best of our knowledge. We also release a corpus of 39,728 manually labeled web resources and 659 queries from NLP, Computer Vision (CV), and Statistics (STATS).
Decision problem of some bundled FOML fragments
Venue: arXiv cs.LO 2022
Year: 2022

Over increasing domain interpretations, \exists\Box and \forall\Box bundled fragments are decidable and over constant domain interpretations, \exists\Box bundled fragment is decidable while \forall\Box bundled fragment is undecidable. Based on the existing results,we show that over increasing domain interpretations, \Box\exists and \Box\forall bundled fragments are decidable as well. On the other hand, over constant domain interpretations, \Box\forall bundled fragment is undecidable and \Box\exists^2 bundled fragment, an extension of \Box\exists bundled fragment, is undecidable neither.
SaL-Lightning Dataset: Search and Eye Gaze Behavior, Resource Interactions and Knowledge Gain during Web Search
Venue: arXiv cs.IR 2022
Year: 2022

The emerging research field Search as Learning investigates how the Web facilitates learning through modern information retrieval systems. SAL research requires significant amounts of data that capture both search behavior of users and their acquired knowledge in order to obtain conclusive insights or train supervised machine learning models. However, the creation of such datasets is costly and requires interdisciplinary efforts in order to design studies and capture a wide range of features. In this paper, we address this issue and introduce an extensive dataset based on a user study, in which $114$ participants were asked to learn about the formation of lightning and thunder. Participants' knowledge states were measured before and after Web search through multiple-choice questionnaires and essay-based free recall tasks. To enable future research in SAL-related tasks we recorded a plethora of features and person-related attributes. Besides the screen recordings, visited Web pages, and detailed browsing histories, a large number of behavioral features and resource features were monitored. We underline the usefulness of the dataset by describing three, already published, use cases.
Methods for Increasing the Resistance of Cryptographic Designs against Horizontal DPA Attacks
Venue: arXiv cs.CR 2022
Year: 2022

Side-channel analysis attacks, especially horizontal DPA and DEMA attacks, are significant threats for cryptographic designs. In this paper we investigate to which extend different multiplication formulae and randomization of the field multiplier increase the resistance of an ECC design against horizontal attacks. We implemented a randomized sequence of the calculation of partial products for the field multiplication in order to increase the security features of the field multiplier. Additionally, we use the partial polynomial multiplier itself as a kind of countermeasure against DPA attacks. We demonstrate that the implemented classical multiplication formula can increase the inherent resistance of the whole ECC design. We also investigate the impact of the combination of these two approaches. For the evaluation we synthesized all these designs for a 250 nm gate library technologies, and analysed the simulated power traces. All investigated protection means help to decrease the success rate of attacks significantly: the correctness of the revealed key was decreased from 99% to 69%.
Bregman divergence based em algorithm and its application to classical and quantum rate distortion theory
Venue: arXiv cs.IT 2022
Year: 2022

We formulate em algorithm in the framework of Bregman divergence, which is a general problem setting of information geometry. That is, we address the minimization problem of the Bregman divergence between an exponential subfamily and a mixture subfamily in a Bregman divergence system. Then, we show the convergence and its speed under several conditions. We apply this algorithm to rate distortion and its variants including the quantum setting, and show the usefulness of our general algorithm. In fact, existing applications of Arimoto-Blahut algorithm to rate distortion theory make the optimization of the weighted sum of the mutual information and the cost function by using the Lagrange multiplier. However, in the rate distortion theory, it is needed to minimize the mutual information under the constant constraint for the cost function. Our algorithm directly solves this minimization. In addition, we have numerically checked the convergence speed of our algorithm in the classical case of rate distortion problem.
Semantic-based Data Augmentation for Math Word Problems
Venue: arXiv cs.CL 2022
Year: 2022

It's hard for neural MWP solvers to deal with tiny local variances. In MWP task, some local changes conserve the original semantic while the others may totally change the underlying logic. Currently, existing datasets for MWP task contain limited samples which are key for neural models to learn to disambiguate different kinds of local variances in questions and solve the questions correctly. In this paper, we propose a set of novel data augmentation approaches to supplement existing datasets with such data that are augmented with different kinds of local variances, and help to improve the generalization ability of current neural models. New samples are generated by knowledge guided entity replacement, and logic guided problem reorganization. The augmentation approaches are ensured to keep the consistency between the new data and their labels. Experimental results have shown the necessity and the effectiveness of our methods.