OpenRaiser/Intern-Atlas huggingface dataset:
R2P2: Reactive Routing and Payment Protocol for Named Data Network using Blockchain
Venue: arXiv cs.NI 2021
Year: 2021

With the continuous emergence of new mobile devices which support new communication paradigms such as D2D and V2V, Internet users can take advantage of these devices to achieve better Internet connectivity and improve service quality. Meanwhile, packet forwarding brings extra costs to devices (e.g. electricity consumption), that hinders the realisation of successful ad-hoc networks. This paper proposes Reactive Routing and Payment Protocol (R2P2) to incentivise mobile devices to contribute idle networking resources and gain monetary returns. The routing and payment protocol is developed for Named-Data Network (NDN) because its content-centric nature can better support the intermittent and ephemeral communication requirements in ad-hoc networks. Blockchain is used as the settlement platform for transactions between devices because of its neutrality, robustness and trust. R2P2 is still an on-going project. The content of this paper focuses on the design of R2P2.
Generating a Doppelganger Graph: Resembling but Distinct
Venue: arXiv cs.LG 2021
Year: 2021

Deep generative models, since their inception, have become increasingly more capable of generating novel and perceptually realistic signals (e.g., images and sound waves). With the emergence of deep models for graph structured data, natural interests seek extensions of these generative models for graphs. Successful extensions were seen recently in the case of learning from a collection of graphs (e.g., protein data banks), but the learning from a single graph has been largely under explored. The latter case, however, is important in practice. For example, graphs in financial and healthcare systems contain so much confidential information that their public accessibility is nearly impossible, but open science in these fields can only advance when similar data are available for benchmarking. In this work, we propose an approach to generating a doppelganger graph that resembles a given one in many graph properties but nonetheless can hardly be used to reverse engineer the original one, in the sense of a near zero edge overlap. The approach is an orchestration of graph representation learning, generative adversarial networks, and graph realization algorithms. Through comparison with several graph generative models (either parameterized by neural networks or not), we demonstrate that our result barely reproduces the given graph but closely matches its properties. We further show that downstream tasks, such as node classification, on the generated graphs reach similar performance to the use of the original ones.
GST: Group-Sparse Training for Accelerating Deep Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Deep reinforcement learning (DRL) has shown remarkable success in sequential decision-making problems but suffers from a long training time to obtain such good performance. Many parallel and distributed DRL training approaches have been proposed to solve this problem, but it is difficult to utilize them on resource-limited devices. In order to accelerate DRL in real-world edge devices, memory bandwidth bottlenecks due to large weight transactions have to be resolved. However, previous iterative pruning not only shows a low compression ratio at the beginning of training but also makes DRL training unstable. To overcome these shortcomings, we propose a novel weight compression method for DRL training acceleration, named group-sparse training (GST). GST selectively utilizes block-circulant compression to maintain a high weight compression ratio during all iterations of DRL training and dynamically adapt target sparsity through reward-aware pruning for stable training. Thanks to the features, GST achieves a 25 \%p $\sim$ 41.5 \%p higher average compression ratio than the iterative pruning method without reward drop in Mujoco Halfcheetah-v2 and Mujoco humanoid-v2 environment with TD3 training.
Grad-CAM guided channel-spatial attention module for fine-grained visual classification
Venue: arXiv cs.CV 2021
Year: 2021

Fine-grained visual classification (FGVC) is becoming an important research field, due to its wide applications and the rapid development of computer vision technologies. The current state-of-the-art (SOTA) methods in the FGVC usually employ attention mechanisms to first capture the semantic parts and then discover their subtle differences between distinct classes. The channel-spatial attention mechanisms, which focus on the discriminative channels and regions simultaneously, have significantly improved the classification performance. However, the existing attention modules are poorly guided since part-based detectors in the FGVC depend on the network learning ability without the supervision of part annotations. As obtaining such part annotations is labor-intensive, some visual localization and explanation methods, such as gradient-weighted class activation mapping (Grad-CAM), can be utilized for supervising the attention mechanism. We propose a Grad-CAM guided channel-spatial attention module for the FGVC, which employs the Grad-CAM to supervise and constrain the attention weights by generating the coarse localization maps. To demonstrate the effectiveness of the proposed method, we conduct comprehensive experiments on three popular FGVC datasets, including CUB-$200$-$2011$, Stanford Cars, and FGVC-Aircraft datasets. The proposed method outperforms the SOTA attention modules in the FGVC task. In addition, visualizations of feature maps also demonstrate the superiority of the proposed method against the SOTA approaches.
A Linear Reduction Method for Local Differential Privacy and Log-lift
Venue: arXiv cs.IT 2021
Year: 2021

This paper considers the problem of publishing data $X$ while protecting correlated sensitive information $S$. We propose a linear method to generate the sanitized data $Y$ with the same alphabet $\mathcal{Y} = \mathcal{X}$ that attains local differential privacy (LDP) and log-lift at the same time. It is revealed that both LDP and log-lift are inversely proportional to the statistical distance between conditional probability $P_{Y|S}(x|s)$ and marginal probability $P_{Y}(x)$: the closer the two probabilities are, the more private $Y$ is. Specifying $P_{Y|S}(x|s)$ that linearly reduces this distance $|P_{Y|S}(x|s) - P_Y(x)| = (1-α)|P_{X|S}(x|s) - P_X(x)|,\forall s,x$ for some $α\in (0,1]$, we study the problem of how to generate $Y$ from the original data $S$ and $X$. The Markov randomization/sanitization scheme $P_{Y|X}(x|x') = P_{Y|S,X}(x|s,x')$ is obtained by solving linear equations. The optimal non-Markov sanitization, the transition probability $P_{Y|S,X}(x|s,x')$ that depends on $S$, can be determined by maximizing the data utility subject to linear equality constraints. We compute the solution for two linear utility function: the expected distance and total variance distance. It is shown that the non-Markov randomization significantly improves data utility and the marginal probability $P_X(x)$ remains the same after the linear sanitization method: $P_Y(x) = P_X(x), \forall x \in \mathcal{X}$.
Fast Sequence Generation with Multi-Agent Reinforcement Learning
Venue: arXiv cs.CL 2021
Year: 2021

Autoregressive sequence Generation models have achieved state-of-the-art performance in areas like machine translation and image captioning. These models are autoregressive in that they generate each word by conditioning on previously generated words, which leads to heavy latency during inference. Recently, non-autoregressive decoding has been proposed in machine translation to speed up the inference time by generating all words in parallel. Typically, these models use the word-level cross-entropy loss to optimize each word independently. However, such a learning process fails to consider the sentence-level consistency, thus resulting in inferior generation quality of these non-autoregressive models. In this paper, we propose a simple and efficient model for Non-Autoregressive sequence Generation (NAG) with a novel training paradigm: Counterfactuals-critical Multi-Agent Learning (CMAL). CMAL formulates NAG as a multi-agent reinforcement learning system where element positions in the target sequence are viewed as agents that learn to cooperatively maximize a sentence-level reward. On MSCOCO image captioning benchmark, our NAG method achieves a performance comparable to state-of-the-art autoregressive models, while brings 13.9x decoding speedup. On WMT14 EN-DE machine translation dataset, our method outperforms cross-entropy trained baseline by 6.0 BLEU points while achieves the greatest decoding speedup of 17.46x.
Standalone Deployment of a Dynamic Drone Cell for Wireless Connectivity of Two Services
Venue: arXiv cs.IT 2021
Year: 2021

We treat a setting in which two priority wireless service classes are offered in a given area by a drone small cell (DSC). Specifically, we consider broadband (BB) user with high priority and reliability requirements that coexists with random access machine-type-communications (MTC) devices. The drone serves both connectivity types with a combination of orthogonal slicing of the wireless resources and dynamic horizontal opportunistic positioning (D-HOP). We treat the D-HOP as a computational geometry function over stochastic BB user locations which requires careful adjustment in the deployment parameters to ensure MTC service at all times. Using an information theoretic approach, we optimize DSC deployment properties and radio resource allocation for the purpose of maximizing the average rate of BB users. While respecting the strict dual service requirements we analyze how system performance is affected by stochastic user positioning and density, topology, and reliability constraints combinations. The numerical results show that this approach outperforms static DSCs that fit the same coverage constraints, with outstanding performance in the urban setting.
A Novel Two-stage Framework for Extracting Opinionated Sentences from News Articles
Venue: arXiv cs.CL 2021
Year: 2021

This paper presents a novel two-stage framework to extract opinionated sentences from a given news article. In the first stage, Naive Bayes classifier by utilizing the local features assigns a score to each sentence - the score signifies the probability of the sentence to be opinionated. In the second stage, we use this prior within the HITS (Hyperlink-Induced Topic Search) schema to exploit the global structure of the article and relation between the sentences. In the HITS schema, the opinionated sentences are treated as Hubs and the facts around these opinions are treated as the Authorities. The algorithm is implemented and evaluated against a set of manually marked data. We show that using HITS significantly improves the precision over the baseline Naive Bayes classifier. We also argue that the proposed method actually discovers the underlying structure of the article, thus extracting various opinions, grouped with supporting facts as well as other supporting opinions from the article.
Deployable, Data-Driven Unmanned Vehicle Navigation System in GPS-Denied, Feature-Deficient Environments
Venue: arXiv cs.RO 2021
Year: 2021

This paper presents a novel data-driven navigation system to navigate an Unmanned Vehicle (UV) in GPS-denied, feature-deficient environments such as tunnels, or mines. The method utilizes landmarks that vehicle can deploy and measure range from to enable localization as the vehicle traverses its pre-defined path through the tunnel. A key question that arises in such scenario is to estimate and reduce the number of landmarks that needs to be deployed for localization before the start of the mission, given some information about the environment. The main focus is to keep the maximum position uncertainty at a desired value. In this article, we develop a novel vehicle navigation system in GPS-denied, feature-deficient environment by combining techniques from estimation, machine learning, and mixed-integer convex optimization. This article develops a novel, systematic method to perform localization and navigate the UV through the environment with minimum number of landmarks while maintaining desired localization accuracy. We also present extensive simulation experiments on different scenarios that corroborate the effectiveness of the proposed navigation system.
A Joint Representation Learning and Feature Modeling Approach for One-class Recognition
Venue: arXiv cs.CV 2021
Year: 2021

One-class recognition is traditionally approached either as a representation learning problem or a feature modeling problem. In this work, we argue that both of these approaches have their own limitations; and a more effective solution can be obtained by combining the two. The proposed approach is based on the combination of a generative framework and a one-class classification method. First, we learn generative features using the one-class data with a generative framework. We augment the learned features with the corresponding reconstruction errors to obtain augmented features. Then, we qualitatively identify a suitable feature distribution that reduces the redundancy in the chosen classifier space. Finally, we force the augmented features to take the form of this distribution using an adversarial framework. We test the effectiveness of the proposed method on three one-class classification tasks and obtain state-of-the-art results.
Information Bottleneck for an Oblivious Relay with Channel State Information: the Vector Case
Venue: arXiv cs.IT 2021
Year: 2021

This paper considers the information bottleneck (IB) problem of a Rayleigh fading multiple-input multiple-out (MIMO) channel. Due to the bottleneck constraint, it is impossible for the oblivious relay to inform the destination node of the perfect channel state information (CSI) in each channel realization. To evaluate the bottleneck rate, we provide an upper bound by assuming that the destination node can get the perfect CSI at no cost and two achievable schemes with simple symbol-by-symbol relay processing and compression. Numerical results show that the lower bounds obtained by the proposed achievable schemes can come close to the upper bound on a wide range of relevant system parameters.
Equitable Division of a Path
Venue: arXiv cs.GT 2021
Year: 2021

We study fair resource allocation under a connectedness constraint wherein a set of indivisible items are arranged on a path and only connected subsets of items may be allocated to the agents. An allocation is deemed fair if it satisfies equitability up to one good (EQ1), which requires that agents' utilities are approximately equal. We show that achieving EQ1 in conjunction with well-studied measures of economic efficiency (such as Pareto optimality, non-wastefulness, maximum egalitarian or utilitarian welfare) is computationally hard even for binary additive valuations. On the algorithmic side, we show that by relaxing the efficiency requirement, a connected EQ1 allocation can be computed in polynomial time for any given ordering of agents, even for general monotone valuations. Interestingly, the allocation computed by our algorithm has the highest egalitarian welfare among all allocations consistent with the given ordering. On the other hand, if efficiency is required, then tractability can still be achieved for binary additive valuations with interval structure. On our way, we strengthen some of the existing results in the literature for other fairness notions such as envy-freeness up to one good (EF1), and also provide novel results for negatively-valued items or chores.
Independent Spanning Trees in Eisenstein-Jacobi Networks
Venue: arXiv cs.DC 2021
Year: 2021

Spanning trees are widely used in networks for broadcasting, fault-tolerance, and securely delivering messages. Hexagonal interconnection networks have a number of real life applications. Examples are cellular networks, computer graphics, and image processing. Eisenstein-Jacobi (EJ) networks are a generalization of hexagonal mesh topology. They have a wide range of potential applications, and thus they have received researchers' attention in different areas among which interconnection networks and coding theory. In this paper, we present two spanning trees' constructions for Eisenstein-Jacobi (EJ). The first constructs three edge-disjoint node-independent spanning trees, while the second constructs six node-independent spanning trees but not edge disjoint. Based on the constructed trees, we develop routing algorithms that can securely deliver a message and tolerate a number of faults in point-to-point or in broadcast communications. The proposed work is also applied on higher dimensional EJ networks.
Annealed Stein Variational Gradient Descent
Venue: arXiv cs.LG 2021
Year: 2021

Particle based optimization algorithms have recently been developed as sampling methods that iteratively update a set of particles to approximate a target distribution. In particular Stein variational gradient descent has gained attention in the approximate inference literature for its flexibility and accuracy. We empirically explore the ability of this method to sample from multi-modal distributions and focus on two important issues: (i) the inability of the particles to escape from local modes and (ii) the inefficacy in reproducing the density of the different regions. We propose an annealing schedule to solve these issues and show, through various experiments, how this simple solution leads to significant improvements in mode coverage, without invalidating any theoretical properties of the original algorithm.
Deep Learning Generalization and the Convex Hull of Training Sets
Venue: arXiv cs.LG 2021
Year: 2021

We study the generalization of deep learning models in relation to the convex hull of their training sets. A trained image classifier basically partitions its domain via decision boundaries and assigns a class to each of those partitions. The location of decision boundaries inside the convex hull of training set can be investigated in relation to the training samples. However, our analysis shows that in standard image classification datasets, all testing images are considerably outside that convex hull, in the pixel space, in the wavelet space, and in the internal representations learned by deep networks. Therefore, the performance of a trained model partially depends on how its decision boundaries are extended outside the convex hull of its training data. From this perspective which is not studied before, over-parameterization of deep learning models may be considered a necessity for shaping the extension of decision boundaries. At the same time, over-parameterization should be accompanied by a specific training regime, in order to yield a model that not only fits the training set, but also its decision boundaries extend desirably outside the convex hull. To illustrate this, we investigate the decision boundaries of a neural network, with various degrees of parameters, inside and outside the convex hull of its training set. Moreover, we use a polynomial decision boundary to study the necessity of over-parameterization and the influence of training regime in shaping its extensions outside the convex hull of training set.
Federated Intrusion Detection for IoT with Heterogeneous Cohort Privacy
Venue: arXiv cs.LG 2021
Year: 2021

Internet of Things (IoT) devices are becoming increasingly popular and are influencing many application domains such as healthcare and transportation. These devices are used for real-world applications such as sensor monitoring, real-time control. In this work, we look at differentially private (DP) neural network (NN) based network intrusion detection systems (NIDS) to detect intrusion attacks on networks of such IoT devices. Existing NN training solutions in this domain either ignore privacy considerations or assume that the privacy requirements are homogeneous across all users. We show that the performance of existing differentially private stochastic methods degrade for clients with non-identical data distributions when clients' privacy requirements are heterogeneous. We define a cohort-based $(ε,δ)$-DP framework that models the more practical setting of IoT device cohorts with non-identical clients and heterogeneous privacy requirements. We propose two novel continual-learning based DP training methods that are designed to improve model performance in the aforementioned setting. To the best of our knowledge, ours is the first system that employs a continual learning-based approach to handle heterogeneity in client privacy requirements. We evaluate our approach on real datasets and show that our techniques outperform the baselines. We also show that our methods are robust to hyperparameter changes. Lastly, we show that one of our proposed methods can easily adapt to post-hoc relaxations of client privacy requirements.
Domain-Dependent Speaker Diarization for the Third DIHARD Challenge
Venue: arXiv cs.SD 2021
Year: 2021

This report presents the system developed by the ABSP Laboratory team for the third DIHARD speech diarization challenge. Our main contribution in this work is to develop a simple and efficient solution for acoustic domain dependent speech diarization. We explore speaker embeddings for \emph{acoustic domain identification} (ADI) task. Our study reveals that i-vector based method achieves considerably better performance than x-vector based approach in the third DIHARD challenge dataset. Next, we integrate the ADI module with the diarization framework. The performance substantially improved over that of the baseline when we optimized the thresholds for agglomerative hierarchical clustering and the parameters for dimensionality reduction during scoring for individual acoustic domains. We achieved a relative improvement of $9.63\%$ and $10.64\%$ in DER for core and full conditions, respectively, for Track 1 of the DIHARD III evaluation set.
Is Phase Shift Keying Optimal for Channels with Phase-Quantized Output?
Venue: arXiv cs.IT 2021
Year: 2021

This paper establishes the capacity of additive white Gaussian noise (AWGN) channels with phase-quantized output. We show that a rotated $2^b$-phase shift keying scheme is the capacity-achieving input distribution for a complex AWGN channel with $b$-bit phase quantization. The result is then used to establish the expression for the channel capacity as a function of average power constraint $P$ and quantization bits $b$. The outage performance of phase-quantized system is also investigated for the case of Rayleigh fading when the channel state information (CSI) is only known at the receiver. Our findings suggest the existence of a threshold in the rate $R$, above which the outage exponent of the outage probability changes abruptly. In fact, this threshold effect in the outage exponent causes $2^b$-PSK to have suboptimal outage performance at high SNR.