OpenRaiser/Intern-Atlas huggingface dataset:
3D Two-Hop Cellular Networks with Wireless Backhauled UAVs: Modeling and Fundamentals
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we characterize the performance of a three-dimensional (3D) two-hop cellular network in which terrestrial base stations (BSs) coexist with unmanned aerial vehicles (UAVs) to serve a set of ground user equipment (UE). In particular, a UE connects either directly to its serving terrestrial BS by an access link or connects first to its serving UAV which is then wirelessly backhauled to a terrestrial BS (joint access and backhaul). We consider realistic antenna radiation patterns for both BSs and UAVs using practical models developed by the third generation partnership project (3GPP). We assume a probabilistic channel model for the air-to-ground transmission, which incorporates both line-of-sight (LoS) and non-line-of-sight (NLoS) links. Assuming the max-power association policy, we study the performance of the network in both amplify-and-forward (AF) and decode-and-forward (DF) relaying protocols. Using tools from stochastic geometry, we analyze the joint distribution of distance and zenith angle of the closest (and serving) UAV to the origin in a 3D setting. Further, we identify and extensively study key mathematical constructs as the building blocks of characterizing the received signal-to-interference-plus-noise ratio (SINR) distribution. Using these results, we obtain exact mathematical expressions for the coverage probability in both AF and DF relaying protocols. Furthermore, considering the fact that backhaul links could be quite weak because of the downtilted antennas at the BSs, we propose and analyze the addition of a directional uptilted antenna at the BS that is solely used for backhaul purposes. The superiority of having directional antennas with wirelessly backhauled UAVs is further demonstrated via simulation.
Lightweight Compression of Intermediate Neural Network Features for Collaborative Intelligence
Venue: arXiv cs.LG 2021
Year: 2021

In collaborative intelligence applications, part of a deep neural network (DNN) is deployed on a lightweight device such as a mobile phone or edge device, and the remaining portion of the DNN is processed where more computing resources are available, such as in the cloud. This paper presents a novel lightweight compression technique designed specifically to quantize and compress the features output by the intermediate layer of a split DNN, without requiring any retraining of the network weights. Mathematical models for estimating the clipping and quantization error of ReLU and leaky-ReLU activations at this intermediate layer are developed and used to compute optimal clipping ranges for coarse quantization. We also present a modified entropy-constrained design algorithm for quantizing clipped activations. When applied to popular object-detection and classification DNNs, we were able to compress the 32-bit floating point intermediate activations down to 0.6 to 0.8 bits, while keeping the loss in accuracy to less than 1%. When compared to HEVC, we found that the lightweight codec consistently provided better inference accuracy, by up to 1.3%. The performance and simplicity of this lightweight compression technique makes it an attractive option for coding an intermediate layer of a split neural network for edge/cloud applications.
FDDH: Fast Discriminative Discrete Hashing for Large-Scale Cross-Modal Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

Cross-modal hashing, favored for its effectiveness and efficiency, has received wide attention to facilitating efficient retrieval across different modalities. Nevertheless, most existing methods do not sufficiently exploit the discriminative power of semantic information when learning the hash codes, while often involving time-consuming training procedure for handling the large-scale dataset. To tackle these issues, we formulate the learning of similarity-preserving hash codes in terms of orthogonally rotating the semantic data so as to minimize the quantization loss of mapping such data to hamming space, and propose an efficient Fast Discriminative Discrete Hashing (FDDH) approach for large-scale cross-modal retrieval. More specifically, FDDH introduces an orthogonal basis to regress the targeted hash codes of training examples to their corresponding semantic labels, and utilizes "-dragging technique to provide provable large semantic margins. Accordingly, the discriminative power of semantic information can be explicitly captured and maximized. Moreover, an orthogonal transformation scheme is further proposed to map the nonlinear embedding data into the semantic subspace, which can well guarantee the semantic consistency between the data feature and its semantic representation. Consequently, an efficient closed form solution is derived for discriminative hash code learning, which is very computationally efficient. In addition, an effective and stable online learning strategy is presented for optimizing modality-specific projection functions, featuring adaptivity to different training sizes and streaming data. The proposed FDDH approach theoretically approximates the bi-Lipschitz continuity, runs sufficiently fast, and also significantly improves the retrieval performance over the state-of-the-art methods. The source code is released at: https://github.com/starxliu/FDDH.
Drill the Cork of Information Bottleneck by Inputting the Most Important Data
Venue: arXiv cs.LG 2021
Year: 2021

Deep learning has become the most powerful machine learning tool in the last decade. However, how to efficiently train deep neural networks remains to be thoroughly solved. The widely used minibatch stochastic gradient descent (SGD) still needs to be accelerated. As a promising tool to better understand the learning dynamic of minibatch SGD, the information bottleneck (IB) theory claims that the optimization process consists of an initial fitting phase and the following compression phase. Based on this principle, we further study typicality sampling, an efficient data selection method, and propose a new explanation of how it helps accelerate the training process of the deep networks. We show that the fitting phase depicted in the IB theory will be boosted with a high signal-to-noise ratio of gradient approximation if the typicality sampling is appropriately adopted. Furthermore, this finding also implies that the prior information of the training set is critical to the optimization process and the better use of the most important data can help the information flow through the bottleneck faster. Both theoretical analysis and experimental results on synthetic and real-world datasets demonstrate our conclusions.
An even-load-distribution design for composite bolted joints using a novel circuit model and artificial neural networks
Venue: arXiv cs.LG 2021
Year: 2021

Due to the brittle feature of carbon fiber reinforced plastic laminates, mechanical multi-joint within these composite components show uneven load distribution for each bolt, which weaken the strength advantage of composite laminates. In order to reduce this defect and achieve the goal of even load distribution in mechanical joints, we propose a machine learning-based framework as an optimization method. Since that the friction effect has been proven to be a significant factor in determining bolt load distribution, our framework aims at providing optimal parameters including bolt-hole clearances and tightening torques for a minimum unevenness of bolt load. A novel circuit model is established to generate data samples for the training of artificial networks at a relatively low computational cost. A database for all the possible inputs in the design space is built through the machine learning model. The optimal dataset of clearances and torques provided by the database is validated by both the finite element method, circuit model, and an experimental measurement based on the linear superposition principle, which shows the effectiveness of this general framework for the optimization problem. Then, our machine learning model is further compared and worked in collaboration with commonly used optimization algorithms, which shows the potential of greatly increasing computational efficiency for the inverse design problem.
On Converse Results for Secure Index Coding
Venue: arXiv cs.IT 2021
Year: 2021

In this work, we study the secure index coding problem where there are security constraints on both legitimate receivers and eavesdroppers. We develop two performance bounds (i.e., converse results) on the symmetric secure capacity. The first one is an extended version of the basic acyclic chain bound (Liu and Sadeghi, 2019) that takes security constraints into account. The second converse result is a novel information-theoretic lower bound on the symmetric secure capacity, which is interesting as all the existing converse results in the literature for secure index coding give upper bounds on the capacity.
String Theories involving Regular Membership Predicates: From Practice to Theory and Back
Venue: arXiv cs.CL 2021
Year: 2021

Widespread use of string solvers in formal analysis of string-heavy programs has led to a growing demand for more efficient and reliable techniques which can be applied in this context, especially for real-world cases. Designing an algorithm for the (generally undecidable) satisfiability problem for systems of string constraints requires a thorough understanding of the structure of constraints present in the targeted cases. In this paper, we investigate benchmarks presented in the literature containing regular expression membership predicates, extract different first order logic theories, and prove their decidability, resp. undecidability. Notably, the most common theories in real-world benchmarks are PSPACE-complete and directly lead to the implementation of a more efficient algorithm to solving string constraints.
Bilevel Programs Meet Deep Learning: A Unifying View on Inference Learning Methods
Venue: arXiv cs.LG 2021
Year: 2021

In this work we unify a number of inference learning methods, that are proposed in the literature as alternative training algorithms to the ones based on regular error back-propagation. These inference learning methods were developed with very diverse motivations, mainly aiming to enhance the biological plausibility of deep neural networks and to improve the intrinsic parallelism of training methods. We show that these superficially very different methods can all be obtained by successively applying a particular reformulation of bilevel optimization programs. As a by-product it becomes also evident that all considered inference learning methods include back-propagation as a special case, and therefore at least approximate error back-propagation in typical settings. Finally, we propose Fenchel back-propagation, that replaces the propagation of infinitesimal corrections performed in standard back-propagation with finite targets as the learning signal. Fenchel back-propagation can therefore be seen as an instance of learning via explicit target propagation.
Comparison of HPC Architectures for Computing All-Pairs Shortest Paths. Intel Xeon Phi KNL vs NVIDIA Pascal
Venue: arXiv cs.DC 2021
Year: 2021

Today, one of the main challenges for high-performance computing systems is to improve their performance by keeping energy consumption at acceptable levels. In this context, a consolidated strategy consists of using accelerators such as GPUs or many-core Intel Xeon Phi processors. In this work, devices of the NVIDIA Pascal and Intel Xeon Phi Knights Landing architectures are described and compared. Selecting the Floyd-Warshall algorithm as a representative case of graph and memory-bound applications, optimized implementations were developed to analyze and compare performance and energy efficiency on both devices. As it was expected, Xeon Phi showed superior when considering double-precision data. However, contrary to what was considered in our preliminary analysis, it was found that the performance and energy efficiency of both devices were comparable using single-precision datatype.
Distributed Resilient Submodular Action Selection in Adversarial Environments
Venue: arXiv cs.RO 2021
Year: 2021

In this letter, we consider a distributed submodular maximization problem for multi-robot systems when attacked by adversaries. One of the major challenges for multi-robot systems is to increase resilience against failures or attacks. This is particularly important for distributed systems under attack as there is no central point of command that can detect, mitigate, and recover from attacks. Instead, a distributed multi-robot system must coordinate effectively to overcome adversarial attacks. In this work, our distributed submodular action selection problem models a broad set of scenarios where each robot in a multi-robot system has multiple action selections that may fulfill a global objective, such as exploration or target tracking. To increase resilience in this context, we propose a fully distributed algorithm to guide each robot's action selection when the system is attacked. The proposed algorithm guarantees performance in a worst-case scenario where up to a portion of the robots malfunction due to attacks. Importantly, the proposed algorithm is also consistent, as it is shown to converge to the same solution as a centralized method. Finally, a distributed resilient multi-robot exploration problem is presented to confirm the performance of the proposed algorithm.
Expressive Explanations of DNNs by Combining Concept Analysis with ILP
Venue: arXiv cs.LG 2021
Year: 2021

Explainable AI has emerged to be a key component for black-box machine learning approaches in domains with a high demand for reliability or transparency. Examples are medical assistant systems, and applications concerned with the General Data Protection Regulation of the European Union, which features transparency as a cornerstone. Such demands require the ability to audit the rationale behind a classifier's decision. While visualizations are the de facto standard of explanations, they come short in terms of expressiveness in many ways: They cannot distinguish between different attribute manifestations of visual features (e.g. eye open vs. closed), and they cannot accurately describe the influence of absence of, and relations between features. An alternative would be more expressive symbolic surrogate models. However, these require symbolic inputs, which are not readily available in most computer vision tasks. In this paper we investigate how to overcome this: We use inherent features learned by the network to build a global, expressive, verbal explanation of the rationale of a feed-forward convolutional deep neural network (DNN). The semantics of the features are mined by a concept analysis approach trained on a set of human understandable visual concepts. The explanation is found by an Inductive Logic Programming (ILP) method and presented as first-order rules. We show that our explanation is faithful to the original black-box model. The code for our experiments is available at https://github.com/mc-lovin-mlem/concept-embeddings-and-ilp/tree/ki2020.
How is BERT surprised? Layerwise detection of linguistic anomalies
Venue: arXiv cs.CL 2021
Year: 2021

Transformer language models have shown remarkable ability in detecting when a word is anomalous in context, but likelihood scores offer no information about the cause of the anomaly. In this work, we use Gaussian models for density estimation at intermediate layers of three language models (BERT, RoBERTa, and XLNet), and evaluate our method on BLiMP, a grammaticality judgement benchmark. In lower layers, surprisal is highly correlated to low token frequency, but this correlation diminishes in upper layers. Next, we gather datasets of morphosyntactic, semantic, and commonsense anomalies from psycholinguistic studies; we find that the best performing model RoBERTa exhibits surprisal in earlier layers when the anomaly is morphosyntactic than when it is semantic, while commonsense anomalies do not exhibit surprisal at any intermediate layer. These results suggest that language models employ separate mechanisms to detect different types of linguistic anomalies.
A Synchronized Action Framework for Responsible Detection of Coordination on Social Media
Venue: arXiv cs.SI 2021
Year: 2021

The study of coordinated manipulation of conversations on social media has become more prevalent as social media's role in amplifying misinformation, hate, and polarization has come under scrutiny. We discuss the implications of successful coordination detection algorithms based on shifts of power, and consider how responsible coordination detection may be carried out through synchronized action. We then propose a Synchronized Action Framework for detection of automated coordination through construction and analysis of multi-view networks. We validate our framework by examining the Reopen America conversation on Twitter, discovering three coordinated campaigns. We further investigate covert coordination surrounding the protests and find the task to be far more complex than examples seen in prior work, demonstrating the need for our multi-view approach. A cluster of suspicious users is identified and the activity of three members is detailed. These users amplify protest messages using the same hashtags at very similar times, though they all focus on different states. Through this analysis, we emphasize both the potential usefulness of coordination detection algorithms in investigating amplification, and the need for careful and responsible deployment of such tools.
Capsule GAN for Prostate MRI Super-Resolution
Venue: arXiv cs.LG 2021
Year: 2021

Prostate cancer is a very common disease among adult men. One in seven Canadian men is diagnosed with this cancer in their lifetime. Super-Resolution (SR) can facilitate early diagnosis and potentially save many lives. In this paper, a robust and accurate model is proposed for prostate MRI SR. The model is trained on the Prostate-Diagnosis and PROSTATEx datasets. The proposed model outperformed the state-of-the-art prostate SR model in all similarity metrics with notable margins. A new task-specific similarity assessment is introduced as well. A classifier is trained for severe cancer detection and the drop in the accuracy of this model when dealing with super-resolved images is used for evaluating the ability of medical detail reconstruction of the SR models. The proposed SR model is a step towards an efficient and accurate general medical SR platform.
Analysis of Bitcoin Vulnerability to Bribery Attacks Launched Through Large Transactions
Venue: arXiv cs.CR 2021
Year: 2021

Bitcoin uses blockchain technology to maintain transactions order and provides probabilistic guarantee to prevent double-spending, assuming that an attacker's computational power does not exceed %50 of the network power. In this paper, we design a novel bribery attack and show that this guarantee can be hugely undermined. Miners are assumed to be rational in this setup and they are given incentives that are dynamically calculated. In this attack, the adversary misuses the Bitcoin protocol to bribe miners and maximize their gained advantage. We will reformulate the bribery attack to propose a general mathematical foundation upon which we build multiple strategies. We show that, unlike Whale Attack, these strategies are practical. If the rationality assumption holds, this shows how vulnerable blockchain-based systems like Bitcoin are. We suggest a soft fork on Bitcoin to fix this issue at the end.
Adaptive Interference Coordination over Channels with Unknown State at the Encoder and the Decoder
Venue: arXiv cs.IT 2021
Year: 2021

We generalize the problem of controlling the interference created to an external observer while communicating over a discrete memoryless channel (DMC) which was studied in \cite{serrano:2014}. In particular, we consider the scenario where the transmission is established over a compound DMC channel with unknown state at both the encoder and the decoder. Depending on the exact state $s$ of the channel, we ask for a different level of average precision $Δ_s$ on the establishment of the interference coordination with the external observer. For this set-up, we fully characterize the capacity region.
Dynamic Routing and Spectrum Assignment based on the Availability of Consecutive Sub-channels in Flexi-grid Optical Networks
Venue: arXiv cs.NI 2021
Year: 2021

Using Optical Orthogonal Frequency Multiplexing (O-OFDM), variable bandwidth channels can be created in Elastic Optical Networks (EON). This allows the use of spectrum more efficiently by allocating integral multiple of basic bandwidth slots to the lightpath requests. Consequently, such networks are also called flexible grid optical networks. It also adds a constraint of keeping all the allocated slots together when deciding the routes for the requests. This constraint called the contiguity constraint makes the routing and spectrum algorithms more challenging. In any network, the lightpath requests will arrive and depart dynamically and will invariably lead to spectrum fragmentation, and hence network will have a reduction in maximum possible utilization due to increased blocking probability. In this paper, we have presented an improvised RSA algorithm that leads to lesser fragmentation. It is evident from the results that the presented RSA algorithm uses adaptive parameters to reduce the blocking probability and fragmentation compared to other algorithms reported in the recent past.
Vision Transformers are Robust Learners
Venue: arXiv cs.CV 2021
Year: 2021

Transformers, composed of multiple self-attention layers, hold strong promises toward a generic learning primitive applicable to different data modalities, including the recent breakthroughs in computer vision achieving state-of-the-art (SOTA) standard accuracy. What remains largely unexplored is their robustness evaluation and attribution. In this work, we study the robustness of the Vision Transformer (ViT) against common corruptions and perturbations, distribution shifts, and natural adversarial examples. We use six different diverse ImageNet datasets concerning robust classification to conduct a comprehensive performance comparison of ViT models and SOTA convolutional neural networks (CNNs), Big-Transfer. Through a series of six systematically designed experiments, we then present analyses that provide both quantitative and qualitative indications to explain why ViTs are indeed more robust learners. For example, with fewer parameters and similar dataset and pre-training combinations, ViT gives a top-1 accuracy of 28.10% on ImageNet-A which is 4.3x higher than a comparable variant of BiT. Our analyses on image masking, Fourier spectrum sensitivity, and spread on discrete cosine energy spectrum reveal intriguing properties of ViT attributing to improved robustness. Code for reproducing our experiments is available at https://git.io/J3VO0.