OpenRaiser/Intern-Atlas huggingface dataset:
Dataflow-Architecture Co-Design for 2.5D DNN Accelerators using Wireless Network-on-Package
Venue: arXiv cs.AR 2020
Year: 2020

Deep neural network (DNN) models continue to grow in size and complexity, demanding higher computational power to enable real-time inference. To efficiently deliver such computational demands, hardware accelerators are being developed and deployed across scales. This naturally requires an efficient scale-out mechanism for increasing compute density as required by the application. 2.5D integration over interposer has emerged as a promising solution, but as we show in this work, the limited interposer bandwidth and multiple hops in the Network-on-Package (NoP) can diminish the benefits of the approach. To cope with this challenge, we propose WIENNA, a wireless NoP-based 2.5D DNN accelerator. In WIENNA, the wireless NoP connects an array of DNN accelerator chiplets to the global buffer chiplet, providing high-bandwidth multicasting capabilities. Here, we also identify the dataflow style that most efficienty exploits the wireless NoP's high-bandwidth multicasting capability on each layer. With modest area and power overheads, WIENNA achieves 2.2X--5.1X higher throughput and 38.2% lower energy than an interposer-based NoP design.
Workload Characterization of Programmable Metasurfaces
Venue: arXiv cs.NI 2020
Year: 2020

Metasurfaces are envisaged to play a key role in next-generation wireless systems due to their powerful control over electromagnetic waves. The last decade has witnessed huge advances in this regard, shifting from static to programmable metasurfaces. The HyperSurface (HSF) paradigm takes one step further by integrating a network of controllers within the device with the aim of adding intelligence, connectivity, and autonomy. However, little is known about the traffic that this network will have to support as the target electromagnetic function or boundary conditions change. In this paper, we lay down the foundations of a methodology to characterize the workload of programmable metasurfaces and then employ it to analyze the case of beam steering HSFs. We observe that traffic is bursty and highly dependent on the position of the target. These results will enable the early-stage evaluation of intra-HSF networks, as well as the estimation of the system performance and cost.
Media Content Delivery Protocols Performance and Reliability Evaluation in Cellular Mobile Networks
Venue: arXiv cs.NI 2020
Year: 2020

Currently, tens of millions of devices around the world communicate with/ each other via cellular networks. In this paper, we study the stability of network content delivery protocols to the effects of network interference. To conduct the research, a tool was developed that allows testing of protocols, such as TCP, UDP, and QUIC. The analysis and comparison of the obtained test results were carried out. In the conclusion, the best protocols for content delivery were shown
State complexity of halting, returning and reversible graph-walking automata
Venue: arXiv cs.FL 2020
Year: 2020

Graph-walking automata (GWA) traverse graphs by moving between the nodes following the edges, using a finite-state control to decide where to go next. It is known that every GWA can be transformed to a GWA that halts on every input, to a GWA returning to the initial node in order to accept, and to a reversible GWA. This paper establishes lower bounds on the state blow-up of these transformations, as well as closely matching upper bounds. It is shown that making an $n$-state GWA traversing $k$-ary graphs halt on every input requires at most $2nk+1$ states and at least $2(n-1)(k-3)$ states in the worst case; making a GWA return to the initial node before acceptance takes at most $2nk+n$ and at least $2(n-1)(k-3)$ states in the worst case; Automata satisfying both properties at once have at most $4nk+1$ and at least $4(n-1)(k-3)$ states in the worst case. Reversible automata have at most $4nk+1$ and at least $4(n-1)(k-3)-1$ states in the worst case.
Impacts of the Space Technology Evolution in the V\&V of Embedded Software-Intensive Systems
Venue: arXiv cs.OH 2020
Year: 2020

CubeSat-based nanosatellites are composed of COTS components and rely on its structure and standardized interfaces. A challenge in the nanosatellites context is to adapt the V\&V (Verification and Validation) process to answer to the increase importance of the embedded software, to reduce the artefacts to be delivered aiming at cutting cost and time and still complying with international standards. This work presents an analysis of the strategy adopted in a real nanosatellite for the development of the OBDH software embedded in NanosatC-BR2 mission. The goal is to discuss the impact that the standardization of the structure and interfaces of the CubeSat impose on the V\&V process of the SiS and to highlight the challenges of ``New Space Age`` for the use of existing V\&V techniques and methods.
Autonomous Graph Mining Algorithm Search with Best Speed/Accuracy Trade-off
Venue: arXiv cs.LG 2020
Year: 2020

Graph data is ubiquitous in academia and industry, from social networks to bioinformatics. The pervasiveness of graphs today has raised the demand for algorithms that can answer various questions: Which products would a user like to purchase given her order list? Which users are buying fake followers to increase their public reputation? Myriads of new graph mining algorithms are proposed every year to answer such questions - each with a distinct problem formulation, computational time, and memory footprint. This lack of unity makes it difficult for a practitioner to compare different algorithms and pick the most suitable one for a specific application. These challenges - even more severe for non-experts - create a gap in which state-of-the-art techniques developed in academic settings fail to be optimally deployed in real-world applications. To bridge this gap, we propose AUTOGM, an automated system for graph mining algorithm development. We first define a unified framework UNIFIEDGM that integrates various message-passing based graph algorithms, ranging from conventional algorithms like PageRank to graph neural networks. Then UNIFIEDGM defines a search space in which five parameters are required to determine a graph algorithm. Under this search space, AUTOGM explicitly optimizes for the optimal parameter set of UNIFIEDGM using Bayesian Optimization. AUTOGM defines a novel budget-aware objective function for the optimization to incorporate a practical issue - finding the best speed-accuracy trade-off under a computation budget - into the graph algorithm generation problem. Experiments on real-world benchmark datasets demonstrate that AUTOGM generates novel graph mining algorithms with the best speed/accuracy trade-off compared to existing models with heuristic parameters.
Flood Detection via Twitter Streams using Textual and Visual Features
Venue: arXiv cs.CV 2020
Year: 2020

The paper presents our proposed solutions for the MediaEval 2020 Flood-Related Multimedia Task, which aims to analyze and detect flooding events in multimedia content shared over Twitter. In total, we proposed four different solutions including a multi-modal solution combining textual and visual information for the mandatory run, and three single modal image and text-based solutions as optional runs. In the multimodal method, we rely on a supervised multimodal bitransformer model that combines textual and visual features in an early fusion, achieving a micro F1-score of .859 on the development data set. For the text-based flood events detection, we use a transformer network (i.e., pretrained Italian BERT model) achieving an F1-score of .853. For image-based solutions, we employed multiple deep models, pre-trained on both, the ImageNet and places data sets, individually and combined in an early fusion achieving F1-scores of .816 and .805 on the development set, respectively.
A proposal and evaluation of new timbre visualisation methods for audio sample browsers
Venue: arXiv cs.HC 2020
Year: 2020

Searching through vast libraries of sound samples can be a daunting and time-consuming task. Modern audio sample browsers use mappings between acoustic properties and visual attributes to visually differentiate displayed items. There are few studies focused on how well these mappings help users search for a specific sample. We propose new methods for generating textural labels and positioning samples based on perceptual representations of timbre. We perform a series of studies to evaluate the benefits of using shape, color or texture as labels in a known-item search task. We describe the motivation and implementation of the study, and present an in-depth analysis of results. We find that shape significantly improves task performance, while color and texture have little effect. We also compare results between in-person and online participants and propose research directions for further studies.
Task Allocation for Asynchronous Mobile Edge Learning with Delay and Energy Constraints
Venue: arXiv cs.LG 2020
Year: 2020

This paper extends the paradigm of "mobile edge learning (MEL)" by designing an optimal task allocation scheme for training a machine learning model in an asynchronous manner across mutiple edge nodes or learners connected via a resource-constrained wireless edge network. The optimization is done such that the portion of the task allotted to each learner is completed within a given global delay constraint and a local maximum energy consumption limit. The time and energy consumed are related directly to the heterogeneous communication and computational capabilities of the learners; i.e. the proposed model is heterogeneity aware (HA). Because the resulting optimization is an NP-hard quadratically-constrained integer linear program (QCILP), a two-step suggest-and-improve (SAI) solution is proposed based on using the solution of the relaxed synchronous problem to obtain the solution to the asynchronous problem. The proposed HA asynchronous (HA-Asyn) approach is compared against the HA synchronous (HA-Sync) scheme and the heterogeneity unaware (HU) equal batch allocation scheme. Results from a system of 20 learners tested for various completion time and energy consumption constraints show that the proposed HA-Asyn method works better than the HU synchronous/asynchronous (HU-Sync/Asyn) approach and can provide gains of up-to 25\% compared to the HA-Sync scheme.
A Contemporary Survey on Free Space Optical Communication: Potential, Technical Challenges, Recent Advances and Research Direction
Venue: arXiv cs.IT 2020
Year: 2020

Optical wireless communication (OWC) covering an ultra-wide range of unlicensed spectrum has emerged as an extent efficient solution to mitigate conventional RF spectrum scarcity ranging from communication distances from nm to several kilometers. Free space optical (FSO) systems operating near IR (NIR) band in OWC links has received substantial attention for enormous data transmission between fixed transceivers covering few kilometers path distance due to high optical bandwidth and higher bit rate as well. Despite the potential benefits of FSO technology, its widespread link reliability suffers especially in the long-range deployment due to atmospheric turbulence, cloud induced fading, some other environmental factors such as fog, aerosol, temperature variations, storms, heavy rain, cloud, pointing error, and scintillation. FSO has the potential to offloading massive traffic demands from RF networks, consequently the combined application of FSO/RF and radio over FSO (RoFSO) systems is regarded as an excellent solution to support 5G and beyond for improving the limitations of an individual system. This survey presents the overview of several key technologies and implications of state-of-the-art criteria in terms of spectrum reuse, classification, architecture and applications are described for understanding FSO. This paper provides principle, significance, demonstration, and recent technological development of FSO technology among different appealing optical wireless technologies. The opportunities in the near future, the potential challenges that need to be addressed to realize the successful deployment of FSO schemes are outlined.
How to fine-tune deep neural networks in few-shot learning?
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning has been widely used in data-intensive applications. However, training a deep neural network often requires a large data set. When there is not enough data available for training, the performance of deep learning models is even worse than that of shallow networks. It has been proved that few-shot learning can generalize to new tasks with few training samples. Fine-tuning of a deep model is simple and effective few-shot learning method. However, how to fine-tune deep learning models (fine-tune convolution layer or BN layer?) still lack deep investigation. Hence, we study how to fine-tune deep models through experimental comparison in this paper. Furthermore, the weight of the models is analyzed to verify the feasibility of the fine-tuning method.
Toward Accurate Platform-Aware Performance Modeling for Deep Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we provide a fine-grain machine learning-based method, PerfNetV2, which improves the accuracy of our previous work for modeling the neural network performance on a variety of GPU accelerators. Given an application, the proposed method can be used to predict the inference time and training time of the convolutional neural networks used in the application, which enables the system developer to optimize the performance by choosing the neural networks and/or incorporating the hardware accelerators to deliver satisfactory results in time. Furthermore, the proposed method is capable of predicting the performance of an unseen or non-existing device, e.g. a new GPU which has a higher operating frequency with less processor cores, but more memory capacity. This allows a system developer to quickly search the hardware design space and/or fine-tune the system configuration. Compared to the previous works, PerfNetV2 delivers more accurate results by modeling detailed host-accelerator interactions in executing the full neural networks and improving the architecture of the machine learning model used in the predictor. Our case studies show that PerfNetV2 yields a mean absolute percentage error within 13.1% on LeNet, AlexNet, and VGG16 on NVIDIA GTX-1080Ti, while the error rate on a previous work published in ICBD 2018 could be as large as 200%.
NEAT: Non-linearity Aware Training for Accurate and Energy-Efficient Implementation of Neural Networks on 1T-1R Memristive Crossbars
Venue: arXiv cs.ET 2020
Year: 2020

Memristive crossbars suffer from non-idealities (such as, sneak paths) that degrade computational accuracy of the Deep Neural Networks (DNNs) mapped onto them. A 1T-1R synapse, adding a transistor (1T) in series with the memristive synapse (1R), has been proposed to mitigate such non-idealities. We observe that the non-linear characteristics of the transistor affect the overall conductance of the 1T-1R cell which in turn affects the Matrix-Vector-Multiplication (MVM) operation in crossbars. This 1T-1R non-ideality arising from the input voltage-dependent non-linearity is not only difficult to model or formulate, but also causes a drastic performance degradation of DNNs when mapped onto crossbars. In this paper, we analyse the non-linearity of the 1T-1R crossbar and propose a novel Non-linearity Aware Training (NEAT) method to address the non-idealities. Specifically, we first identify the range of network weights, which can be mapped into the 1T-1R cell within the linear operating region of the transistor. Thereafter, we regularize the weights of the DNNs to exist within the linear operating range by using iterative training algorithm. Our iterative training significantly recovers the classification accuracy drop caused by the non-linearity. Moreover, we find that each layer has a different weight distribution and in turn requires different gate voltage of transistor to guarantee linear operation. Based on this observation, we achieve energy efficiency while preserving classification accuracy by applying heterogeneous gate voltage control to the 1T-1R cells across different layers. Finally, we conduct various experiments on CIFAR10 and CIFAR100 benchmark datasets to demonstrate the effectiveness of our non-linearity aware training. Overall, NEAT yields ~20% energy gain with less than 1% accuracy loss (with homogeneous gate control) when mapping ResNet18 networks on 1T-1R crossbars.
Synchronized Traveling Salesman Problem
Venue: arXiv cs.DM 2020
Year: 2020

We consider a variation of the well-known traveling salesman problem in which there are multiple agents who all have to tour the whole set of nodes of the same graph, while obeying node- and edge-capacity constraints require that agents must not "crash". We consider the simplest model in which the input is an undirected graph with all capacities equal to one. A solution to the synchronized traveling salesman problem is called an "agency". Our model puts the synchronized traveling salesman problem in a similar relation with the traveling salesman problem as the so-called evacuation problem, or the well-known dynamic flow (flow-over-time) problem is in relation with the minimum cost flow problem. We measure the strength of an agency in terms of number of agents which should be as large as possible, and the time horizon which should be as small as possible. Beside some elementary discussion of the notions introduced, we establish several upper and lower bounds for the strength of an agency under the assumption that the input graph is a tree, or a 3-connected 3-regular graph.
Decentralized Multi-Agent Linear Bandits with Safety Constraints
Venue: arXiv cs.LG 2020
Year: 2020

We study decentralized stochastic linear bandits, where a network of $N$ agents acts cooperatively to efficiently solve a linear bandit-optimization problem over a $d$-dimensional space. For this problem, we propose DLUCB: a fully decentralized algorithm that minimizes the cumulative regret over the entire network. At each round of the algorithm each agent chooses its actions following an upper confidence bound (UCB) strategy and agents share information with their immediate neighbors through a carefully designed consensus procedure that repeats over cycles. Our analysis adjusts the duration of these communication cycles ensuring near-optimal regret performance $\mathcal{O}(d\log{NT}\sqrt{NT})$ at a communication rate of $\mathcal{O}(dN^2)$ per round. The structure of the network affects the regret performance via a small additive term - coined the regret of delay - that depends on the spectral gap of the underlying graph. Notably, our results apply to arbitrary network topologies without a requirement for a dedicated agent acting as a server. In consideration of situations with high communication cost, we propose RC-DLUCB: a modification of DLUCB with rare communication among agents. The new algorithm trades off regret performance for a significantly reduced total communication cost of $\mathcal{O}(d^3N^{2.5})$ over all $T$ rounds. Finally, we show that our ideas extend naturally to the emerging, albeit more challenging, setting of safe bandits. For the recently studied problem of linear bandits with unknown linear safety constraints, we propose the first safe decentralized algorithm. Our study contributes towards applying bandit techniques in safety-critical distributed systems that repeatedly deal with unknown stochastic environments. We present numerical simulations for various network topologies that corroborate our theoretical findings.
Semi-Supervised Noisy Student Pre-training on EfficientNet Architectures for Plant Pathology Classification
Venue: arXiv cs.CV 2020
Year: 2020

In recent years, deep learning has vastly improved the identification and diagnosis of various diseases in plants. In this report, we investigate the problem of pathology classification using images of a single leaf. We explore the use of standard benchmark models such as VGG16, ResNet101, and DenseNet 161 to achieve a 0.945 score on the task. Furthermore, we explore the use of the newer EfficientNet model, improving the accuracy to 0.962. Finally, we introduce the state-of-the-art idea of semi-supervised Noisy Student training to the EfficientNet, resulting in significant improvements in both accuracy and convergence rate. The final ensembled Noisy Student model performs very well on the task, achieving a test score of 0.982.
Design and Implementation of Fair Congestion Control for Data Centers Networks
Venue: arXiv cs.NI 2020
Year: 2020

In data centers, the nature of the composite bursty traffic along with the small bandwidth-delay product and switch buffers lead to several congestion problems that are not handled well by traditional congestion control mechanisms such as TCP. Existing work try to address the problem by modifying TCP to suit the operational nature of data centers. This is practically feasible in private settings, however, in public environments, such modifications are prohibited. Therefore, in this work, we design simple switch-based queue management to deal with such congestion issues adequately. This approach entails no modification to the TCP sender and receiver algorithms which enables easy and seamless deployment in public data centers. We present a theoretical analysis to show the stability and effectiveness of the scheme. We also present, three different real implementations (as a Linux kernel module and as an added feature to OpenvSwitch) and give numerical results from both NS-2 simulation and experiments of real deployment in a small test-bed cluster to show its effectiveness in achieving high throughput overall, a good fairness and short flow completion times for delay-sensitive flows.
Deep Learning-Based Arrhythmia Detection Using RR-Interval Framed Electrocardiograms
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning applied to electrocardiogram (ECG) data can be used to achieve personal authentication in biometric security applications, but it has not been widely used to diagnose cardiovascular disorders. We developed a deep learning model for the detection of arrhythmia in which time-sliced ECG data representing the distance between successive R-peaks are used as the input for a convolutional neural network (CNN). The main objective is developing the compact deep learning based detect system which minimally uses the dataset but delivers the confident accuracy rate of the Arrhythmia detection. This compact system can be implemented in wearable devices or real-time monitoring equipment because the feature extraction step is not required for complex ECG waveforms, only the R-peak data is needed. The results of both tests indicated that the Compact Arrhythmia Detection System (CADS) matched the performance of conventional systems for the detection of arrhythmia in two consecutive test runs. All features of the CADS are fully implemented and publicly available in MATLAB.
HORAE: an annotated dataset of books of hours
Venue: arXiv cs.CV 2020
Year: 2020

We introduce in this paper a new dataset of annotated pages from books of hours, a type of handwritten prayer books owned and used by rich lay people in the late middle ages. The dataset was created for conducting historical research on the evolution of the religious mindset in Europe at this period since the book of hours represent one of the major sources of information thanks both to their rich illustrations and the different types of religious sources they contain. We first describe how the corpus was collected and manually annotated then present the evaluation of a state-of-the-art system for text line detection and for zone detection and typing. The corpus is freely available for research.