OpenRaiser/Intern-Atlas huggingface dataset:
PruneNet: Channel Pruning via Global Importance
Venue: arXiv cs.LG 2020
Year: 2020

Channel pruning is one of the predominant approaches for accelerating deep neural networks. Most existing pruning methods either train from scratch with a sparsity inducing term such as group lasso, or prune redundant channels in a pretrained network and then fine tune the network. Both strategies suffer from some limitations: the use of group lasso is computationally expensive, difficult to converge and often suffers from worse behavior due to the regularization bias. The methods that start with a pretrained network either prune channels uniformly across the layers or prune channels based on the basic statistics of the network parameters. These approaches either ignore the fact that some CNN layers are more redundant than others or fail to adequately identify the level of redundancy in different layers. In this work, we investigate a simple-yet-effective method for pruning channels based on a computationally light-weight yet effective data driven optimization step that discovers the necessary width per layer. Experiments conducted on ILSVRC-$12$ confirm effectiveness of our approach. With non-uniform pruning across the layers on ResNet-$50$, we are able to match the FLOP reduction of state-of-the-art channel pruning results while achieving a $0.98\%$ higher accuracy. Further, we show that our pruned ResNet-$50$ network outperforms ResNet-$34$ and ResNet-$18$ networks, and that our pruned ResNet-$101$ outperforms ResNet-$50$.
Spatial Indexing for System-Level Evaluation of 5G Heterogeneous Cellular Networks
Venue: arXiv cs.IT 2020
Year: 2020

System level simulations of large 5G networks are essential to evaluate and design algorithms related to network issues such as scheduling, mobility management, interference management, and cell planning. In this paper, we look back to the idea of spatial indexing and its advantages, applications, and future potentials in accelerating large 5G network simulations. We introduce a multi-level inheritance based architecture which is used to index all elements of a heterogeneous network (HetNet) on a single geometry tree. Then, we define spatial queries to accelerate searches in distance, azimuth, and elevation. We demonstrate that spatial indexing can accelerate location-based searches by 3 orders of magnitude. Further, the proposed design is implemented as an open source platform freely available to all.
SentPWNet: A Unified Sentence Pair Weighting Network for Task-specific Sentence Embedding
Venue: arXiv cs.CL 2020
Year: 2020

Pair-based metric learning has been widely adopted to learn sentence embedding in many NLP tasks such as semantic text similarity due to its efficiency in computation. Most existing works employed a sequence encoder model and utilized limited sentence pairs with a pair-based loss to learn discriminating sentence representation. However, it is known that the sentence representation can be biased when the sampled sentence pairs deviate from the true distribution of all sentence pairs. In this paper, our theoretical analysis shows that existing works severely suffered from a good pair sampling and instance weighting strategy. Instead of one time pair selection and learning on equal weighted pairs, we propose a unified locality weighting and learning framework to learn task-specific sentence embedding. Our model, SentPWNet, exploits the neighboring spatial distribution of each sentence as locality weight to indicate the informative level of sentence pair. Such weight is updated along with pair-loss optimization in each round, ensuring the model keep learning the most informative sentence pairs. Extensive experiments on four public available datasets and a self-collected place search benchmark with 1.4 million places clearly demonstrate that our model consistently outperforms existing sentence embedding methods with comparable efficiency.
Accelerate Cycle-Level Full-System Simulation of Multi-Core RISC-V Systems with Binary Translation
Venue: arXiv cs.AR 2020
Year: 2020

It has always been difficult to balance the accuracy and performance of ISSs. RTL simulators or systems such as gem5 are used to execute programs in a cycle-accurate manner but are often prohibitively slow. In contrast, functional simulators such as QEMU can run large benchmarks to completion in a reasonable time yet capture few performance metrics and fail to model complex interactions between multiple cores. This paper presents a novel multi-purpose simulator that exploits binary translation to offer fast cycle-level full-system simulations. Its functional simulation mode outperforms QEMU and, if desired, it is possible to switch between functional and timing modes at run-time. Cycle-level simulations of RISC-V multi-core processors are possible at more than 20 MIPS, a useful middle ground in terms of accuracy and performance with simulation speeds nearly 100 times those of more detailed cycle-accurate models.
Image Translation by Latent Union of Subspaces for Cross-Domain Plaque Detection
Venue: arXiv cs.CV 2020
Year: 2020

Calcified plaque in the aorta and pelvic arteries is associated with coronary artery calcification and is a strong predictor of heart attack. Current calcified plaque detection models show poor generalizability to different domains (ie. pre-contrast vs. post-contrast CT scans). Many recent works have shown how cross domain object detection can be improved using an image translation model which translates between domains using a single shared latent space. However, while current image translation models do a good job preserving global/intermediate level structures they often have trouble preserving tiny structures. In medical imaging applications, preserving small structures is important since these structures can carry information which is highly relevant for disease diagnosis. Recent works on image reconstruction show that complex real-world images are better reconstructed using a union of subspaces approach. Since small image patches are used to train the image translation model, it makes sense to enforce that each patch be represented by a linear combination of subspaces which may correspond to the different parts of the body present in that patch. Motivated by this, we propose an image translation network using a shared union of subspaces constraint and show our approach preserves subtle structures (plaques) better than the conventional method. We further applied our method to a cross domain plaque detection task and show significant improvement compared to the state-of-the art method.
Data as Infrastructure for Smart Cities: Linking Data Platforms to Business Strategies
Venue: arXiv cs.CY 2020
Year: 2020

The systems that operate the infrastructure of cities have evolved in a fragmented fashion across several generations of technology, causing city utilities and services to operate sub-optimally and limiting the creation of new value-added services and restrict opportunities for cost-saving. The integration of cross-domain city data offers a new wave of opportunities to mitigate some of these impacts and enables city systems to draw effectively on interoperable data that will be used to deliver smarter cities. Despite the considerable potential of city data, current smart cities initiatives have mainly addressed the problem of data management from a technology perspective, and have disregarded stakeholders and data needs. As a consequence, such initiatives are susceptible to failure from inadequate stakeholder input, requirements neglecting, and information fragmentation and overload. They are also likely to be limited in terms of both scalability and future proofing against technological, commercial and legislative change. This paper proposes a systematic business-modeldriven framework to guide the design of large and highly interconnected data infrastructures which are provided and supported by multiple stakeholders. The framework is used to model, elicit and reason about the requirements of the service, technology, organization, value, and governance aspects of smart cities. The requirements serve as an input to a closed-loop supply chain model, which is designed and managed to explicitly consider the activities and processes that enables the stakeholders of smart cities to efficiently leverage their collective knowledge. We demonstrate how our approach can be used to design data infrastructures by examining a series of exemplary scenarios and by demonstrating how our approach handles the holistic design of a data infrastructure and informs the decision making process.
FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data
Venue: arXiv cs.LG 2020
Year: 2020

Federated Learning (FL) has become a popular paradigm for learning from distributed data. To effectively utilize data at different devices without moving them to the cloud, algorithms such as the Federated Averaging (FedAvg) have adopted a "computation then aggregation" (CTA) model, in which multiple local updates are performed using local data, before sending the local models to the cloud for aggregation. However, these schemes typically require strong assumptions, such as the local data are identically independent distributed (i.i.d), or the size of the local gradients are bounded. In this paper, we first explicitly characterize the behavior of the FedAvg algorithm, and show that without strong and unrealistic assumptions on the problem structure, the algorithm can behave erratically for non-convex problems (e.g., diverge to infinity). Aiming at designing FL algorithms that are provably fast and require as few assumptions as possible, we propose a new algorithm design strategy from the primal-dual optimization perspective. Our strategy yields a family of algorithms that take the same CTA model as existing algorithms, but they can deal with the non-convex objective, achieve the best possible optimization and communication complexity while being able to deal with both the full batch and mini-batch local computation models. Most importantly, the proposed algorithms are {\it communication efficient}, in the sense that the communication pattern can be adaptive to the level of heterogeneity among the local data. To the best of our knowledge, this is the first algorithmic framework for FL that achieves all the above properties.
Find Unique Usages: Helping Developers Understand Common Usages
Venue: arXiv cs.SE 2020
Year: 2020

When working in large and complex codebases, developers face challenges using \textit{Find Usages} to understand how to reuse classes and methods. To better understand these challenges, we conducted a small exploratory study with 4 participants. We found that developers often wasted time reading long lists of similar usages or prematurely focused on a single usage. Based on these findings, we hypothesized that clustering usages by the similarity of their surrounding context might enable developers to more rapidly understand how to use a function. To explore this idea, we designed and implemented \textit{Find Unique Usages}, which extracts usages, computes a diff between pairs of usages, generates similarity scores, and uses these scores to form usage clusters. To evaluate this approach, we conducted a controlled experiment with 12 participants. We found that developers with Find Unique Usages were significantly faster, completing their task in 35% less time.
Short-term Load Forecasting Based on Hybrid Strategy Using Warm-start Gradient Tree Boosting
Venue: arXiv cs.LG 2020
Year: 2020

A deep-learning-based hybrid strategy for short-term load forecasting is presented. The strategy proposes a novel tree-based ensemble method Warm-start Gradient Tree Boosting (WGTB). Current strategies either ensemble submodels of a single type, which fail to take advantage of the statistical strengths of different inference models. Or they simply sum the outputs from completely different inference models, which doesn't maximize the potential of ensemble. Inspired by the bias-variance trade-off, WGTB is proposed and tailored to the great disparity among different inference models on accuracy, volatility and linearity. The complete strategy integrates four different inference models of different capacities. WGTB then ensembles their outputs by a warm-start and a hybrid of bagging and boosting, which lowers bias and variance concurrently. It is validated on two real datasets from State Grid Corporation of China of hourly resolution. The result demonstrates the effectiveness of the proposed strategy that hybridizes the statistical strengths of both low-bias and low-variance inference models.
Fluid Antenna Systems
Venue: arXiv cs.IT 2020
Year: 2020

Over the past decades, multiple antenna technologies have appeared in many different forms, most notably as multiple-input multiple-output (MIMO), to transform wireless communications for extraordinary diversity and multiplexing gains. The variety of technologies has been based on placing a number of antennas at fixed locations which dictates the fundamental limit on the achievable performance. By contrast, this paper envisages the scenario where the physical position of an antenna can be switched freely to one of the N positions over a fixed-length line space to pick up the strongest signal in the manner of traditional selection combining. We refer to this system as a fluid antenna system (FAS) for tremendous flexibility in its possible shape and position. The aim of this paper is to study the achievable performance of a single-antenna FAS system with a fixed length and N in arbitrarily correlated Rayleigh fading channels. Our contributions include exact and approximate closed-form expressions for the outage probability of FAS. We also derive an upper bound for the outage probability, from which it is shown that a single-antenna FAS given any arbitrarily small space can outperform an L-antenna maximum ratio combining (MRC) system if N is large enough. Our analysis also reveals the minimum required size of the FAS, and how large N is considered enough for the FAS to surpass MRC.
Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks
Venue: arXiv cs.NE 2020
Year: 2020

The emergence of brain-inspired neuromorphic computing as a paradigm for edge AI is motivating the search for high-performance and efficient spiking neural networks to run on this hardware. However, compared to classical neural networks in deep learning, current spiking neural networks lack competitive performance in compelling areas. Here, for sequential and streaming tasks, we demonstrate how a novel type of adaptive spiking recurrent neural network (SRNN) is able to achieve state-of-the-art performance compared to other spiking neural networks and almost reach or exceed the performance of classical recurrent neural networks (RNNs) while exhibiting sparse activity. From this, we calculate a $>$100x energy improvement for our SRNNs over classical RNNs on the harder tasks. To achieve this, we model standard and adaptive multiple-timescale spiking neurons as self-recurrent neural units, and leverage surrogate gradients and auto-differentiation in the PyTorch Deep Learning framework to efficiently implement backpropagation-through-time, including learning of the important spiking neuron parameters to adapt our spiking neurons to the tasks.
Master-Auxiliary: an efficient aggregation strategy for video anomaly detection
Venue: arXiv cs.CV 2020
Year: 2020

The aim of surveillance video anomaly detection is to detect events that rarely or never happened in a certain scene. Generally, different detectors can detect different anomalies. This paper proposes an efficient strategy to aggregate multiple detectors. First, the aggregation strategy chooses one detector as master detector by experience, and sets the remaining detectors as auxiliary detectors. Then, the aggregation strategy extracts credible information from auxiliary detectors, including credible abnormal (Cred-a) frames and credible normal (Cred-n) frames. After that, the frequencies that each video frame being judged as Cred-a and Cred-n are counted. Applying the events' time continuity property, more Cred-a and Cred-n frames can be inferred. Finally, the aggregation strategy utilizes the Cred-a and Cred-n frequencies to vote to calculate soft weights, and uses the soft weights to assist the master detector. Experiments are carried out on multiple datasets. Comparing with existing aggregation strategies, the proposed strategy achieves state-of-the-art performance.
Fast Beam Training for IRS-Assisted Multiuser Communications
Venue: arXiv cs.IT 2020
Year: 2020

In this letter, we consider an intelligent reflecting surface (IRS)-assisted multiuser communication system, where an IRS is deployed to provide virtual line-of-sight (LoS) links between an access point (AP) and multiple users. We consider the practical codebook-based IRS passive beamforming and study efficient design for IRS reflect beam training, which is challenging due to the large number of IRS reflecting elements. In contrast to the conventional single-beam training, we propose a new multi-beam training method by dividing the IRS reflecting elements into multiple sub-arrays and designing their simultaneous multi-beam steering over time. By simply comparing the received signal power over time, each user can detect its optimal IRS beam direction with a high probability, even without searching over all possible beam directions as the single-beam training. Simulation results show that our proposed multi-beam training significantly reduces the training time of conventional single-beam training and yet achieves comparable IRS passive beamforming performance for data transmission.
Robust Spatial-spread Deep Neural Image Watermarking
Venue: arXiv cs.MM 2020
Year: 2020

Watermarking is an operation of embedding an information into an image in a way that allows to identify ownership of the image despite applying some distortions on it. In this paper, we presented a novel end-to-end solution for embedding and recovering the watermark in the digital image using convolutional neural networks. The method is based on spreading the message over the spatial domain of the image, hence reducing the "local bits per pixel" capacity. To obtain the model we used adversarial training and applied noiser layers between the encoder and the decoder. Moreover, we broadened the spectrum of typically considered attacks on the watermark and by grouping the attacks according to their scope, we achieved high general robustness, most notably against JPEG compression, Gaussian blurring, subsampling or resizing. To help us in the models training we also proposed a precise differentiable approximation of JPEG.
Adversarial NLI for Factual Correctness in Text Summarisation Models
Venue: arXiv cs.CL 2020
Year: 2020

We apply the Adversarial NLI dataset to train the NLI model and show that the model has the potential to enhance factual correctness in abstract summarization. We follow the work of Falke et al. (2019), which rank multiple generated summaries based on the entailment probabilities between an source document and summaries and select the summary that has the highest entailment probability. The authors' earlier study concluded that current NLI models are not sufficiently accurate for the ranking task. We show that the Transformer models fine-tuned on the new dataset achieve significantly higher accuracy and have the potential of selecting a coherent summary.
Custody Protocols Using Bitcoin Vaults
Venue: arXiv cs.CR 2020
Year: 2020

A bitcoin \textit{covenant} is a mechanism to enforce conditions on future bitcoin transactions. A bitcoin \textit{vault} is a specific type of covenant transaction that enforces a time-lock on the transfer of control of funds to a hot wallet, but enables an immediate transfer of funds into a deep cold recovery wallet. This paper demonstrates how to integrate a bitcoin vault into a custody protocol and demonstrates the security properties of that protocol. The vault is implemented using pre-signed transactions with secure key deletion (as proposed in \cite{Swambo2020cov}). It is shown that vault-custody protocols enable the wallet owner to specify their desired balance for an inherent trade-off between the security of and accessibility of bitcoin holdings by adjusting the length of time-locks used. It is also demonstrated that wallet owners have increased control of risk-management by compartmentalizing funds across numerous vault transactions. While it isn't realistic to completely prevent theft, the most likely theft scenarios (compromising the hot wallet) have severely limited profitability for an attacker, deterring attempts at theft from the beginning. The proposed architecture was designed to offer defence-in-depth through redundancy and fault-tolerant functionality as well as countermeasures for class breaks through diversity across hardware and software layers. Finally, the architecture employs a detection (a watchtower) and response system that enables fail-safe recovery from attempted or partial thefts through a second type of covenant transaction, a push-to-recovery-wallet transaction.
RTAMT: Online Robustness Monitors from STL
Venue: arXiv cs.LO 2020
Year: 2020

We present RTAMT, an online monitoring library for Signal Temporal Logic (STL) and its interface-aware variant (IA-STL), providing both discrete- and dense-time interpretation of the logic. We also introduce RTAMT4ROS, a tool that integrates RTAMT with Robotic Operating System (ROS), a common environment for developing robotic applications. We evaluate RTAMT and RTAMT4ROS on two robotic case studies.
SecureABC: Secure AntiBody Certificates for COVID-19
Venue: arXiv cs.CR 2020
Year: 2020

COVID-19 has resulted in unprecedented social distancing policies being enforced worldwide. As governments seek to restore their economies, open workplaces and permit travel there is a demand for technologies that may alleviate the requirement for social distancing whilst also protecting healthcare services. In this work we explore the controversial technique of so-called immunity passports and present SecureABC: a decentralised, privacy-preserving protocol for issuing and verifying antibody certificates. We consider the implications of antibody certificate systems, develop a set of risk-minimising principles and a security framework for their evaluation, and show that these may be satisfied in practice. Finally, we also develop two additional protocols that minimise individual discrimination but which still allow for collective transmission risk to be estimated. We use these two protocols to illustrate the utility-privacy trade-offs of antibody certificates and their alternatives.