OpenRaiser/Intern-Atlas huggingface dataset:
Incidents Are Meant for Learning, Not Repeating: Sharing Knowledge About Security Incidents in Cyber-Physical Systems
Venue: arXiv cs.CR 2019
Year: 2019

Cyber-physical systems (CPSs) are part of most critical infrastructures such as industrial automation and transportation systems. Thus, security incidents targeting CPSs can have disruptive consequences to assets and people. As prior incidents tend to re-occur, sharing knowledge about these incidents can help organizations be more prepared to prevent, mitigate or investigate future incidents. This paper proposes a novel approach to enable representation and sharing of knowledge about CPS incidents across different organizations. To support sharing, we represent incident knowledge (incident patterns) capturing incident characteristics that can manifest again, such as incident activities or vulnerabilities exploited by offenders. Incident patterns are a more abstract representation of specific incident instances and, thus, are general enough to be applicable to various systems - different than the one in which the incident occurred. They can also avoid disclosing potentially sensitive information about an organization's assets and resources. We provide an automated technique to extract an incident pattern from a specific incident instance. To understand how an incident pattern can manifest again in other cyber-physical systems, we also provide an automated technique to instantiate incident patterns to specific systems. We demonstrate the feasibility of our approach in the application domain of smart buildings. We evaluate correctness, scalability, and performance using two substantive scenarios inspired by real-world systems and incidents.
Towards Forward Secure Internet Traffic
Venue: arXiv cs.CR 2019
Year: 2019

Forward Secrecy (FS) is a security property in key-exchange algorithms which guarantees that a compromise in the secrecy of a long-term private-key does not compromise the secrecy of past session keys. With a growing awareness of long-term mass surveillance programs by governments and others, FS has become widely regarded as a highly desirable property. This is particularly true in the TLS protocol, which is used to secure Internet communication. In this paper, we investigate FS in pre-TLS 1.3 protocols, which do not mandate FS, but still widely used today. We conduct an empirical analysis of over 10 million TLS servers from three different datasets using a novel heuristic approach. Using a modern TLS client handshake algorithms, our results show 5.37% of top domains, 7.51% of random domains, and 26.16% of random IPs do not select FS key-exchange algorithms. Surprisingly, 39.20% of the top domains, 24.40% of the random domains, and 14.46% of the random IPs that do not select FS, do support FS. In light of this analysis, we discuss possible paths toward forward secure Internet traffic. As an improvement of the current state, we propose a new client-side mechanism that we call "Best Effort Forward Secrecy" (BEFS), and an extension of it that we call "Best Effort Forward Secrecy and Authenticated Encryption" (BESAFE), which aims to guide (force) misconfigured servers to FS using a best effort approach. Finally, within our analysis, we introduce a novel adversarial model that we call "discriminatory" adversary, which is applicable to the TLS protocol.
Joint Functional Splitting and Content Placement for Green Hybrid CRAN
Venue: arXiv cs.NI 2019
Year: 2019

A hybrid cloud radio access network (H-CRAN) architecture has been proposed to alleviate the midhaul capacity limitation in C-RAN. In this architecture, functional splitting is utilized to distribute the processing functions between a central cloud and edge clouds. The flexibility of selecting specific split point enables the H-CRAN designer to reduce midhaul bandwidth, or reduce latency, or save energy, or distribute the computation task depending on equipment availability. Meanwhile, techniques for caching are proposed to reduce content delivery latency and the required bandwidth. However, caching imposes new constraints on functional splitting. In this study, considering H-CRAN, a constraint programming problem is formulated to minimize the overall power consumption by selecting the optimal functional split point and content placement, taking into account the content access delay constraint. We also investigate the trade-off between the overall power consumption and occupied midhaul bandwidth in the network. Our results demonstrate that functional splitting together with enabling caching at edge clouds reduces not only content access delays but also fronthaul bandwidth consumption but at the expense of higher power consumption.
Dissecting Pruned Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Pruning is a standard technique for removing unnecessary structure from a neural network to reduce its storage footprint, computational demands, or energy consumption. Pruning can reduce the parameter-counts of many state-of-the-art neural networks by an order of magnitude without compromising accuracy, meaning these networks contain a vast amount of unnecessary structure. In this paper, we study the relationship between pruning and interpretability. Namely, we consider the effect of removing unnecessary structure on the number of hidden units that learn disentangled representations of human-recognizable concepts as identified by network dissection. We aim to evaluate how the interpretability of pruned neural networks changes as they are compressed. We find that pruning has no detrimental effect on this measure of interpretability until so few parameters remain that accuracy beings to drop. Resnet-50 models trained on ImageNet maintain the same number of interpretable concepts and units until more than 90% of parameters have been pruned.
X-CHANGR: Changing Memristive Crossbar Mapping for Mitigating Line-Resistance Induced Accuracy Degradation in Deep Neural Networks
Venue: arXiv cs.ET 2019
Year: 2019

There is widespread interest in emerging technologies, especially resistive crossbars for accelerating Deep Neural Networks (DNNs). Resistive crossbars offer a highly-parallel and efficient matrix-vector-multiplication (MVM) operation. MVM being the most dominant operation in DNNs makes crossbars ideally suited. However, various sources of device and circuit non-idealities lead to errors in the MVM output, thereby reducing DNN accuracy. Towards that end, we propose crossbar re-mapping strategies to mitigate line-resistance induced accuracy degradation in DNNs, without having to re-train the learned weights, unlike most prior works. Line-resistances degrade the voltage levels along the crossbar columns, thereby inducing more errors at the columns away from the drivers. We rank the DNN weights and kernels based on a sensitivity analysis, and re-arrange the columns such that the most sensitive kernels are mapped closer to the drivers, thereby minimizing the impact of errors on the overall accuracy. We propose two algorithms $-$ static remapping strategy (SRS) and dynamic remapping strategy (DRS), to optimize the crossbar re-arrangement of a pre-trained DNN. We demonstrate the benefits of our approach on a standard VGG16 network trained using CIFAR10 dataset. Our results show that SRS and DRS limit the accuracy degradation to 2.9\% and 2.1\%, respectively, compared to a 5.6\% drop from an as it is mapping of weights and kernels to crossbars. We believe this work brings an additional aspect for optimization, which can be used in tandem with existing mitigation techniques, such as in-situ compensation, technology aware training and re-training approaches, to enhance system performance.
Bonded Mining: Difficulty Adjustment by Miner Commitment
Venue: arXiv cs.CR 2019
Year: 2019

Proof-of-work blockchains must implement a difficulty adjustment algorithm (DAA) in order to maintain a consistent inter-arrival time between blocks. Conventional DAAs are essentially feedback controllers, and as such, they are inherently reactive. This approach leaves them susceptible to manipulation and often causes them to either under- or over-correct. We present Bonded Mining, a proactive DAA that works by collecting hash rate commitments secured by bond from miners. The difficulty is set directly from the commitments and the bond is used to penalize miners who deviate from their commitment. We devise a statistical test that is capable of detecting hash rate deviations by utilizing only on-blockchain data. The test is sensitive enough to detect a variety of deviations from commitments, while almost never misclassifying honest miners. We demonstrate in simulation that, under reasonable assumptions, Bonded Mining is more effective at maintaining a target block time than the Bitcoin Cash DAA, one of the newest and most dynamic DAAs currently deployed. In this preliminary work, the lowest hash rate miner our approach supports is 1% of the total and we directly consider only two types of fundamental attacks. Future work will address these limitations.
Multiple Landmark Detection using Multi-Agent Reinforcement Learning
Venue: arXiv cs.CV 2019
Year: 2019

The detection of anatomical landmarks is a vital step for medical image analysis and applications for diagnosis, interpretation and guidance. Manual annotation of landmarks is a tedious process that requires domain-specific expertise and introduces inter-observer variability. This paper proposes a new detection approach for multiple landmarks based on multi-agent reinforcement learning. Our hypothesis is that the position of all anatomical landmarks is interdependent and non-random within the human anatomy, thus finding one landmark can help to deduce the location of others. Using a Deep Q-Network (DQN) architecture we construct an environment and agent with implicit inter-communication such that we can accommodate K agents acting and learning simultaneously, while they attempt to detect K different landmarks. During training the agents collaborate by sharing their accumulated knowledge for a collective gain. We compare our approach with state-of-the-art architectures and achieve significantly better accuracy by reducing the detection error by 50%, while requiring fewer computational resources and time to train compared to the naive approach of training K agents separately.
Learning to Find Correlated Features by Maximizing Information Flow in Convolutional Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

Training convolutional neural networks for image classification tasks usually causes information loss. Although most of the time the information lost is redundant with respect to the target task, there are still cases where discriminative information is also discarded. For example, if the samples that belong to the same category have multiple correlated features, the model may only learn a subset of the features and ignore the rest. This may not be a problem unless the classification in the test set highly depends on the ignored features. We argue that the discard of the correlated discriminative information is partially caused by the fact that the minimization of the classification loss doesn't ensure to learn the overall discriminative information but only the most discriminative information. To address this problem, we propose an information flow maximization (IFM) loss as a regularization term to find the discriminative correlated features. With less information loss the classifier can make predictions based on more informative features. We validate our method on the shiftedMNIST dataset and show the effectiveness of IFM loss in learning representative and discriminative features.
FVA: Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics
Venue: arXiv cs.HC 2019
Year: 2019

We present a new approach for improving the friendliness and warmth of a virtual agent in an AR environment by generating appropriate movement characteristics. Our algorithm is based on a novel data-driven friendliness model that is computed using a user-study and psychological characteristics. We use our model to control the movements corresponding to the gaits, gestures, and gazing of friendly virtual agents (FVAs) as they interact with the user's avatar and other agents in the environment. We have integrated FVA agents with an AR environment using with a Microsoft HoloLens. Our algorithm can generate plausible movements at interactive rates to increase the social presence. We also investigate the perception of a user in an AR setting and observe that an FVA has a statistically significant improvement in terms of the perceived friendliness and social presence of a user compared to an agent without the friendliness modeling. We observe an increment of 5.71% in the mean responses to a friendliness measure and an improvement of 4.03% in the mean responses to a social presence measure.
Study of Rate-Splitting Techniques with Block Diagonalization for Multiuser MIMO Systems
Venue: arXiv cs.IT 2019
Year: 2019

In this work, we investigate Block Diagonalization (BD) techniques for multiuser multiple-antenna systems using rate-splitting (RS) multiple access. In RS multiple access the messages of the users are split into a common part and a private part in order to mitigate multiuser interference. We present the system model for a RS multiple access system operating in a broadcast channel scenario where the receivers are equipped with multiple antennas. We also develop linear precoders based on BD for the RS multiple access systems along with combining techniques, such as the min-max criterion and the maximum ratio combining criterion, to enhance the common rate. Closed-form expressions to describe the sum rate performance of the proposed scheme are also derived. The performance of the system is evaluated via simulations considering imperfect channel state information at the transmitter. The results show that the proposed schemes outperform conventional linear precoding methods.
Machine Learning for Intelligent Authentication in 5G-and-Beyond Wireless Networks
Venue: arXiv cs.CR 2019
Year: 2019

The fifth generation (5G) and beyond wireless networks are critical to support diverse vertical applications by connecting heterogeneous devices and machines, which directly increase vulnerability for various spoofing attacks. Conventional cryptographic and physical layer authentication techniques are facing some challenges in complex dynamic wireless environments, including significant security overhead, low reliability, as well as difficulty in pre-designing authentication model, providing continuous protections, and learning time-varying attributes. In this article, we envision new authentication approaches based on machine learning techniques by opportunistically leveraging physical layer attributes, and introduce intelligence to authentication for more efficient security provisioning. Machine learning paradigms for intelligent authentication design are presented, namely for parametric/non-parametric and supervised/unsupervised/reinforcement learning algorithms. In a nutshell, the machine learning-based intelligent authentication approaches utilize specific features in the multi-dimensional domain for achieving cost-effective, more reliable, model-free, continuous and situation-aware device validation under unknown network conditions and unpredictable dynamics.
Detecting Spiky Corruption in Markov Decision Processes
Venue: arXiv cs.LG 2019
Year: 2019

Current reinforcement learning methods fail if the reward function is imperfect, i.e. if the agent observes reward different from what it actually receives. We study this problem within the formalism of Corrupt Reward Markov Decision Processes (CRMDPs). We show that if the reward corruption in a CRMDP is sufficiently "spiky", the environment is solvable. We fully characterize the regret bound of a Spiky CRMDP, and introduce an algorithm that is able to detect its corrupt states. We show that this algorithm can be used to learn the optimal policy with any common reinforcement learning algorithm. Finally, we investigate our algorithm in a pair of simple gridworld environments, finding that our algorithm can detect the corrupt states and learn the optimal policy despite the corruption.
Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
Venue: arXiv cs.LG 2019
Year: 2019

Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment. These are critical shortcomings for applying RL to real-world problems where collecting data is expensive, and models must be tested offline before being deployed to interact with the environment -- e.g. systems that learn from human interaction. Thus, we develop a novel class of off-policy batch RL algorithms, which are able to effectively learn offline, without exploring, from a fixed batch of human interaction data. We leverage models pre-trained on data as a strong prior, and use KL-control to penalize divergence from this prior during RL training. We also use dropout-based uncertainty estimates to lower bound the target Q-values as a more efficient alternative to Double Q-Learning. The algorithms are tested on the problem of open-domain dialog generation -- a challenging reinforcement learning problem with a 20,000-dimensional action space. Using our Way Off-Policy algorithm, we can extract multiple different reward functions post-hoc from collected human interaction data, and learn effectively from all of these. We test the real-world generalization of these systems by deploying them live to converse with humans in an open-domain setting, and demonstrate that our algorithm achieves significant improvements over prior methods in off-policy batch RL.
Inter and Intra Document Attention for Depression Risk Assessment
Venue: arXiv cs.CL 2019
Year: 2019

We take interest in the early assessment of risk for depression in social media users. We focus on the eRisk 2018 dataset, which represents users as a sequence of their written online contributions. We implement four RNN-based systems to classify the users. We explore several aggregations methods to combine predictions on individual posts. Our best model reads through all writings of a user in parallel but uses an attention mechanism to prioritize the most important ones at each timestep.
Analyzing Utility of Visual Context in Multimodal Speech Recognition Under Noisy Conditions
Venue: arXiv cs.CL 2019
Year: 2019

Multimodal learning allows us to leverage information from multiple sources (visual, acoustic and text), similar to our experience of the real world. However, it is currently unclear to what extent auxiliary modalities improve performance over unimodal models, and under what circumstances the auxiliary modalities are useful. We examine the utility of the auxiliary visual context in Multimodal Automatic Speech Recognition in adversarial settings, where we deprive the models from partial audio signal during inference time. Our experiments show that while MMASR models show significant gains over traditional speech-to-text architectures (upto 4.2% WER improvements), they do not incorporate visual information when the audio signal has been corrupted. This shows that current methods of integrating the visual modality do not improve model robustness to noise, and we need better visually grounded adaptation techniques.
Effects of Foraging in Personalized Content-based Image Recommendation
Venue: arXiv cs.IR 2019
Year: 2019

A major challenge of recommender systems is to help users locating interesting items. Personalized recommender systems have become very popular as they attempt to predetermine the needs of users and provide them with recommendations to personalize their navigation. However, few studies have addressed the question of what drives the users' attention to specific content within the collection and what influences the selection of interesting items. To this end, we employ the lens of Information Foraging Theory (IFT) to image recommendation to demonstrate how the user could utilize visual bookmarks to locate interesting images. We investigate a personalized content-based image recommendation system to understand what affects user attention by reinforcing visual attention cues based on IFT. We further find that visual bookmarks (cues) lead to a stronger scent of the recommended image collection. Our evaluation is based on the Pinterest image collection.
Topic Modeling the Reading and Writing Behavior of Information Foragers
Venue: arXiv cs.CL 2019
Year: 2019

The general problem of "information foraging" in an environment about which agents have incomplete information has been explored in many fields, including cognitive psychology, neuroscience, economics, finance, ecology, and computer science. In all of these areas, the searcher aims to enhance future performance by surveying enough of existing knowledge to orient themselves in the information space. Individuals can be viewed as conducting a cognitive search in which they must balance exploration of ideas that are novel to them against exploitation of knowledge in domains in which they are already expert. In this dissertation, I present several case studies that demonstrate how reading and writing behaviors interact to construct personal knowledge bases. These studies use LDA topic modeling to represent the information environment of the texts each author read and wrote. Three studies revolve around Charles Darwin. Darwin left detailed records of every book he read for 23 years, from disembarking from the H.M.S. Beagle to just after publication of The Origin of Species. Additionally, he left copies of his drafts before publication. I characterize his reading behavior, then show how that reading behavior interacted with the drafts and subsequent revisions of The Origin of Species, and expand the dataset to include later readings and writings. Then, through a study of Thomas Jefferson's correspondence, I expand the study to non-book data. Finally, through an examination of neuroscience citation data, I move from individual behavior to collective behavior in constructing an information environment. Together, these studies reveal "the interplay between individual and collective phenomena where innovation takes place" (Tria et al. 2014).
The University of Sydney's Machine Translation System for WMT19
Venue: arXiv cs.CL 2019
Year: 2019

This paper describes the University of Sydney's submission of the WMT 2019 shared news translation task. We participated in the Finnish$\rightarrow$English direction and got the best BLEU(33.0) score among all the participants. Our system is based on the self-attentional Transformer networks, into which we integrated the most recent effective strategies from academic research (e.g., BPE, back translation, multi-features data selection, data augmentation, greedy model ensemble, reranking, ConMBR system combination, and post-processing). Furthermore, we propose a novel augmentation method $Cycle Translation$ and a data mixture strategy $Big$/$Small$ parallel construction to entirely exploit the synthetic corpus. Extensive experiments show that adding the above techniques can make continuous improvements of the BLEU scores, and the best result outperforms the baseline (Transformer ensemble model trained with the original parallel corpus) by approximately 5.3 BLEU score, achieving the state-of-the-art performance.