OpenRaiser/Intern-Atlas huggingface dataset:
Domain Name Encryption Is Not Enough: Privacy Leakage via IP-based Website Fingerprinting
Venue: arXiv cs.CR 2021
Year: 2021

Although the security benefits of domain name encryption technologies such as DNS over TLS (DoT), DNS over HTTPS (DoH), and Encrypted Client Hello (ECH) are clear, their positive impact on user privacy is weakened by--the still exposed--IP address information. However, content delivery networks, DNS-based load balancing, co-hosting of different websites on the same server, and IP address churn, all contribute towards making domain-IP mappings unstable, and prevent straightforward IP-based browsing tracking. In this paper, we show that this instability is not a roadblock (assuming a universal DoT/DoH and ECH deployment), by introducing an IP-based website fingerprinting technique that allows a network-level observer to identify at scale the website a user visits. Our technique exploits the complex structure of most websites, which load resources from several domains besides their primary one. Using the generated fingerprints of more than 200K websites studied, we could successfully identify 84% of them when observing solely destination IP addresses. The accuracy rate increases to 92% for popular websites, and 95% for popular and sensitive websites. We also evaluated the robustness of the generated fingerprints over time, and demonstrate that they are still effective at successfully identifying about 70% of the tested websites after two months. We conclude by discussing strategies for website owners and hosting providers towards hindering IP-based website fingerprinting and maximizing the privacy benefits offered by DoT/DoH and ECH.
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering
Venue: arXiv cs.CL 2021
Year: 2021

When Question-Answering (QA) systems are deployed in the real world, users query them through a variety of interfaces, such as speaking to voice assistants, typing questions into a search engine, or even translating questions to languages supported by the QA system. While there has been significant community attention devoted to identifying correct answers in passages assuming a perfectly formed question, we show that components in the pipeline that precede an answering engine can introduce varied and considerable sources of error, and performance can degrade substantially based on these upstream noise sources even for powerful pre-trained QA models. We conclude that there is substantial room for progress before QA systems can be effectively deployed, highlight the need for QA evaluation to expand to consider real-world use, and hope that our findings will spur greater community interest in the issues that arise when our systems actually need to be of utility to humans.
Topological Deep Learning: Classification Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Topological deep learning is a formalism that is aimed at introducing topological language to deep learning for the purpose of utilizing the minimal mathematical structures to formalize problems that arise in a generic deep learning problem. This is the first of a sequence of articles with the purpose of introducing and studying this formalism. In this article, we define and study the classification problem in machine learning in a topological setting. Using this topological framework, we show when the classification problem is possible or not possible in the context of neural networks. Finally, we demonstrate how our topological setting immediately illuminates aspects of this problem that are not as readily apparent using traditional tools.
Revisiting Language Encoding in Learning Multilingual Representations
Venue: arXiv cs.CL 2021
Year: 2021

Transformer has demonstrated its great power to learn contextual word representations for multiple languages in a single model. To process multilingual sentences in the model, a learnable vector is usually assigned to each language, which is called "language embedding". The language embedding can be either added to the word embedding or attached at the beginning of the sentence. It serves as a language-specific signal for the Transformer to capture contextual representations across languages. In this paper, we revisit the use of language embedding and identify several problems in the existing formulations. By investigating the interaction between language embedding and word embedding in the self-attention module, we find that the current methods cannot reflect the language-specific word correlation well. Given these findings, we propose a new approach called Cross-lingual Language Projection (XLP) to replace language embedding. For a sentence, XLP projects the word embeddings into language-specific semantic space, and then the projected embeddings will be fed into the Transformer model to process with their language-specific meanings. In such a way, XLP achieves the purpose of appropriately encoding "language" in a multilingual Transformer model. Experimental results show that XLP can freely and significantly boost the model performance on extensive multilingual benchmark datasets. Codes and models will be released at https://github.com/lsj2408/XLP.
Darknet Traffic Big-Data Analysis and Network Management to Real-Time Automating the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework
Venue: arXiv cs.CR 2021
Year: 2021

Attackers are perpetually modifying their tactics to avoid detection and frequently leverage legitimate credentials with trusted tools already deployed in a network environment, making it difficult for organizations to proactively identify critical security risks. Network traffic analysis products have emerged in response to attackers relentless innovation, offering organizations a realistic path forward for combatting creative attackers. Additionally, thanks to the widespread adoption of cloud computing, Device Operators processes, and the Internet of Things, maintaining effective network visibility has become a highly complex and overwhelming process. What makes network traffic analysis technology particularly meaningful is its ability to combine its core capabilities to deliver malicious intent detection. In this paper, we propose a novel darknet traffic analysis and network management framework to real-time automating the malicious intent detection process, using a weight agnostic neural networks architecture. It is an effective and accurate computational intelligent forensics tool for network traffic analysis, the demystification of malware traffic, and encrypted traffic identification in real-time. Based on Weight Agnostic Neural Networks methodology, we propose an automated searching neural net architectures strategy that can perform various tasks such as identify zero-day attacks. By automating the malicious intent detection process from the darknet, the advanced proposed solution is reducing the skills and effort barrier that prevents many organizations from effectively protecting their most critical assets.
Complex Momentum for Optimization in Games
Venue: arXiv cs.LG 2021
Year: 2021

We generalize gradient descent with momentum for optimization in differentiable games to have complex-valued momentum. We give theoretical motivation for our method by proving convergence on bilinear zero-sum games for simultaneous and alternating updates. Our method gives real-valued parameter updates, making it a drop-in replacement for standard optimizers. We empirically demonstrate that complex-valued momentum can improve convergence in realistic adversarial games - like generative adversarial networks - by showing we can find better solutions with an almost identical computational cost. We also show a practical generalization to a complex-valued Adam variant, which we use to train BigGAN to better inception scores on CIFAR-10.
A Bayesian Approach for Inferring Sea Ice Loads
Venue: arXiv cs.CE 2021
Year: 2021

The Earth's climate is rapidly changing and some of the most drastic changes can be seen in the Arctic, where sea ice extent has diminished considerably in recent years. As the Arctic climate continues to change, gathering in situ sea ice measurements is increasingly important for understanding the complex evolution of the Arctic ice pack. To date, observations of ice stresses in the Arctic have been spatially and temporally sparse. We propose a measurement framework that would instrument existing sea ice buoys with strain gauges. This measurement framework uses a Bayesian inference approach to infer ice loads acting on the buoy from a set of strain gauge measurements. To test our framework, strain measurements were collected from an experiment where a buoy was frozen into ice that was subsequently compressed to simulate convergent sea ice conditions. A linear elastic finite element model was used to describe the response of the deformable buoy to mechanical loading, allowing us to link the observed strain on the buoy interior to the applied load on the buoy exterior. The approach presented in this paper presents an instrumentation framework that could use existing buoy platforms as in situ sensors of internal stresses in the ice pack.
THIA: Accelerating Video Analytics using Early Inference and Fine-Grained Query Planning
Venue: arXiv cs.DB 2021
Year: 2021

To efficiently process visual data at scale, researchers have proposed two techniques for lowering the computational overhead associated with the underlying deep learning models. The first approach consists of leveraging a specialized, lightweight model to directly answer the query. The second approach focuses on filtering irrelevant frames using a lightweight model and processing the filtered frames using a heavyweight model. These techniques suffer from two limitations. With the first approach, the specialized model is unable to provide accurate results for hard-to-detect events. With the second approach, the system is unable to accelerate queries focusing on frequently occurring events as the filter is unable to eliminate a significant fraction of frames in the video. In this paper, we present THIA, a video analytics system for tackling these limitations. The design of THIA is centered around three techniques. First, instead of using a cascade of models, it uses a single object detection model with multiple exit points for short-circuiting the inference. This early inference technique allows it to support a range of throughput-accuracy tradeoffs. Second, it adopts a fine-grained approach to planning and processes different chunks of the video using different exit points to meet the user's requirements. Lastly, it uses a lightweight technique for directly estimating the exit point for a chunk to lower the optimization time. We empirically show that these techniques enable THIA to outperform two state-of-the-art video analytics systems by up to 6.5X, while providing accurate results even on queries focusing on hard-to-detect events.
Automatic Detection of Five API Documentation Smells: Practitioners' Perspectives
Venue: arXiv cs.SE 2021
Year: 2021

The learning and usage of an API is supported by official documentation. Like source code, API documentation is itself a software product. Several research results show that bad design in API documentation can make the reuse of API features difficult. Indeed, similar to code smells or code antipatterns, poorly designed API documentation can also exhibit 'smells'. Such documentation smells can be described as bad documentation styles that do not necessarily produce an incorrect documentation but nevertheless make the documentation difficult to properly understand and to use. Recent research on API documentation has focused on finding content inaccuracies in API documentation and to complement API documentation with external resources (e.g., crowd-shared code examples). We are aware of no research that focused on the automatic detection of API documentation smells. This paper makes two contributions. First, we produce a catalog of five API documentation smells by consulting literature on API documentation presentation problems. We create a benchmark dataset of 1,000 API documentation units by exhaustively and manually validating the presence of the five smells in Java official API reference and instruction documentation. Second, we conduct a survey of 21 professional software developers to validate the catalog. The developers agreed that they frequently encounter all five smells in API official documentation and 95.2% of them reported that the presence of the documentation smells negatively affects their productivity. The participants wished for tool support to automatically detect and fix the smells in API official documentation. We develop a suite of rule-based, deep and shallow machine learning classifiers to automatically detect the smells. The best performing classifier BERT, a deep learning model, achieves F1-scores of 0.75 - 0.97.
Pattern Sampling for Shapelet-based Time Series Classification
Venue: arXiv cs.LG 2021
Year: 2021

Subsequence-based time series classification algorithms provide accurate and interpretable models, but training these models is extremely computation intensive. The asymptotic time complexity of subsequence-based algorithms remains a higher-order polynomial, because these algorithms are based on exhaustive search for highly discriminative subsequences. Pattern sampling has been proposed as an effective alternative to mitigate the pattern explosion phenomenon. Therefore, we employ pattern sampling to extract discriminative features from discretized time series data. A weighted trie is created based on the discretized time series data to sample highly discriminative patterns. These sampled patterns are used to identify the shapelets which are used to transform the time series classification problem into a feature-based classification problem. Finally, a classification model can be trained using any off-the-shelf algorithm. Creating a pattern sampler requires a small number of patterns to be evaluated compared to an exhaustive search as employed by previous approaches. Compared to previously proposed algorithms, our approach requires considerably less computational and memory resources. Experiments demonstrate how the proposed approach fares in terms of classification accuracy and runtime performance.
Automatic API Usage Scenario Documentation from Technical Q&A Sites
Venue: arXiv cs.SE 2021
Year: 2021

The online technical Q&A site Stack Overflow (SO) is popular among developers to support their coding and diverse development needs. To address shortcomings in API official documentation resources, several research has thus focused on augmenting official API documentation with insights (e.g., code examples) from SO. The techniques propose to add code examples/insights about APIs into its official documentation. Reviews are opinionated sentences with positive/negative sentiments. However, we are aware of no previous research that attempts to automatically produce API documentation from SO by considering both API code examples and reviews. In this paper, we present two novel algorithms that can be used to automatically produce API documentation from SO by combining code examples and reviews towards those examples. The first algorithm is called statistical documentation, which shows the distribution of positivity and negativity around the code examples of an API using different metrics (e.g., star ratings). The second algorithm is called concept-based documentation, which clusters similar and conceptually relevant usage scenarios. An API usage scenario contains a code example, a textual description of the underlying task addressed by the code example, and the reviews (i.e., opinions with positive and negative sentiments) from other developers towards the code example. We deployed the algorithms in Opiner, a web-based platform to aggregate information about APIs from online forums. We evaluated the algorithms by mining all Java JSON-based posts in SO and by conducting three user studies based on produced documentation from the posts.
NEAT: A Framework for Automated Exploration of Floating Point Approximations
Venue: arXiv cs.DC 2021
Year: 2021

Much recent research is devoted to exploring tradeoffs between computational accuracy and energy efficiency at different levels of the system stack. Approximation at the floating point unit (FPU) allows saving energy by simply reducing the number of computed floating point bits in return for accuracy loss. Although, finding the most energy efficient approximation for various applications with minimal effort is the main challenge. To address this issue, we propose NEAT: a pin tool that helps users automatically explore the accuracy-energy tradeoff space induced by various floating point implementations. NEAT helps programmers explore the effects of simultaneously using multiple floating point implementations to achieve the lowest energy consumption for an accuracy constraint or vice versa. NEAT accepts one or more user-defined floating point implementations and programmable placement rules for where/when to apply them. NEAT then automatically replaces floating point operations with different implementations based on the user-specified rules during the runtime and explores the resulting tradeoff space to find the best use of approximate floating point implementations for the precision tuning throughout the program. We evaluate NEAT by enforcing combinations of 24/53 different floating point implementations with three sets of placement rules on a wide range of benchmarks. We find that heuristic precision tuning at the function level provides up to 22% and 48% energy savings at 1% and 10% accuracy loss comparing to applying a single implementation for the whole application. Also, NEAT is applicable to neural networks where it finds the optimal precision level for each layer considering an accuracy target for the model.
First Target and Opinion then Polarity: Enhancing Target-opinion Correlation for Aspect Sentiment Triplet Extraction
Venue: arXiv cs.CL 2021
Year: 2021

Aspect Sentiment Triplet Extraction (ASTE) aims to extract triplets from a sentence, including target entities, associated sentiment polarities, and opinion spans which rationalize the polarities. Existing methods are short on building correlation between target-opinion pairs, and neglect the mutual interference among different sentiment triplets. To address these issues, we utilize a two-stage framework to enhance the correlation between targets and opinions: at stage one, we extract targets and opinions through sequence tagging; then we append a group of artificial tags named Perceivable Pair, which indicate the span of a specific target-opinion tuple, to the input sentence to obtain closer correlated target-opinion pair representation. Meanwhile, we reduce the negative interference between triplets by restricting tokens' attention field. Finally, the polarity is identified according to the representation of the Perceivable Pair. We conduct experiments on four datasets, and the experimental results show the effectiveness of our model.
Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

We propose firefly neural architecture descent, a general framework for progressively and dynamically growing neural networks to jointly optimize the networks' parameters and architectures. Our method works in a steepest descent fashion, which iteratively finds the best network within a functional neighborhood of the original network that includes a diverse set of candidate network structures. By using Taylor approximation, the optimal network structure in the neighborhood can be found with a greedy selection procedure. We show that firefly descent can flexibly grow networks both wider and deeper, and can be applied to learn accurate but resource-efficient neural architectures that avoid catastrophic forgetting in continual learning. Empirically, firefly descent achieves promising results on both neural architecture search and continual learning. In particular, on a challenging continual image classification task, it learns networks that are smaller in size but have higher average accuracy than those learned by the state-of-the-art methods.
NODE-SELECT: A Graph Neural Network Based On A Selective Propagation Technique
Venue: arXiv cs.LG 2021
Year: 2021

While there exists a wide variety of graph neural networks (GNN) for node classification, only a minority of them adopt mechanisms that effectively target noise propagation during the message-passing procedure. Additionally, a very important challenge that significantly affects graph neural networks is the issue of scalability which limits their application to larger graphs. In this paper we propose our method named NODE-SELECT: an efficient graph neural network that uses subsetting layers which only allow the best sharing-fitting nodes to propagate their information. By having a selection mechanism within each layer which we stack in parallel, our proposed method NODE-SELECT is able to both reduce the amount noise propagated and adapt the restrictive sharing concept observed in real world graphs. Our NODE-SELECT significantly outperformed existing GNN frameworks in noise experiments and matched state-of-the art results in experiments without noise over different benchmark datasets.
Client-Based Intelligence for Resource Efficient Vehicular Big Data Transfer in Future 6G Network
Venue: arXiv cs.NI 2021
Year: 2021

Vehicular big data is anticipated to become the "new oil" of the automotive industry which fuels the development of novel crowdsensing-enabled services. However, the tremendous amount of transmitted vehicular sensor data represents a massive challenge for the cellular network. A promising method for achieving relief which allows to utilize the existing network resources in a more efficient way is the utilization of intelligence on the end-edge-cloud devices. Through machine learning-based identification and exploitation of highly resource efficient data transmission opportunities, the client devices are able to participate in overall network resource optimization process. In this work, we present a novel client-based opportunistic data transmission method for delay-tolerant applications which is based on a hybrid machine learning approach: Supervised learning is applied to forecast the currently achievable data rate which serves as the metric for the reinforcement learning-based data transfer scheduling process. In addition, unsupervised learning is applied to uncover geospatially-dependent uncertainties within the prediction model. In a comprehensive real world evaluation in the public cellular networks of three German Mobile Network Operators (MNOs), we show that the average data rate can be improved by up to 223 % while simultaneously reducing the amount of occupied network resources by up to 89 %. As a side-effect of preferring more robust network conditions for the data transfer, the transmission-related power consumption is reduced by up to 73 %. The price to pay is an increased Age of Information (AoI) of the sensor data.
Linear Time Runs over General Ordered Alphabets
Venue: arXiv cs.DS 2021
Year: 2021

A run in a string is a maximal periodic substring. For example, the string $\texttt{bananatree}$ contains the runs $\texttt{anana} = (\texttt{an})^{3/2}$ and $\texttt{ee} = \texttt{e}^2$. There are less than $n$ runs in any length-$n$ string, and computing all runs for a string over a linearly-sortable alphabet takes $\mathcal{O}(n)$ time (Bannai et al., SODA 2015). Kosolobov conjectured that there also exists a linear time runs algorithm for general ordered alphabets (Inf. Process. Lett. 2016). The conjecture was almost proven by Crochemore et al., who presented an $\mathcal{O}(nα(n))$ time algorithm (where $α(n)$ is the extremely slowly growing inverse Ackermann function). We show how to achieve $\mathcal{O}(n)$ time by exploiting combinatorial properties of the Lyndon array, thus proving Kosolobov's conjecture.