OpenRaiser/Intern-Atlas huggingface dataset:
Scaling Laws for Neural Language Models
Venue: arXiv cs.LG 2020
Year: 2020

We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of magnitude. Other architectural details such as network width or depth have minimal effects within a wide range. Simple equations govern the dependence of overfitting on model/dataset size and the dependence of training speed on model size. These relationships allow us to determine the optimal allocation of a fixed compute budget. Larger models are significantly more sample-efficient, such that optimally compute-efficient training involves training very large models on a relatively modest amount of data and stopping significantly before convergence.
Stacked Boosters Network Architecture for Short Term Load Forecasting in Buildings
Venue: arXiv cs.LG 2020
Year: 2020

This paper presents a novel deep learning architecture for short term load forecasting of building energy loads. The architecture is based on a simple base learner and multiple boosting systems that are modelled as a single deep neural network. The architecture transforms the original multivariate time series into multiple cascading univariate time series. Together with sparse interactions, parameter sharing and equivariant representations, this approach makes it possible to combat against overfitting while still achieving good presentation power with a deep network architecture. The architecture is evaluated in several short-term load forecasting tasks with energy data from an office building in Finland. The proposed architecture outperforms state-of-the-art load forecasting model in all the tasks.
Relational Thematic Clustering with Mutually Preferred Neighbors
Venue: arXiv cs.SI 2020
Year: 2020

Automatically learning thematic clusters in network data has long been a challenging task in machine learning community. A number of approaches have been proposed to accomplish it, utilizing edges, vertex features, or both aforementioned. However, few of them consider how the quantification of dichotomous inclination w.r.t. network topology and vertex features may influence vertex-cluster preferences, which deters previous methods from uncovering more interpretable latent groups in network data. To fill this void, we propose a novel probabilistic model, dubbed Relational Thematic Clustering with Mutually Preferred Neighbors (RTCMPN). Different from prevalent approaches which predetermine the learning significance of edge structure and vertex features, RTCMPN can further learn the latent preferences indicating which neighboring vertices are more possible to be in the same cluster, and the dichotomous inclinations describing how relative significance w.r.t. edge structure and vertex features may impact the association between pairwise vertices. Therefore, cluster structure implanted with edge structure, vertex features, neighboring preferences, and vertex-vertex dichotomous inclinations can be learned by RTCMPN. We additionally derive an effective Expectation-Maximization algorithm for RTCMPN to infer the optimal model parameters. RTCMPN has been compared with several strong baselines on various network data. The remarkable results validate the effectiveness of RTCMPN.
Sorting Permutations with Fixed Pinnacle Set
Venue: arXiv cs.DS 2020
Year: 2020

We give a positive answer to a question raised by Davis et al. ({\em Discrete Mathematics} 341, 2018), concerning permutations with the same pinnacle set. Given $π\in S_n$, a {\em pinnacle} of $π$ is an element $π_i$ ($i\neq 1,n$) such that $π_{i-1}<π_i>π_{i+1}$. The question is: given $π,π'\in S_n$ with the same pinnacle set $S$, is there a sequence of operations that transforms $π$ into $π'$ such that all the intermediate permutations have pinnacle set $S$? We introduce {\em balanced reversals}, defined as reversals that do not modify the pinnacle set of the permutation to which they are applied. Then we show that $π$ may be sorted by balanced reversals (i.e. transformed into a standard permutation $\Id_S$), implying that $π$ may be transformed into $π'$ using at most $4n-2\min\{p,3\}$ balanced reversals, where $p=|S|\geq 1$. In case $p=0$, at most $2n-1$ balanced reversals are needed.
Crushing the Wave -- new Z-Wave vulnerabilities exposed
Venue: arXiv cs.CR 2020
Year: 2020

This paper describes two denial of service attacks against the Z-Wave protocol and their effects on smart home gateways. Both utilize modified unencrypted packets, which are used in the inclusion phase and during normal operation. These are the commands Nonce Get/S2 Nonce Get and Find Nodes In Range. This paper shows how both can be manipulated and used to block a Z-Wave gateway's communication processing which in turn disables the whole Z-Wave network connected to it
DCT-Conv: Coding filters in convolutional networks with Discrete Cosine Transform
Venue: arXiv cs.NE 2020
Year: 2020

Convolutional neural networks are based on a huge number of trained weights. Consequently, they are often data-greedy, sensitive to overtraining, and learn slowly. We follow the line of research in which filters of convolutional neural layers are determined on the basis of a smaller number of trained parameters. In this paper, the trained parameters define a frequency spectrum which is transformed into convolutional filters with Inverse Discrete Cosine Transform (IDCT, the same is applied in decompression from JPEG). We analyze how switching off selected components of the spectra, thereby reducing the number of trained weights of the network, affects its performance. Our experiments show that coding the filters with trained DCT parameters leads to improvement over traditional convolution. Also, the performance of the networks modified this way decreases very slowly with the increasing extent of switching off these parameters. In some experiments, a good performance is observed when even 99.9% of these parameters are switched off.
Receive Antenna Selection for Secure Pre-coding Aided Spatial Modulation
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we make an investigation of receive antenna selection (RAS) strategies in the secure pre-coding aided spatial modulation (PSM) system with the aid of artificial noise. Due to a lack of the closed-form expression for secrecy rate (SR) in secure PSM systems, it is hard to optimize the RAS. To address this issue, the cut-off rate is used as an approximation of the SR. Further, two low-complexity RAS schemes for maximizing SR, called Max-SR-L and Max-SR-H, are derived in the low and high signal-to-noise ratio (SNR) regions, respectively. Due to the fact that the former works well in the low SNR region but becomes worse in the medium and high SNR regions while the latter also has the similar problem, a novel RAS strategy Max-SR-A is proposed to cover all SNR regions. Simulation results show that the proposed Max-SR-H and Max-SR-L schemes approach the optimal SR performances of the exhaustive search (ES) in the high and low SNR regions, respectively. In particular, the SR performance of the proposed Max-SR-A is close to that of the optimal ES and better than that of the random method in almost all SNR regions.
Design, Validation, and Case Studies of 2D-VSR-Sim, an Optimization-friendly Simulator of 2-D Voxel-based Soft Robots
Venue: arXiv cs.RO 2020
Year: 2020

Voxel-based soft robots (VSRs) are aggregations of soft blocks whose design is amenable to optimization. We here present a software, 2D-VSR-Sim, for facilitating research concerning the optimization of VSRs body and brain. The software, written in Java, provides consistent interfaces for all the VSRs aspects suitable for optimization and considers by design the presence of sensing, i.e., the possibility of exploiting the feedback from the environment for controlling the VSR. We experimentally characterize, from a mechanical point of view, the VSRs that can be simulated with 2D-VSR-Sim and we discuss the computational burden of the simulation. Finally, we show how 2D-VSR-Sim can be used to repeat the experiments of significant previous studies and, in perspective, to provide experimental answers to a variety of research questions.
Whose Tweets are Surveilled for the Police: An Audit of Social-Media Monitoring Tool via Log Files
Venue: arXiv cs.CY 2020
Year: 2020

Social media monitoring by law enforcement is becoming commonplace, but little is known about what software packages for it do. Through public records requests, we obtained log files from the Corvallis (Oregon) Police Department's use of social media monitoring software called DigitalStakeout. These log files include the results of proprietary searches by DigitalStakeout that were running over a period of 13 months and include 7240 social media posts. In this paper, we focus on the Tweets logged in this data and consider the racial and ethnic identity (through manual coding) of the users that are therein flagged by DigitalStakeout. We observe differences in the demographics of the users whose Tweets are flagged by DigitalStakeout compared to the demographics of the Twitter users in the region, however, our sample size is too small to determine significance. Further, the demographics of the Twitter users in the region do not seem to reflect that of the residents of the region, with an apparent higher representation of Black and Hispanic people. We also reconstruct the keywords related to a Narcotics report set up by DigitalStakeout for the Corvallis Police Department and find that these keywords flag Tweets unrelated to narcotics or flag Tweets related to marijuana, a drug that is legal for recreational use in Oregon. Almost all of the keywords have a common meaning unrelated to narcotics (e.g.\ broken, snow, hop, high) that call into question the utility that such a keyword based search could have to law enforcement.
A Dynamical Systems Perspective Reveals Coordination in Russian Twitter Operations
Venue: arXiv cs.SI 2020
Year: 2020

We study Twitter data from a dynamical systems perspective. In particular, we focus on the large set of data released by Twitter Inc. and asserted to represent a Russian influence operation. We propose a mathematical model to describe the per-day tweet production that can be extracted using spectral analysis. We show that this mathematical model allows us to construct families (clusters) of users with common harmonics. We define a labeling scheme describing user strategy in an information operation and show that the resulting strategies correspond to the behavioral clusters identified from their harmonics. We then compare these user clusters to the ones derived from text data using a graph-based topic analysis method. We show that spectral properties of the user clusters are related to the number of user-topic groups represented in a spectral cluster. Bulk data analysis also provides new insights into the data set in the context of prior work.
Wibson Protocol for Secure Data Exchange and Batch Payments
Venue: arXiv cs.CR 2020
Year: 2020

Wibson is a blockchain-based, decentralized data marketplace that provides individuals a way to securely and anonymously sell information in a trusted environment. The combination of the Wibson token and blockchain-enabled smart contracts hopes to allow Data Sellers and Data Buyers to transact with each other directly while providing individuals the ability to maintain anonymity as desired. The Wibson marketplace will provide infrastructure and financial incentives for individuals to securely sell personal information without sacrificing personal privacy. Data Buyers receive information from willing and actively participating individuals with the benefit of knowing that the personal information should be accurate and current. We present here two different components working together to achieve an efficient decentralized marketplace. The first is a smart contract called Data Exchange, which stores references to Data Orders that different Buyers open in order to show to the market that they are interested in buying certain types of data, and provides secure mechanisms to perform the transactions. The second is used to process payments from Buyers to Sellers and intermediaries, and is called Batch Payments.
SeCloak: ARM Trustzone-based Mobile Peripheral Control
Venue: arXiv cs.CR 2020
Year: 2020

Reliable on-off control of peripherals on smart devices is a key to security and privacy in many scenarios. Journalists want to reliably turn off radios to protect their sources during investigative reporting. Users wish to ensure cameras and microphones are reliably off during private meetings. In this paper, we present SeCloak, an ARM TrustZone-based solution that ensures reliable on-off control of peripherals even when the platform software is compromised. We design a secure kernel that co-exists with software running on mobile devices (e.g., Android and Linux) without requiring any code modifications. An Android prototype demonstrates that mobile peripherals like radios, cameras, and microphones can be controlled reliably with a very small trusted computing base and with minimal performance overhead.
Genome Reconstruction Attacks Against Genomic Data-Sharing Beacons
Venue: arXiv cs.CR 2020
Year: 2020

Sharing genome data in a privacy-preserving way stands as a major bottleneck in front of the scientific progress promised by the big data era in genomics. A community-driven protocol named genomic data-sharing beacon protocol has been widely adopted for sharing genomic data. The system aims to provide a secure, easy to implement, and standardized interface for data sharing by only allowing yes/no queries on the presence of specific alleles in the dataset. However, beacon protocol was recently shown to be vulnerable against membership inference attacks. In this paper, we show that privacy threats against genomic data sharing beacons are not limited to membership inference. We identify and analyze a novel vulnerability of genomic data-sharing beacons: genome reconstruction. We show that it is possible to successfully reconstruct a substantial part of the genome of a victim when the attacker knows the victim has been added to the beacon in a recent update. We also show that even if multiple individuals are added to the beacon during the same update, it is possible to identify the victim's genome with high confidence using traits that are easily accessible by the attacker (e.g., eye and hair color). Moreover, we show how the reconstructed genome using a beacon that is not associated with a sensitive phenotype can be used for membership inference attacks to beacons with sensitive phenotypes (i.e., HIV+). The outcome of this work will guide beacon operators on when and how to update the content of the beacon. Thus, this work will be an important attempt at helping beacon operators and participants make informed decisions.
Learning Multi-instrument Classification with Partial Labels
Venue: arXiv cs.SD 2020
Year: 2020

Multi-instrument recognition is the task of predicting the presence or absence of different instruments within an audio clip. A considerable challenge in applying deep learning to multi-instrument recognition is the scarcity of labeled data. OpenMIC is a recent dataset containing 20K polyphonic audio clips. The dataset is weakly labeled, in that only the presence or absence of instruments is known for each clip, while the onset and offset times are unknown. The dataset is also partially labeled, in that only a subset of instruments are labeled for each clip. In this work, we investigate the use of attention-based recurrent neural networks to address the weakly-labeled problem. We also use different data augmentation methods to mitigate the partially-labeled problem. Our experiments show that our approach achieves state-of-the-art results on the OpenMIC multi-instrument recognition task.
Compiling Crossing-free Geometric Graphs with Connectivity Constraint for Fast Enumeration, Random Sampling, and Optimization
Venue: arXiv cs.CG 2020
Year: 2020

Given $n$ points in the plane, we propose algorithms to compile connected crossing-free geometric graphs into directed acyclic graphs (DAGs). The DAGs allow efficient counting, enumeration, random sampling, and optimization. Our algorithms rely on Wettstein's framework to compile several crossing-free geometric graphs. One of the remarkable contributions of Wettstein is to allow dealing with geometric graphs with connectivity, since it is known to be difficult to efficiently represent geometric graphs with such global property. To achieve this, Wettstein proposed specialized techniques for crossing-free spanning trees and crossing-free spanning cycles and invented compiling algorithms running in $\mathrm{O}(7.044^n)$ time and $\mathrm{O}(5.619^n)$ time, respectively. Our first contribution is to propose a technique to deal with the connectivity constraint more simply and efficiently. It makes the design and analysis of algorithms easier, and yields improved time complexity. Our algorithms achieve $\mathrm{O}(6^n)$ time and $\mathrm{O}(4^n)$ time for compiling crossing-free spanning trees and crossing-free spanning cycles, respectively. As the second contribution, we propose an algorithm to optimize the area surrounded by crossing-free spanning cycles. To achieve this, we modify the DAG so that it has additional information. Our algorithm runs in $\mathrm{O}(4.829^n)$ time to find an area-minimized (or maximized) crossing-free spanning cycle of a given point set. Although the problem was shown to be NP-complete in 2000, as far as we know, there were no known algorithms faster than the obvious $\mathrm{O}(n!)$ time algorithm for 20 years.
K-resolver: Towards Decentralizing Encrypted DNS Resolution
Venue: arXiv cs.NI 2020
Year: 2020

Centralized DNS over HTTPS/TLS (DoH/DoT) resolution, which has started being deployed by major hosting providers and web browsers, has sparked controversy among Internet activists and privacy advocates due to several privacy concerns. This design decision causes the trace of all DNS resolutions to be exposed to a third-party resolver, different than the one specified by the user's access network. In this work we propose K-resolver, a DNS resolution mechanism that disperses DNS queries across multiple DoH resolvers, reducing the amount of information about a user's browsing activity exposed to each individual resolver. As a result, none of the resolvers can learn a user's entire web browsing history. We have implemented a prototype of our approach for Mozilla Firefox, and used it to evaluate the performance of web page load time compared to the default centralized DoH approach. While our K-resolver mechanism has some effect on DNS resolution time and web page load time, we show that this is mainly due to the geographical location of the selected DoH servers. When more well-provisioned anycast servers are available, our approach incurs negligible overhead while improving user privacy.
The Impact of Content Commenting on User Continuance in Online Q&A Communities: An Affordance Perspective
Venue: arXiv cs.HC 2020
Year: 2020

Online question-and-answer (Q&A) communities provide convenient and innovative ways for participants to share information and collaboratively solve problems with others. A growing challenge for those Q&A communities is to encourage and maintain ongoing user participation. From the perspective of motivational affordances, this study proposes a research framework to explain the effect of content commenting on user continuance behavior in online Q&A communities. The moderating role of participant's tenure in the relationship between content commenting and user continuance is also explored. Using a longitudinal panel dataset collected from a large online Q&A community, this research empirically tests the effect of content commenting on continued user participation in the Q&A community. The results show that both comment receipt and comment provisioning are important motivating factors for user continuance in the community. Specifically, received comments on questions submitted by a participant have a positive effect on the participant's continuance of posting questions, while answer comments both received and posted by a participant have positive impact on user continuance of posting answers in the community. In addition, tenure in the community is indeed found to have a significant negative moderating effect on the relationship between content commenting and user continuance. This research not only offers a more nuanced theoretical understanding of how content commenting affects continued user involvement and how participants' tenure in the community moderates the impact of content commenting, but also provides implications for improving user continuance in online Q&A communities.
A Framework for Cloud Security Risk Management Based on the Business Objectives of Organizations
Venue: arXiv cs.CR 2020
Year: 2020

Security is considered one of the top ranked risks of Cloud Computing (CC) due to the outsourcing of sensitive data onto a third party. In addition, the complexity of the cloud model results in a large number of heterogeneous security controls that must be consistently managed. Hence, no matter how strongly the cloud model is secured, organizations continue suffering from lack of trust on CC and remain uncertain about its security risk consequences. Traditional risk management frameworks do not consider the impact of CC security risks on the business objectives of the organizations. In this paper, we propose a novel Cloud Security Risk Management Framework (CSRMF) that helps organizations adopting CC identify, analyze, evaluate, and mitigate security risks in their Cloud platforms. Unlike traditional risk management frameworks, CSRMF is driven by the business objectives of the organizations. It allows any organization adopting CC to be aware of cloud security risks and align their low-level management decisions according to high-level business objectives. In essence, it is designed to address impacts of cloud-specific security risks into business objectives in a given organization. Consequently, organizations are able to conduct a cost-value analysis regarding the adoption of CC technology and gain an adequate level of confidence in Cloud technology. On the other hand, Cloud Service Providers (CSP) are able to improve productivity and profitability by managing cloud-related risks. The proposed framework has been validated and evaluated through a use-case scenario.