OpenRaiser/Intern-Atlas huggingface dataset:
Secrecy Performance of RIS-assisted Wireless Networks under Rician fading
Venue: arXiv cs.IT 2021
Year: 2021

Secrecy outage probability (SOP) and secrecy rate (SR) of the reconfigurable intelligent surface (RIS) assisted wireless networks under Rician fading are investigated in this paper. More precisely, we enhance the secrecy performance of the considered networks by suppressing the wiretap channel instead of maximizing the main channel. We propose a simple heuristic algorithm to find out the optimal phase-shift of each RIS's element. Simulation results based on the Monte-Carlo method are given to verify the superiority of the proposed optimal phase-shifts compared to the random phase-shifts design.
Spectrum Trading for Device-to-Device Communication In Cellular Networks using Incomplete Information Bandwidth-Auction Game
Venue: arXiv cs.IT 2021
Year: 2021

Device-to-device (D2D) communication that allows proximity users to communicate directly has been recently proposed to improve spectral efficiency of cellular networks. In this paper, we assume a cellular network consisting of multiple cellular user equipments (CUEs), which are the primary users, and a cognitive D2D pair, which is the secondary user. The D2D pair needs a bandwidth for data transmission that can be obtained via spectrum trading. We introduce a bandwidth-auction game for the spectrum trading problem. The base station (BS) and CUEs are able to sell their spectrum or share it with the D2D pair, which allows the D2D pair to operate in orthogonal sharing, cellular, or non-orthogonal sharing (NOS) modes. Operation of the D2D pair in the NOS mode causes interference to the CUEs, which is possible under low interference condition. In the auction, the D2D pair can buy its required spectrum from three different service providers (SPs) corresponding to each mode that operateon different frequency spectrums. The D2D pair bids a price bandwidth demand curve and the SPs offer a price-demand supply curve. Since each player is not aware of the strategy of other players in practical scenarios, the game is assumed to be an incomplete information repeated one. A best response based learning method is proposed for the decision making procedure of all players, the D2D pair and SPs. It is shown that the proposed method converges to the Nash equilibrium (NE) point of the game more rapidly than the state-of-the-art methods when the game is played repeatedly. The sensitivity of the proposed method to the learning rate variable is also less than the state-of-the-art methods and hence can be considered as a robust one.
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
Venue: arXiv cs.CL 2021
Year: 2021

What are the units of text that we want to model? From bytes to multi-word expressions, text can be analyzed and generated at many granularities. Until recently, most natural language processing (NLP) models operated over words, treating those as discrete and atomic tokens, but starting with byte-pair encoding (BPE), subword-based approaches have become dominant in many areas, enabling small vocabularies while still allowing for fast inference. Is the end of the road character-level model or byte-level processing? In this survey, we connect several lines of work from the pre-neural and neural era, by showing how hybrid approaches of words and characters as well as subword-based approaches based on learned segmentation have been proposed and evaluated. We conclude that there is and likely will never be a silver bullet singular solution for all applications and that thinking seriously about tokenization remains important for many applications.
An ensemble deep learning technique for detecting suicidal ideation from posts in social media platforms
Venue: arXiv cs.IR 2021
Year: 2021

Suicidal ideation detection from social media is an evolving research with great challenges. Many of the people who have the tendency to suicide share their thoughts and opinions through social media platforms. As part of many researches it is observed that the publicly available posts from social media contain valuable criteria to effectively detect individuals with suicidal thoughts. The most difficult part to prevent suicide is to detect and understand the complex risk factors and warning signs that may lead to suicide. This can be achieved by identifying the sudden changes in a user behavior automatically. Natural language processing techniques can be used to collect behavioral and textual features from social media interactions and these features can be passed to a specially designed framework to detect anomalies in human interactions that are indicators of suicidal intentions. We can achieve fast detection of suicidal ideation using deep learning and/or machine learning based classification approaches. For such a purpose, we can employ the combination of LSTM and CNN models to detect such emotions from posts of the users. In order to improve the accuracy, some approaches like using more data for training, using attention model to improve the efficiency of existing models etc. could be done. This paper proposes a LSTM-Attention-CNN combined model to analyze social media submissions to detect any underlying suicidal intentions. During evaluations, the proposed model demonstrated an accuracy of 90.3 percent and an F1-score of 92.6 percent, which is greater than the baseline models.
Intelligent Online Selling Point Extraction for E-Commerce Recommendation
Venue: arXiv cs.IR 2021
Year: 2021

In the past decade, automatic product description generation for e-commerce have witnessed significant advancement. As the services provided by e-commerce platforms become diverse, it is necessary to dynamically adapt the patterns of descriptions generated. The selling point of products is an important type of product description for which the length should be as short as possible while still conveying key information. In addition, this kind of product description should be eye-catching to the readers. Currently, product selling points are normally written by human experts. Thus, the creation and maintenance of these contents incur high costs. These costs can be significantly reduced if product selling points can be automatically generated by machines. In this paper, we report our experience developing and deploying the Intelligent Online Selling Point Extraction (IOSPE) system to serve the recommendation system in the JD.com e-commerce platform. Since July 2020, IOSPE has become a core service for 62 key categories of products (covering more than 4 million products). So far, it has generated more than 0.1 billion selling points, thereby significantly scaling up the selling point creation operation and saving human labour. These IOSPE generated selling points have increased the click-through rate (CTR) by 1.89\% and the average duration the customers spent on the products by more than 2.03\% compared to the previous practice, which are significant improvements for such a large-scale e-commerce platform.
Evolutionary Hierarchical Harvest Schedule Optimization for Food Waste Prevention
Venue: arXiv cs.CY 2021
Year: 2021

In order to avoid disadvantages of monocropping for soil and environment, it is advisable to practice intercropping of various plant species whenever possible. However, intercropping is challenging as it requires a balanced planting schedule due to individual cultivation time frames. Maintaining a continuous harvest reduces logistical costs and related greenhouse gas emissions, and contributes to food waste prevention. In this work, we address these issues and propose an optimization method for a full harvest season of large crop ensembles that complies with given constraints. By using an approach based on an evolutionary algorithm combined with a novel hierarchical loss function and adaptive mutation rate, we transfer the multi-objective into a pseudo-single-objective optimization problem and obtain faster convergence and better solutions than for conventional approaches.
Learning Physics Properties of Fabrics and Garments with a Physics Similarity Neural Network
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose to predict the physics parameters of real fabrics and garments by learning their physics similarities between simulated fabrics via a Physics Similarity Network (PhySNet). For this, we estimate wind speeds generated by an electric fan and the area weight to predict bending stiffness of simulated and real fabrics and garments. We found that PhySNet coupled with a Bayesian optimiser can predict physics parameters and improve the state-of-art by 34%for real fabrics and 68% for real garments.
Improving Learning-to-Defer Algorithms Through Fine-Tuning
Venue: arXiv cs.LG 2021
Year: 2021

The ubiquity of AI leads to situations where humans and AI work together, creating the need for learning-to-defer algorithms that determine how to partition tasks between AI and humans. We work to improve learning-to-defer algorithms when paired with specific individuals by incorporating two fine-tuning algorithms and testing their efficacy using both synthetic and image datasets. We find that fine-tuning can pick up on simple human skill patterns, but struggles with nuance, and we suggest future work that uses robust semi-supervised to improve learning.
Checkpoint-Restart Libraries Must Become More Fault Tolerant
Venue: arXiv cs.DC 2021
Year: 2021

Production MPI codes need checkpoint-restart (CPR) support. Clearly, checkpoint-restart libraries must be fault tolerant lest they open up a window of vulnerability for failures with byzantine outcomes. But, certain popular libraries that leverage MPI are evidently not fault tolerant. Nowadays, fault detection with automatic recovery without batch requeueing is a strong requirement for production environments. Thus, allowing deadlock and setting long timeouts are suboptimal for fault detection even when paired with conservative recovery from the penultimate checkpoint. When MPI is used as a communication mechanism within a CPR library, such libraries must offer fault-tolerant extensions with minimal detection, isolation, mitigation, and potential recovery semantics to aid the CPR's library fail-backward. Communication between MPI and the checkpoint library regarding system health may be valuable. For fault-tolerant MPI programs (e.g., using APIs like FA-MPI, Stages/Reinit, or ULFM), the checkpoint library must cooperate with the extended model or else invalidate fault-tolerant operation.
Load-balanced Gather-scatter Patterns for Sparse Deep Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks (DNNs) have been proven to be effective in solving many real-life problems, but its high computation cost prohibits those models from being deployed to edge devices. Pruning, as a method to introduce zeros to model weights, has shown to be an effective method to provide good trade-offs between model accuracy and computation efficiency, and is a widely-used method to generate compressed models. However, the granularity of pruning makes important trade-offs. At the same sparsity level, a coarse-grained structured sparse pattern is more efficient on conventional hardware but results in worse accuracy, while a fine-grained unstructured sparse pattern can achieve better accuracy but is inefficient on existing hardware. On the other hand, some modern processors are equipped with fast on-chip scratchpad memories and gather/scatter engines that perform indirect load and store operations on such memories. In this work, we propose a set of novel sparse patterns, named gather-scatter (GS) patterns, to utilize the scratchpad memories and gather/scatter engines to speed up neural network inferences. Correspondingly, we present a compact sparse format. The proposed set of sparse patterns, along with a novel pruning methodology, address the load imbalance issue and result in models with quality close to unstructured sparse models and computation efficiency close to structured sparse models. Our experiments show that GS patterns consistently make better trade-offs between accuracy and computation efficiency compared to conventional structured sparse patterns. GS patterns can reduce the runtime of the DNN components by two to three times at the same accuracy levels. This is confirmed on three different deep learning tasks and popular models, namely, GNMT for machine translation, ResNet50 for image recognition, and Japser for acoustic speech recognition.
The Cascading Metric Tree
Venue: arXiv cs.DS 2021
Year: 2021

This paper presents the Cascaded Metric Tree (CMT) for efficient satisfaction of metric search queries over a dataset of N objects. It provides extra information that permits query algorithms to exploit all distance calculations performed along each path in the tree for pruning purposes. In addition to improving standard metric range (ball) query algorithms, we present a new algorithm for exploiting the CMT cascaded information to achieve near-optimal performance for k-nearest neighbor (kNN) queries. We demonstrate the performance advantage of CMT over classical metric search structures on synthetic datasets of up to 10 million objects and on the 564K Swiss-Prot protein sequence dataset containing over $200$ million amino acids. As a supplement to the paper, we provide reference implementations of the empirically-examined algorithms to encourage improvements and further applications of CMT to practical scientific and engineering problems
Accelerating Clique Counting in Sparse Real-World Graphs via Communication-Reducing Optimizations
Venue: arXiv cs.DS 2021
Year: 2021

Counting instances of specific subgraphs in a larger graph is an important problem in graph mining. Finding cliques of size k (k-cliques) is one example of this NP-hard problem. Different algorithms for clique counting avoid counting the same clique multiple times by pivoting or ordering the graph. Ordering-based algorithms include an ordering step to direct the edges in the input graph, and a counting step, which is dominated by building node or edge-induced subgraphs. Of the ordering-based algorithms, kClist is the state-of-the art algorithm designed to work on sparse real-world graphs. Despite its leading overall performance, kClist's vertex-parallel implementation does not scale well in practice on graphs with a few million vertices. We present CITRON (Clique counting with Traffic Reducing Optimizations) to improve the parallel scalability and thus overall performance of clique counting. We accelerate the ordering phase by abandoning kClist's sequential core ordering and using a parallelized degree ordering. We accelerate the counting phase with our reorganized subgraph data structures that reduce memory traffic to improve scaling bottlenecks. Our sorted, compact neighbor lists improve locality and communication efficiency which results in near-linear parallel scaling. CITRON significantly outperforms kClist while counting moderately sized cliques, and thus increases the size of graph practical for clique counting. We have recently become aware of ArbCount (arXiv:2002.10047), which often outperforms us. However, we believe that the analysis included in this paper will be helpful for anyone who wishes to understand the performance characteristics of k-clique counting.
On the Size and Width of the Decoder of a Boolean Threshold Autoencoder
Venue: arXiv cs.NE 2021
Year: 2021

In this paper, we study the size and width of autoencoders consisting of Boolean threshold functions, where an autoencoder is a layered neural network whose structure can be viewed as consisting of an encoder, which compresses an input vector to a lower dimensional vector, and a decoder which transforms the low-dimensional vector back to the original input vector exactly (or approximately). We focus on the decoder part, and show that $Ω(\sqrt{Dn/d})$ and $O(\sqrt{Dn})$ nodes are required to transform $n$ vectors in $d$-dimensional binary space to $D$-dimensional binary space. We also show that the width can be reduced if we allow small errors, where the error is defined as the average of the Hamming distance between each vector input to the encoder part and the resulting vector output by the decoder.
ACGNet: Action Complement Graph Network for Weakly-supervised Temporal Action Localization
Venue: arXiv cs.CV 2021
Year: 2021

Weakly-supervised temporal action localization (WTAL) in untrimmed videos has emerged as a practical but challenging task since only video-level labels are available. Existing approaches typically leverage off-the-shelf segment-level features, which suffer from spatial incompleteness and temporal incoherence, thus limiting their performance. In this paper, we tackle this problem from a new perspective by enhancing segment-level representations with a simple yet effective graph convolutional network, namely action complement graph network (ACGNet). It facilitates the current video segment to perceive spatial-temporal dependencies from others that potentially convey complementary clues, implicitly mitigating the negative effects caused by the two issues above. By this means, the segment-level features are more discriminative and robust to spatial-temporal variations, contributing to higher localization accuracies. More importantly, the proposed ACGNet works as a universal module that can be flexibly plugged into different WTAL frameworks, while maintaining the end-to-end training fashion. Extensive experiments are conducted on the THUMOS'14 and ActivityNet1.2 benchmarks, where the state-of-the-art results clearly demonstrate the superiority of the proposed approach.
Lower Bounds for Sparse Oblivious Subspace Embeddings
Venue: arXiv cs.DS 2021
Year: 2021

An oblivious subspace embedding (OSE), characterized by parameters $m,n,d,ε,δ$, is a random matrix $Π\in \mathbb{R}^{m\times n}$ such that for any $d$-dimensional subspace $T\subseteq \mathbb{R}^n$, $\Pr_Π[\forall x\in T, (1-ε)\|x\|_2 \leq \|Πx\|_2\leq (1+ε)\|x\|_2] \geq 1-δ$. For $ε$ and $δ$ at most a small constant, we show that any OSE with one nonzero entry in each column must satisfy that $m = Ω(d^2/(ε^2δ))$, establishing the optimality of the classical Count-Sketch matrix. When an OSE has $1/(9ε)$ nonzero entries in each column, we show it must hold that $m = Ω(ε^{O(δ)} d^2)$, improving on the previous $Ω(ε^2 d^2)$ lower bound due to Nelson and Nguyen (ICALP 2014).
Mapping industrial poultry operations at scale with deep learning and aerial imagery
Venue: arXiv cs.CV 2021
Year: 2021

Concentrated Animal Feeding Operations (CAFOs) pose serious risks to air, water, and public health, but have proven to be challenging to regulate. The U.S. Government Accountability Office notes that a basic challenge is the lack of comprehensive location information on CAFOs. We use the USDA's National Agricultural Imagery Program (NAIP) 1m/pixel aerial imagery to detect poultry CAFOs across the continental United States. We train convolutional neural network (CNN) models to identify individual poultry barns and apply the best performing model to over 42 TB of imagery to create the first national, open-source dataset of poultry CAFOs. We validate the model predictions against held-out validation set on poultry CAFO facility locations from 10 hand-labeled counties in California and demonstrate that this approach has significant potential to fill gaps in environmental monitoring.
Robust Recommendation with Implicit Feedback via Eliminating the Effects of Unexpected Behaviors
Venue: arXiv cs.AI 2021
Year: 2021

In the implicit feedback recommendation, incorporating short-term preference into recommender systems has attracted increasing attention in recent years. However, unexpected behaviors in historical interactions like clicking some items by accident don't well reflect users' inherent preferences. Existing studies fail to model the effects of unexpected behaviors, thus achieve inferior recommendation performance. In this paper, we propose a Multi-Preferences Model (MPM) to eliminate the effects of unexpected behaviors. MPM first extracts the users' instant preferences from their recent historical interactions by a fine-grained preference module. Then an unexpected-behaviors detector is trained to judge whether these instant preferences are biased by unexpected behaviors. We also integrate user's general preference in MPM. Finally, an output module is performed to eliminate the effects of unexpected behaviors and integrates all the information to make a final recommendation. We conduct extensive experiments on two datasets of a movie and an e-retailing, demonstrating significant improvements in our model over the state-of-the-art methods. The experimental results show that MPM gets a massive improvement in HR@10 and NDCG@10, which relatively increased by 3.643% and 4.107% compare with AttRec model on average. We publish our code at https://github.com/chenjie04/MPM/.
Distributed Machine Learning and the Semblance of Trust
Venue: arXiv cs.LG 2021
Year: 2021

The utilisation of large and diverse datasets for machine learning (ML) at scale is required to promote scientific insight into many meaningful problems. However, due to data governance regulations such as GDPR as well as ethical concerns, the aggregation of personal and sensitive data is problematic, which prompted the development of alternative strategies such as distributed ML (DML). Techniques such as Federated Learning (FL) allow the data owner to maintain data governance and perform model training locally without having to share their data. FL and related techniques are often described as privacy-preserving. We explain why this term is not appropriate and outline the risks associated with over-reliance on protocols that were not designed with formal definitions of privacy in mind. We further provide recommendations and examples on how such algorithms can be augmented to provide guarantees of governance, security, privacy and verifiability for a general ML audience without prior exposure to formal privacy techniques.
Can We Use Neural Regularization to Solve Depth Super-Resolution?
Venue: arXiv cs.CV 2021
Year: 2021

Depth maps captured with commodity sensors often require super-resolution to be used in applications. In this work we study a super-resolution approach based on a variational problem statement with Tikhonov regularization where the regularizer is parametrized with a deep neural network. This approach was previously applied successfully in photoacoustic tomography. We experimentally show that its application to depth map super-resolution is difficult, and provide suggestions about the reasons for that.
Before and after GDPR: tracking in mobile apps
Venue: arXiv cs.CY 2021
Year: 2021

Third-party tracking, the collection and sharing of behavioural data about individuals, is a significant and ubiquitous privacy threat in mobile apps. The EU General Data Protection Regulation (GDPR) was introduced in 2018 to protect personal data better, but there exists, thus far, limited empirical evidence about its efficacy. This paper studies tracking in nearly two million Android apps from before and after the introduction of the GDPR. Our analysis suggests that there has been limited change in the presence of third-party tracking in apps, and that the concentration of tracking capabilities among a few large gatekeeper companies persists. However, change might be imminent.