OpenRaiser/Intern-Atlas huggingface dataset:
Exponentially Improved Dimensionality Reduction for $\ell_1$: Subspace Embeddings and Independence Testing
Venue: arXiv cs.DS 2021
Year: 2021

Despite many applications, dimensionality reduction in the $\ell_1$-norm is much less understood than in the Euclidean norm. We give two new oblivious dimensionality reduction techniques for the $\ell_1$-norm which improve exponentially over prior ones: 1. We design a distribution over random matrices $S \in \mathbb{R}^{r \times n}$, where $r = 2^{\tilde O(d/(\varepsilon δ))}$, such that given any matrix $A \in \mathbb{R}^{n \times d}$, with probability at least $1-δ$, simultaneously for all $x$, $\|SAx\|_1 = (1 \pm \varepsilon)\|Ax\|_1$. Note that $S$ is linear, does not depend on $A$, and maps $\ell_1$ into $\ell_1$. Our distribution provides an exponential improvement on the previous best known map of Wang and Woodruff (SODA, 2019), which required $r = 2^{2^{Ω(d)}}$, even for constant $\varepsilon$ and $δ$. Our bound is optimal, up to a polynomial factor in the exponent, given a known $2^{\sqrt d}$ lower bound for constant $\varepsilon$ and $δ$. 2. We design a distribution over matrices $S \in \mathbb{R}^{k \times n}$, where $k = 2^{O(q^2)}(\varepsilon^{-1} q \log d)^{O(q)}$, such that given any $q$-mode tensor $A \in (\mathbb{R}^{d})^{\otimes q}$, one can estimate the entrywise $\ell_1$-norm $\|A\|_1$ from $S(A)$. Moreover, $S = S^1 \otimes S^2 \otimes \cdots \otimes S^q$ and so given vectors $u_1, \ldots, u_q \in \mathbb{R}^d$, one can compute $S(u_1 \otimes u_2 \otimes \cdots \otimes u_q)$ in time $2^{O(q^2)}(\varepsilon^{-1} q \log d)^{O(q)}$, which is much faster than the $d^q$ time required to form $u_1 \otimes u_2 \otimes \cdots \otimes u_q$. Our linear map gives a streaming algorithm for independence testing using space $2^{O(q^2)}(\varepsilon^{-1} q \log d)^{O(q)}$, improving the previous doubly exponential $(\varepsilon^{-1} \log d)^{q^{O(q)}}$ space bound of Braverman and Ostrovsky (STOC, 2010).
3D Placement and Orientation of mmWave-based UAVs for Guaranteed LoS Coverage
Venue: arXiv cs.IT 2021
Year: 2021

Unmanned aerial vehicles (UAVs), as aerial base stations, are a promising solution for providing wireless communications, thanks to their high flexibility and autonomy. Moreover, emerging services, such as extended reality, require high-capacity communications. To achieve this, millimeter wave (mmWave), and recently, terahertz bands have been considered for UAV communications. However, communication at these high frequencies requires a line-of-sight (LoS) to the terminals, which may be located in 3D space and may have extremely limited direct-line-of-view (LoV) due to blocking objects, like buildings and trees. In this paper, we investigate the problem of determining 3D placement and orientation of UAVs such that users have guaranteed LoS coverage by at least one UAV and the signal-to-noise ratio (SNR) between the UAV-user pairs are maximized. We formulate the problem as an integer linear programming(ILP) problem and prove its NP-hardness. Next, we propose a low-complexity geometry-based greedy algorithm to solve the problem efficiently. Our simulation results show that the proposed algorithm (almost) always guarantees LoS coverage to all users in all considered simulation settings.
A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation
Venue: arXiv cs.IR 2021
Year: 2021

Influenced by the great success of deep learning in computer vision and language understanding, research in recommendation has shifted to inventing new recommender models based on neural networks. In recent years, we have witnessed significant progress in developing neural recommender models, which generalize and surpass traditional recommender models owing to the strong representation power of neural networks. In this survey paper, we conduct a systematic review on neural recommender models from the perspective of recommendation modeling with the accuracy goal, aiming to summarize this field to facilitate researchers and practitioners working on recommender systems. Specifically, based on the data usage during recommendation modeling, we divide the work into collaborative filtering and information-rich recommendation: 1) collaborative filtering, which leverages the key source of user-item interaction data; 2) content enriched recommendation, which additionally utilizes the side information associated with users and items, like user profile and item knowledge graph; and 3) temporal/sequential recommendation, which accounts for the contextual information associated with an interaction, such as time, location, and the past interactions. After reviewing representative work for each type, we finally discuss some promising directions in this field.
Toward Blockchain for Edge-of-Things: A New Paradigm, Opportunities, and Future Directions
Venue: arXiv cs.CR 2021
Year: 2021

Blockchain is gaining momentum as a promising technology for many application domains, one of them being the Edge-of- Things (EoT) that is enabled by the integration of edge computing and the Internet-of-Things (IoT). Particularly, the amalgamation of blockchain and EoT leads to a new paradigm, called blockchain enabled EoT (BEoT) that is crucial for enabling future low-latency and high-security services and applications. This article envisions a novel BEoT architecture for supporting industrial applications under the management of blockchain at the network edge in a wide range of IoT use cases such as smart home, smart healthcare, smart grid, and smart transportation. The potentials of BEoT in providing security services are also explored, including access authentication, data privacy preservation, attack detection, and trust management. Finally, we point out some key research challenges and future directions in this emerging area.
Three-stream network for enriched Action Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Understanding accurate information on human behaviours is one of the most important tasks in machine intelligence. Human Activity Recognition that aims to understand human activities from a video is a challenging task due to various problems including background, camera motion and dataset variations. This paper proposes two CNN based architectures with three streams which allow the model to exploit the dataset under different settings. The three pathways are differentiated in frame rates. The single pathway, operates at a single frame rate captures spatial information, the slow pathway operates at low frame rates captures the spatial information and the fast pathway operates at high frame rates that capture fine temporal information. Post CNN encoders, we add bidirectional LSTM and attention heads respectively to capture the context and temporal features. By experimenting with various algorithms on UCF-101, Kinetics-600 and AVA dataset, we observe that the proposed models achieve state-of-art performance for human action recognition task.
Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework
Venue: arXiv cs.SD 2021
Year: 2021

The field of automatic music composition has seen great progress in the last few years, much of which can be attributed to advances in deep neural networks. There are numerous studies that present different strategies for generating sheet music from scratch. The inclusion of high-level musical characteristics (e.g., perceived emotional qualities), however, as conditions for controlling the generation output remains a challenge. In this paper, we present a novel approach for calculating the valence (the positivity or negativity of the perceived emotion) of a chord progression within a lead sheet, using pre-defined mood tags proposed by music experts. Based on this approach, we propose a novel strategy for conditional lead sheet generation that allows us to steer the music generation in terms of valence, phrasing, and time signature. Our approach is similar to a Neural Machine Translation (NMT) problem, as we include high-level conditions in the encoder part of the sequence-to-sequence architectures used (i.e., long-short term memory networks, and a Transformer network). We conducted experiments to thoroughly analyze these two architectures. The results show that the proposed strategy is able to generate lead sheets in a controllable manner, resulting in distributions of musical attributes similar to those of the training dataset. We also verified through a subjective listening test that our approach is effective in controlling the valence of a generated chord progression.
Property Inference Attacks on Convolutional Neural Networks: Influence and Implications of Target Model's Complexity
Venue: arXiv cs.CR 2021
Year: 2021

Machine learning models' goal is to make correct predictions for specific tasks by learning important properties and patterns from data. By doing so, there is a chance that the model learns properties that are unrelated to its primary task. Property Inference Attacks exploit this and aim to infer from a given model (\ie the target model) properties about the training dataset seemingly unrelated to the model's primary goal. If the training data is sensitive, such an attack could lead to privacy leakage. This paper investigates the influence of the target model's complexity on the accuracy of this type of attack, focusing on convolutional neural network classifiers. We perform attacks on models that are trained on facial images to predict whether someone's mouth is open. Our attacks' goal is to infer whether the training dataset is balanced gender-wise. Our findings reveal that the risk of a privacy breach is present independently of the target model's complexity: for all studied architectures, the attack's accuracy is clearly over the baseline. We discuss the implication of the property inference on personal data in the light of Data Protection Regulations and Guidelines.
Interdisciplinary researchers attain better performance in funding
Venue: arXiv cs.DL 2021
Year: 2021

Interdisciplinary research is fundamental when it comes to tackling complex problems in our highly interlinked world, and is on the rise globally. Yet, it is unclear why--in an increasingly competitive academic environment--one should pursue an interdisciplinary career given its recent negative press. Several studies have indeed shown that interdisciplinary research often achieves lower impact compared to more specialized work, and is less likely to attract funding. We seek to reconcile such evidence by analyzing a dataset of 44,419 research grants awarded between 2006 and 2018 from the seven national research councils in the UK. We compared the research performance of researchers with an interdisciplinary funding track record with those who have a specialized profile. We found that the former dominates the network of academic collaborations, both in terms of centrality and knowledge brokerage; but such a competitive advantage does not immediately translate into impact. Indeed, by means of a matched pair experimental design, we found that researchers who transcend between disciplines on average achieve lower impacts in their publications than the subject specialists in the short run, but eventually outperform them in funding performance, both in terms of volume and value. Our results suggest that launching an interdisciplinary career may require more time and persistence to overcome extra challenges, but can pave the way for a more successful endeavour.
Combinatorial Proofs and Decomposition Theorems for First-order Logic
Venue: arXiv cs.LO 2021
Year: 2021

We uncover a close relationship between combinatorial and syntactic proofs for first-order logic (without equality). Whereas syntactic proofs are formalized in a deductive proof system based on inference rules, a combinatorial proof is a syntax-free presentation of a proof that is independent from any set of inference rules. We show that the two proof representations are related via a deep inference decomposition theorem that establishes a new kind of normal form for syntactic proofs. This yields (a) a simple proof of soundness and completeness for first-order combinatorial proofs, and (b) a full completeness theorem: every combinatorial proof is the image of a syntactic proof.
Watershed of Artificial Intelligence: Human Intelligence, Machine Intelligence, and Biological Intelligence
Venue: arXiv cs.AI 2021
Year: 2021

This article reviews the "Once learning" mechanism that was proposed 23 years ago and the subsequent successes of "One-shot learning" in image classification and "You Only Look Once - YOLO" in objective detection. Analyzing the current development of Artificial Intelligence (AI), the proposal is that AI should be clearly divided into the following categories: Artificial Human Intelligence (AHI), Artificial Machine Intelligence (AMI), and Artificial Biological Intelligence (ABI), which will also be the main directions of theory and application development for AI. As a watershed for the branches of AI, some classification standards and methods are discussed: 1) Human-oriented, machine-oriented, and biological-oriented AI R&D; 2) Information input processed by Dimensionality-up or Dimensionality-reduction; 3) The use of one/few or large samples for knowledge learning.
Rapid feasibility assessment of components formed through hot stamping: A deep learning approach
Venue: arXiv cs.CE 2021
Year: 2021

The novel non-isothermal Hot Forming and cold die Quenching (HFQ) process can enable the cost-effective production of complex shaped, high strength aluminium alloy panel components. However, the unfamiliarity of designing for the new process prevents its widescale adoption in industrial settings. Recent research efforts focus on the development of advanced material models for finite element simulations, used to assess the feasibility of new component designs for the HFQ process. However, FE simulations take place late in design processes, require forming process expertise and are unsuitable for early-stage design explorations. To address these limitations, this study presents a novel application of a Convolutional Neural Network (CNN) based surrogate as a means of rapid manufacturing feasibility assessment for components to be formed using the HFQ process. A diverse dataset containing variations in component geometry, blank shapes, and processing parameters, together with corresponding physical fields is generated and used to train the model. The results show that near indistinguishable full field predictions are obtained in real time from the model when compared with HFQ simulations. This technique provides an invaluable tool to aid component design and decision making at the onset of a design process for complex-shaped components formed under HFQ conditions.
A Graph Federated Architecture with Privacy Preserving Learning
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning involves a central processor that works with multiple agents to find a global model. The process consists of repeatedly exchanging estimates, which results in the diffusion of information pertaining to the local private data. Such a scheme can be inconvenient when dealing with sensitive data, and therefore, there is a need for the privatization of the algorithms. Furthermore, the current architecture of a server connected to multiple clients is highly sensitive to communication failures and computational overloads at the server. Thus in this work, we develop a private multi-server federated learning scheme, which we call graph federated learning. We use cryptographic and differential privacy concepts to privatize the federated learning algorithm that we extend to the graph structure. We study the effect of privatization on the performance of the learning algorithm for general private schemes that can be modeled as additive noise. We show under convexity and Lipschitz conditions, that the privatized process matches the performance of the non-private algorithm, even when we increase the noise variance.
Influence Based Defense Against Data Poisoning Attacks in Online Learning
Venue: arXiv cs.LG 2021
Year: 2021

Data poisoning is a type of adversarial attack on training data where an attacker manipulates a fraction of data to degrade the performance of machine learning model. Therefore, applications that rely on external data-sources for training data are at a significantly higher risk. There are several known defensive mechanisms that can help in mitigating the threat from such attacks. For example, data sanitization is a popular defensive mechanism wherein the learner rejects those data points that are sufficiently far from the set of training instances. Prior work on data poisoning defense primarily focused on offline setting, wherein all the data is assumed to be available for analysis. Defensive measures for online learning, where data points arrive sequentially, have not garnered similar interest. In this work, we propose a defense mechanism to minimize the degradation caused by the poisoned training data on a learner's model in an online setup. Our proposed method utilizes an influence function which is a classic technique in robust statistics. Further, we supplement it with the existing data sanitization methods for filtering out some of the poisoned data points. We study the effectiveness of our defense mechanism on multiple datasets and across multiple attack strategies against an online learner.
Throughput Maximization for IRS-Assisted Wireless Powered Hybrid NOMA and TDMA
Venue: arXiv cs.IT 2021
Year: 2021

The high reflect beamforming gain of the intelligent reflecting surface (IRS) makes it appealing not only for wireless information transmission but also for wireless power transfer. In this letter, we consider an IRS-assisted wireless powered communication network, where a base station (BS) transmits energy to multiple users grouped into multiple clusters in the downlink, and the clustered users transmit information to the BS in the manner of hybrid non-orthogonal multiple access and time division multiple access in the uplink. We investigate optimizing the reflect beamforming of the IRS and the time allocation among the BS's power transfer and different user clusters' information transmission to maximize the throughput of the network, and we propose an efficient algorithm based on the block coordinate ascent, semidefinite relaxation, and sequential rank-one constraint relaxation techniques to solve the resultant problem. Simulation results have verified the effectiveness of the proposed algorithm and have shown the impact of user clustering setup on the throughput performance of the network.
A Dual Process Model for Optimizing Cross Entropy in Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Minimizing cross-entropy is a widely used method for training artificial neural networks. Many training procedures based on backpropagation use cross-entropy directly as their loss function. Instead, this theoretical essay investigates a dual process model with two processes, in which one process minimizes the Kullback-Leibler divergence while its dual counterpart minimizes the Shannon entropy. Postulating that learning consists of two dual processes complementing each other, the model defines an equilibrium state for both processes in which the loss function assumes its minimum. An advantage of the proposed model is that it allows deriving the optimal learning rate and momentum weight to update network weights for backpropagation. Furthermore, the model introduces the golden ratio and complex numbers as important new concepts in machine learning.
Smells and Refactorings for Microservices Security: A Multivocal Literature Review
Venue: arXiv cs.SE 2021
Year: 2021

Context: Securing microservice-based applications is crucial, as many IT companies are delivering their businesses through microservices. If security smells affect microservice-based applications, they can possibly suffer from security leaks and need to be refactored to mitigate the effects of security smells therein. Objective: As the currently available knowledge on securing microservices is scattered across different pieces of white and grey literature, our objective here is to distill well-known smells for securing microservices, together with the refactorings enabling to mitigate the effects of such smells. Method: To capture the state of the art and practice in securing microservices, we conducted a multivocal review of the existing white and grey literature on the topic. We systematically analyzed 58 studies published from 2014 until the end of 2020. Results: Ten bad smells for securing microservices are identified, which we organized in a taxonomy, associating each smell with the security properties it may violate and the refactorings enabling to mitigate its effects. Conclusions: The security smells and the corresponding refactorings have pragmatic value for practitioners, who can exploit them in their daily work on securing microservices. They also serve as a starting point for researchers wishing to establish new research directions on securing microservices.
Inductive Program Synthesis over Noisy Datasets using Abstraction Refinement Based Optimization
Venue: arXiv cs.PL 2021
Year: 2021

We present a new synthesis algorithm to solve program synthesis over noisy datasets, i.e., data that may contain incorrect/corrupted input-output examples. Our algorithm uses an abstraction refinement based optimization process to synthesize programs which optimize the tradeoff between the loss over the noisy dataset and the complexity of the synthesized program. The algorithm uses abstractions to divide the search space of programs into subspaces by computing an abstract value that represents outputs for all programs in a subspace. The abstract value allows our algorithm to compute, for each subspace, a sound approximate lower bound of the loss over all programs in the subspace. It iteratively refines these abstractions to further subdivide the space into smaller subspaces, prune subspaces that do not contain an optimal program, and eventually synthesize an optimal program. We implemented this algorithm in a tool called Rose. We compare Rose to a current state-of-the-art noisy program synthesis system using the SyGuS 2018 benchmark suite. Our evaluation demonstrates that Rose significantly outperforms this previous system: on two noisy benchmark program synthesis problems sets drawn from the SyGus 2018 benchmark suite, Rose delivers speedups of up to 1587 and 81.7, with median speedups of 20.5 and 81.7. Rose also terminates on 20 (out of 54) and 4 (out of 11) more benchmark problems than the previous system. Both Rose and the previous system synthesize programs that are optimal over the provided noisy data sets. For the majority of the problems in the benchmark sets ($272$ out of $286$), the synthesized programs also produce correct outputs for all inputs in the original (unseen) noise-free data set. These results highlight the benefits that Rose can deliver for effective noisy program synthesis.
MVS2D: Efficient Multi-view Stereo via Attention-Driven 2D Convolutions
Venue: arXiv cs.CV 2021
Year: 2021

Deep learning has made significant impacts on multi-view stereo systems. State-of-the-art approaches typically involve building a cost volume, followed by multiple 3D convolution operations to recover the input image's pixel-wise depth. While such end-to-end learning of plane-sweeping stereo advances public benchmarks' accuracy, they are typically very slow to compute. We present \ouralg, a highly efficient multi-view stereo algorithm that seamlessly integrates multi-view constraints into single-view networks via an attention mechanism. Since \ouralg only builds on 2D convolutions, it is at least $2\times$ faster than all the notable counterparts. Moreover, our algorithm produces precise depth estimations and 3D reconstructions, achieving state-of-the-art results on challenging benchmarks ScanNet, SUN3D, RGBD, and the classical DTU dataset. our algorithm also out-performs all other algorithms in the setting of inexact camera poses. Our code is released at \url{https://github.com/zhenpeiyang/MVS2D}