OpenRaiser/Intern-Atlas huggingface dataset:
Deep Technology Tracing for High-tech Companies
Venue: arXiv cs.LG 2020
Year: 2020

Technological change and innovation are vitally important, especially for high-tech companies. However, factors influencing their future research and development (R&D) trends are both complicated and various, leading it a quite difficult task to make technology tracing for high-tech companies. To this end, in this paper, we develop a novel data-driven solution, i.e., Deep Technology Forecasting (DTF) framework, to automatically find the most possible technology directions customized to each high-tech company. Specially, DTF consists of three components: Potential Competitor Recognition (PCR), Collaborative Technology Recognition (CTR), and Deep Technology Tracing (DTT) neural network. For one thing, PCR and CTR aim to capture competitive relations among enterprises and collaborative relations among technologies, respectively. For another, DTT is designed for modeling dynamic interactions between companies and technologies with the above relations involved. Finally, we evaluate our DTF framework on real-world patent data, and the experimental results clearly prove that DTF can precisely help to prospect future technology emphasis of companies by exploiting hybrid factors.
Knowledge Graphs for Innovation Ecosystems
Venue: arXiv cs.IR 2020
Year: 2020

Innovation ecosystems can be naturally described as a collection of networked entities, such as experts, institutions, projects, technologies and products. Representing in a machine-readable form these entities and their relations is not entirely attainable, due to the existence of abstract concepts such as knowledge and due to the confidential, non-public nature of this information, but even its partial depiction is of strong interest. The representation of innovation ecosystems incarnated as knowledge graphs would enable the generation of reports with new insights, the execution of advanced data analysis tasks. An ontology to capture the essential entities and relations is presented, as well as the description of data sources, which can be used to populate innovation knowledge graphs. Finally, the application case of the Universidad Politecnica de Madrid is presented, as well as an insight of future applications.
A Study of the Tasks and Models in Machine Reading Comprehension
Venue: arXiv cs.CL 2020
Year: 2020

To provide a survey on the existing tasks and models in Machine Reading Comprehension (MRC), this report reviews: 1) the dataset collection and performance evaluation of some representative simple-reasoning and complex-reasoning MRC tasks; 2) the architecture designs, attention mechanisms, and performance-boosting approaches for developing neural-network-based MRC models; 3) some recently proposed transfer learning approaches to incorporating text-style knowledge contained in external corpora into the neural networks of MRC models; 4) some recently proposed knowledge base encoding approaches to incorporating graph-style knowledge contained in external knowledge bases into the neural networks of MRC models. Besides, according to what has been achieved and what are still deficient, this report also proposes some open problems for the future research.
Discovering the IPv6 Network Periphery
Venue: arXiv cs.NI 2020
Year: 2020

We consider the problem of discovering the IPv6 network periphery, i.e., the last hop router connecting endhosts in the IPv6 Internet. Finding the IPv6 periphery using active probing is challenging due to the IPv6 address space size, wide variety of provider addressing and subnetting schemes, and incomplete topology traces. As such, existing topology mapping systems can miss the large footprint of the IPv6 periphery, disadvantaging applications ranging from IPv6 census studies to geolocation and network resilience. We introduce "edgy," an approach to explicitly discover the IPv6 network periphery, and use it to find >~64M IPv6 periphery router addresses and >~87M links to these last hops -- several orders of magnitude more than in currently available IPv6 topologies. Further, only 0.2% of edgy's discovered addresses are known to existing IPv6 hitlists.
Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
Venue: arXiv cs.CV 2020
Year: 2020

Few-shot classification aims to recognize novel categories with only few labeled images in each class. Existing metric-based few-shot classification algorithms predict categories by comparing the feature embeddings of query images with those from a few labeled images (support examples) using a learned metric function. While promising performance has been demonstrated, these methods often fail to generalize to unseen domains due to large discrepancy of the feature distribution across domains. In this work, we address the problem of few-shot classification under domain shifts for metric-based methods. Our core idea is to use feature-wise transformation layers for augmenting the image features using affine transforms to simulate various feature distributions under different domains in the training stage. To capture variations of the feature distributions under different domains, we further apply a learning-to-learn approach to search for the hyper-parameters of the feature-wise transformation layers. We conduct extensive experiments and ablation studies under the domain generalization setting using five few-shot classification datasets: mini-ImageNet, CUB, Cars, Places, and Plantae. Experimental results demonstrate that the proposed feature-wise transformation layer is applicable to various metric-based models, and provides consistent improvements on the few-shot classification performance under domain shift.
Compressive MRI quantification using convex spatiotemporal priors and deep auto-encoders
Venue: arXiv cs.CV 2020
Year: 2020

We propose a dictionary-matching-free pipeline for multi-parametric quantitative MRI image computing. Our approach has two stages based on compressed sensing reconstruction and deep learned quantitative inference. The reconstruction phase is convex and incorporates efficient spatiotemporal regularisations within an accelerated iterative shrinkage algorithm. This minimises the under-sampling (aliasing) artefacts from aggressively short scan times. The learned quantitative inference phase is purely trained on physical simulations (Bloch equations) that are flexible for producing rich training samples. We propose a deep and compact auto-encoder network with residual blocks in order to embed Bloch manifold projections through multiscale piecewise affine approximations, and to replace the nonscalable dictionary-matching baseline. Tested on a number of datasets we demonstrate effectiveness of the proposed scheme for recovering accurate and consistent quantitative information from novel and aggressively subsampled 2D/3D quantitative MRI acquisition protocols.
Reducing Non-Normative Text Generation from Language Models
Venue: arXiv cs.CL 2020
Year: 2020

Large-scale, transformer-based language models such as GPT-2 are pretrained on diverse corpora scraped from the internet. Consequently, they are prone to generating non-normative text (i.e. in violation of social norms). We introduce a technique for fine-tuning GPT-2, using a policy gradient reinforcement learning technique and a normative text classifier to produce reward and punishment values. We evaluate our technique on five data sets using automated and human participant experiments. The normative text classifier is 81-90% accurate when compared to gold-standard human judgments of normative and non-normative generated text. Our normative fine-tuning technique is able to reduce non-normative text by 27-61%, depending on the data set.
Semi-Autoregressive Training Improves Mask-Predict Decoding
Venue: arXiv cs.CL 2020
Year: 2020

The recently proposed mask-predict decoding algorithm has narrowed the performance gap between semi-autoregressive machine translation models and the traditional left-to-right approach. We introduce a new training method for conditional masked language models, SMART, which mimics the semi-autoregressive behavior of mask-predict, producing training examples that contain model predictions as part of their inputs. Models trained with SMART produce higher-quality translations when using mask-predict decoding, effectively closing the remaining performance gap with fully autoregressive models.
Inexpensive and Portable System for Dexterous High-Density Myoelectric Control of Multiarticulate Prostheses
Venue: arXiv cs.RO 2020
Year: 2020

Multiarticulate bionic arms are now capable of mimicking the endogenous movements of the human hand. 3D-printing has reduced the cost of prosthetic hands themselves, but there is currently no low-cost alternative to dexterous electromyographic (EMG) control systems. To address this need, we developed an inexpensive (~$675) and portable EMG control system by integrating low-cost microcontrollers with an EMG acquisition device. We validated signal acquisition by comparing the signal-to-noise ratio (SNR) of our system with that of a high-end research-grade system. We also demonstrate the ability to use the low-cost control system for proportional and independent control of various prosthetic hands in real-time. We found that the SNR of the low-cost control system was statistically no worse than 44% of the SNR of a research-grade control system. The RMSEs of predicted hand movements (from a modified Kalman filter) were typically a few percent better than, and not more than 6% worse than, RMSEs of a research-grade system for up to six degrees of freedom when only relatively few (six) EMG electrodes were used. However, RMSEs were generally higher than RMSEs of research-grade systems that utilize considerably more (32) EMG electrodes, guiding future work towards increasing electrode count. Successful instantiation of this low-cost control system constitutes an important step towards the commercialization and wide-spread availability of dexterous bionic hands.
Reliable and Energy Efficient MLC STT-RAM Buffer for CNN Accelerators
Venue: arXiv cs.LG 2020
Year: 2020

We propose a lightweight scheme where the formation of a data block is changed in such a way that it can tolerate soft errors significantly better than the baseline. The key insight behind our work is that CNN weights are normalized between -1 and 1 after each convolutional layer, and this leaves one bit unused in half-precision floating-point representation. By taking advantage of the unused bit, we create a backup for the most significant bit to protect it against the soft errors. Also, considering the fact that in MLC STT-RAMs the cost of memory operations (read and write), and reliability of a cell are content-dependent (some patterns take larger current and longer time, while they are more susceptible to soft error), we rearrange the data block to minimize the number of costly bit patterns. Combining these two techniques provides the same level of accuracy compared to an error-free baseline while improving the read and write energy by 9% and 6%, respectively.
Bilaterally Mirrored Movements Improve the Accuracy and Precision of Training Data for Supervised Learning of Neural or Myoelectric Prosthetic Control
Venue: arXiv cs.RO 2020
Year: 2020

Intuitive control of prostheses relies on training algorithms to correlate biological recordings to motor intent. The quality of the training dataset is critical to run-time performance, but it is difficult to label hand kinematics accurately after the hand has been amputated. We quantified the accuracy and precision of labeling hand kinematics for two different approaches: 1) assuming a participant is perfectly mimicking predetermined motions of a prosthesis (mimicked training), and 2) assuming a participant is perfectly mirroring their contralateral hand during identical bilateral movements (mirrored training). We compared these approaches in non-amputee individuals, using an infrared camera to track eight different joint angles of the hands in real-time. Aggregate data revealed that mimicked training does not account for biomechanical coupling or temporal changes in hand posture. Mirrored training was significantly more accurate and precise at labeling hand kinematics. However, when training a modified Kalman filter to estimate motor intent, the mimicked and mirrored training approaches were not significantly different. The results suggest that the mirrored training approach creates a more faithful but more complex dataset. Advanced algorithms, more capable of learning the complex mirrored training dataset, may yield better run-time prosthetic control.
Universal Data Anomaly Detection via Inverse Generative Adversary Network
Venue: arXiv cs.LG 2020
Year: 2020

The problem of detecting data anomaly is considered. Under the null hypothesis that models anomaly-free data, measurements are assumed to be from an unknown distribution with some authenticated historical samples. Under the composite alternative hypothesis, measurements are from an unknown distribution positive distance away from the distribution under the null hypothesis. No training data are available for the distribution of anomaly data. A semi-supervised deep learning technique based on an inverse generative adversary network is proposed.
Wirelessly-powered Sensor Networks Power Allocation for Channel Estimation and Energy Beamforming
Venue: arXiv cs.NI 2020
Year: 2020

Wirelessly-powered sensor networks (WPSNs) are becoming increasingly important in different monitoring applications. We consider a WPSN where a multiple-antenna base station, which is dedicated for energy transmission, sends pilot signals to estimate the channel state information and consequently shapes the energy beams toward the sensor nodes. Given a fixed energy budget at the base station, in this paper, we investigate the novel problem of optimally allocating the power for the channel estimation and for the energy transmission. We formulate this non-convex optimization problem for general channel estimation and beamforming schemes that satisfy some qualification conditions. We provide a new solution approach and a performance analysis in terms of optimality and complexity. We also present a closed-form solution for the case where the channels are estimated based on a least square channel estimation and a maximum ratio transmit beamforming scheme. The analysis and simulations indicate a significant gain in terms of the network sensing rate, compared to the fixed power allocation, and the importance of improving the channel estimation efficiency.
Families of Multidimensional Arrays with Good Autocorrelation and Asymptotically Optimal Cross-correlation
Venue: arXiv cs.IT 2020
Year: 2020

We introduce a construction for families of 2n-dimensional arrays with asymptotically optimal pairwise cross-correlation. These arrays are constructed using a circulant array of n-dimensional Legendre arrays. We also introduce an application of these higher-dimensional arrays to high-capacity digital watermarking of images and video.
Intelligent Reflecting Surface-Assisted Multiple Access with User Pairing: NOMA or OMA?
Venue: arXiv cs.IT 2020
Year: 2020

The integration of intelligent reflecting surface (IRS) to multiple access networks is a cost-effective solution for boosting spectrum/energy efficiency and enlarging network coverage/connections. However, due to the new capability of IRS in reconfiguring the wireless propagation channels, it is fundamentally unknown which multiple access scheme is superior in the IRS-assisted wireless network. In this letter, we pursue a theoretical performance comparison between non-orthogonal multiple access (NOMA) and orthogonal multiple access (OMA) in the IRS-assisted downlink communication, for which the transmit power minimization problems are formulated under the discrete unit-modulus reflection constraint on each IRS element. We analyze the minimum transmit powers required by different multiple access schemes and compare them numerically, which turn out to not fully comply with the stereotyped superiority of NOMA over OMA in conventional systems without IRS. Moreover, to avoid the exponential complexity of the brute-force search for the optimal discrete IRS phase shifts, we propose a low-complexity solution to achieve near-optimal performance.
On the Performance of Metaheuristics: A Different Perspective
Venue: arXiv cs.NE 2020
Year: 2020

Nowadays, we are immersed in tens of newly-proposed evolutionary and swam-intelligence metaheuristics, which makes it very difficult to choose a proper one to be applied on a specific optimization problem at hand. On the other hand, most of these metaheuristics are nothing but slightly modified variants of the basic metaheuristics. For example, Differential Evolution (DE) or Shuffled Frog Leaping (SFL) are just Genetic Algorithms (GA) with a specialized operator or an extra local search, respectively. Therefore, what comes to the mind is whether the behavior of such newly-proposed metaheuristics can be investigated on the basis of studying the specifications and characteristics of their ancestors. In this paper, a comprehensive evaluation study on some basic metaheuristics i.e. Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC), Teaching-Learning-Based Optimization (TLBO), and Cuckoo Optimization algorithm (COA) is conducted, which give us a deeper insight into the performance of them so that we will be able to better estimate the performance and applicability of all other variations originated from them. A large number of experiments have been conducted on 20 different combinatorial optimization benchmark functions with different characteristics, and the results reveal to us some fundamental conclusions besides the following ranking order among these metaheuristics, {ABC, PSO, TLBO, GA, COA} i.e. ABC and COA are the best and the worst methods from the performance point of view, respectively. In addition, from the convergence perspective, PSO and ABC have significant better convergence for unimodal and multimodal functions, respectively, while GA and COA have premature convergence to local optima in many cases needing alternative mutation mechanisms to enhance diversification and global search.
Joint Geographical and Temporal Modeling based on Matrix Factorization for Point-of-Interest Recommendation
Venue: arXiv cs.IR 2020
Year: 2020

With the popularity of Location-based Social Networks, Point-of-Interest (POI) recommendation has become an important task, which learns the users' preferences and mobility patterns to recommend POIs. Previous studies show that incorporating contextual information such as geographical and temporal influences is necessary to improve POI recommendation by addressing the data sparsity problem. However, existing methods model the geographical influence based on the physical distance between POIs and users, while ignoring the temporal characteristics of such geographical influences. In this paper, we perform a study on the user mobility patterns where we find out that users' check-ins happen around several centers depending on their current temporal state. Next, we propose a spatio-temporal activity-centers algorithm to model users' behavior more accurately. Finally, we demonstrate the effectiveness of our proposed contextual model by incorporating it into the matrix factorization model under two different settings: i) static and ii) temporal. To show the effectiveness of our proposed method, which we refer to as STACP, we conduct experiments on two well-known real-world datasets acquired from Gowalla and Foursquare LBSNs. Experimental results show that the STACP model achieves a statistically significant performance improvement, compared to the state-of-the-art techniques. Also, we demonstrate the effectiveness of capturing geographical and temporal information for modeling users' activity centers and the importance of modeling them jointly.
Intelligent Reflecting Surface Assisted Secure Wireless Communications with Multiple-Transmit and Multiple-Receive Antennas
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we propose intelligent reflecting surfaces (IRS) assisted secure wireless communications with multi-input and multi-output antennas (IRS-MIMOME). The considered scenario is an access point (AP) equipped with multiple antennas communicates with a multi-antenna enabled legitimate user in the downlink at the present of an eavesdropper configured with multiple antennas. Particularly, the joint optimization of the transmit covariance matrix at the AP and the reflecting coefficients at the IRS to maximize the secrecy rate for the IRS-MIMOME system is investigated, with two different assumptions on the phase shifting capabilities at the IRS, i.e., the IRS has the continuous reflecting coefficients and the IRS has the discrete reflecting coefficients. For the former case, due to the non-convexity of the formulated problem, an alternating optimization (AO)-based algorithm is proposed, i.e., for given the reflecting coefficients at the IRS, the successive convex approximation (SCA)-based algorithm is used to solve the transmit covariance matrix optimization, while given the transmit covariance matrix at the AP, alternative optimization is used again in individually optimizing of each reflecting coefficient at the IRS with other fixed reflecting coefficients. For the individual reflecting coefficient optimization, the close-form or an interval of the optimal solution is provided. Then, the proposed algorithm is extended to the discrete reflecting coefficient model at the IRS. Finally, some numerical simulations have been done to demonstrate that the proposed algorithm outperforms other benchmark schemes.
Mechanism Design for Multi-Party Machine Learning
Venue: arXiv cs.MA 2020
Year: 2020

In a multi-party machine learning system, different parties cooperate on optimizing towards better models by sharing data in a privacy-preserving way. A major challenge in learning is the incentive issue. For example, if there is competition among the parties, one may strategically hide his data to prevent other parties from getting better models. In this paper, we study the problem through the lens of mechanism design and incorporate the features of multi-party learning in our setting. First, each agent's valuation has externalities that depend on others' types and actions. Second, each agent can only misreport a type lower than his true type, but not the other way round. We call this setting interdependent value with type-dependent action spaces. We provide the optimal truthful mechanism in the quasi-monotone utility setting. We also provide necessary and sufficient conditions for truthful mechanisms in the most general case. Finally, we show the existence of such mechanisms is highly affected by the market growth rate and provide empirical analysis.