OpenRaiser/Intern-Atlas huggingface dataset:
Characterizing Delay and Control Traffic of the Cellular MME with IoT Support
Venue: arXiv cs.NI 2020
Year: 2020

One of the main use cases for advanced cellular networks is represented by massive Internet-of-things (MIoT), i.e., an enormous number of IoT devices that transmit data toward the cellular network infrastructure. To make cellular MIoT a reality, data transfer and control procedures specifically designed for the support of IoT are needed. For this reason, 3GPP has introduced the Control Plane Cellular IoT optimization, which foresees a simplified bearer instantiation, with the Mobility Management Entity (MME) handling both control and data traffic. The performance of the MME has therefore become critical, and properly scaling its computational capability can determine the ability of the whole network to tackle MIoT effectively. In particular, considering virtualized networks and the need for an efficient allocation of computing resources, it is paramount to characterize the MME performance as the MIoT traffic load changes. We address this need by presenting compact, closed-form expressions linking the number of IoT sources with the rate at which bearers are requested, and such a rate with the delay incurred by the IoT data. We show that our analysis, supported by testbed experiments and verified through large-scale simulations, represents a valuable tool to make effective scaling decisions in virtualized cellular core networks.
NeuroBoun: An inquiry-based approach for exploring scientific literature -- a use case in neuroscience
Venue: arXiv cs.IR 2020
Year: 2020

Online scientific publications provide vast opportunities for researchers. Alas, the quantity and the rate of increase in the articles make the utilization of these resources very challenging. This work presents as inquiry-based approach to support the articulation of complex inter-related queries to gain insights regarding how these subjects have been studied in conjunction with one another as reported in the scientific literature. For this purpose we introduce inquiries that represent inter-related subqueries that are of interest to a researcher. The inquiries are expanded to better capture the intent of the inquirer, from which several queries are generated that represent various juxtapositions of the subjects in consideration. The sets of queries are used to search repositories to yield results that reveal quantitative and temporal relations among the subjects of the inquiry. A web-based tool, NeuroBoun, is developed as a proof of concept for medical publications found in PubMed. A use case related to the asymmetry of amygdala is presented to illustrate the potentials of the proposed approach.
Multi-lane Detection Using Instance Segmentation and Attentive Voting
Venue: arXiv cs.CV 2020
Year: 2020

Autonomous driving is becoming one of the leading industrial research areas. Therefore many automobile companies are coming up with semi to fully autonomous driving solutions. Among these solutions, lane detection is one of the vital driver-assist features that play a crucial role in the decision-making process of the autonomous vehicle. A variety of solutions have been proposed to detect lanes on the road, which ranges from using hand-crafted features to the state-of-the-art end-to-end trainable deep learning architectures. Most of these architectures are trained in a traffic constrained environment. In this paper, we propose a novel solution to multi-lane detection, which outperforms state of the art methods in terms of both accuracy and speed. To achieve this, we also offer a dataset with a more intuitive labeling scheme as compared to other benchmark datasets. Using our approach, we are able to obtain a lane segmentation accuracy of 99.87% running at 54.53 fps (average).
A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

In recent years, plenty of metrics have been proposed to identify networks that are free of gradient explosion and vanishing. However, due to the diversity of network components and complex serial-parallel hybrid connections in modern DNNs, the evaluation of existing metrics usually requires strong assumptions, complex statistical analysis, or has limited application fields, which constraints their spread in the community. In this paper, inspired by the Gradient Norm Equality and dynamical isometry, we first propose a novel metric called Block Dynamical Isometry, which measures the change of gradient norm in individual block. Because our Block Dynamical Isometry is norm-based, its evaluation needs weaker assumptions compared with the original dynamical isometry. To mitigate the challenging derivation, we propose a highly modularized statistical framework based on free probability. Our framework includes several key theorems to handle complex serial-parallel hybrid connections and a library to cover the diversity of network components. Besides, several sufficient prerequisites are provided. Powered by our metric and framework, we analyze extensive initialization, normalization, and network structures. We find that Gradient Norm Equality is a universal philosophy behind them. Then, we improve some existing methods based on our analysis, including an activation function selection strategy for initialization techniques, a new configuration for weight normalization, and a depth-aware way to derive coefficients in SeLU. Moreover, we propose a novel normalization technique named second moment normalization, which is theoretically 30% faster than batch normalization without accuracy loss. Last but not least, our conclusions and methods are evidenced by extensive experiments on multiple models over CIFAR10 and ImageNet.
A Feature Comparison of Modern Digital Forensic Imaging Software
Venue: arXiv cs.CY 2020
Year: 2020

Fundamental processes in digital forensic investigation, such as disk imaging, were developed when digital investigation was relatively young. As digital forensic processes and procedures matured, these fundamental tools, that are the pillars of the reset of the data processing and analysis phases of an investigation, largely stayed the same. This work is a study of modern digital forensic imaging software tools. Specifically, we will examine the feature sets of modern digital forensic imaging tools, as well as their development and release cycles to understand patterns of fundamental tool development. Based on this survey, we show the weakness in current digital investigation fundamental software development and maintenance over time. We also provide recommendations on how to improve fundamental tools.
Learning-Aided Deep Path Prediction for Sphere Decoding in Large MIMO Systems
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we propose a novel learning-aided sphere decoding (SD) scheme for large multiple-input--multiple-output systems, namely, deep path prediction-based sphere decoding (DPP-SD). In this scheme, we employ a neural network (NN) to predict the minimum metrics of the ``deep'' paths in sub-trees before commencing the tree search in SD. To reduce the complexity of the NN, we employ the input vector with a reduced dimension rather than using the original received signals and full channel matrix. The outputs of the NN, i.e., the predicted minimum path metrics, are exploited to determine the search order between the sub-trees, as well as to optimize the initial search radius, which may reduce the computational complexity of SD. For further complexity reduction, an early termination scheme based on the predicted minimum path metrics is also proposed. Our simulation results show that the proposed DPP-SD scheme provides a significant reduction in computational complexity compared with the conventional SD algorithm, despite achieving near-optimal performance.
The Demise of Single-Authored Publications in Computer Science: A Citation Network Analysis
Venue: arXiv cs.DL 2020
Year: 2020

In this study, I analyze the DBLP bibliographic database to study role of single author publications in the computer science literature between 1940 and 2019. I examine the demographics and reception by computing the population fraction, citation statistics, and PageRank scores of single author publications over the years. Both the population fraction and reception have been continuously declining since the 1940s. The overall decaying trend of single author publications is qualitatively consistent with those observed in other scientific disciplines, though the diminution is taking place several decades later than those in the natural sciences. Additionally, I analyze the scope and volume of single author publications, using page length and reference count as first-order approximations of the scope of publications. Although both metrics on average show positive correlations with citation count, single author papers show no significant difference in page or reference counts compared to the rest of the publications, suggesting that there exist other factors that impact the citations of single author publications.
Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation
Venue: arXiv cs.SD 2020
Year: 2020

Target speech separation refers to extracting the target speaker's speech from mixed signals. Despite the recent advances in deep learning based close-talk speech separation, the applications to real-world are still an open issue. Two main challenges are the complex acoustic environment and the real-time processing requirement. To address these challenges, we propose a temporal-spatial neural filter, which directly estimates the target speech waveform from multi-speaker mixture in reverberant environments, assisted with directional information of the speaker(s). Firstly, against variations brought by complex environment, the key idea is to increase the acoustic representation completeness through the jointly modeling of temporal, spectral and spatial discriminability between the target and interference source. Specifically, temporal, spectral, spatial along with the designed directional features are integrated to create a joint acoustic representation. Secondly, to reduce the latency, we design a fully-convolutional autoencoder framework, which is purely end-to-end and single-pass. All the feature computation is implemented by the network layers and operations to speed up the separation procedure. Evaluation is conducted on simulated reverberant dataset WSJ0-2mix and WSJ0-3mix under speaker-independent scenario. Experimental results demonstrate that the proposed method outperforms state-of-the-art deep learning based multi-channel approaches with fewer parameters and faster processing speed. Furthermore, the proposed temporal-spatial neural filter can handle mixtures with varying and unknown number of speakers and exhibits persistent performance even when existing a direction estimation error. Codes and models will be released soon.
On permutation quadrinomials and $4$-uniform BCT
Venue: arXiv cs.IT 2020
Year: 2020

We study a class of general quadrinomials over the field of size $2^{2m}$ with odd $m$ and characterize conditions under which they are permutations with the best boomerang uniformity, a new and important parameter related to boomerang-style attacks. This vastly extends previous results from several recent papers.
Thresholds of descending algorithms in inference problems
Venue: arXiv cs.LG 2020
Year: 2020

We review recent works on analyzing the dynamics of gradient-based algorithms in a prototypical statistical inference problem. Using methods and insights from the physics of glassy systems, these works showed how to understand quantitatively and qualitatively the performance of gradient-based algorithms. Here we review the key results and their interpretation in non-technical terms accessible to a wide audience of physicists in the context of related works.
The Real-World-Weight Cross-Entropy Loss Function: Modeling the Costs of Mislabeling
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we propose a new metric to measure goodness-of-fit for classifiers, the Real World Cost function. This metric factors in information about a real world problem, such as financial impact, that other measures like accuracy or F1 do not. This metric is also more directly interpretable for users. To optimize for this metric, we introduce the Real-World- Weight Crossentropy loss function, in both binary and single-label classification variants. Both variants allow direct input of real world costs as weights. For single-label, multicategory classification, our loss function also allows direct penalization of probabilistic false positives, weighted by label, during the training of a machine learning model. We compare the design of our loss function to the binary crossentropy and categorical crossentropy functions, as well as their weighted variants, to discuss the potential for improvement in handling a variety of known shortcomings of machine learning, ranging from imbalanced classes to medical diagnostic error to reinforcement of social bias. We create scenarios that emulate those issues using the MNIST data set and demonstrate empirical results of our new loss function. Finally, we sketch a proof of this function based on Maximum Likelihood Estimation and discuss future directions.
Read Beyond the Lines: Understanding the Implied Textual Meaning via a Skim and Intensive Reading Model
Venue: arXiv cs.CL 2020
Year: 2020

The nonliteral interpretation of a text is hard to be understood by machine models due to its high context-sensitivity and heavy usage of figurative language. In this study, inspired by human reading comprehension, we propose a novel, simple, and effective deep neural framework, called Skim and Intensive Reading Model (SIRM), for figuring out implied textual meaning. The proposed SIRM consists of two main components, namely the skim reading component and intensive reading component. N-gram features are quickly extracted from the skim reading component, which is a combination of several convolutional neural networks, as skim (entire) information. An intensive reading component enables a hierarchical investigation for both local (sentence) and global (paragraph) representation, which encapsulates the current embedding and the contextual information with a dense connection. More specifically, the contextual information includes the near-neighbor information and the skim information mentioned above. Finally, besides the normal training loss function, we employ an adversarial loss function as a penalty over the skim reading component to eliminate noisy information arisen from special figurative words in the training data. To verify the effectiveness, robustness, and efficiency of the proposed architecture, we conduct extensive comparative experiments on several sarcasm benchmarks and an industrial spam dataset with metaphors. Experimental results indicate that (1) the proposed model, which benefits from context modeling and consideration of figurative language, outperforms existing state-of-the-art solutions, with comparable parameter scale and training speed; (2) the SIRM yields superior robustness in terms of parameter size sensitivity; (3) compared with ablation and addition variants of the SIRM, the final framework is efficient enough.
Assessment of Audio Features for Automatic Cough Detection
Venue: arXiv cs.SD 2020
Year: 2020

This paper addresses the issue of cough detection using only audio recordings, with the ultimate goal of quantifying and qualifying the degree of pathology for patients suffering from respiratory diseases, notably mucoviscidosis. A large set of audio features describing various aspects of the audio signal is proposed. These features are assessed in two steps. First, their intrisic potential and redundancy are evaluated using mutual information-based measures. Secondly, their efficiency is confirmed relying on three classifiers: Artificial Neural Network, Gaussian Mixture Model and Support Vector Machine. The influence of both the feature dimension and the classifier complexity are also investigated.
On the Mutual Information between Source and Filter Contributions for Voice Pathology Detection
Venue: arXiv cs.SD 2020
Year: 2020

This paper addresses the problem of automatic detection of voice pathologies directly from the speech signal. For this, we investigate the use of the glottal source estimation as a means to detect voice disorders. Three sets of features are proposed, depending on whether they are related to the speech or the glottal signal, or to prosody. The relevancy of these features is assessed through mutual information-based measures. This allows an intuitive interpretation in terms of discrimation power and redundancy between the features, independently of any subsequent classifier. It is discussed which characteristics are interestingly informative or complementary for detecting voice pathologies.
Zero-Shot Reinforcement Learning with Deep Attention Convolutional Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Simulation-to-simulation and simulation-to-real world transfer of neural network models have been a difficult problem. To close the reality gap, prior methods to simulation-to-real world transfer focused on domain adaptation, decoupling perception and dynamics and solving each problem separately, and randomization of agent parameters and environment conditions to expose the learning agent to a variety of conditions. While these methods provide acceptable performance, the computational complexity required to capture a large variation of parameters for comprehensive scenarios on a given task such as autonomous driving or robotic manipulation is high. Our key contribution is to theoretically prove and empirically demonstrate that a deep attention convolutional neural network (DACNN) with specific visual sensor configuration performs as well as training on a dataset with high domain and parameter variation at lower computational complexity. Specifically, the attention network weights are learned through policy optimization to focus on local dependencies that lead to optimal actions, and does not require tuning in real-world for generalization. Our new architecture adapts perception with respect to the control objective, resulting in zero-shot learning without pre-training a perception network. To measure the impact of our new deep network architecture on domain adaptation, we consider autonomous driving as a use case. We perform an extensive set of experiments in simulation-to-simulation and simulation-to-real scenarios to compare our approach to several baselines including the current state-of-art models.
On Large-Scale Dynamic Topic Modeling with Nonnegative CP Tensor Decomposition
Venue: arXiv cs.LG 2020
Year: 2020

There is currently an unprecedented demand for large-scale temporal data analysis due to the explosive growth of data. Dynamic topic modeling has been widely used in social and data sciences with the goal of learning latent topics that emerge, evolve, and fade over time. Previous work on dynamic topic modeling primarily employ the method of nonnegative matrix factorization (NMF), where slices of the data tensor are each factorized into the product of lower-dimensional nonnegative matrices. With this approach, however, information contained in the temporal dimension of the data is often neglected or underutilized. To overcome this issue, we propose instead adopting the method of nonnegative CANDECOMP/PARAPAC (CP) tensor decomposition (NNCPD), where the data tensor is directly decomposed into a minimal sum of outer products of nonnegative vectors, thereby preserving the temporal information. The viability of NNCPD is demonstrated through application to both synthetic and real data, where significantly improved results are obtained compared to those of typical NMF-based methods. The advantages of NNCPD over such approaches are studied and discussed. To the best of our knowledge, this is the first time that NNCPD has been utilized for the purpose of dynamic topic modeling, and our findings will be transformative for both applications and further developments.
From Kinematics To Dynamics: Estimating Center of Pressure and Base of Support from Video Frames of Human Motion
Venue: arXiv cs.CV 2020
Year: 2020

To gain an understanding of the relation between a given human pose image and the corresponding physical foot pressure of the human subject, we propose and validate two end-to-end deep learning architectures, PressNet and PressNet-Simple, to regress foot pressure heatmaps (dynamics) from 2D human pose (kinematics) derived from a video frame. A unique video and foot pressure data set of 813,050 synchronized pairs, composed of 5-minute long choreographed Taiji movement sequences of 6 subjects, is collected and used for leaving-one-subject-out cross validation. Our initial experimental results demonstrate reliable and repeatable foot pressure prediction from a single image, setting the first baseline for such a complex cross modality mapping problem in computer vision. Furthermore, we compute and quantitatively validate the Center of Pressure (CoP) and Base of Support (BoS) from predicted foot pressure distribution, obtaining key components in pose stability analysis from images with potential applications in kinesiology, medicine, sports and robotics.
HandAugment: A Simple Data Augmentation Method for Depth-Based 3D Hand Pose Estimation
Venue: arXiv cs.CV 2020
Year: 2020

Hand pose estimation from 3D depth images, has been explored widely using various kinds of techniques in the field of computer vision. Though, deep learning based method improve the performance greatly recently, however, this problem still remains unsolved due to lack of large datasets, like ImageNet or effective data synthesis methods. In this paper, we propose HandAugment, a method to synthesize image data to augment the training process of the neural networks. Our method has two main parts: First, We propose a scheme of two-stage neural networks. This scheme can make the neural networks focus on the hand regions and thus to improve the performance. Second, we introduce a simple and effective method to synthesize data by combining real and synthetic image together in the image space. Finally, we show that our method achieves the first place in the task of depth-based 3D hand pose estimation in HANDS 2019 challenge.