OpenRaiser/Intern-Atlas huggingface dataset:
PcmNet: Position-Sensitive Context Modeling Network for Temporal Action Localization
Venue: arXiv cs.CV 2021
Year: 2021

Temporal action localization is an important and challenging task that aims to locate temporal regions in real-world untrimmed videos where actions occur and recognize their classes. It is widely acknowledged that video context is a critical cue for video understanding, and exploiting the context has become an important strategy to boost localization performance. However, previous state-of-the-art methods focus more on exploring semantic context which captures the feature similarity among frames or proposals, and neglect positional context which is vital for temporal localization. In this paper, we propose a temporal-position-sensitive context modeling approach to incorporate both positional and semantic information for more precise action localization. Specifically, we first augment feature representations with directed temporal positional encoding, and then conduct attention-based information propagation, in both frame-level and proposal-level. Consequently, the generated feature representations are significantly empowered with the discriminative capability of encoding the position-aware context information, and thus benefit boundary detection and proposal evaluation. We achieve state-of-the-art performance on both two challenging datasets, THUMOS-14 and ActivityNet-1.3, demonstrating the effectiveness and generalization ability of our method.
Bio-Inspired Representation Learning for Visual Attention Prediction
Venue: arXiv cs.CV 2021
Year: 2021

Visual Attention Prediction (VAP) is a significant and imperative issue in the field of computer vision. Most of existing VAP methods are based on deep learning. However, they do not fully take advantage of the low-level contrast features while generating the visual attention map. In this paper, a novel VAP method is proposed to generate visual attention map via bio-inspired representation learning. The bio-inspired representation learning combines both low-level contrast and high-level semantic features simultaneously, which are developed by the fact that human eye is sensitive to the patches with high contrast and objects with high semantics. The proposed method is composed of three main steps: 1) feature extraction, 2) bio-inspired representation learning and 3) visual attention map generation. Firstly, the high-level semantic feature is extracted from the refined VGG16, while the low-level contrast feature is extracted by the proposed contrast feature extraction block in a deep network. Secondly, during bio-inspired representation learning, both the extracted low-level contrast and high-level semantic features are combined by the designed densely connected block, which is proposed to concatenate various features scale by scale. Finally, the weighted-fusion layer is exploited to generate the ultimate visual attention map based on the obtained representations after bio-inspired representation learning. Extensive experiments are performed to demonstrate the effectiveness of the proposed method.
Energy-Efficient MAC for Cellular IoT: State-of-the-Art, Challenges, and Standardization
Venue: arXiv cs.NI 2021
Year: 2021

In the modern world, the connectivity-as-we-go model is gaining popularity. Internet-of-Things (IoT) envisions a future in which human beings communicate with each other and with devices that have identities and virtual personalities, as well as sensing, processing, and networking capabilities, which will allow the developing of smart environments that operate with little or no human intervention. In such IoT environments, that will have battery-operated sensors and devices, energy efficiency becomes a fundamental concern. Thus, energy-efficient (EE) connectivity is gaining significant attention from the industrial and academic communities. This work aims to provide a comprehensive state-of-the-art survey on the energy efficiency of medium access control (MAC) protocols for cellular IoT. we provide a detailed discussion on the sources of energy dissipation at the MAC layer and then propose solutions. In addition to reviewing the proposed MAC designs, we also provide insights and suggestions that can guide practitioners and researchers in designing EE MAC protocols that extend the battery life of IoT devices. Finally, we identify a range of challenging open problems that should be solved for providing EE MAC services for IoT devices, along with corresponding opportunities and future research ideas to address these challenges.
CNNATT: Deep EEG & fNIRS Real-Time Decoding of bimanual forces
Venue: arXiv cs.LG 2021
Year: 2021

Non-invasive cortical neural interfaces have only achieved modest performance in cortical decoding of limb movements and their forces, compared to invasive brain-computer interfaces (BCIs). While non-invasive methodologies are safer, cheaper and vastly more accessible technologies, signals suffer from either poor resolution in the space domain (EEG) or the temporal domain (BOLD signal of functional Near Infrared Spectroscopy, fNIRS). The non-invasive BCI decoding of bimanual force generation and the continuous force signal has not been realised before and so we introduce an isometric grip force tracking task to evaluate the decoding. We find that combining EEG and fNIRS using deep neural networks works better than linear models to decode continuous grip force modulations produced by the left and the right hand. Our multi-modal deep learning decoder achieves 55.2 FVAF[%] in force reconstruction and improves the decoding performance by at least 15% over each individual modality. Our results show a way to achieve continuous hand force decoding using cortical signals obtained with non-invasive mobile brain imaging has immediate impact for rehabilitation, restoration and consumer applications.
A Mask R-CNN approach to counting bacterial colony forming units in pharmaceutical development
Venue: arXiv cs.LG 2021
Year: 2021

We present an application of the well-known Mask R-CNN approach to the counting of different types of bacterial colony forming units that were cultured in Petri dishes. Our model was made available to lab technicians in a modern SPA (Single-Page Application). Users can upload images of dishes, after which the Mask R-CNN model that was trained and tuned specifically for this task detects the number of BVG- and BVG+ colonies and displays these in an interactive interface for the user to verify. Users can then check the model's predictions, correct them if deemed necessary, and finally validate them. Our adapted Mask R-CNN model achieves a mean average precision (mAP) of 94\% at an intersection-over-union (IoU) threshold of 50\%. With these encouraging results, we see opportunities to bring the benefits of improved accuracy and time saved to related problems, such as generalising to other bacteria types and viral foci counting.
Inter-subject Deep Transfer Learning for Motor Imagery EEG Decoding
Venue: arXiv cs.LG 2021
Year: 2021

Convolutional neural networks (CNNs) have become a powerful technique to decode EEG and have become the benchmark for motor imagery EEG Brain-Computer-Interface (BCI) decoding. However, it is still challenging to train CNNs on multiple subjects' EEG without decreasing individual performance. This is known as the negative transfer problem, i.e. learning from dissimilar distributions causes CNNs to misrepresent each of them instead of learning a richer representation. As a result, CNNs cannot directly use multiple subjects' EEG to enhance model performance directly. To address this problem, we extend deep transfer learning techniques to the EEG multi-subject training case. We propose a multi-branch deep transfer network, the Separate-Common-Separate Network (SCSN) based on splitting the network's feature extractors for individual subjects. We also explore the possibility of applying Maximum-mean discrepancy (MMD) to the SCSN (SCSN-MMD) to better align distributions of features from individual feature extractors. The proposed network is evaluated on the BCI Competition IV 2a dataset (BCICIV2a dataset) and our online recorded dataset. Results show that the proposed SCSN (81.8%, 53.2%) and SCSN-MMD (81.8%, 54.8%) outperformed the benchmark CNN (73.4%, 48.8%) on both datasets using multiple subjects. Our proposed networks show the potential to utilise larger multi-subject datasets to train an EEG decoder without being influenced by negative transfer.
Streaming Hypergraph Partitioning Algorithms on Limited Memory Environments
Venue: arXiv cs.DS 2021
Year: 2021

Many well-known, real-world problems involve dynamic data which describe the relationship among the entities. Hypergraphs are powerful combinatorial structures that are frequently used to model such data. For many of today's data-centric applications, this data is streaming; new items arrive continuously, and the data grows with time. With paradigms such as Internet of Things and Edge Computing, such applications become more natural and more practical. In this work, we assume a streaming model where the data is modeled as a hypergraph, which is generated at the edge. This data then partitioned and sent to remote nodes via an algorithm running on a memory-restricted device such as a single board computer. Such a partitioning is usually performed by taking a connectivity metric into account to minimize the communication cost of later analyses that will be performed in a distributed fashion. Although there are many offline tools that can partition static hypergraphs excellently, algorithms for the streaming settings are rare. We analyze a well-known algorithm from the literature and significantly improve its running time by altering its inner data structure. For instance, on a medium-scale hypergraph, the new algorithm reduces the runtime from 17800 seconds to 10 seconds. We then propose sketch- and hash-based algorithms, as well as ones that can leverage extra memory to store a small portion of the data to enable the refinement of partitioning when possible. We experimentally analyze the performance of these algorithms and report their run times, connectivity metric scores, and memory uses on a high-end server and four different single-board computer architectures.
Efficient learning of goal-oriented push-grasping synergy in clutter
Venue: arXiv cs.RO 2021
Year: 2021

We focus on the task of goal-oriented grasping, in which a robot is supposed to grasp a pre-assigned goal object in clutter and needs some pre-grasp actions such as pushes to enable stable grasps. However, in this task, the robot gets positive rewards from environment only when successfully grasping the goal object. Besides, joint pushing and grasping elongates the action sequence, compounding the problem of reward delay. Thus, sample inefficiency remains a main challenge in this task. In this paper, a goal-conditioned hierarchical reinforcement learning formulation with high sample efficiency is proposed to learn a push-grasping policy for grasping a specific object in clutter. In our work, sample efficiency is improved by two means. First, we use a goal-conditioned mechanism by goal relabeling to enrich the replay buffer. Second, the pushing and grasping policies are respectively regarded as a generator and a discriminator and the pushing policy is trained with supervision of the grasping discriminator, thus densifying pushing rewards. To deal with the problem of distribution mismatch caused by different training settings of two policies, an alternating training stage is added to learn pushing and grasping in turn. A series of experiments carried out in simulation and real world indicate that our method can quickly learn effective pushing and grasping policies and outperforms existing methods in task completion rate and goal grasp success rate by less times of motion. Furthermore, we validate that our system can also adapt to goal-agnostic conditions with better performance. Note that our system can be transferred to the real world without any fine-tuning. Our code is available at https://github.com/xukechun/Efficient_goal-oriented_push-grasping_synergy.
Analytically Tractable Inference in Deep Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Since its inception, deep learning has been overwhelmingly reliant on backpropagation and gradient-based optimization algorithms in order to learn weight and bias parameter values. Tractable Approximate Gaussian Inference (TAGI) algorithm was shown to be a viable and scalable alternative to backpropagation for shallow fully-connected neural networks. In this paper, we are demonstrating how TAGI matches or exceeds the performance of backpropagation, for training classic deep neural network architectures. Although TAGI's computational efficiency is still below that of deterministic approaches relying on backpropagation, it outperforms them on classification tasks and matches their performance for information maximizing generative adversarial networks while using smaller architectures trained with fewer epochs.
Generating Reliable Process Event Streams and Time Series Data based on Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Domains such as manufacturing and medicine crave for continuous monitoring and analysis of their processes, especially in combination with time series as produced by sensors. Time series data can be exploited to, for example, explain and predict concept drifts during runtime. Generally, a certain data volume is required in order to produce meaningful analysis results. However, reliable data sets are often missing, for example, if event streams and times series data are collected separately, in case of a new process, or if it is too expensive to obtain a sufficient data volume. Additional challenges arise with preparing time series data from multiple event sources, variations in data collection frequency, and concept drift. This paper proposes the GENLOG approach to generate reliable event and time series data that follows the distribution of the underlying input data set. GENLOG employs data resampling and enables the user to select different parts of the log data to orchestrate the training of a recurrent neural network for stream generation. The generated data is sampled back to its original sample rate and is embedded into the originating log data file. Overall, GENLOG can boost small data sets and consequently the application of online process mining.
PEAK SHIFT ESTIMATION A novel method to estimate ranking of selectively omitted examination data
Venue: arXiv cs.CY 2021
Year: 2021

In this paper, we focus on examination results when examinees selectively skip examinations, to compare the difficulty levels of these examinations. We call the resultant data 'selectively omitted examination data' Examples of this type of examination are university entrance examinations, certification examinations, and the outcome of students' job-hunting activities. We can learn the number of students accepted for each examination and organization but not the examinees' identity. No research has focused on this type of data. When we know the difficulty level of these examinations, we can obtain a new index to assess organization ability, how many students pass, and the difficulty of the examinations. This index would reflect the outcomes of their education corresponding to perspectives on examinations. Therefore, we propose a novel method, Peak Shift Estimation, to estimate the difficulty level of an examination based on selectively omitted examination data. First, we apply Peak Shift Estimation to the simulation data and demonstrate that Peak Shift Estimation estimates the rank order of the difficulty level of university entrance examinations very robustly. Peak Shift Estimation is also suitable for estimating a multi-level scale for universities, that is, A, B, C, and D rank university entrance examinations. We apply Peak Shift Estimation to real data of the Tokyo metropolitan area and demonstrate that the rank correlation coefficient between difficulty level ranking and true ranking is 0.844 and that the difference between 80 percent of universities is within 25 ranks. The accuracy of Peak Shift Estimation is thus low and must be improved; however, this is the first study to focus on ranking selectively omitted examination data, and therefore, one of our contributions is to shed light on this method.
Doubly Contrastive Deep Clustering
Venue: arXiv cs.CV 2021
Year: 2021

Deep clustering successfully provides more effective features than conventional ones and thus becomes an important technique in current unsupervised learning. However, most deep clustering methods ignore the vital positive and negative pairs introduced by data augmentation and further the significance of contrastive learning, which leads to suboptimal performance. In this paper, we present a novel Doubly Contrastive Deep Clustering (DCDC) framework, which constructs contrastive loss over both sample and class views to obtain more discriminative features and competitive results. Specifically, for the sample view, we set the class distribution of the original sample and its augmented version as positive sample pairs and set one of the other augmented samples as negative sample pairs. After that, we can adopt the sample-wise contrastive loss to pull positive sample pairs together and push negative sample pairs apart. Similarly, for the class view, we build the positive and negative pairs from the sample distribution of the class. In this way, two contrastive losses successfully constrain the clustering results of mini-batch samples in both sample and class level. Extensive experimental results on six benchmark datasets demonstrate the superiority of our proposed model against state-of-the-art methods. Particularly in the challenging dataset Tiny-ImageNet, our method leads 5.6\% against the latest comparison method. Our code will be available at \url{https://github.com/ZhiyuanDang/DCDC}.
Software Development During COVID-19 Pandemic: an Analysis of Stack Overflow and GitHub
Venue: arXiv cs.SE 2021
Year: 2021

The new coronavirus became a severe health issue for the world. This situation has motivated studies of different areas to combat this pandemic. In software engineering, we point out data visualization projects to follow the disease evolution, machine learning to estimate the pandemic behavior, and computer vision processing radiologic images. Most of these projects are stored in version control systems, and there are discussions about them in Question & Answer websites. In this work, we conducted a Mining Software Repository on a large number of questions and projects aiming to find trends that could help researchers and practitioners to fight against the coronavirus. We analyzed 1,190 questions from Stack Overflow and Data Science Q\&A and 60,352 GitHub projects. We identified a correlation between the questions and projects throughout the pandemic. The main questions about coronavirus are how-to, related to web scraping and data visualization, using Python, JavaScript, and R. The most recurrent GitHub projects are machine learning projects, using JavaScript, Python, and Java.
140 GHz Urban Microcell Propagation Measurements for Spatial Consistency Modeling
Venue: arXiv cs.IT 2021
Year: 2021

Sub-Terahertz frequencies (frequencies above 100 GHz) have the potential to satisfy the unprecedented demand on data rate on the order of hundreds of Gbps for sixth-generation (6G) wireless communications and beyond. Accurate beam tracking and rapid beam selection are increasingly important since antenna arrays with more elements generate narrower beams to compensate for additional path loss within the first meter of propagation distance at sub-THz frequencies. Realistic channel models for above 100 GHz are needed, and should include spatial consistency to model the spatial and temporal channel evolution along the user trajectory. This paper introduces recent outdoor urban microcell (UMi) propagation measurements at 142 GHz along a 39 m $\times$ 12 m rectangular route (102 m long), where each consecutive and adjacent receiver location is 3 m apart from each other. The measured power delay profiles and angular power spectrum at each receiver location are used to study spatial autocorrelation properties of various channel parameters such as shadow fading, delay spread, and angular spread along the track. Compared to the correlation distances reported in the 3GPP TR 38.901 for frequencies below 100 GHz, the measured correlation distance of shadow fading at 142 GHz (3.8 m) is much shorter than the 10-13 m as specified in 3GPP; the measured correlation distances of delay spread and angular spread at 142 GHz (both 12 m) are comparable to the 7-10 m as specified in 3GPP. This result may guide the development of a statistical spatially consistent channel model for frequencies above 100 GHz in the UMi street canyon environment.
Symbolic integration by integrating learning models with different strengths and weaknesses
Venue: arXiv cs.LG 2021
Year: 2021

Integration is indispensable, not only in mathematics, but also in a wide range of other fields. A deep learning method has recently been developed and shown to be capable of integrating mathematical functions that could not previously be integrated on a computer. However, that method treats integration as equivalent to natural language translation and does not reflect mathematical information. In this study, we adjusted the learning model to take mathematical information into account and developed a wide range of learning models that learn the order of numerical operations more robustly. In this way, we achieved a 98.80% correct answer rate with symbolic integration, a higher rate than that of any existing method. We judged the correctness of the integration based on whether the derivative of the primitive function was consistent with the integrand. By building an integrated model based on this strategy, we achieved a 99.79% rate of correct answers with symbolic integration.
Constrained Contextual Bandit Learning for Adaptive Radar Waveform Selection
Venue: arXiv cs.IT 2021
Year: 2021

A sequential decision process in which an adaptive radar system repeatedly interacts with a finite-state target channel is studied. The radar is capable of passively sensing the spectrum at regular intervals, which provides side information for the waveform selection process. The radar transmitter uses the sequence of spectrum observations as well as feedback from a collocated receiver to select waveforms which accurately estimate target parameters. It is shown that the waveform selection problem can be effectively addressed using a linear contextual bandit formulation in a manner that is both computationally feasible and sample efficient. Stochastic and adversarial linear contextual bandit models are introduced, allowing the radar to achieve effective performance in broad classes of physical environments. Simulations in a radar-communication coexistence scenario, as well as in an adversarial radar-jammer scenario, demonstrate that the proposed formulation provides a substantial improvement in target detection performance when Thompson Sampling and EXP3 algorithms are used to drive the waveform selection process. Further, it is shown that the harmful impacts of pulse-agile behavior on coherently processed radar data can be mitigated by adopting a time-varying constraint on the radar's waveform catalog.