OpenRaiser/Intern-Atlas huggingface dataset:
OLED: One-Class Learned Encoder-Decoder Network with Adversarial Context Masking for Novelty Detection
Venue: arXiv cs.CV 2021
Year: 2021

Novelty detection is the task of recognizing samples that do not belong to the distribution of the target class. During training, the novelty class is absent, preventing the use of traditional classification approaches. Deep autoencoders have been widely used as a base of many unsupervised novelty detection methods. In particular, context autoencoders have been successful in the novelty detection task because of the more effective representations they learn by reconstructing original images from randomly masked images. However, a significant drawback of context autoencoders is that random masking fails to consistently cover important structures of the input image, leading to suboptimal representations - especially for the novelty detection task. In this paper, to optimize input masking, we have designed a framework consisting of two competing networks, a Mask Module and a Reconstructor. The Mask Module is a convolutional autoencoder that learns to generate optimal masks that cover the most important parts of images. Alternatively, the Reconstructor is a convolutional encoder-decoder that aims to reconstruct unperturbed images from masked images. The networks are trained in an adversarial manner in which the Mask Module generates masks that are applied to images given to the Reconstructor. In this way, the Mask Module seeks to maximize the reconstruction error that the Reconstructor is minimizing. When applied to novelty detection, the proposed approach learns semantically richer representations compared to context autoencoders and enhances novelty detection at test time through more optimal masking. Novelty detection experiments on the MNIST and CIFAR-10 image datasets demonstrate the proposed approach's superiority over cutting-edge methods. In a further experiment on the UCSD video dataset for novelty detection, the proposed approach achieves state-of-the-art results.
Effective GPU Parallelization of Distributed and Localized Model Predictive Control
Venue: arXiv cs.DC 2021
Year: 2021

To effectively control large-scale distributed systems online, model predictive control (MPC) has to swiftly solve the underlying high-dimensional optimization. There are multiple techniques applied to accelerate the solving process in the literature, mainly attributed to software-based algorithmic advancements and hardware-assisted computation enhancements. However, those methods focus on arithmetic accelerations and overlook the benefits of the underlying system's structure. In particular, the existing decoupled software-hardware algorithm design that naively parallelizes the arithmetic operations by the hardware does not tackle the hardware overheads such as CPU-GPU and thread-to-thread communications in a principled manner. Also, the advantages of parallelizable subproblem decomposition in distributed MPC are not well recognized and exploited. As a result, we have not reached the full potential of hardware acceleration for MPC. In this paper, we explore those opportunities by leveraging GPU to parallelize the distributed and localized MPC (DLMPC) algorithm. We exploit the locality constraints embedded in the DLMPC formulation to reduce the hardware-intrinsic communication overheads. Our parallel implementation achieves up to 50x faster runtime than its CPU counterparts under various parameters. Furthermore, we find that the locality-aware GPU parallelization could halve the optimization runtime comparing to the naive acceleration. Overall, our results demonstrate the performance gains brought by software-hardware co-design with the information exchange structure in mind.
Thermal transmittance prediction based on the application of artificial neural networks on heat flux method results
Venue: arXiv cs.LG 2021
Year: 2021

Deep energy renovation of building stock came more into focus in the European Union due to energy efficiency related directives. Many buildings that must undergo deep energy renovation are old and may lack design/renovation documentation, or possible degradation of materials might have occurred in building elements over time. Thermal transmittance (i.e. U-value) is one of the most important parameters for determining the transmission heat losses through building envelope elements. It depends on the thickness and thermal properties of all the materials that form a building element. In-situ U-value can be determined by ISO 9869-1 standard (Heat Flux Method - HFM). Still, measurement duration is one of the reasons why HFM is not widely used in field testing before the renovation design process commences. This paper analyzes the possibility of reducing the measurement time by conducting parallel measurements with one heat-flux sensor. This parallelization could be achieved by applying a specific class of the Artificial Neural Network (ANN) on HFM results to predict unknown heat flux based on collected interior and exterior air temperatures. After the satisfying prediction is achieved, HFM sensor can be relocated to another measuring location. Paper shows a comparison of four ANN cases applied to HFM results for a measurement held on one multi-layer wall - multilayer perceptron with three neurons in one hidden layer, long short-term memory with 100 units, gated recurrent unit with 100 units and combination of 50 long short-term memory units and 50 gated recurrent units. The analysis gave promising results in term of predicting the heat flux rate based on the two input temperatures. Additional analysis on another wall showed possible limitations of the method that serves as a direction for further research on this topic.
MT-lib: A Topology-aware Message Transfer Library for Graph500 on Supercomputers
Venue: arXiv cs.DC 2021
Year: 2021

We present MT-lib, an efficient message transfer library for messages gather and scatter in benchmarks like Graph500 for Supercomputers. Our library includes MST version as well as new-MST version. The MT-lib is deliberately kept light-weight, efficient and friendly interfaces for massive graph traverse. MST provides (1) a novel non-blocking communication scheme with sending and receiving messages asynchronously to overlap calculation and communication;(2) merging messages according to the target process for reducing communication overhead;(3) a new communication mode of gathering intra-group messages before forwarding between groups for reducing communication traffic. In MT-lib, there are (1) one-sided message; (2) two-sided messages; and (3) two-sided messages with buffer, in which dynamic buffer expansion is built for messages delivery. We experimented with MST and then testing Graph500 with MST on Tianhe supercomputers. Experimental results show high communication efficiency and high throughputs for both BFS and SSSP communication operations.
An Affective Approach for Behavioral Performance Estimation and Induction
Venue: arXiv cs.HC 2021
Year: 2021

Emotions have a major interactive role in defining how humans interact with their environment by encoding their perception to external events and accordingly, influencing their cognition and decision-making process. Therefore, increasing attention has been directed toward integrating human affective states into system design in order to optimize the quality of task performance. In this work, we seize on the significant correlation between emotions and behavioral performance that is reported in several psychological studies and develop an online closed-loop design framework for Human-Robot Interaction (HRI). The proposed approach monitors the behavioral performance based on the levels of Pleasure, Arousal, and Dominance (PAD) states for the human operator and when required, applies an external stimulus which is selected to induce an improvement in performance. The framework is implemented on an HRI task involving a human operator teleoperating an articulated robotic manipulator. Our statistical analysis shows a significant decrease in pleasure, arousal, and dominance states as the behavioral performance deteriorates $(p < 0.05)$. Our closed-loop experiment that uses an audio stimulus to improve emotional state shows a significant improvement in the behavioral performance of certain subjects.
Friends and Foes in Learning from Noisy Labels
Venue: arXiv cs.CV 2021
Year: 2021

Learning from examples with noisy labels has attracted increasing attention recently. But, this paper will show that the commonly used CIFAR-based datasets and the accuracy evaluation metric used in the literature are both inappropriate in this context. An alternative valid evaluation metric and new datasets are proposed in this paper to promote proper research and evaluation in this area. Then, friends and foes are identified from existing methods as technical components that are either beneficial or detrimental to deep learning from noisy labeled examples, respectively, and this paper improves and combines technical components from the friends category, including self-supervised learning, new warmup strategy, instance filtering and label correction. The resulting F&F method significantly outperforms existing methods on the proposed nCIFAR datasets and the real-world Clothing1M dataset.
Symbolic regression outperforms other models for small data sets
Venue: arXiv cs.LG 2021
Year: 2021

Machine learning is often applied in health science to obtain predictions and new understandings of complex phenomena and relationships, but an availability of sufficient data for model training is a widespread problem. Traditional machine learning techniques, such as random forests and gradient boosting, tend to overfit when working with data sets of only a few hundred observations. This study demonstrates that for small training sets of 250 observations, symbolic regression generalises better to out-of-sample data than traditional machine learning frameworks, as measured by the coefficient of determination R2 on the validation set. In 132 out of 240 cases, symbolic regression achieves a higher R2 than any of the other models on the out-of-sample data. Furthermore, symbolic regression also preserves the interpretability of linear models and decision trees, an added benefit to its superior generalisation. The second best algorithm was found to be a random forest, which performs best in 37 of the 240 cases. When restricting the comparison to interpretable models, symbolic regression performs best in 184 out of 240 cases.
Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Rendering
Venue: arXiv cs.CV 2021
Year: 2021

Reconstructing the shape and appearance of real-world objects using measured 2D images has been a long-standing problem in computer vision. In this paper, we introduce a new analysis-by-synthesis technique capable of producing high-quality reconstructions through robust coarse-to-fine optimization and physics-based differentiable rendering. Unlike most previous methods that handle geometry and reflectance largely separately, our method unifies the optimization of both by leveraging image gradients with respect to both object reflectance and geometry. To obtain physically accurate gradient estimates, we develop a new GPU-based Monte Carlo differentiable renderer leveraging recent advances in differentiable rendering theory to offer unbiased gradients while enjoying better performance than existing tools like PyTorch3D and redner. To further improve robustness, we utilize several shape and material priors as well as a coarse-to-fine optimization strategy to reconstruct geometry. We demonstrate that our technique can produce reconstructions with higher quality than previous methods such as COLMAP and Kinect Fusion.
Euler Meets GPU: Practical Graph Algorithms with Theoretical Guarantees
Venue: arXiv cs.DS 2021
Year: 2021

The Euler tour technique is a classical tool for designing parallel graph algorithms, originally proposed for the PRAM model. We ask whether it can be adapted to run efficiently on GPU. We focus on two established applications of the technique: (1) the problem of finding lowest common ancestors (LCA) of pairs of nodes in trees, and (2) the problem of finding bridges in undirected graphs. In our experiments, we compare theoretically optimal algorithms using the Euler tour technique against simpler heuristics supposed to perform particularly well on typical instances. We show that the Euler tour-based algorithms not only fulfill their theoretical promises and outperform practical heuristics on hard instances, but also perform on par with them on easy instances.
TULVCAN: Terahertz Ultra-broadband Learning Vehicular Channel-Aware Networking
Venue: arXiv cs.NI 2021
Year: 2021

Due to spectrum scarcity and increasing wireless capacity demands, terahertz (THz) communications at 0.1-10THz and the corresponding spectrum characterization have emerged to meet diverse service requirements in future 5G and 6G wireless systems. However, conventional compressed sensing techniques to reconstruct the original wideband spectrum with under-sampled measurements become inefficient as local spectral correlation is deliberately omitted. Recent works extend communication methods with deep learning-based algorithms but lack strong ties to THz channel properties. This paper introduces novel THz channel-aware spectrum learning solutions that fully disclose the uniqueness of THz channels when performing such ultra-broadband sensing in vehicular environments. Specifically, a joint design of spectrum compression and reconstruction is proposed through a structured sensing matrix and two-phase reconstruction based on high spreading loss and molecular absorption at THz frequencies. An end-to-end learning framework, namely compression and reconstruction network (CRNet), is further developed with the mean-square-error loss function to improve sensing accuracy while significantly reducing computational complexity. Numerical results show that the CRNet solutions outperform the latest generative adversarial network (GAN) realization with a much higher cosine and structure similarity measures, smaller learning errors, and 56% less required training overheads. This THz Ultra-broadband Learning Vehicular Channel-Aware Networking (TULVCAN) work successfully achieves effective THz spectrum learning and hence allows frequency-agile access.
Low-Fidelity End-to-End Video Encoder Pre-training for Temporal Action Localization
Venue: arXiv cs.CV 2021
Year: 2021

Temporal action localization (TAL) is a fundamental yet challenging task in video understanding. Existing TAL methods rely on pre-training a video encoder through action classification supervision. This results in a task discrepancy problem for the video encoder -- trained for action classification, but used for TAL. Intuitively, end-to-end model optimization is a good solution. However, this is not operable for TAL subject to the GPU memory constraints, due to the prohibitive computational cost in processing long untrimmed videos. In this paper, we resolve this challenge by introducing a novel low-fidelity end-to-end (LoFi) video encoder pre-training method. Instead of always using the full training configurations for TAL learning, we propose to reduce the mini-batch composition in terms of temporal, spatial or spatio-temporal resolution so that end-to-end optimization for the video encoder becomes operable under the memory conditions of a mid-range hardware budget. Crucially, this enables the gradient to flow backward through the video encoder from a TAL loss supervision, favourably solving the task discrepancy problem and providing more effective feature representations. Extensive experiments show that the proposed LoFi pre-training approach can significantly enhance the performance of existing TAL methods. Encouragingly, even with a lightweight ResNet18 based video encoder in a single RGB stream, our method surpasses two-stream ResNet50 based alternatives with expensive optical flow, often by a good margin.
KNN, An Underestimated Model for Regional Rainfall Forecasting
Venue: arXiv cs.LG 2021
Year: 2021

Regional rainfall forecasting is an important issue in hydrology and meteorology. This paper aims to design an integrated tool by applying various machine learning algorithms, especially the state-of-the-art deep learning algorithms including Deep Neural Network, Wide Neural Network, Deep and Wide Neural Network, Reservoir Computing, Long Short Term Memory, Support Vector Machine, K-Nearest Neighbor for forecasting regional precipitations over different catchments in Upstate New York. Through the experimental results and the comparison among machine learning models including classification and regression, we find that KNN is an outstanding model over other models to handle the uncertainty in the precipitation data. The data normalization methods such as ZScore and MinMax are also evaluated and discussed.
Rethinking ResNets: Improved Stacking Strategies With High Order Schemes
Venue: arXiv cs.CV 2021
Year: 2021

Various deep neural network architectures (DNNs) maintain massive vital records in computer vision. While drawing attention worldwide, the design of the overall structure lacks general guidance. Based on the relationship between DNN design and numerical differential equations, we performed a fair comparison of the residual design with higher-order perspectives. We show that the widely used DNN design strategy, constantly stacking a small design (usually 2-3 layers), could be easily improved, supported by solid theoretical knowledge and with no extra parameters needed. We reorganise the residual design in higher-order ways, which is inspired by the observation that many effective networks can be interpreted as different numerical discretisations of differential equations. The design of ResNet follows a relatively simple scheme, which is Euler forward; however, the situation becomes complicated rapidly while stacking. We suppose that stacked ResNet is somehow equalled to a higher-order scheme; then, the current method of forwarding propagation might be relatively weak compared with a typical high-order method such as Runge-Kutta. We propose HO-ResNet to verify the hypothesis of widely used CV benchmarks with sufficient experiments. Stable and noticeable increases in performance are observed, and convergence and robustness are also improved. Our stacking strategy improved ResNet-30 by 2.15 per cent and ResNet-58 by 2.35 per cent on CIFAR-10, with the same settings and parameters. The proposed strategy is fundamental and theoretical and can therefore be applied to any network as a general guideline.
A More Fine-Grained Aspect-Sentiment-Opinion Triplet Extraction Task
Venue: arXiv cs.CL 2021
Year: 2021

Aspect Sentiment Triplet Extraction (ASTE) aims to extract aspect term, sentiment and opinion term triplets from sentences and tries to provide a complete solution for aspect-based sentiment analysis (ABSA). However, some triplets extracted by ASTE are confusing, since the sentiment in a triplet extracted by ASTE is the sentiment that the sentence expresses toward the aspect term rather than the sentiment of the aspect term and opinion term pair. In this paper, we introduce a more fine-grained Aspect-Sentiment-Opinion Triplet Extraction (ASOTE) Task. ASOTE also extracts aspect term, sentiment and opinion term triplets. However, the sentiment in a triplet extracted by ASOTE is the sentiment of the aspect term and opinion term pair. We build four datasets for ASOTE based on several popular ABSA benchmarks. We propose a Position-aware BERT-based Framework (PBF) to address this task. PBF first extracts aspect terms from sentences. For each extracted aspect term, PBF first generates aspect term-specific sentence representations considering both the meaning and the position of the aspect term, then extracts associated opinion terms and predicts the sentiments of the aspect term and opinion term pairs based on the sentence representations. Experimental results on the four datasets show the effectiveness of PBF.
Personalized Affect-Aware Socially Assistive Robot Tutors Aimed at Fostering Social Grit in Children with Autism
Venue: arXiv cs.HC 2021
Year: 2021

Affect-aware socially assistive robotics (SAR) tutors have great potential to augment and democratize professional therapeutic interventions for children with autism spectrum disorders (ASD) from different socioeconomic backgrounds. However, the majority of research on SAR for ASD has been on teaching cognitive and/or social skills, not on addressing users' emotional needs for real-world social situations. To bridge that gap, this work aims to develop personalized affect-aware SAR tutors to help alleviate social anxiety and foster social grit-the growth mindset for social skill development-in children with ASD. We propose a novel paradigm to incorporate clinically validated Acceptance and Commitment Training (ACT) with personalized SAR interventions. This work paves the way toward developing personalized affect-aware SAR interventions to support the unique and diverse socio-emotional needs and challenges of children with ASD.
A cooperative partial snapshot algorithm for checkpoint-rollback recovery of large-scale and dynamic distributed systems and experimental evaluations
Venue: arXiv cs.DC 2021
Year: 2021

A distributed system consisting of a huge number of computational entities is prone to faults, because faults in a few nodes cause the entire system to fail. Consequently, fault tolerance of distributed systems is a critical issue. Checkpoint-rollback recovery is a universal and representative technique for fault tolerance; it periodically records the entire system state (configuration) to non-volatile storage, and the system restores itself using the recorded configuration when the system fails. To record a configuration of a distributed system, a specific algorithm known as a snapshot algorithm is required. However, many snapshot algorithms require coordination among all nodes in the system; thus, frequent executions of snapshot algorithms require unacceptable communication cost, especially if the systems are large. As a sophisticated snapshot algorithm, a partial snapshot algorithm has been introduced that takes a partial snapshot (instead of a global snapshot). However, if two or more partial snapshot algorithms are concurrently executed, and their snapshot domains overlap, they should coordinate, so that the partial snapshots (taken by the algorithms) are consistent. In this paper, we propose a new efficient partial snapshot algorithm with the aim of reducing communication for the coordination. In a simulation, we show that the proposed algorithm drastically outperforms the existing partial snapshot algorithm, in terms of message and time complexity.
Variational Rejection Particle Filtering
Venue: arXiv cs.LG 2021
Year: 2021

We present a variational inference (VI) framework that unifies and leverages sequential Monte-Carlo (particle filtering) with \emph{approximate} rejection sampling to construct a flexible family of variational distributions. Furthermore, we augment this approach with a resampling step via Bernoulli race, a generalization of a Bernoulli factory, to obtain a low-variance estimator of the marginal likelihood. Our framework, Variational Rejection Particle Filtering (VRPF), leads to novel variational bounds on the marginal likelihood, which can be optimized efficiently with respect to the variational parameters and generalizes several existing approaches in the VI literature. We also present theoretical properties of the variational bound and demonstrate experiments on various models of sequential data, such as the Gaussian state-space model and variational recurrent neural net (VRNN), on which VRPF outperforms various existing state-of-the-art VI methods.