OpenRaiser/Intern-Atlas huggingface dataset:
Past, Present and Future of Computational Storage: A Survey
Venue: arXiv cs.DC 2021
Year: 2021

We live in a data-centric world where we are heading to generate close to 200 Zettabytes of data by the year 2025. Our data processing requirements have also increased as we push to build data processing frameworks that can process large volumes of data in a short duration, a few milli- and even micro-seconds. In the prevalent computer systems designs, data is stored passively in storage devices which is brought in for processing and then the results are written out. As the volume of data explodes this constant data movement has led to a "data movement wall" which hinders further process and optimizations in data processing systems designs. One promising alternative to this architecture is to push computation to the data (instead of the other way around), and design a computational-storage device or CSD. The idea of CSD is not new and can trace its root to the pioneering work done in the 1970s and 1990s. More recently, with the emergence of non-volatile memory (NVM) storage in the mainstream computing (e.g., NAND flash and Optane), the idea has again gained a lot of traction with multiple academic and commercial prototypes being available now. In this brief survey we present a systematic analysis of work done in the area of computation storage and present future directions.
Optimization Study of Hydraulic Power Take-off System for an Ocean Wave Energy Converter
Venue: arXiv cs.NE 2021
Year: 2021

Ocean wave renewable energy is fast becoming a key part of renewable energy industries over the recent decades. By developing wave energy converters as the main converter technology in this process, their power take-off (PTO) systems have been investigated. Adjusting PTO parameters is a challenging optimization problem because there is a complex and nonlinear relationship between these parameters and the absorbed power output. In this regard, this study aims to optimize the PTO system parameters of a point absorber wave energy converter in the wave scenario in Perth, on Western Australian coasts. The converter is numerically designed to oscillate against irregular and multi-dimensional waves and sensitivity analysis for PTO settings is performed. Then, to find the optimal PTO system parameters which lead to the highest power output, ten optimization algorithms are incorporated to solve the non-linear problem, Including Nelder-Mead search method, Active-set method, Sequential quadratic Programming method (SQP), Multi-Verse Optimizer (MVO), and six modified combination of Genetic, Surrogate and fminsearch algorithms. After a feasibility landscape analysis, the optimization outcome is carried out and gives us the best answer in terms of PTO system settings. Finally, the investigation shows that the modified combinations of Genetic, Surrogate, and fminsearch algorithms can outperform the others in the studied wave scenario, as well as the interaction between PTO system variables.
Gradient-based Novelty Detection Boosted by Self-supervised Binary Classification
Venue: arXiv cs.LG 2021
Year: 2021

Novelty detection aims to automatically identify out-of-distribution (OOD) data, without any prior knowledge of them. It is a critical step in data monitoring, behavior analysis and other applications, helping enable continual learning in the field. Conventional methods of OOD detection perform multi-variate analysis on an ensemble of data or features, and usually resort to the supervision with OOD data to improve the accuracy. In reality, such supervision is impractical as one cannot anticipate the anomalous data. In this paper, we propose a novel, self-supervised approach that does not rely on any pre-defined OOD data: (1) The new method evaluates the Mahalanobis distance of the gradients between the in-distribution and OOD data. (2) It is assisted by a self-supervised binary classifier to guide the label selection to generate the gradients, and maximize the Mahalanobis distance. In the evaluation with multiple datasets, such as CIFAR-10, CIFAR-100, SVHN and TinyImageNet, the proposed approach consistently outperforms state-of-the-art supervised and unsupervised methods in the area under the receiver operating characteristic (AUROC) and area under the precision-recall curve (AUPR) metrics. We further demonstrate that this detector is able to accurately learn one OOD class in continual learning.
Anomaly Discovery in Semantic Segmentation via Distillation Comparison Networks
Venue: arXiv cs.CV 2021
Year: 2021

This paper aims to address the problem of anomaly discovery in semantic segmentation. Our key observation is that semantic classification plays a critical role in existing approaches, while the incorrectly classified pixels are easily regarded as anomalies. Such a phenomenon frequently appears and is rarely discussed, which significantly reduces the performance of anomaly discovery. To this end, we propose a novel Distillation Comparison Network (DiCNet). It comprises of a teacher branch which is a semantic segmentation network that removed the semantic classification head, and a student branch that is distilled from the teacher branch through a distribution distillation. We show that the distillation guarantees the semantic features of the two branches hold consistency in the known classes, while reflect inconsistency in the unknown class. Therefore, we leverage the semantic feature discrepancy between the two branches to discover the anomalies. DiCNet abandons the semantic classification head in the inference process, and hence significantly alleviates the issue caused by incorrect semantic classification. Extensive experimental results on StreetHazards dataset and BDD-Anomaly dataset are conducted to verify the superior performance of DiCNet. In particular, DiCNet obtains a 6.3% improvement in AUPR and a 5.2% improvement in FPR95 on StreetHazards dataset, achieves a 4.2% improvement in AUPR and a 6.8% improvement in FPR95 on BDD-Anomaly dataset. Codes are available at https://github.com/zhouhuan-hust/DiCNet.
Decentralizing Information Technology: The Advent of Resource Based Systems
Venue: arXiv cs.CR 2021
Year: 2021

The growth of the Bitcoin network during the first decade of its operation to a global scale system is a singular event in the deployment of Information Technology systems. Can this approach serve as a wider paradigm for Information Technology services beyond the use case of digital currencies? We investigate this question by introducing the concept of resource based systems and their four fundamental characteristics: (i) resource-based operation, (ii) tokenomics, (iii) decentralized service provision, and (iv) rewards sharing. We explore these characteristics, identify design goals and challenges and investigate some crucial game theoretic aspects of reward sharing that can be decisive for their effective operation.
Learning-based methods to model small body gravity fields for proximity operations: Safety and Robustness
Venue: arXiv cs.RO 2021
Year: 2021

Accurate gravity field models are essential for safe proximity operations around small bodies. State-of-the-art techniques use spherical harmonics or high-fidelity polyhedron shape models. Unfortunately, these techniques can become inaccurate near the surface of the small body or have high computational costs, especially for binary or heterogeneous small bodies. New learning-based techniques do not encode a predefined structure and are more versatile. In exchange for versatility, learning-based techniques can be less robust outside the training data domain. In deployment, the spacecraft trajectory is the primary source of dynamics data. Therefore, the training data domain should include spacecraft trajectories to accurately evaluate the learned model's safety and robustness. We have developed a novel method for learning-based gravity models that directly uses the spacecraft's past trajectories. We further introduce a method to evaluate the safety and robustness of learning-based techniques via comparing accuracy within and outside of the training domain. We demonstrate this safety and robustness method for two learning-based frameworks: Gaussian processes and neural networks. Along with the detailed analysis provided, we empirically establish the need for robustness verification of learned gravity models when used for proximity operations.
Controlling the Quality of Distillation in Response-Based Network Compression
Venue: arXiv cs.CV 2021
Year: 2021

The performance of a distillation-based compressed network is governed by the quality of distillation. The reason for the suboptimal distillation of a large network (teacher) to a smaller network (student) is largely attributed to the gap in the learning capacities of given teacher-student pair. While it is hard to distill all the knowledge of a teacher, the quality of distillation can be controlled to a large extent to achieve better performance. Our experiments show that the quality of distillation is largely governed by the quality of teacher's response, which in turn is heavily affected by the presence of similarity information in its response. A well-trained large capacity teacher loses similarity information between classes in the process of learning fine-grained discriminative properties for classification. The absence of similarity information causes the distillation process to be reduced from one example-many class learning to one example-one class learning, thereby throttling the flow of diverse knowledge from the teacher. With the implicit assumption that only the instilled knowledge can be distilled, instead of focusing only on the knowledge distilling process, we scrutinize the knowledge inculcation process. We argue that for a given teacher-student pair, the quality of distillation can be improved by finding the sweet spot between batch size and number of epochs while training the teacher. We discuss the steps to find this sweet spot for better distillation. We also propose the distillation hypothesis to differentiate the behavior of the distillation process between knowledge distillation and regularization effect. We conduct all our experiments on three different datasets.
Reasoning Structural Relation for Occlusion-Robust Facial Landmark Localization
Venue: arXiv cs.CV 2021
Year: 2021

In facial landmark localization tasks, various occlusions heavily degrade the localization accuracy due to the partial observability of facial features. This paper proposes a structural relation network (SRN) for occlusion-robust landmark localization. Unlike most existing methods that simply exploit the shape constraint, the proposed SRN aims to capture the structural relations among different facial components. These relations can be considered a more powerful shape constraint against occlusion. To achieve this, a hierarchical structural relation module (HSRM) is designed to hierarchically reason the structural relations that represent both long- and short-distance spatial dependencies. Compared with existing network architectures, HSRM can efficiently model the spatial relations by leveraging its geometry-aware network architecture, which reduces the semantic ambiguity caused by occlusion. Moreover, the SRN augments the training data by synthesizing occluded faces. To further extend our SRN for occluded video data, we formulate the occluded face synthesis as a Markov decision process (MDP). Specifically, it plans the movement of the dynamic occlusion based on an accumulated reward associated with the performance degradation of the pre-trained SRN. This procedure augments hard samples for robust facial landmark tracking. Extensive experimental results indicate that the proposed method achieves outstanding performance on occluded and masked faces. Code is available at https://github.com/zhuccly/SRN.
Investigation of Densely Connected Convolutional Networks with Domain Adversarial Learning for Noise Robust Speech Recognition
Venue: arXiv cs.CL 2021
Year: 2021

We investigate densely connected convolutional networks (DenseNets) and their extension with domain adversarial training for noise robust speech recognition. DenseNets are very deep, compact convolutional neural networks which have demonstrated incredible improvements over the state-of-the-art results in computer vision. Our experimental results reveal that DenseNets are more robust against noise than other neural network based models such as deep feed forward neural networks and convolutional neural networks. Moreover, domain adversarial learning can further improve the robustness of DenseNets against both, known and unknown noise conditions.
An Experimental and Comparative Benchmark Study Examining Resource Utilization in Managed Hadoop Context
Venue: arXiv cs.DC 2021
Year: 2021

Transitioning cloud-based Hadoop from IaaS to PaaS, which are commercially conceptualized as pay-as-you-go or pay-per-use, often reduces the associated system costs. However, managed Hadoop systems do present a black-box behavior to the end-users who cannot be clear on the inner performance dynamics, hence, on the benefits of leveraging them. In the study, we aimed to understand managed Hadoop context in terms of resource utilization. We utilized three experimental Hadoop-on-PaaS proposals as they come out-of-the-box and conducted Hadoop specific workloads of the HiBench Benchmark Suite. During the benchmark executions, we collected system resource utilization data on the worker nodes. The results indicated that the same property specifications among cloud services do not guarantee nearby performance outputs, nor consistent results within themselves. We assume that the managed systems' architectures and pre-configurations play a significant role in the performance.
Modelling of Received Signals in Molecular Communication Systems based machine learning: Comparison of azure machine learning and Python tools
Venue: arXiv cs.LG 2021
Year: 2021

Molecular communication (MC) implemented on Nano networks has extremely attractive characteristics in terms of energy efficiency, dependability, and robustness. Even though, the impact of incredibly slow molecule diffusion and high variability environments remains unknown. Analysis and designs of communication systems usually rely on developing mathematical models that describe the communication channel. However, the underlying channel models are unknown in some systems, such as MC systems, where chemical signals are used to transfer information. In these cases, a new method to analyze and design is needed. In this paper, we concentrate on one critical aspect of the MC system, modelling MC received signal until time t , and demonstrate that using tools from ML makes it promising to train detectors that can be executed well without any information about the channel model. Machine learning (ML) is one of the intelligent methodologies that has shown promising results in the domain. This paper applies Azure Machine Learning (Azure ML) for flexible pavement maintenance regressions problems and solutions. For prediction, four parameters are used as inputs: the receiver radius, transmitter radius, distance between receiver and transmitter, and diffusion coefficient, while the output is mAP (mean average precision) of the received signal. Azure ML enables algorithms that can learn from data and experiences and accomplish tasks without having to be coded. In the established Azure ML, the regression algorithms such as, boost decision tree regression, Bayesian linear regression, neural network, and decision forest regression are selected. The best performance is chosen as an optimality criterion. Finally, a comparison that shows the potential benefits of Azure ML tool over programmed based tool (Python), used by developers on local PCs, is demonstrated
SSDNet: State Space Decomposition Neural Network for Time Series Forecasting
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we present SSDNet, a novel deep learning approach for time series forecasting. SSDNet combines the Transformer architecture with state space models to provide probabilistic and interpretable forecasts, including trend and seasonality components and previous time steps important for the prediction. The Transformer architecture is used to learn the temporal patterns and estimate the parameters of the state space model directly and efficiently, without the need for Kalman filters. We comprehensively evaluate the performance of SSDNet on five data sets, showing that SSDNet is an effective method in terms of accuracy and speed, outperforming state-of-the-art deep learning and statistical methods, and able to provide meaningful trend and seasonality components.
Active Reconfigurable Intelligent Surface Aided Secure Transmission
Venue: arXiv cs.IT 2021
Year: 2021

Reconfigurable Intelligent Surface (RIS) draws great attentions in academic and industry due to its passive and low power consumption nature, and has currently been used in physical layer security to enhance the secure transmission. However, due to the existence of double fading effect on the reflecting channel link between transmitter and user, RIS helps achieve limited secrecy performance gain compared with the case without RIS. In this correspondence, we propose a novel active RIS design to enhance the secure wireless transmission, where the reflecting elements in RIS not only adjust the phase shift but also amplify the amplitude of signals. To solve the non convex secrecy rate optimization based on this design, an efficient alternating optimization algorithm is proposed to jointly optimize the beamformer at transmitter and reflecting coefficient matrix at RIS. Simulation results show that with the aid of active RIS design, the impact of double fading effect can be effectively relieved, resulting in a significantly higher secrecy performance gain compared with existing solutions with passive RIS and without RIS design.
May the Force Be with Your Copy Mechanism: Enhanced Supervised-Copy Method for Natural Language Generation
Venue: arXiv cs.CL 2021
Year: 2021

Recent neural sequence-to-sequence models with a copy mechanism have achieved remarkable progress in various text generation tasks. These models addressed out-of-vocabulary problems and facilitated the generation of rare words. However, the identification of the word which needs to be copied is difficult, as observed by prior copy models, which suffer from incorrect generation and lacking abstractness. In this paper, we propose a novel supervised approach of a copy network that helps the model decide which words need to be copied and which need to be generated. Specifically, we re-define the objective function, which leverages source sequences and target vocabularies as guidance for copying. The experimental results on data-to-text generation and abstractive summarization tasks verify that our approach enhances the copying quality and improves the degree of abstractness.
Multimodal Adversarially Learned Inference with Factorized Discriminators
Venue: arXiv cs.LG 2021
Year: 2021

Learning from multimodal data is an important research topic in machine learning, which has the potential to obtain better representations. In this work, we propose a novel approach to generative modeling of multimodal data based on generative adversarial networks. To learn a coherent multimodal generative model, we show that it is necessary to align different encoder distributions with the joint decoder distribution simultaneously. To this end, we construct a specific form of the discriminator to enable our model to utilize data efficiently, which can be trained constrastively. By taking advantage of contrastive learning through factorizing the discriminator, we train our model on unimodal data. We have conducted experiments on the benchmark datasets, whose promising results show that our proposed approach outperforms the-state-of-the-art methods on a variety of metrics. The source code will be made publicly available.
A Novel Iterative Soft-Decision Decoding Algorithm for RS-SPC Product Codes
Venue: arXiv cs.IT 2021
Year: 2021

This paper presents a generalized construction of RS-SPC product codes. A low-complexity joint-decoding scheme is proposed for these codes, in which a BP-based iterative decoding is performed based on the binary expansion of the whole parity-check matrix. Various powerful RS codes can be used as the component codes for RS-SPC product codes, which gives a good performance for local decoding (decode a single component codeword). The proposed BP-based iterative decoding is a global decoding, and it achieves an error-correcting capability comparable to codes of large blocklengths. This two-phase decoding scheme preserves the low decoding latency and complexity of the local decoding while achieves high reliability through the global decoding. The complexity of the proposed iterative decoding is discussed, and the simulation results show the proposed scheme offers a good trade-off between the complexity and the error performance.
Relational Models of Microarchitectures for Formal Security Analyses
Venue: arXiv cs.CR 2021
Year: 2021

There is a growing need for hardware-software contracts which precisely define the implications of microarchitecture on software security-i.e., security contracts. It is our view that such contracts should explicitly account for microarchitecture-level implementation details that underpin hardware leakage, thereby establishing a direct correspondence between a contract and the microarchitecture it represents. At the same time, these contracts should remain as abstract as possible so as to support efficient formal analyses. With these goals in mind, we propose leakage containment models (LCMs)-novel axiomatic security contracts which support formally reasoning about the security guarantees of programs when they run on particular microarchitectures. Our core contribution is an axiomatic vocabulary for formally defining LCMs, derived from the established axiomatic vocabulary used to formalize processor memory consistency models. Using this vocabulary, we formalize microarchitectural leakage-focusing on leakage through hardware memory systems-so that it can be automatically detected in programs. To illustrate the efficacy of LCMs, we present two case studies. First, we demonstrate that our leakage definition faithfully captures a sampling of (transient and non-transient) microarchitectural attacks from the literature. Second, we develop a static analysis tool based on LCMs which automatically identifies Spectre vulnerabilities in programs and scales to analyze realistic-sized codebases, like libsodium.
Object Recognition as Classification via Visual Properties
Venue: arXiv cs.CV 2021
Year: 2021

We base our work on the teleosemantic modelling of concepts as abilities implementing the distinct functions of recognition and classification. Accordingly, we model two types of concepts - substance concepts suited for object recognition exploiting visual properties, and classification concepts suited for classification of substance concepts exploiting linguistically grounded properties. The goal in this paper is to demonstrate that object recognition can be construed as classification via visual properties, as distinct from work in mainstream computer vision. Towards that, we present an object recognition process based on Ranganathan's four-phased faceted knowledge organization process, grounded in the teleosemantic distinctions of substance concept and classification concept. We also briefly introduce the ongoing project MultiMedia UKC, whose aim is to build an object recognition resource following our proposed process