Reviewerly/exHarmony huggingface dataset:
On the Effect of Standing and Seated Viewing of 360° Videos on Subjective Quality Assessment: A Pilot Study

Due to the advances in head-mounted displays (HMDs), hardware and software technologies, and mobile connectivity, virtual reality (VR) applications such as viewing 360° videos on HMDs have seen an increased interest in a wide range of consumer and vertical markets. Quality assessment of digital media systems and services related to immersive visual stimuli has been one of the challenging problems of multimedia signal processing. Specifically, subjective quality assessment of 360° videos presented on HMDs is needed to obtain a ground truth on the visual quality as perceived by humans. Standardized test methodologies to assess the subjective quality of 360° videos on HMDs are currently not as developed as for conventional videos and are subject to further study. In addition, subjective tests related to quality assessment of 360° videos are commonly conducted with participants seated on a chair but neglect other options of consumption such as standing viewing. In this paper, we compare the effect that standing and seated viewing of 360° videos on an HMD has on subjective quality assessment. A pilot study was conducted to obtain psychophysical and psychophysiological data that covers explicit and implicit responses of the participants to the shown 360° video stimuli with different quality levels. The statistical analysis of the data gathered in the pilot study is reported in terms of average rating times, mean opinion scores, standard deviation of opinion scores, head movements, pupil diameter, galvanic skin response (GSR), and simulator sickness scores. The results indicate that the average rating times consumed for 360° video quality assessment are similar for standing and seated viewing. Further, the participants showed higher resolving power among different 360° video quality levels and were more confident about the given opinion scores for seated viewing. On the other hand, a larger scene exploration of 360° videos was observed for standing viewing which appears to distract from the quality assessment task. A slightly higher pupil dilation was recorded for standing viewing which suggests a slightly more immersed experience compared to seated viewing. GSR data indicate a lower degree of emotional arousal in seated viewing which seems to allow the participants to better conduct the quality assessment task. Similarly, simulator sickness symptoms are kept significantly lower when seated. The pilot study also contributes to a holistic view of subjective quality assessment and provides indicative ground truth that can guide the design of large-scale subjective tests.
Adversarial Disentanglement and Correlation Network for Rgb-Infrared Person Re-Identification

RGB-infrared person re-identification is a challenging task for intelligent video surveillance. Compared to traditional person re-identification, it concerns the additional modality discrepancy between RGB and infrared images originated from the different imaging processes of spectrum cameras, as well as the pedestrian’s appearance discrepancy. In this work, we propose a novel Adversarial Disentanglement and Correlation Network (ADCNet) towards learning modality-invariant and discriminative representations of pedestrians for RGB-infrared person re-identification. ADCNet consists of a feature disentanglement network and a feature alignment network. The feature disentanglement network is designed with an auto-encoder and an adversarial learning module to unify the representations for images across modalities, and the feature alignment network is developed with multiple second-order correlation blocks to employ the second-order non-local position-wise operations for refining the representations and shrinking the intra-modality variations. Extensive experimental results on two challenging benchmarks have demonstrated the effectiveness of the proposed method.
Degrade Is Upgrade: Learning Degradation for Low-Light Image Enhancement

Low-light image enhancement aims to improve an image's visibility while keeping its visual naturalness. Different from existing methods, which tend to accomplish the relighting task directly, we investigate the intrinsic degradation and relight the low-light image while refining the details and color in two steps. Inspired by the color image formulation (diffuse illumination color plus environment illumination color), we first estimate the degradation from low-light inputs to simulate the distortion of environment illumination color, and then refine the content to recover the loss of diffuse illumination color. To this end, we propose a novel Degradation-to-Refinement Generation Network (DRGN). Its distinctive features can be summarized as 1) A novel two-step generation network for degradation learning and content refinement. It is not only superior to one-step methods, but also capable of synthesizing sufficient paired samples to benefit the model training; 2) A multi-resolution fusion network to represent the target information (degradation or contents) in a multi-scale cooperative manner, which is more effective to address the complex unmixing problems. Extensive experiments on both the enhancement task and the joint detection task have verified the effectiveness and efficiency of our proposed method, surpassing the SOTA by 1.59dB on average and 3.18\% in mAP on the ExDark dataset. The code will be available soon.
Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue Generation

Visual dialogue is a challenging task since it needs to answer a series of coherent questions on the basis of understanding the visual environment.Previous studies focus on the implicit exploration of multimodal coreference by implicitly attending to spatial image features or object-level image features but neglect the importance of locating the objects explicitly in the visual content, which is associated with entities in the textual content.Therefore, in this paper we propose a Multimodal Incremental Transformer with Visual Grounding, named MITVG, which consists of two key parts: visual grounding and multimodal incremental transformer.Visual grounding aims to explicitly locate related objects in the image guided by textual entities, which helps the model exclude the visual content that does not need attention.On the basis of visual grounding, the multimodal incremental transformer encodes the multi-turn dialogue history combined with visual scene step by step according to the order of the dialogue and then generates a contextually and visually coherent response.Experimental results on the VisDial v0.9 and v1.0 datasets demonstrate the superiority of the proposed model, which achieves comparable performance.
Proof of Learning (PoLe): Empowering Machine Learning with Consensus Building on Blockchains (Demo)

The consensus algorithm is the core component of a blockchain system, which determines the efficiency, security, and scalability of the blockchain network. The representative consensus algorithm is the proof of work (PoW) proposed in Bitcoin, where the consensus process consumes large amount of compute in solving meaningless Hash puzzel. Meanwhile, the deep learning (DL) has brought unprecedented performance gains at heavy computate cost. In this demo, we channels the otherwise wasted computational power to the practical purpose of training neural network models, through the proposed proof of learning (PoL) consensus algorithm. In PoLe, the training/testing data are released to the entire blockchain network (BCN) and the consensus nodes train NN models on the data, which serves as the proof of learning. When the consensus on the BCN considers a NN model to be valid, a new block is appended to the blockchain. Through our system, we investigate the potential of enpowering machine learning with consensus building on blockchains.
VRU Pose-SSD: Multiperson Pose Estimation For Automated Driving

We present a fast and efficient approach for joint person detection and pose estimation optimized for automated driving (AD) in urban scenarios. We use a multitask weight sharing architecture to jointly train detection and pose estimation. This modular architecture allows us to accommodate different downstream tasks in the future. By systematic large-scale experiments on the Tsinghua-Daimler Urban Pose Dataset (TDUP), we obtain multiple models with varying accuracy-speed trade-offs. We then quantize and optimize our network for deployment and present a detailed analysis of the efficacy of the algorithm. We introduce a two-stage evaluation strategy, which is more suitable for AD and achieve a significant performance improvement in comparison to state-of-the-art approaches. Our optimized model runs at 52~fps on full HD images and still reaches a competitive performance of 32.25~LAMR. We are confident that our work serves as an enabler to tackle higher-level tasks like VRU intention estimation and gesture recognition, which rely on stable pose estimates and will play a crucial role in future AD systems.
PNNPU: A 11.9 TOPS/W High-speed 3D Point Cloud-based Neural Network Processor with Block-based Point Processing for Regular DRAM Access

An efficient and high-speed 3D point cloud-based neural network processing unit (PNNPU) is proposed using the block-based point processing. It has three key features: 1) page-based point block memory management unit (PMMU) with linked list-based page table (LLPT) for on-chip memory footprint reduction, 2) hierarchical block-wise farthest point sampling (HFPS), and block skipping ball-query (BSBQ) for fast and efficient point processing, 3) Skipping-based max-pooling prediction (SMPP) for throughput enhancement. The PNNPU is fabricated in 65nm CMOS process and evaluated on the 3D object detection (3D OD) application. As a result, it shows 84.8 fps at 266.8mW power consumption and achieving 6.6-11.9 TOPS/W energy efficiency.
SPPS: A Search Pattern Privacy System for Approximate Shortest Distance Query of Encrypted Graphs in IIoT

In recent years, Industrial Internet of Things (IIoT) has gradually attracted the attention of the industry owing to its accurate time synchronization, communication accuracy, and high adaptability. As an important data structure, graphs are widely used in IIoT applications, where entities and their relationships can be expressed in the form of graphs. With the widespread adoption of IIoT and cloud computing, an increasing number of individuals or organizations are outsourcing their IIoT graph data to cloud servers to enjoy the unlimited storage space and fast computing service. To protect the privacy of graph data, graphs are usually encrypted before being outsourced. In this article, we propose a search pattern privacy system for approximate shortest distance query of encrypted graphs in IIoT. To realize search pattern privacy, we adopt two noncolluded cloud servers to accomplish different tasks. We leverage the first server to store the encrypted data and perform query operations, and use the second one to rerandomize the contents and shuffle the locations of the queried records. Before queries, we generate the trapdoors by using different random numbers. After queries, we ask the second server to rerandomize the contents of the records that the first server touched. In addition, we shuffle the physical locations of original records by inserting some fake records. In this way, all contents and physical locations of the touched records change, so that the first server cannot distinguish whether two queries are the same or not. To enhance the efficiency on the user side, we further improve this system by moving some heavy workloads from the user to the cloud. The security analysis and the performance evaluation show that our work is secure and efficient.
Interpretable Compositional Convolutional Neural Networks

This paper proposes a method to modify a traditional convolutional neural network (CNN) into an interpretable compositional CNN, in order to learn filters that encode meaningful visual patterns in intermediate convolutional layers. In a compositional CNN, each filter is supposed to consistently represent a specific compositional object part or image region with a clear meaning. The compositional CNN learns from image labels for classification without any annotations of parts or regions for supervision. Our method can be broadly applied to different types of CNNs. Experiments have demonstrated the effectiveness of our method. The code will be released when the paper is accepted.
A Payload Based Malicious HTTP Traffic Detection Method Using Transfer Semi-Supervised Learning

Malicious HTTP traffic detection plays an important role in web application security. Most existing work applies machine learning and deep learning techniques to build the malicious HTTP traffic detection model. However, they still suffer from the problems of huge training data collection cost and low cross-dataset generalization ability. Aiming at these problems, this paper proposes DeepPTSD, a deep learning method for payload based malicious HTTP traffic detection. First, it treats the malicious HTTP traffic detection as a text classification problem and trains the initial detection model using TextCNN on a public dataset, and then adapts the initial detection model to the target dataset based on a transfer learning algorithm. Second, in the transfer learning procedure, it uses a semi-supervised learning algorithm to accomplish the model adaptation task. The semi-supervised learning algorithm enhances the target dataset based on a HTTP payload data augmentation mechanism to exploit both the labeled and unlabeled data. We evaluate DeepPTSD on two real HTTP traffic datasets. The results show that DeepPTSD has competitive performance under the small data condition.
Scalability improvement and analysis of permissioned-blockchain

Through the Bitcoin application, the innovative technology was miraculously launched in the markets, influencing numerous industries. Bitcoin is nothing but a form of digital currency (cryptocurrency) that can be used for trading in place of fiat money, where the underlying infrastructure is called Blockchain. The Blockchain is an open ledger that provides decentralization, transparency, immutability, and confidentiality. Blockchain can be used in massive, beneficial applications such as healthcare, logistics, supply chain management, the Internet of Things (IoT), etc. Most of the industrial applications rely on the permissioned Blockchain. However, the permissioned Blockchain fails in some aspects, such as scalability and throughput. This paper suggests a system to solve the scalability issue of permissioned Blockchain by incorporating data science techniques. The scalability analysis of the proposed solution is done in the hyperledger fabric framework with a variable number of transactions and results in scalability improvement.
Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech

Whilst recent neural text-to-speech (TTS) approaches produce high-quality speech, they typically require a large amount of recordings from the target speaker. In previous work, a 3-step method was proposed to generate high-quality TTS while greatly reducing the amount of data required for training. However, we have observed a ceiling effect in the level of naturalness achievable for highly expressive voices when using this approach. In this paper, we present a method for building highly expressive TTS voices with as little as 15 minutes of speech data from the target speaker. Compared to the current state-of-the-art approach, our proposed improvements close the gap to recordings by 23.3% for naturalness of speech and by 16.3% for speaker similarity. Further, we match the naturalness and speaker similarity of a Tacotron2-based full-data (~10 hours) model using only 15 minutes of target speaker data, whereas with 30 minutes or more, we significantly outperform it. The following improvements are proposed: 1) changing from an autoregressive, attention-based TTS model to a non-autoregressive model replacing attention with an external duration model and 2) an additional Conditional Generative Adversarial Network (cGAN) based fine-tuning step.
AlloST: Low-Resource Speech Translation Without Source Transcription

The end-to-end architecture has made promising progress in speech translation (ST). However, the ST task is still challenging under low-resource conditions. Most ST models have shown unsatisfactory results, especially in the absence of word information from the source speech utterance. In this study, we survey methods to improve ST performance without using source transcription, and propose a learning framework that utilizes a language-independent universal phone recognizer. The framework is based on an attention-based sequence-to-sequence model, where the encoder generates the phonetic embeddings and phone-aware acoustic representations, and the decoder controls the fusion of the two embedding streams to produce the target token sequence. In addition to investigating different fusion strategies, we explore the specific usage of byte pair encoding (BPE), which compresses a phone sequence into a syllable-like segmented sequence. Due to the conversion of symbols, a segmented sequence represents not only pronunciation but also language-dependent information lacking in phones. Experiments conducted on the Fisher Spanish-English and Taigi-Mandarin drama corpora show that our method outperforms the conformer-based baseline, and the performance is close to that of the existing best method using source transcription.
COVID-19 Impact on Education and Work in the Kingdom of Bahrain: Survey Study

The Coronavirus (COVID-19) pandemic has disrupted teaching and work in the entire world. The COVID-19 reshaped the management of education and work leading to a rethinking of new approaches in the education system or the business environment. A lot of studies and reports were published over the world analyzing and discussing the current situation and looking for solutions and alternatives to overcome this crisis. This paper discusses the impact of the COVID-19 pandemic in education and work environments in the Kingdom of Bahrain using the survey method. Also, it reviewed the precautionary measures that were taken by the Kingdom to confront this dangerous pandemic. In addition, it discussed the use of digital tools and technology, which will help to overcome the problems initiated during the pandemic. An online survey was performed by distributing 12 research questions to different people in the Kingdom of Bahrain. 210 answered the questionnaire, and the data were analyzed with statistical software. Overall findings of this study indicated that the online learning transition in the Kingdom of Bahrain was successful because most of the respondents agreed with the use of online learning. Moreover, the respondents who had jobs revealed that COVID-19 affected their work. © 2021 NSP Natural Sciences Publishing Cor.
Performance Analysis of Fully Actuated Multirotor Unmanned Aerial Vehicle Configurations with Passively Tilted Rotors

Unmanned aerial vehicles (UAVs) nowadays represent an interesting tool capable of performing various missions. The multirotor type of UAV is proven to be a potential solution in missions that require precise movements, such as environmental objects manipulation. In this paper, a procedure for the performance analysis of fully actuated multirotor UAV configurations is proposed. For this purpose, a configuration is described by a control allocation scheme and implemented in the software package which enables the analysis and control implementation of a real system. The parameter analysis of the passively tilted multirotor configurations is performed based on the characteristics of the electric propulsion units, and the allocation of propulsion forces is graphically shown. The results of the proposed procedure provide an insight into the capabilities of configurations and can ultimately be used to select the propulsion system components and parameters according to the requirements and constraints associated with the specific mission profile. An experimental aircraft was built, and custom firmware was created, which enable us to experimentally prove the feasibility of fully actuated and passively tilted configurations.
Contrastive Quantization with Code Memory for Unsupervised Image Retrieval

The high efficiency in computation and storage makes hashing (including binary hashing and quantization) a common strategy in large-scale retrieval systems. To alleviate the reliance on expensive annotations, unsupervised deep hashing becomes an important research problem. This paper provides a novel solution to unsupervised deep quantization, namely Contrastive Quantization with Code Memory (MeCoQ). Different from existing reconstruction-based strategies, we learn unsupervised binary descriptors by contrastive learning, which can better capture discriminative visual semantics. Besides, we uncover that codeword diversity regularization is critical to prevent contrastive learning-based quantization from model degeneration. Moreover, we introduce a novel quantization code memory module that boosts contrastive learning with lower feature drift than conventional feature memories. Extensive experiments on benchmark datasets show that MeCoQ outperforms state-of-the-art methods. Code and configurations are publicly released.
FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding

The few-shot natural language understanding (NLU) task has attracted much recent attention.However, prior methods have been evaluated under a disparate set of protocols, which hinders fair comparison and measuring progress of the field.To address this issue, we introduce an evaluation framework that improves previous evaluation procedures in three key aspects, i.e., test performance, dev-test correlation, and stability.Under this new evaluation framework, we re-evaluate several stateof-the-art few-shot methods for NLU tasks.Our framework reveals new insights: (1) both the absolute performance and relative gap of the methods were not accurately estimated in prior literature; (2) no single method dominates most tasks with consistent performance;(3) improvements of some methods diminish with a larger pretrained model; and (4) gains from different methods are often complementary and the best combined model performs close to a strong fully-supervised baseline.We open-source our toolkit, FewNLU, that implements our evaluation framework along with a number of state-of-the-art methods.
Nonadiabatic quantum state engineering by time-dependent decoherence-free subspaces in open quantum systems

We extend the theory of quantum state engineering (QSE) to the open quantum systems which simultaneously suffer time-dependent decoherence and time-independent decoherence. A hybrid approach is proposed to effectively realize QSE by combination of time-dependent decoherence-free subspaces and shortcuts to adiabaticity theories. As an application, a concrete single-qubit QSE example with an open five-level system is presented. Numerical simulations confirm that the approach is reliable and robust against both of time-dependent decoherence and time-independent decoherence. Certain dissipation effects are no longer harmful but play a positive role in the QSE. In addition, the approach can be generalized to realize QSE in high-dimensional quantum systems. Therefore, this proposal is quite useful for quantum computation and quantum information processing in open systems.