Reviewerly/exHarmony huggingface dataset:
On Recognizing Texts of Arbitrary Shapes with 2D Self-Attention

Scene text recognition (STR) is the task of recognizing character sequences in natural scenes. While there have been great advances in STR methods, current methods which convert two-dimensional (2D) image to one-dimensional (1D) feature map still fail to recognize texts in arbitrary shapes, such as heavily curved, rotated or vertically aligned texts, which are abundant in daily life (e.g. restaurant signs, product labels, company logos, etc). This paper introduces an architecture to recognize texts of arbitrary shapes, named Self-Attention Text Recognition Network (SATRN). SATRN utilizes the self-attention mechanism, which is originally proposed to capture the dependency between word tokens in a sentence, to describe 2D spatial dependencies of characters in a scene text image. Exploiting the full-graph propagation of self-attention, SATRN can recognize texts with arbitrary arrangements and large inter-character spacing. As a result, our model outperforms all existing STR models by a large margin of 4.5pp on average in "irregular text" benchmarks and also achieved state-of-the-art performance in two "regular text" benchmarks. We provide empirical analyses that illustrate the inner mechanisms and the extent to which the model is applicable (e.g. rotated and multi-line text). We will open- source the code. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Spherical Space Domain Adaptation With Robust Pseudo-Label Loss

Adversarial domain adaptation (DA) has been an effective approach for learning domain-invariant features by adversarial training. In this paper, we propose a novel adversarial DA approach completely defined in spherical feature space, in which we define spherical classifier for label prediction and spherical domain discriminator for discriminating domain labels. To utilize pseudo-label robustly, we develop a robust pseudo-label loss in the spherical feature space, which weights the importance of estimated labels of target data by posterior probability of correct labeling, modeled by Gaussian-uniform mixture model in spherical feature space. Extensive experiments show that our method achieves state-of-the-art results, and also confirm effectiveness of spherical classifier, spherical discriminator and spherical robust pseudo-label loss.
How virtual reality affects consumer choice

With high-immersive virtual reality (VR) systems approaching mass markets, companies are seeking to better understand how consumers behave when shopping in VR. A key feature of high-immersive VR environments is that they can create a strong illusion of reality to the senses, which could substantially change consumer choice behavior compared to online shopping. We compare consumer choice from virtual shelves in two environments: (i) a high-immersive VR environment using a head-mounted display and hand-held controllers with (ii) a low-immersive environment showing products as rotatable 3-D models on a desktop computer screen. We use an incentive-aligned choice experiment to investigate how immersion affects consumer choice. Our investigation comprises three key choice characteristics: variety-seeking, price-sensitivity, and satisfaction with the choice made. The empirical results provide evidence that consumers in high-immersive VR choose a larger variety of products and are less price-sensitive. Choice satisfaction, however, did not increase in high-immersive VR.
Creating Something From Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing

In recent years, cross-modal hashing (CMH) has attracted increasing attentions, mainly because its potential ability of mapping contents from different modalities, especially in vision and language, into the same space, so that it becomes efficient in cross-modal data retrieval. There are two main frameworks for CMH, differing from each other in whether semantic supervision is required. Compared to the unsupervised methods, the supervised methods often enjoy more accurate results, but require much heavier labors in data annotation. In this paper, we propose a novel approach that enables guiding a supervised method using outputs produced by an unsupervised method. Specifically, we make use of teacher-student optimization for propagating knowledge. Experiments are performed on two popular CMH benchmarks, i.e., the MIRFlickr and NUS-WIDE datasets. Our approach outperforms all existing unsupervised methods by a large margin.
Heterogeneous Knowledge Distillation Using Information Flow Modeling

Knowledge Distillation (KD) methods are capable of transferring the knowledge encoded in a large and complex teacher into a smaller and faster student. Early methods were usually limited to transferring the knowledge only between the last layers of the networks, while latter approaches were capable of performing multi-layer KD, further increasing the accuracy of the student. However, despite their improved performance, these methods still suffer from several limitations that restrict both their efficiency and flexibility. First, existing KD methods typically ignore that neural networks undergo through different learning phases during the training process, which often requires different types of supervision for each one. Furthermore, existing multi-layer KD methods are usually unable to effectively handle networks with significantly different architectures (heterogeneous KD). In this paper we propose a novel KD method that works by modeling the information flow through the various layers of the teacher model and then train a student model to mimic this information flow. The proposed method is capable of overcoming the aforementioned limitations by using an appropriate supervision scheme during the different phases of the training process, as well as by designing and training an appropriate auxiliary teacher model that acts as a proxy model capable of “explaining” the way the teacher works to the student. The effectiveness of the proposed method is demonstrated using four image datasets and several different evaluation setups.
Can Weight Sharing Outperform Random Architecture Search? An Investigation With TuNAS

Efficient Neural Architecture Search methods based on weight sharing have shown good promise in democratizing Neural Architecture Search for computer vision models. There is, however, an ongoing debate whether these efficient methods are significantly better than random search. Here we perform a thorough comparison between efficient and random search methods on a family of progressively larger and more challenging search spaces for image classification and detection on ImageNet and COCO. While the efficacies of both methods are problem-dependent, our experiments demonstrate that there are large, realistic tasks where efficient search methods can provide substantial gains over random search. In addition, we propose and evaluate techniques which improve the quality of searched architectures and reduce the need for manual hyper-parameter tuning.
A Consent Model for Blockchain-Based Health Data Sharing Platforms

In modern healthcare systems, being able to share electronic health records is crucial for providing quality care and for enabling a larger spectrum of health services. Health data sharing is dependent on obtaining individual consent which, in turn, is hindered by a lack of resources. To this extend, blockchain-based platforms facilitate data sharing by inherently creating a trusted distributed network of users. These users are enabled to share their data without depending on the time and resources of specific players (such as the health services). In blockchain-based platforms, data governance mechanisms become very important due to the need to specify and monitor data sharing and data use conditions. In this paper, we present a blockchain-based data sharing consent model for access control over individual health data. We use smart contracts to dynamically represent the individual consent over health data and to enable data requesters to search and access them. The dynamic consent model extends upon two ontologies: the Data Use Ontology (DUO) which models the individual consent of users and the Automatable Discovery and Access Matrix (ADA-M) which describes queries from data requesters. We deploy the model on Ethereum blockchain and evaluate different data sharing scenarios. The contribution of this paper is to create an individual consent model for health data sharing platforms. Such a model guarantees that individual consent is respected and that there is accountability for all the participants in the data sharing platform. The evaluation of our solution indicates that such a data sharing model provides a flexible approach to decide how the data is used by data requesters. Our experimental evaluation shows that the proposed model is efficient and adapts to personalized access control policies in data sharing.
A Fast Defogging Image Recognition Algorithm Based on Bilateral Hybrid Filtering

With the rapid advancement of video and image processing technologies in the Internet of Things, it is urgent to address the issues in real-time performance, clarity, and reliability of image recognition technology for a monitoring system in foggy weather conditions. In this work, a fast defogging image recognition algorithm is proposed based on bilateral hybrid filtering. First, the mathematical model based on bilateral hybrid filtering is established. The dark channel is used for filtering and denoising the defogging image. Next, a bilateral hybrid filtering method is proposed by using a combination of guided filtering and median filtering, as it can effectively improve the robustness and transmittance of defogging images. On this basis, the proposed algorithm dramatically decreases the computation complexity of defogging image recognition and reduces the image execution time. Experimental results show that the defogging effect and speed are promising, with the image recognition rate reaching to 98.8% after defogging.
The monarch butterfly optimization algorithm for solving feature selection problems

Feature selection (FS) is considered to be a hard optimization problem in data mining and some artificial intelligence fields. It is a process where rather than studying all of the features of a whole dataset, some associated features of a problem are selected, the aim of which is to increase classification accuracy and reduce computational time. In this paper, a recent optimization algorithm, the monarch butterfly optimization (MBO) algorithm, is implemented with a wrapper FS method that uses the k-nearest neighbor (KNN) classifier. Experiments were implemented on 18 benchmark datasets. The results showed that, in comparison with four metaheuristic algorithms (WOASAT, ALO, GA and PSO), MBO was superior, giving a high rate of classification accuracy of, on average, 93% for all datasets as well as reducing the selection size significantly. Therefore, the use of the MBO to solve the FS problems has been proven through the results obtained to be effective and highly efficient in this field, and the results have also proven the strength of the balance between global and local search of MBO.
Streaming 360-Degree Videos Using Super-Resolution

360° videos provide an immersive experience to users, but require considerably more bandwidth to stream compared to regular videos. State-of-the-art 360° video streaming systems use viewport prediction to reduce bandwidth requirement, that involves predicting which part of the video the user will view and only fetching that content. However, viewport prediction is error prone resulting in poor user Quality of Experience (QoE). We design PARSEC, a 360° video streaming system that reduces bandwidth requirement while improving video quality. PARSEC trades off bandwidth for additional client-side computation to achieve its goals. PARSEC uses an approach based on super-resolution, where the video is significantly compressed at the server and the client runs a deep learning model to enhance the video to a much higher quality. PARSEC addresses a set of challenges associated with using super-resolution for 360° video streaming: large deep learning models, slow inference rate, and variance in the quality of the enhanced videos. To this end, PAR-SEC trains small micro-models over shorter video segments, and then combines traditional video encoding with super-resolution techniques to overcome the challenges. We evaluate PARSEC on a real WiFi network, over a broadband network trace released by FCC, and over a 4G/LTE network trace. PARSEC significantly outperforms the state-of-art 360° video streaming systems while reducing the bandwidth requirement.
Evaluating immersive experiences during Covid-19 and beyond

column Share on Evaluating immersive experiences during Covid-19 and beyond Authors: Anthony Steed University College London University College LondonView Profile , Francisco R. Ortega Colorado State University Colorado State UniversityView Profile , Adam S. Williams Colorado State University Colorado State UniversityView Profile , Ernst Kruijff Bonn-Rhein-Sieg University of Applied Sciences Bonn-Rhein-Sieg University of Applied SciencesView Profile , Wolfgang Stuerzlinger Simon Fraser University Simon Fraser UniversityView Profile , Anil Ufuk Batmaz Simon Fraser University Simon Fraser UniversityView Profile , Andrea Stevenson Won Cornell University Cornell UniversityView Profile , Evan Suma Rosenberg University of Minnesota University of MinnesotaView Profile , Adalberto L. Simeone KU Leuven KU LeuvenView Profile , Aleshia Hayes University of North Texas University of North TexasView Profile Authors Info & Claims InteractionsVolume 27Issue 4July - August 2020pp 62–67https://doi.org/10.1145/3406098Published:09 July 2020Publication History 66citation2,488DownloadsMetricsTotal Citations66Total Downloads2,488Last 12 Months383Last 6 weeks82 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Publisher SiteGet Access
AGLNet: Towards real-time semantic segmentation of self-driving images via attention-guided lightweight network

The extensive computational burden limits the usage of convolutional neural networks (CNNs) in edge devices for image semantic segmentation, which plays a significant role in many real-world applications, such as augmented reality, robotics, and self-driving. To address this problem, this paper presents an attention-guided lightweight network, namely AGLNet, which employs an encoder–decoder architecture for real-time semantic segmentation. Specifically, the encoder adopts a novel residual module to abstract feature representations, where two new operations, channel split and shuffle, are utilized to greatly reduce computation cost while maintaining higher segmentation accuracy. On the other hand, instead of using complicated dilated convolution and artificially designed architecture, two types of attention mechanism are subsequently employed in the decoder to upsample features to match input resolution. Specifically, a factorized attention pyramid module (FAPM) is used to explore hierarchical spatial attention from high-level output, still remaining fewer model parameters. To delineate object shapes and boundaries, a global attention upsample module (GAUM) is adopted as global guidance for high-level features. The comprehensive experiments demonstrate that our approach achieves state-of-the-art results in terms of speed and accuracy on three self-driving datasets: CityScapes, CamVid, and Mapillary Vistas. AGLNet achieves 71.3%, 69.4%, and 30.7% mean IoU on these datasets with only 1.12M model parameters. Our method also achieves 52 FPS, 90 FPS, and 53 FPS inference speed, respectively, using a single GTX 1080Ti GPU. Our code is open-source and available at https://github.com/xiaoyufenfei/Efficient-Segmentation-Networks.
A Distributed framework for detecting DDoS attacks in smart contract‐based Blockchain‐IoT Systems by leveraging Fog computing

Abstract With the advancement of blockchain technology, and the proliferation of Internet of things (IoT)‐driven devices, the blockchain‐IoT applications is changing the perception and working infrastructure of smart networks. Blockchain supports decentralized architecture and provides secure management, authentication, and access to IoT systems by deploying smart contracts provided by Ethereum. The growing demand and expansion of blockchain‐IoT systems is generating large volume of sensitive data. Moreover, distributed denial‐of‐service (DDoS) attacks are the most challenging threats to smart contracts in blockchain‐IoT systems. The 2016 decentralized autonomous organization and 2017 parity wallet attacks exposed the critical fault‐lines among Ethereum smart contracts. Currently, there is no security mechanism available for smart contracts after its deployment in blockchain‐IoT systems. In order to address these challenges, first we use two artificial intelligence techniques, random forest (RF) and XGBoost that gives full autonomy in decision making capabilities in the proposed security framework. Second, for data load balancing and distributed file storage of IoT data, interplanetary file system is suggested. Finally, we are the first to propose a distributed framework based on fog computing to detect DDoS attacks in smart contracts. The performance of the detection system is evaluated using actual IoT dataset, namely, BoT‐IoT. The proposed system is evaluated in terms of accuracy (AC), detection rate (DR), and false alarm rate (FAR). The results confirms the superiority of the proposed framework over some of the recent state‐of‐art techniques in detecting rare attacks. The proposed framework has achieved DR up to 99.99% using RF by using 10 features of BoT‐IoT dataset.
Classification of white blood cells using deep features obtained from Convolutional Neural Network models based on the combination of feature selection methods

White blood cells are cells in the blood and lymph tissue produced by the bone marrow in the human body. White blood cells are an important part of the immune system. The most important task of these cells is to protect the human body against foreign invaders and infectious diseases. When the number of white blood cells in the blood is not enough for the human body, it can cause leukopenia. As a result of this situation, the resistance of the human body against infections and diseases decreases. In this respect, determining the number of these cells in the human body is a specialist task. Detection and treatment of this symptom is a labor-intensive process carried out by specialist doctors and radiologists. Image processing techniques have recently been widely used in biomedical systems for the diagnosis of various diseases. In this study, it is aimed to use image processing techniques to improve the classification performance of deep learning models in white blood cells classification. To perform the classification process more efficiently, the Maximal Information Coefficient and Ridge feature selection methods were used in conjunction with the Convolutional Neural Network models. The Maximal Information Coefficient and Ridge feature selection methods extracted the most relevant features. Afterward, the classification process was realized by using this feature set. In this study, AlexNet, GoogLeNet, and ResNet-50 were used as feature extractor and quadratic discriminant analysis was used as a classifier. As a result, the overall success rate was obtained as 97.95% in the classification of white blood cells. The experimental results showed that the use of the convolutional neural network models with feature selection methods contributed to improving the classification success of white blood cell types.
Image-Text Multimodal Emotion Classification via Multi-View Attentional Network

Compared with single-modal content, multimodal data can express users’ feelings and sentiments more vividly and interestingly. Therefore, multimodal sentiment analysis has become a popular research topic. However, most existing methods either learn modal sentiment feature independently, without considering their correlations, or they simply integrate multimodal features. In addition, most publicly available multimodal datasets are labeled by sentiment polarities, while the emotions expressed by users are specific. Based on this observation, in this paper, we build a large-scale image-text emotion dataset (i.e., labeled by different emotions), called TumEmo, with more than <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">190,000</i> instances from Tumblr. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> We further propose a novel multimodal emotion analysis model based on the Multi-view Attentional Network (MVAN), which utilizes a memory network that is continually updated to obtain the deep semantic features of image-text. The model includes three stages: feature mapping, interactive learning, and feature fusion. In the feature mapping stage, we leverage image features from an object viewpoint and a scene viewpoint to capture effective information for multimodal emotion analysis. Then, an interactive learning mechanism is adopted that uses the memory network; this mechanism extracts single-modal emotion features and interactively models the cross-view dependencies between the image and text. In the feature fusion stage, multiple features are deeply fused using a multilayer perceptron and a stacking-pooling module. The experimental results on the MVSA-Single, MVSA-Multiple, and TumEmo datasets show that the proposed MVAN outperforms strong baseline models by large margins.
A concise review of recent few-shot meta-learning methods

Few-shot meta-learning has been recently reviving with expectations to mimic humanity’s fast adaption to new concepts based on prior knowledge. In this short communication, we give a concise review on recent representative methods in few-shot meta-learning, which are categorized into four branches according to their technical characteristics. We conclude this review with some vital current challenges and future prospects in few-shot meta-learning.