Reviewerly/exHarmony huggingface dataset:
DocVQA: A Dataset for VQA on Document Images

We present a new dataset for Visual Question Answering (VQA) on document images called DocVQA. The dataset consists of 50,000 questions defined on 12,000+ document images. Detailed analysis of the dataset in comparison with similar datasets for VQA and reading comprehension is presented. We report several baseline results by adopting existing VQA and reading comprehension models. Although the existing models perform reasonably well on certain types of questions, there is large performance gap compared to human performance (94.36% accuracy). The models need to improve specifically on questions where understanding structure of the document is crucial. The dataset, code and leaderboard are available at docvqa.org.
Collaborative Federated Learning for Healthcare: Multi-Modal COVID-19 Diagnosis at the Edge

Despite significant improvements over the last few years, cloud-based healthcare applications continue to suffer from poor adoption due to their limitations in meeting stringent security, privacy, and quality of service requirements (such as low latency). The edge computing trend, along with techniques for distributed machine learning such as federated learning, has gained popularity as a viable solution in such settings. In this paper, we leverage the capabilities of edge computing in medicine by evaluating the potential of intelligent processing of clinical data at the edge. We utilized the emerging concept of clustered federated learning (CFL) for an automatic COVID-19 diagnosis. We evaluate the performance of the proposed framework under different experimental setups on two benchmark datasets. Promising results are obtained on both datasets resulting in comparable results against the central baseline where the specialized models (i.e., each on a specific image modality) are trained with central data, and improvements of 16% and 11% in overall F1-Scores have been achieved over the trained model trained (using multi-modal COVID-19 data) in the CFL setup on X-ray and Ultrasound datasets, respectively. We also discussed the associated challenges, technologies, and techniques available for deploying ML at the edge in such privacy and delay-sensitive applications.
Developing a formative scale to measure consumers’ trust toward interaction with artificially intelligent (AI) social robots in service delivery

This study develops and validates a scale of Social Service Robot Interaction Trust (SSRIT) that measures consumers’ trust toward interaction with AI social robots in service delivery. Through a systematic literature review, semi-structured interviews, a focus group study, and rigorous quantitative studies, this study conceptualizes the interaction-based trust and proposes a third-order reflective-formative scale, which suggests that trust in interaction is measured by 3 s-order indicators: propensity to trust in robot, trustworthy robot function and design, and trustworthy service task and context. Propensity to trust in robot is predicted by familiarity, robot use self-efficacy, social influence, technology attachment, and trust stance in technology. Trustworthy robot function and design is formed by anthropomorphism, robot performance, and effort expectancy. Trustworthy service task and context is determined by perceived service risk, robot-service fit, and facilitating robot-use condition. The convergent, discriminant, external, concurrent, and predictive validities of the scale are validated. Theoretical contributions and managerial implications are provided.
Attitudes, current behaviours and barriers to public health measures that reduce COVID-19 transmission: A qualitative study to inform public health messaging

Public health measures to reduce COVID-19 transmission include masking in public places, physical distancing, staying home when ill, avoiding high-risk locations, using a contact tracing app, and being willing to take a COVID-19 vaccine. However, adoption of these measures varies greatly. We aimed to improve health messaging to increase adherence to public health behaviours to reduce COVID-19 transmission by: 1) determining attitudes towards public health measures and current behaviours; 2) identifying barriers to following public health measures; and, 3) identifying public health communication strategies. We recruited participants from a random panel of 3000 phone numbers across Alberta to fill a predetermined quota: age (18–29; 30–59; 60+ years), geographic location (urban; rural), and whether they had school-age children. Two researchers coded and themed all transcripts. We performed content analysis and in-depth thematic analysis. Nine focus groups were conducted with 2–8 participants/group in August-September, 2020. Several themes were identified: 1) importance of public health measures; 2) compliance with public health measures; 3) critiques of public health messaging; and 4) suggestions for improving public health messaging. Physical distancing and masking were seen as more important than using a contact tracing app. There were mixed views around willingness to take COVID-19 vaccine. Current public health messaging was perceived as conflicting. Participants felt that consistent messaging and using social media to reach younger people would be helpful. In conclusion, these findings provide insights that can be used to inform targeted (e.g., by age, current behaviour) public health communications to encourage behaviors that reduce COVID-19 transmission.
A Survey on Multi-Access Edge Computing Applied to Video Streaming: Some Research Issues and Challenges

Driven by the quality of experience (QoE) requirement of video streaming applications in the smart city, smart education, immersive service, and connected vehicle scenarios, the existing network poses significant challenges, including ultra-high bandwidth, ultra-large storage, and ultra-low latency requirements, etc. Multi-access edge computing (MEC) is a potential technology, which can provide computation-intensive and caching-intensive services for video streaming applications to satisfy the requirement of QoE. Thus, focusing on video streaming schemes, a comprehensive summary of the state of the art applying MEC to video streaming is surveyed. Firstly, the related overview and background knowledge are reviewed. Secondly, resource allocation issues have been discussed. Thirdly, the enabling technologies for video streaming are summarized by taking account of caching, computing, and networking. Then, a taxonomy of MEC enabled video streaming applications is classified. Finally, challenges and future research directions are given.
A Systematic Review of AR and VR Enhanced Language Learning

This paper provided a systematic review of previous Augmented Reality (AR) and Virtual Reality (VR) studies on language learning. A total of 88 articles were selected and analyzed from five perspectives: their ways of integrating AR or VR tools in language learning; main users of AR and VR technologies; major research findings; why AR and VR tools are effective in promoting language learning; and the implications. It was found that (1) immersing learners into virtual worlds is the main approach to language learning in AR and VR studies; (2) university students were the main users of AR/VR technologies; (3) the major research findings concerning the benefits of AR and VR included improvement of students’ learning outcomes, enhancement of motivation, and positive perceptions towards using AR and VR; (4) AR and VR tools promoted language learning through providing immersive learning experience, enhancing motivation, creating interaction, and reducing learning anxiety; and (5) implications identified from previous research include the need of providing training for teachers, enlarging sample sizes, and exploring learner factors such as learner engagement and satisfaction.
Emergency education effects on teacher abilities and motivation to use digital technologies

Abstract To identify factors that can contribute toward supporting educator adoption of digital technologies beyond the emergency remote teaching response to COVID‐19, we investigated how teachers’ motivation and abilities related to the use of digital technologies for teaching changed since the onset of the pandemic. Two surveys and interviews were conducted with school teachers in Spain. The first survey was completed at the onset of the COVID‐19 lockdown, the second survey and interviews in the weeks leading up to the school year that followed. Survey questions were from SELFIE and the Work Tasks Motivation Scale for Teachers. Moreover we analysed the type of advice teachers sought on Twitter during the lockdown and post‐lockdown periods. Results indicate that teachers believe their proficiency in using digital technologies for teaching has improved. Teacher confidence in using technology for preparing lessons, class teaching, assessing and providing feedback, and for communicating with students and families has increased along with teacher motivation to improve their digital skills and use digital technologies for teaching. Teacher advice seeking on Twitter seemed to shift from serving immediate instructional needs to focussing on professional development and the creation of their own digital content. Practitioner notes What is already known about this topic There is a need to enhance educator digital skills and competences for a digital transformation of education. The emergency remote teaching response to COVID‐19 made educators increase their usage of technology. What this paper adds Teachers’ motivation and abilities related to the use of digital technologies for teaching have changed since the onset of the pandemic. Teachers believe their proficiency in using digital technologies for teaching has improved. Teacher confidence in using technology for preparing lessons, class teaching, assessing and providing feedback, and communication has increased. Teacher motivation to use digital technologies in their teaching practice increased during the pandemic. Teacher advice seeking on Twitter shifted from serving immediate instructional needs to focusing on professional development and the creation of their own digital content. Implications for practice and/or policy COVID‐19 has rapidly advanced teacher digital skills and has altered their relationships with digital technologies for teaching and learning. Teachers have acquired a range of new experiences related to using digital technologies for teaching from which future initiatives can build upon.
Dense Relation Distillation with Context-aware Aggregation for Few-Shot Object Detection

Conventional deep learning based methods for object detection require a large amount of bounding box annotations for training, which is expensive to obtain such high quality annotated data. Few-shot object detection, which learns to adapt to novel classes with only a few annotated examples, is very challenging since the fine-grained feature of novel object can be easily overlooked with only a few data available. In this work, aiming to fully exploit features of annotated novel object and capture fine-grained features of query object, we propose Dense Relation Distillation with Context-aware Aggregation (DCNet) to tackle the few-shot detection problem. Built on the meta-learning based framework, Dense Relation Distillation module targets at fully exploiting support features, where support features and query feature are densely matched, covering all spatial locations in a feed-forward fashion. The abundant usage of the guidance information endows model the capability to handle common challenges such as appearance changes and occlusions. Moreover, to better capture scale-aware features, Context-aware Aggregation module adaptively harnesses features from different scales for a more comprehensive feature representation. Extensive experiments illustrate that our proposed approach achieves state-of-the-art results on PASCAL VOC and MS COCO datasets. Code will be made available at https://github.com/hzhupku/DCNet.
Learning to Recover 3D Scene Shape from a Single Image

Despite significant progress in monocular depth estimation in the wild, recent state-of-the-art methods cannot be used to recover accurate 3D scene shape due to an unknown depth shift induced by shift-invariant reconstruction losses used in mixed-data depth prediction training, and possible unknown camera focal length. We investigate this problem in detail, and propose a two-stage framework that first predicts depth up to an unknown scale and shift from a single monocular image, and then use 3D point cloud encoders to predict the missing depth shift and focal length that allow us to recover a realistic 3D scene shape. In addition, we propose an image-level normalized regression loss and a normal-based geometry loss to enhance depth prediction models trained on mixed datasets. We test our depth model on nine unseen datasets and achieve state-of-the-art performance on zero-shot dataset generalization. Code is available at: https://git.io/Depth
RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content

Blind or no-reference video quality assessment of user-generated content (UGC) has become a trending, challenging, heretofore unsolved problem. Accurate and efficient video quality predictors suitable for this content are thus in great demand to achieve more intelligent analysis and processing of UGC videos. Previous studies have shown that natural scene statistics and deep learning features are both sufficient to capture spatial distortions, which contribute to a significant aspect of UGC video quality issues. However, these models are either incapable or inefficient for predicting the quality of complex and diverse UGC videos in practical applications. Here we introduce an effective and efficient video quality model for UGC content, which we dub the Rapid and Accurate Video Quality Evaluator (RAPIQUE), which we show performs comparably to state-of-the-art (SOTA) models but with orders-of-magnitude faster runtime. RAPIQUE combines and leverages the advantages of both quality-aware scene statistics features and semantics-aware deep convolutional features, allowing us to design the first general and efficient spatial and temporal (space-time) bandpass statistics model for video quality modeling. Our experimental results on recent large-scale UGC video quality databases show that RAPIQUE delivers top performances on all the datasets at a considerably lower computational expense. We hope this work promotes and inspires further efforts towards practical modeling of video quality problems for potential real-time and low-latency applications. To promote public usage, an implementation of RAPIQUE has been made freely available online: \url{https://github.com/vztu/RAPIQUE}.
Patch2Pix: Epipolar-Guided Pixel-Level Correspondences

The classical matching pipeline used for visual localization typically involves three steps: (i) local feature detection and description, (ii) feature matching, and (iii) outlier rejection. Recently emerged correspondence networks propose to perform those steps inside a single network but suffer from low matching resolution due to the memory bottle-neck. In this work, we propose a new perspective to estimate correspondences in a detect-to-refine manner, where we first predict patch-level match proposals and then refine them. We present Patch2Pix, a novel refinement network that refines match proposals by regressing pixel-level matches from the local regions defined by those proposals and jointly rejecting outlier matches with confidence scores. Patch2Pix is weakly supervised to learn correspondences that are consistent with the epipolar geometry of an input image pair. We show that our refinement network significantly improves the performance of correspondence networks on image matching, homography estimation, and localization tasks. In addition, we show that our learned refinement generalizes to fully-supervised methods without retraining, which leads us to state-of-the-art localization performance. The code is available at https://github.com/GrumpyZhou/patch2pix.
Learning the Best Pooling Strategy for Visual Semantic Embedding

Visual Semantic Embedding (VSE) is a dominant approach for vision-language retrieval, which aims at learning a deep embedding space such that visual data are embedded close to their semantic text labels or descriptions. Recent VSE models use complex methods to better contextualize and aggregate multi-modal features into holistic embeddings. However, we discover that surprisingly simple (but carefully selected) global pooling functions (e.g., max pooling) outperform those complex models, across different feature extractors. Despite its simplicity and effectiveness, seeking the best pooling function for different data modality and feature extractor is costly and tedious, especially when the size of features varies (e.g., text, video). Therefore, we propose a Generalized Pooling Operator (GPO), which learns to automatically adapt itself to the best pooling strategy for different features, requiring no manual tuning while staying effective and efficient. We extend the VSE model using this proposed GPO and denote it as VSE∞.Without bells and whistles, VSE∞ outperforms previous VSE methods significantly on image-text retrieval benchmarks across popular feature extractors. With a simple adaptation, variants of VSE∞ further demonstrate its strength by achieving the new state of the art on two video-text retrieval datasets. Comprehensive experiments and visualizations confirm that GPO always discovers the best pooling strategy and can be a plug-and-play feature aggregation module for standard VSE models. Code and pre-trained models are available at http://jcchen.me/vse_infty/
A Consolidated Review of Path Planning and Optimization Techniques: Technical Perspectives and Future Directions

In this paper, a review on the three most important communication techniques (ground, aerial, and underwater vehicles) has been presented that throws light on trajectory planning, its optimization, and various issues in a summarized way. This kind of extensive research is not often seen in the literature, so an effort has been made for readers interested in path planning to fill the gap. Moreover, optimization techniques suitable for implementing ground, aerial, and underwater vehicles are also a part of this review. This paper covers the numerical, bio-inspired techniques and their hybridization with each other for each of the dimensions mentioned. The paper provides a consolidated platform, where plenty of available research on-ground autonomous vehicle and their trajectory optimization with the extension for aerial and underwater vehicles are documented.
Dynamic Levy Flight Chimp Optimization

The Chimp Optimization Algorithm (ChOA) is a hunting-based model and can be utilized as a set of optimization rules to tackle optimization problems. Due to agents’ insufficient diversity in some complex problems, this algorithm is sometimes exposed to local optima stagnation. This paper introduces a Dynamic Lévy Flight (DLF) technique to smoothly and gradually transit the search agents from the exploration phase to the exploitation phase. To investigate the efficiency of the DLFChOA, this paper evaluates the performance of DLFChOA on twenty-three standard benchmark functions, twenty challenging functions of CEC-2005, ten suit tests of IEEE CEC06-2019, and twelve real-world optimization problems. The results are compared to benchmark optimization algorithms, including CMA-ES, SHADE, ChOA, HGSO, LGWO and ALEP (as the best benchmark Lévy-based algorithms), and eighteen state-of-the-art algorithms (as the winners of the CEC2019, the GECCO2019, and the SEMCCO2019). Among forty-three numerical test functions, DLFChOA and CMA-ES gain the first and second rank with thirty and eleven best results. In the 100-digit challenge, jDE100 with a score of 100 provides the best results, followed by DISHchain1e+12, and DLFChOA with a score of 85.68 is ranked fifth among eighteen state-of-the-art algorithms achieved the best score in seven out of ten problems. Finally, DLFChOA and CMA-ES respectively gain the best results in five and four real-world engineering problems.
ViT-YOLO:Transformer-Based YOLO for Object Detection

Drone captured images have overwhelming characteristics including dramatic scale variance, complicated background filled with distractors, and flexible viewpoints, which pose enormous challenges for general object detectors based on common convolutional networks. Recently, the design of vision backbone architectures that use self-attention is an exciting topic. In this work, an improved backbone MHSA-Darknet is designed to retain sufficient global context information and extract more differentiated features for object detection via multi-head self-attention. Regarding the path-aggregation neck, we present a simple yet highly effective weighted bi-directional feature pyramid network (BiFPN) for effectively cross-scale feature fusion. In addition, other techniques including time-test augmentation (TTA) and wighted boxes fusion (WBF) help to achieve better accuracy and robustness. Our experiments demonstrate that ViT-YOLO significantly outperforms the state-of-the-art detectors and achieve one of the top results in VisDrone-DET 2021 challenge (39.41 mAP for test-challenge data set and 41 mAP for the test-dev data set).
GMFlow: Learning Optical Flow via Global Matching

Learning-based optical flow estimation has been dominated with the pipeline of cost volume with convolutions for flow regression, which is inherently limited to local correlations and thus is hard to address the long-standing challenge of large displacements. To alleviate this, the state-of-the-art framework RAFT gradually improves its prediction quality by using a large number of iterative refinements, achieving remarkable performance but introducing linearly increasing inference time. To enable both high accuracy and efficiency, we completely revamp the dominant flow regression pipeline by reformulating optical flow as a global matching problem, which identifies the correspondences by directly comparing feature similarities. Specifically, we propose a GMFlow framework, which consists of three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. We further introduce a refinement step that reuses GMFlow at higher feature resolution for residual flow prediction. Our new framework outperforms 31-refinements RAFT on the challenging Sintel benchmark, while using only one refinement and running faster, suggesting a new paradigm for accurate and efficient optical flow estimation. Code is available at https://github.com/haofeixu/gmflow.