Reviewerly/exHarmony huggingface dataset:
MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos

User-generated content (UGC) live videos are often bothered by various distortions during capture procedures and thus exhibit diverse visual qualities. Such source videos are further compressed and transcoded by media server providers before being distributed to end-users. Because of the flourishing of UGC live videos, effective video quality assessment (VQA) tools are needed to monitor and perceptually optimize live streaming videos in the distributing process. In this paper, we address UGC Live VQA problems by constructing a first-of-a-kind subjective UGC Live VQA database and developing an effective evaluation tool. Concretely, 418 source UGC videos are collected in real live streaming scenarios and 3,762 compressed ones at different bit rates are generated for the subsequent subjective VQA experiments. Based on the built database, we develop a Multi-12imensional VQA (MD-VQA) evaluator to measure the visual quality of UGC live videos from semantic, distortion, and motion aspects respectively. Extensive experimental results show that MD-VQA achieves state-of-the-art performance on both our UGC Live VQA database and existing compressed UGC VQA databases.
BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision

We present a novel bird's-eye-view (BEV) detector with perspective supervision, which converges faster and bet-suits modern image backbones. Existing state-of-the-art BEV detectors are often tied to certain depth pretrained backbones like Vo Vn et, hindering the synergy between booming image backbones and BEV detectors. To address this limitation, we prioritize easing the optimization of BEV detectors by introducing perspective view supervision. To this end, we propose a two-stage BEV detector; where proposals from the perspective head are fed into the bird’ s-eye-view head for final predictions. To evaluate the effectiveness of our model, we conduct extensive ablation studies focusing on the form of supervision and the gener-ality of the proposed detector. The proposed method is ver-ified with a wide spectrum of traditional and modern image backbones and achieves new SoTA results on the large-scale nuScenes dataset. The code shall be released soon.
Unified Keypoint-Based Action Recognition Framework via Structured Keypoint Pooling

This paper simultaneously addresses three limitations associated with conventional skeleton-based action recognition; skeleton detection and tracking errors, poor variety of the targeted actions, as well as person-wise and framewise action recognition. A point cloud deep-learning paradigm is introduced to the action recognition, and a unified framework along with a novel deep neural network architecture called Structured Keypoint Pooling is proposed. The proposed method sparsely aggregates keypoint features in a cascaded manner based on prior knowledge of the data structure (which is inherent in skeletons), such as the instances and frames to which each keypoint belongs, and achieves robustness against input errors. Its less constrained and tracking-free architecture enables time-series keypoints consisting of human skeletons and nonhuman object contours to be efficiently treated as an input 3D point cloud and extends the variety of the targeted action. Furthermore, we propose a Pooling-Switching Trick inspired by Structured Keypoint Pooling. This trick switches the pooling kernels between the training and inference phases to detect person-wise and framewise actions in a weakly supervised manner using only video-level action labels. This trick enables our training scheme to naturally introduce novel data augmentation, which mixes multiple point clouds extracted from different videos. In the experiments, we comprehensively verify the effectiveness of the proposed method against the limitations, and the method outperforms state-of-the-art skeleton-based action recognition and spatio-temporal action localization methods.
MIME: Human-Aware 3D Scene Generation

Generating realistic 3D worlds occupied by moving humans has many applications in games, architecture, and synthetic data creation. But generating such scenes is expensive and labor intensive. Recent work generates human poses and motions given a 3D scene. Here, we take the opposite approach and generate 3D indoor scenes given 3D human motion. Such motions can come from archival motion capture or from IMU sensors worn on the body, effectively turning human movement into a “scanner” of the 3D world. Intuitively, human movement indicates the free-space in a room and human contact indicates surfaces or objects that support activities such as sitting, lying or touching. We propose MIME (Mining Interaction and Movement to infer 3D Environments), which is a generative model of indoor scenes that produces furniture layouts that are consistent with the human movement. MIME uses an auto-regressive transformer architecture that takes the already generated objects in the scene as well as the human motion as input, and outputs the next plausible object. To train MIME, we build a dataset by populating the 3D FRONT scene dataset with 3D humans. Our experiments show that MIME produces more diverse and plausible 3D scenes than a recent generative scene method that does not know about human movement. Code and data are available for research at https://mime.is.tue.mpg.de.
Adaptive Sparse Convolutional Networks with Global Context Enhancement for Faster Object Detection on Drone Images

Object detection on drone images with low-latency is an important but challenging task on the resource-constrained unmanned aerial vehicle (UAV) platform. This paper investigates optimizing the detection head based on the sparse convolution, which proves effective in balancing the accuracy and efficiency. Nevertheless, it suffers from inadequate integration of contextual information of tiny objects as well as clumsy control of the mask ratio in the presence of foreground with varying scales. To address the issues above, we propose a novel global context-enhanced adaptive sparse convolutional network (CEASC). It first develops a context-enhanced group normalization (CE-GN) layer, by replacing the statistics based on sparsely sampled features with the global contextual ones, and then designs an adaptive multi-layer masking strategy to generate optimal mask ratios at distinct scales for compact foreground coverage, promoting both the accuracy and efficiency. Extensive experimental results on two major benchmarks, i.e. VisDrone and UAVDT, demonstrate that CEASC remarkably reduces the GFLOPs and accelerates the inference procedure when plugging into the typical state-of-the-art detection frameworks (e.g. RetinaNet and GFL V1) with competitive performance. Code is available at https://github.com/Cuogeihong/CEASC.
TransFlow: Transformer as Flow Learner

Optical flow is an indispensable building block for various important computer vision tasks, including motion estimation, object tracking, and disparity measurement. In this work, we propose TransFlow, a pure transformer architecture for optical flow estimation. Compared to dominant CNN-based methods, TransFlow demonstrates three advantages. First, it provides more accurate correlation and trustworthy matching in flow estimation by utilizing spatial self-attention and crossattention mechanisms between adjacent frames to effectively capture global dependencies; Second, it recovers more compromised information (e.g., occlusion and motion blur) in flow estimation through long-range temporal association in dynamic scenes; Third, it enables a concise self-learning paradigm and effectively eliminate the complex and laborious multi-stage pre-training procedures. We achieve the state-of-the-art results on the Sintel, KITTI-15, as well as several downstream tasks, including video object detection, interpolation and stabilization. For its efficacy, we hope TransFlow could serve as a flexible baseline for optical flow estimation.
WIRE: Wavelet Implicit Neural Representations

Implicit neural representations (INRs) have recently advanced numerous vision-related areas. INR performance depends strongly on the choice of activation function employed in its MLP network. A wide range of nonlinearities have been explored, but, unfortunately, current INRs designed to have high accuracy also suffer from poor robustness (to signal noise, parameter variation, etc.). Inspired by harmonic analysis, we develop a new, highly accurate and robust INR that does not exhibit this trade off. Our Wavelet Implicit neural REpresentation (WIRE) uses as its activation function the complex Gabor wavelet that is well-known to be optimally concentrated in space-frequency and to have excellent biases for representing images. A wide range of experiments (image denoising, image inpainting, super-resolution, computed tomography reconstruction, image over fitting, and novel view synthesis with neural radiance fields) demonstrate that WIRE defines the new state of the art in INR accuracy, training time, and robustness.
Benchmarking Self-Supervised Learning on Diverse Pathology Datasets

Computational pathology can lead to saving human lives, but models are annotation hungry and pathology images are notoriously expensive to annotate. Self-supervised learning (SSL) has shown to be an effective method for utilizing unlabeled data, and its application to pathology could greatly benefit its downstream tasks. Yet, there are no principled studies that compare SSL methods and discuss how to adapt them for pathology. To address this need, we execute the largest-scale study of SSL pre-training on pathology image data, to date. Our study is conducted using 4 representative SSL methods on diverse downstream tasks. We establish that large-scale domain-aligned pre-training in pathology consistently out-performs ImageNet pre-training in standard SSL settings such as linear and fine-tuning evaluations, as well as in low-label regimes. Moreover, we propose a set of domain-specific techniques that we experimentally show leads to a performance boost. Lastly, for the first time, we apply SSL to the challenging task of nuclei instance segmentation and show large and consistent performance improvements. We release the pre-trained model weights <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://lunit-io.github.io/research/publications/pathology_ssl.
Potential to use metaverse for future teaching and learning

Abstract Metaverse, a virtual shared space integrating augmented reality and virtual reality technologies, is often hailed as the “Internet of the future” for its potential to revolutionize online communication, collaboration, learning, and work. However, despite its promising strategic and business applications, there is a dearth of empirical evidence for evaluating its educational value. This qualitative study examined the impact of Metaverse on teaching and learning by examining its advantages and challenges. The outcomes highlight Metaverse’s remarkable potential for personalized and adaptive learning thanks to its immersive and interactive capabilities. An analysis of the existing literature suggests that Metaverse can create engaging learning experiences in which students can explore tailored virtual environments and interact within them. Semi-structured in-depth interviews were conducted to confirm the perception of metaverse-based education among students, educators, and program administrators, who view it as an innovative and effective educational delivery method. Several barriers require attention to ensure the successful integration of Metaverse in education, such as network connectivity, reliability, standardization and certification, organizational readiness, and the specialized skills necessary to leverage Metaverse’s potential. This study offers a comprehensive understanding of Metaverse’s potential, limitations, and critical factors contributing to its education success. By providing immersive and interactive learning experiences tailored to individual student needs, Metaverse holds the power to revolutionize teaching and learning. However, addressing challenges related to interoperability and the need for more evidence on its efficacy requires further research and development efforts. The significance of this study hinges on the continued exploration of Metaverse to enhance educational experiences, offering a novel and innovative approach to teaching and learning.
Novel Deep Feature Fusion Framework for Multi-Scenario Violence Detection

Detecting violence in various scenarios is a difficult task that requires a high degree of generalisation. This includes fights in different environments such as schools, streets, and football stadiums. However, most current research on violence detection focuses on a single scenario, limiting its ability to generalise across multiple scenarios. To tackle this issue, this paper offers a new multi-scenario violence detection framework that operates in two environments: fighting in various locations and rugby stadiums. This framework has three main steps. Firstly, it uses transfer learning by employing three pre-trained models from the ImageNet dataset: Xception, Inception, and InceptionResNet. This approach enhances generalisation and prevents overfitting, as these models have already learned valuable features from a large and diverse dataset. Secondly, the framework combines features extracted from the three models through feature fusion, which improves feature representation and enhances performance. Lastly, the concatenation step combines the features of the first violence scenario with the second scenario to train a machine learning classifier, enabling the classifier to generalise across both scenarios. This concatenation framework is highly flexible, as it can incorporate multiple violence scenarios without requiring training from scratch with additional scenarios. The Fusion model, which incorporates feature fusion from multiple models, obtained an accuracy of 97.66% on the RLVS dataset and 92.89% on the Hockey dataset. The Concatenation model accomplished an accuracy of 97.64% on the RLVS and 92.41% on the Hockey datasets with just a single classifier. This is the first framework that allows for the classification of multiple violent scenarios within a single classifier. Furthermore, this framework is not limited to violence detection and can be adapted to different tasks.
“Chatting with ChatGPT”: Analyzing the factors influencing users' intention to Use the Open AI's ChatGPT using the UTAUT model

Open AI's ChatGPT has emerged as a popular AI language model that can engage in natural language conversations with users. Based on a qualitative research approach using semistructured interviews with 32 ChatGPT users from India, this study examined the factors influencing users' acceptance and use of ChatGPT using the unified theory of acceptance and usage of technology (UTAUT) model. The study results demonstrated that the four factors of UTAUT, along with two extended constructs, i.e. perceived interactivity and privacy concerns, can explain users' interaction and engagement with ChatGPT. The study also found that age and experience can moderate the impact of various factors on the use of ChatGPT. The theoretical and practical implications of the study were also discussed.
Midcircuit Operations Using the <i>omg</i> Architecture in Neutral Atom Arrays

Three research groups have exploited the nuclear spins of ytterbium-171 to manipulate qubits before they are read out---an approach that could lead to efficient error-correction schemes for trapped-atom computing platforms.
Coresets Meet EDCS: Algorithms for Matching and Vertex Cover on Massive Graphs

There is a rapidly growing need for scalable algorithms that solve classical graph problems, such as maximum matching and minimum vertex cover, on massive graphs.For massive inputs, several different computational models have been introduced, including the streaming model, the distributed communication model, and the massively parallel computation (MPC) model that is a common abstraction of MapReduce-style computation.In each model, algorithms are analyzed in terms of resources such as space used or rounds of communication needed, in addition to the more traditional approximation ratio.In this paper, we give a single unified approach that yields better approximation algorithms for matching and vertex cover in all these models.The highlights include:• The first one pass, significantly-better-than-2approximation for matching in random arrival streams that uses subquadratic space, namely a (1.5 + ε)-approximation streaming algorithm that uses O(n 1.5 ) space for constant ε > 0.
Blockchain-Based Distributed Marketplace

Developments in Blockchain technology have enabled the creation of smart contracts; i.e., self-executing code that is stored and executed on the Blockchain. This has led to the creation of distributed, decentralised applications, along with frameworks for developing and deploying them easily. This paper describes a proof-of-concept system that implements a distributed online marketplace using the Ethereum framework, where buyers and sellers can engage in e-commerce transactions without the need of a large central entity coordinating the process. The performance of the system was measured in terms of cost of use through the concept of ‘gas usage’. It was determined that such costs are significantly less than that of Amazon and eBay for high volume users. The findings generally support the ability to use Ethereum to create a distributed on-chain market, however, there are still areas that require further research and development.
Hippocampus Segmentation Based on Iterative Local Linear Mapping With Representative and Local Structure-Preserved Feature Embedding

Hippocampus segmentation plays a significant role in mental disease diagnoses, such as Alzheimer's disease, epilepsy, and so on. Patch-based multi-atlas segmentation (PBMAS) approach is a popular method for hippocampus segmentation and has achieved a promising result. However, the PBMAS approach needs high computation cost due to registration and the segmentation accuracy is subject to the registration accuracy. In this paper, we propose a novel method based on iterative local linear mapping (ILLM) with the representative and local structure-preserved feature embedding to achieve accurate and robust hippocampus segmentation with no need for registration. In the proposed approach, semi-supervised deep autoencoder (SSDA) exploits unsupervised deep autoencoder and local structure-preserved manifold regularization to nonlinearly transform the extracted magnetic resonance (MR) patch to embedded feature manifold, whose adjacent relationship is similar to the signed distance map (SDM) patch manifold. Local linear mapping is used to preliminarily predict SDM patch corresponding to the MR patch. Subsequently, threshold segmentation generates a preliminary segmentation. The ILLM refines the segmentation result iteratively by ensuring the local constraints of embedded feature manifold and SDM patch manifold using a space-constrained dictionary update. Thus, a refined segmentation is obtained with no need for registration. The experiments on 135 subjects from ADNI dataset show that the proposed approach is superior to the state-of-the-art PBMAS and classification-based approaches with mean Dice similarity coefficients of 0.8852 ± 0.0203 and 0.8783 ± 0.0251 for bilateral hippocampus segmentation of 1.5T and 3.0T datasets, respectively.
Optical measurement of surface topographies with transparent coatings

Manufacturers nowadays have access to state-of-the-art areal surface topography measurement instruments that allow investigation of surface topography at unprecedented levels of detail and over a wide range of scales. However, high value-added products have demanding requirements, pushing measurement technologies to their limits. Therefore, a deeper insight and more comprehensive understanding of performance and behaviour of current areal surface topography measurement solutions is often needed. In this work, we investigate and compare the results when measuring the same surface with different, state-of-the-art areal surface topography measurement solutions involving the principal optical technologies, notably focus variation microscopy, coherence scanning interferometry, imaging confocal microscopy and point autofocus instrument, operated using different set-ups. The test case is a highly engineered surface obtained through a sequence of mechanical and chemical surface modification processes. The surface has complex topographic formations at micrometre and sub-micrometre scales, and is characterised by the presence of a thin transparent layer, notoriously challenging for optical measurement. The topographies reconstructed from measurement are compared both in terms of visual appearance and texture parameters.
Recognition of Handwritten Digit using Convolutional Neural Network (CNN)

Humans can see and visually sense the world around them by using their eyes and brains. Computer vision works on enabling computers to see and process images in the same way that human vision does. Several algorithms developed in the area of computer vision to recognize images. The goal of our work will be to create a model that will be able to identify and determine the handwritten digit from its image with better accuracy. We aim to complete this by using the concepts of Convolutional Neural Network and MNIST dataset. We will also show how MatConvNet can be used to implement our model with CPU training as well as less training time. Though the goal is to create a model which can recognize the digits, we can extend it for letters and then a person’s handwriting. Through this work, we aim to learn and practically apply the concepts of Convolutional Neural Networks.