Reviewerly/exHarmony huggingface dataset:
Two-stage Co-segmentation Network Based on Discriminative Representation for Recovering Human Mesh from Videos

Recovering 3D human mesh from videos has recently made significant progress. However, most of the existing methods focus on the temporal consistency of videos, while ignoring the spatial representation in complex scenes, thus failing to recover a reasonable and smooth human mesh sequence under extreme illumination and chaotic backgrounds. To alleviate this problem, we propose a two-stage co-segmentation network based on discriminative representationfor recovering human body meshes from videos. Specifically, the first stage of the network segments the video spatial domain to spotlight spatially fine-grained information, and then learns and enhances the intra-frame discriminative representation through a dual-excitation mechanism and a frequency domain enhancement module, while sup-pressing irrelevant information (e.g., background). The second stage focuses on temporal context by segmenting the video temporal domain, and models inter-frame discriminative representation via a dynamic integration strategy. Further, to efficiently generate reasonable human discriminative actions, we carefully elaborate a landmark anchor area loss to constrain the variation of the human motion area. Extensive experimental results on large publicly available datasets indicate superiority in comparison with most state-of-the-art. The Code will be made public.
Accelerating Dataset Distillation via Model Augmentation

Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones. Existing DD methods based on gradient matching achieve leading performance; however, they are extremely computationally intensive as they require continuously optimizing a dataset among thousands of randomly initialized models. In this paper, we assume that training the synthetic data with diverse models leads to better generalization performance. Thus we propose two model augmentation techniques, i.e. using early-stage models and parameter perturbation to learn an informative synthetic set with significantly reduced training cost. Extensive experiments demonstrate that our method achieves up to 20× speedup and comparable performance on par with state-of-the-art methods.
NaQ: Leveraging Narrations as Queries to Supervise Episodic Memory

Searching long egocentric videos with natural language queries (NLQ) has compelling applications in augmented reality and robotics, where a fluid index into everything that a person (agent) has seen before could augment human memory and surface relevant information on demand. However, the structured nature of the learning problem (free-form text query inputs, localized video temporal window outputs) and its needle-in-a-haystack nature makes it both technically challenging and expensive to supervise. We introduce Narrations-as-Queries (NaQ), a data augmentation strategy that transforms standard video-text narrations into training data for a video query localization model. Validating our idea on the Ego4D benchmark, we find it has tremendous impact in practice. NaQ improves multiple top models by substantial margins (even doubling their accuracy), and yields the very best results to date on the Ego4D NLQ challenge, soundly outperforming all challenge winners in the CVPR and ECCV 2022 competitions and topping the current public leaderboard. Beyond achieving the state-of-the-art for NLQ, we also demonstrate unique properties of our approach such as the ability to perform zero-shot and fewshot NLQ, and improved performance on queries about long-tail object categories. Code and models: http://vision.cs.utexas.edu/projects/naq.
GEN: Pushing the Limits of Softmax-Based Out-of-Distribution Detection

Out-of-distribution (OOD) detection has been exten-sively studied in order to successfully deploy neural networks, in particular, for safety-critical applications. More-over, performing OOD detection on large-scale datasets is closer to reality, but is also more challenging. Sev-eral approaches need to either access the training data for score design or expose models to outliers during training. Some post-hoc methods are able to avoid the afore-mentioned constraints, but are less competitive. In this work, we propose Generalized ENtropy score (GEN), a simple but effective entropy-based score function, which can be applied to any pre-trained softmax-based classifier. Its performance is demonstrated on the large-scale ImageNet-lk OOD detection benchmark. It consistently improves the average AUROC across six commonly-used CNN-based and visual transformer classifiers over a num-ber of state-of-the-art post-hoc methods. The average AU- ROC improvement is at least 3.5%. Furthermore, we used GEN on top of feature-based enhancing methods as well as methods using training statistics to further improve the OOD detection performance. The code is available at: https://github.com/XixiLiu95/GEN.
Efficient RGB-T Tracking via Cross-Modality Distillation

Most current RGB-T trackers adopt a two-stream structure to extract unimodal RGB and thermal features and complex fusion strategies to achieve multi-modal feature fusion, which require a huge number of parameters, thus hindering their real-life applications. On the other hand, a compact RGB-T tracker may be computationally efficient but encounter non-negligible performance degradation, due to the weakening of feature representation ability. To remedy this situation, a cross-modality distillation framework is presented to bridge the performance gap between a compact tracker and a powerful tracker. Specifically, a specific-common feature distillation module is proposed to transform the modality-common information as well as the modality-specific information from a deeper two-stream network to a shallower single-stream network. In addition, a multi-path selection distillation module is proposed to instruct a simple fusion module to learn more accurate multi-modal information from a well-designed fusion mechanism by using multiple paths. We validate the effectiveness of our method with extensive experiments on three RGB-T benchmarks, which achieves state-of-the-art performance but consumes much less computational resources.
MVImgNet: A Large-scale Dataset of Multi-view Images

Being data-driven is one of the most iconic properties of deep learning algorithms. The birth of ImageNet [24] drives a remarkable trend of ‘learning from large-scale data’ in computer vision. Pretraining on ImageNet to obtain rich universal representations has been manifested to benefit various 2D visual tasks, and becomes a standard in 2D vision. However, due to the laborious collection of real-world 3D data, there is yet no generic dataset serving as a counterpart of ImageNet in 3D vision, thus how such a dataset can impact the 3D community is unraveled. To remedy this defect, we introduce MVImgNet, a large-scale dataset of multi-view images, which is highly convenient to gain by shooting videos of real-world objects in human daily life. It contains 6.5 million frames from 219,188 videos crossing objects from 238 classes, with rich annotations of object masks, camera parameters, and point clouds. The multi-view attribute endows our dataset with 3D-aware signals, making it a soft bridge between 2D and 3D vision. We conduct pilot studies for probing the potential of MVImgNet on a variety of 3D and 2D visual tasks, including radiance field reconstruction, multi-view stereo, and view-consistent image understanding, where MVImgNet demonstrates promising performance, remaining lots of possibilities for future explorations. Besides, via dense reconstruction on MVImgNet, a 3D object point cloud dataset is derived, called MVPNet, covering 87,200 samples from 150 categories, with the class label on each point cloud. Experiments show that MVP-Net can benefit the real-world 3D object classification while posing new challenges to point cloud understanding. MVImgNet and MVPNet will be public, hoping to inspire the broader vision community.
ScaleFL: Resource-Adaptive Federated Learning with Heterogeneous Clients

Federated learning (FL) is an attractive distributed learning paradigm supporting real-time continuous learning and client privacy by default. In most FL approaches, all edge clients are assumed to have sufficient computation capabilities to participate in the learning of a deep neural network (DNN) model. However, in real-life applications, some clients may have severely limited resources and can only train a much smaller local model. This paper presents ScaleFL, a novel FL approach with two distinctive mechanisms to handle resource heterogeneity and provide an equitable FL framework for all clients. First, ScaleFL adaptively scales down the DNN model along width and depth dimensions by leveraging early exits to find the best-fit models for resource-aware local training on distributed clients. In this way, ScaleFL provides an efficient balance of preserving basic and complex features in local model splits with various sizes for joint training while enabling fast inference for model deployment. Second, ScaleFL utilizes self-distillation among exit predictions during training to improve aggregation through knowledge transfer among subnetworks. We conduct extensive experiments on benchmark CV (CIFAR-10/100, ImageNet) and NLP datasets (SST-2, AgNews). We demonstrate that ScaleFL outperforms existing representative heterogeneous FL approaches in terms of global/local model performance and provides inference efficiency, with up to 2x latency and 4x model size reduction with negligible performance drop below 2%.
One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer

Whole-body mesh recovery aims to estimate the 3D human body, face, and hands parameters from a single image. It is challenging to perform this task with a single network due to resolution issues, i.e., the face and hands are usually located in extremely small regions. Existing works usually detect hands and faces, enlarge their resolution to feed in a specific network to predict the parameter, and finally fuse the results. While this copy-paste pipeline can capture the fine-grained details of the face and hands, the connections between different parts cannot be easily recovered in late fusion, leading to implausible 3D rotation and unnatural pose. In this work, we propose a one-stage pipeline for expressive whole-body mesh recovery, named OSX, without separate networks for each part. Specifically, we design a Component Aware Transformer (CAT) composed of a global body encoder and a local face/hand decoder. The encoder predicts the body parameters and provides a high-quality feature map for the decoder, which performs a feature-level upsample-crop scheme to extract highresolution part-specific features and adopt keypointguided deformable attention to estimate hand and face precisely. The whole pipeline is simple yet effective without any manual post-processing and naturally avoids implausible prediction. Comprehensive experiments demonstrate the effectiveness of OSX. Lastly, we build a large-scale Upper-Body dataset (UBody) with high-quality 2D and 3D whole-body annotations. It contains persons with partially visible bodies in diverse real-life scenarios to bridge the gap between the basic task and downstream applications.
Non-Uniform-Illumination Image Enhancement Algorithm Based on Retinex Theory

To address the issues of fuzzy scene details, reduced definition, and poor visibility in images captured under non-uniform lighting conditions, this paper presents an algorithm for effectively enhancing such images. Firstly, an adaptive color balance method is employed to address the color differences in low-light images, ensuring a more uniform color distribution and yielding a low-light image with improved color consistency. Subsequently, the image obtained is transformed from the RGB space to the HSV space, wherein the multi-scale Gaussian function is utilized in conjunction with the Retinex theory to accurately extract the lighting components and reflection components. To further enhance the image quality, the lighting components are categorized into high-light areas and low-light areas based on their pixel mean values. The low-light areas undergo improvement through an enhanced adaptive gamma correction algorithm, while the high-light areas are enhanced using the Weber–Fechner law for optimal results. Then, each block area of the image is weighted and fused, leading to its conversion back to the RGB space. And a multi-scale detail enhancement algorithm is utilized to further enhance image details. Through comprehensive experiments comparing various methods based on subjective visual perception and objective quality metrics, the algorithm proposed in this paper convincingly demonstrates its ability to effectively enhance the brightness of non-uniformly illuminated areas. Moreover, the algorithm successfully retains details in high-light regions while minimizing the impact of non-uniform illumination on the overall image quality.
Underwater Target Detection Based on Parallel High-Resolution Networks

A parallel high-resolution underwater target detection network is proposed to address the problems of complex underwater scenes and limited target feature extraction capability. First, a high-resolution network (HRNet), a lighter high-resolution human posture estimation network, is used to improve the target feature representation and effectively reduce the semantic information lost in the image during sampling. Then, the attention module (A-CBAM) is improved to capture complex feature distributions by modeling the two-dimensional space in the activation function stage through the introduction of the flexible rectified linear units (FReLU) activation function to achieve pixel-level spatial information modeling capability. Feature enhancement in the spatial and channel dimensions is performed to improve understanding of fuzzy targets and small target objects and to better capture irregular and detailed object layouts. Finally, a receptive field augmentation module (RFAM) is constructed to obtain sufficient semantic information and rich detail information to further enhance the robustness and discrimination of features and improve the detection capability of the model for multi-scale underwater targets. Experimental results show that the method achieves 81.17%, 77.02%, and 82.9% mean average precision (mAP) on three publicly available datasets, specifically underwater robot professional contest (URPC2020, URPC2018) and pattern analysis, statistical modeling, and computational learning visual object classes (PASCAL VOC2007), respectively, demonstrating the effectiveness of the proposed network.
More Than a Bot? The Impact of Disclosing Human Involvement on Customer Interactions with Hybrid Service Agents

To leverage the complementary strengths of humans and artificial intelligence (AI) in online service encounters, firms have begun to use hybrid service agents: combinations of AI agents (e.g., chatbots) and human agents (e.g., service employees) behind a single interface. However, it is unclear whether firms should be transparent about behind-the-scenes employees working in tandem with an AI-based chatbot to serve customers. Against this backdrop, we investigated the impact of human involvement disclosure on customer interactions with hybrid service agents. Our findings suggest that disclosing human involvement before or during an interaction with the hybrid service agent leads customers to adopt a more human-oriented communication style. This effect is driven by impression management concerns that are activated when customers become aware of humans working in tandem with the chatbot. The more human-oriented communication style ultimately increases employee workload because fewer customer requests can be handled automatically by the chatbot and must be delegated to a human. These findings provide novel insights into how and why disclosing human involvement affects customer communication behavior, reveal its negative consequences for employees working in tandem with a chatbot, and highlight the potential costs and benefits of providing transparency in customer–hybrid service agent interactions.
An efficient multi-scale channel attention network for person re-identification

Abstract At present, occlusion and similar appearance pose serious challenges to the task of person re-identification. In this work, we propose an efficient multi-scale channel attention network (EMCA) to learn robust and more discriminative features to solve these problems. Specifically, we designed a novel cross-channel attention module (CCAM) in EMCA and placed it after different layers in the backbone. The CCAM includes local cross-channel interaction (LCI) and channel weight integration (CWI). LCI focuses on both the maximum pooling features and the average pooling features to generate channel weights through convolutional layers, respectively. CWI combines the two channel weights to generate richer and more discriminant channel weights. Experiments on four popular person Re-ID datasets (Market-1501, DukeMTMC-ReID, CUHK-03 (detected) and MSMT17) show that the performance of our EMCA is consistently significantly superior to the existing state-of-the-art methods.
Light-Matter Interaction in the ZXW Calculus

In this paper we develop a graphical calculus to rewrite photonic circuits involving lightmatter interactions and non-linear optical effects.We introduce the infinite ZW calculus, a graphical language for linear operators on the bosonic Fock space which captures both linear and non-linear photonic circuits.This calculus is obtained by combining the QPath calculus, a diagrammatic language for linear optics, and the recently developed qudit ZXW calculus, a complete axiomatisation of linear maps between qudits.It comes with a "lifting" theorem allowing to prove equalities between infinite operators by rewriting in the ZXW calculus.We give a method for representing bosonic and fermionic Hamiltonians in the infinite ZW calculus.This allows us to derive their exponentials by diagrammatic reasoning.Examples include phase shifts and beam splitters, as well as non-linear Kerr media and Jaynes-Cummings light-matter interaction.
Machine Learning Techniques in Predicting Bottom Hole Temperature and Remote Sensing for Assessment of Geothermal Potential in the Kingdom of Saudi Arabia

The global demand for energy is increasing rapidly due to population growth, urbanization, and industrialization, as well as to meet the desire for a higher standard of living. However, environmental concerns, such as air pollution from fossil fuels, are becoming limiting factors for energy sources. Therefore, the appropriate and sustainable solution is to transition towards renewable energy sources to meet global energy demands by using environmentally friendly sources, such as geothermal. The Harrat Rahat volcanic field, located in the western region of the Kingdom of Saudi Arabia (KSA), gets more attention due to its geothermal potential as a viable site for geothermal energy exploration due to its high enthalpy. The prime objective of this study is to present up-to-date and comprehensive information on the utilization of borehole temperature and remote sensing data to identify the most prospective zones with significant geothermal activity favorable for exploration and drilling. A brief description of the selected wells and the methodology used to determine the petrophysical parameters relevant to the geothermal potential assessment are presented. Special emphasis is given to gamma-ray ray and temperature logs for calculating heat production and the geothermal gradient. The effectiveness of various machine learning techniques are assessed throughout this study for predicting the temperature-at-depth to evaluate the suitability of employing machine learning models for temperature prediction, and it is found that XG Boost provided excellent results. It can be observed that some linear anomalies can be traced in the NW, trending on the west side of the Harrat volcanic field based on magnetic data interpretation. The land surface temperature in 2021 exhibited higher temperatures compared to 2000, suggesting potential volcanic activity in the subsurface. It is concluded that the integration of remote sensing data with subsurface data provides the most reliable results.
Deep Learning within a DICOM WSI Viewer for Histopathology

Microscopy scanners and artificial intelligence (AI) techniques have facilitated remarkable advancements in biomedicine. Incorporating these advancements into clinical practice is, however, hampered by the variety of digital file formats used, which poses a significant challenge for data processing. Open-source and commercial software solutions have attempted to address proprietary formats, but they fall short of providing comprehensive access to vital clinical information beyond image pixel data. The proliferation of competing proprietary formats makes the lack of interoperability even worse. DICOM stands out as a standard that transcends internal image formats via metadata-driven image exchange in this context. DICOM defines imaging workflow information objects for images, patients’ studies, reports, etc. DICOM promises standards-based pathology imaging, but its clinical use is limited. No FDA-approved digital pathology system natively generates DICOM, and only one high-performance whole slide images (WSI) device has been approved for diagnostic use in Asia and Europe. In a recent series of Digital Pathology Connectathons, the interoperability of our solution was demonstrated by integrating DICOM digital pathology imaging, i.e., WSI, into PACs and enabling their visualisation. However, no system that incorporates state-of-the-art AI methods and directly applies them to DICOM images has been presented. In this paper, we present the first web viewer system that employs WSI DICOM images and AI models. This approach aims to bridge the gap by integrating AI methods with DICOM images in a seamless manner, marking a significant step towards more effective CAD WSI processing tasks. Within this innovative framework, convolutional neural networks, including well-known architectures such as AlexNet and VGG, have been successfully integrated and evaluated.
Understanding the Continuance Intention for Artificial Intelligence News Anchor: Based on the Expectation Confirmation Theory

The Metaverse accelerates the development of the meta-human industry and human-AI interactions in both traditional media outlets and online platforms. As a typical application of meta-human, artificial intelligence (AI) news anchors have been gradually utilized for program reports instead of newscasters in China. In this paper, through the lens of expectation confirmation theory, we establish a conceptual model consisting of perceived anthropomorphism (ANT), perceived intelligence (PI), perceived attractiveness (PA), perceived novelty (PN), information quality (IQ), confirmation of expectation (CE), trust (TRU), and satisfaction (SAT) to explore continuous intention (CI) of watching news reported by AI anchors among online users. By leveraging on a sample of 598 eligible questionnaires, the partial least square structural equation model is employed and the results show that the holistic continuing intention for AI news anchor is positive but not robust. Further analysis indicates that SAT, PI, and TRU can predict CI directly, meanwhile CE, ANT, and PA associate with CI through the mediation of satisfaction. In addition, trust and satisfaction serve as serial mediators between IQ and CI. There is no direct relationship between CE &amp; CI, ANT &amp; CI, and PN &amp; SAT. Nevertheless, user gender and previous experience can moderate the relationships of ANT &amp; CI and PN &amp; SAT, respectively. It can be seen that the proposed model can explain 80.1% of the variance in CI. The implications are intended to provide references for further commercialization of AI news anchors.
Serverless Computing: What It Is, and What It Is Not?

Dispelling the confusion around serverless computing by capturing its essential and conceptual characteristics.