Reviewerly/exHarmony huggingface dataset:
3D Mesh Reconstruction from 2D Images: A NeRF based Approach

3D object reconstruction is a vital obstacle within computer vision, and several techniques have been proposed to tackle it. However, the automation of the reconstruction process continues to pose a significant challenge, and limited research has been devoted to this problem. We proposed a NeRF-based pipeline for robust multi-view object reconstruction. We conducted an extensive analysis of existing NeRF-based methods and addressed some of those limitations. Our proposed pipeline achieves 17.6% higher quality reconstructions at one-tenth of the time. We evaluated our pipeline using benchmark datasets, and our results show that it outperforms state-of-the-art approaches with respect to quantitative and qualitative evaluations, respectively. Our proposed pipeline offers a robust and efficient solution to the multi-view object reconstruction problem, with potential applications in several domains, including robotics, virtual, and augmented reality.
Governing digital health for infectious disease outbreaks

ABSTRACTHow can governing digital health for infectious disease outbreaks be enhanced? In many ways, the COVID-19 pandemic has simultaneously represented both the potential and marked limitations of digital health practices for infectious disease outbreaks. During the pandemic's initial stages, states along with Big Data and Big Tech actors unleashed a scope of both established and experimental digital technologies for tracking infections, hospitalisations, and deaths from COVID-19 - and sometimes exposure to the virus SARS-CoV-2. Despite the proliferation of these technologies at the global level, transnational and cross-border integration, and cooperation within digital health responses to COVID-19 often faltered, while digital health regulations were fragmented, contested, and uncoordinated. This article presents a critiquing reflection of approaches to conceptualising, understanding, and implementing digital health for infectious disease outbreaks, observed from COVID-19 and previous examples. In assessing the strengths and limitations of existing practices of governing digital health for infectious disease outbreaks, this article particularly examines 'informal' digital health to build upon and consider how digitised responses to addressing and governing infectious disease outbreaks may be reconceptualised, revisited, or revised.
Deep Learning Generalized Structured Component Analysis: An Interpretable Artificial Neural Network Model with Composite Indexes

AbstractGeneralized structured component analysis (GSCA) is a multivariate method for specifying and examining interrelationships between observed variables and components. Despite its data-analytic flexibility honed over the decade, GSCA always defines every component as a linear function of observed variables, which can be less optimal when observed variables for a component are nonlinearly related, often reducing the component's predictive power. To address this issue, we combine deep learning and GSCA into a single framework to allow a component to be a nonlinear function of observed variables without specifying the exact functional form in advance. This new method, termed deep learning generalized structured component analysis (DL-GSCA), aims to maximize the predictive power of components while their directed or undirected network remains interpretable. Our real and simulated data analyses show that DL-GSCA produces components with greater predictive power than those from GSCA in the presence of nonlinear associations between observed variables per component.Keywords: Composite indexdeep learninggeneralized structured component analysisnonlinear component Notes1 The MATLAB code and the HDR data are available at https://osf.io/wdz79/?view_only=39cbbe18a7f442d381e2483b4bced177.2 In the original BLU function, τ1,m and τ2,m are restricted as 1 = τ2,m = –τ1,m.
Disentangling Abstraction from Statistical Pattern Matching in Human and Machine Learning

The ability to acquire abstract knowledge is a hallmark of human intelligence and is believed by many to be one of the core differences between humans and neural network models. Agents can be endowed with an inductive bias towards abstraction through meta-learning, where they are trained on a distribution of tasks that share some abstract structure that can be learned and applied. However, because neural networks are hard to interpret, it can be difficult to tell whether agents have learned the underlying abstraction, or alternatively statistical patterns that are characteristic of that abstraction. In this work, we compare the performance of humans and agents in a meta-reinforcement learning paradigm in which tasks are generated from abstract rules. We define a novel methodology for building "task metamers" that closely match the statistics of the abstract tasks but use a different underlying generative process, and evaluate performance on both abstract and metamer tasks. We find that humans perform better at abstract tasks than metamer tasks whereas common neural network architectures typically perform worse on the abstract tasks than the matched metamers. This work provides a foundation for characterizing differences between humans and machine learning that can be used in future work towards developing machines with more human-like behavior.
Class-aware Variational Auto-encoder for Open Set Recognition

Compared with traditional classification models trained under the closed world assumption, Open Set Recognition (OSR) requires accurate classification for known classes as well as rejection for unknown ones. By modeling the distribution of each known class, Conditional Variational Auto-encoder (CVAE) has achieved great success in OSR, even though it was originally proposed for image generation. In this paper, we propose a novel two-stage learning framework, Class-aware Variational Auto-encoder (CA-VAE) to better adapt CVAE to the OSR task. Pre-derived attention images are taken as the objective target for reconstruction, thus model is implicitly directed to focus on the class-discriminative regions of the image. In this way, the learned latent representation is de-biased towards class-aware. Experiments on standard image datasets demonstrate the outperformance of the proposed method over existing ones, which achieves new state-of-the-art results. Codes are available at https://github.com/roywang021/CA-VAE.
FlowText: Synthesizing Realistic Scene Text Video with Optical Flow Estimation

Current video text spotting methods can achieve preferable performance, powered with sufficient labeled training data. However, labeling data manually is time-consuming and labor-intensive. To overcome this, using low-cost synthetic data is a promising alternative. This paper introduces a novel video text synthesis technique called FlowText, which utilizes optical flow estimation to synthesize a large amount of text video data at a low cost for training robust video text spotters. Unlike existing methods that focus on image-level synthesis, FlowText concentrates on synthesizing temporal information of text instances across consecutive frames using optical flow. This temporal information is crucial for accurately tracking and spotting text in video sequences, including text movement, distortion, appearance, disappearance, shelter, and blur. Experiments show that combining general detectors like TransDETR with the proposed FlowText produces remarkable results on various datasets, such as ICDAR2015video and ICDAR2013video. Code is available at https://github.com/callsys/FlowText.
MTNet: Learning Modality-aware Representation with Transformer for RGBT Tracking

The ability to learn robust multi-modality representation has played a critical role in the development of RGBT tracking. However, the regular fusion paradigm and the invariable tracking template remain restrictive to the feature interaction. In this paper, we propose a modality-aware tracker based on transformer, termed MTNet. Specifically, a modality- aware network is presented to explore modality-specific cues, which contains both channel aggregation and distribution module (CADM) and spatial similarity perception module (SSPM). A transformer fusion network is then applied to capturing global dependencies to reinforce instance representations. To estimate the precise location and tackle the challenges, such as scale variation and deformation, we design a trident prediction head and a dynamic update strategy which jointly maintain a reliable template for facilitating inter-frame communication. Extensive experiments validate that the proposed method achieves satisfactory results compared with the state-of-the-art competitors on three RGBT benchmarks while reaching real-time speed.
Stochastic flows – a primer on early multi-object filtering work with point processes

Multi-object filtering is a generalisation of stochastic filtering to deal with an unknown and time-varying number of targets, largely based on modelling with point processes. Some early works on this topic from the Soviet Union from 1960s-l980s appeared prior to well known results in the contemporary liter-ature. This article reviews some of these historical contributions.
Closing the Ivory Validation Gates: A Historical Analysis of Achkasov’s Data Association Filters

We examine prototypical versions of the probabilistic data association (PDA) filter and the joint PDA filter developed in the Soviet Union in 1972. These filters, created by Y.S. Achkasov, are previously undiscussed in the conventional English-language multitarget tracking (MTT) literature, despite being translated into English from their original Russian soon after publication. We place this work in its appropriate historical context and demonstrate the equivalence of the proto-(J)PDA with the conventional forms of these algorithms, as well as discuss areas where this work anticipates multiple decades of MTT filter development.
A Two-stage hybrid CNN-Transformer Network for RGB Guided Indoor Depth Completion

The indoor captured raw depth images usually contain large in-homogeneous missing regions. Most existing methods are designed for the outdoor sparse depth completion, which struggle in completing the indoor depth with large holes. In this paper, to solve this problem, we propose a hybrid CNN-Transformer network for RGB guided indoor depth completion. The proposed network is composed of two stages to achieve depth completion in a coarse-to-fine manner. In the first stage, we propose a CNN based self-completion module (SCM) with cross scale attention to restore a coarse depth image. In the second stage, we further refine the completed depth image with the guidance of RGB image by proposing a guided completion module (GCM). To fully explore the guidance from the RGB image, we design a cross-modal Transformer (CMT) block to fuse the features from the depth and RGB modalities at different scales. Extensive experiments on NYUv2 and SUN RGB-D datasets demonstrate the superior performance of the proposed method over other state-of-the-art methods both quantitatively and qualitatively. The code is available at https://github.com/eecoder-dyf/ICME-2023-depth-completion.
The Product Multi-Sensor Labeled Multi-Bernoulli Filter

The main challenge in random finite set-based multi-sensor multi-object tracking is the NP-hard association of the sensor measurements with the tracks. Using the Bayes parallel combination rule, we have recently proposed the Product Multi-sensor Generalized Labeled Multi-Bernoulli (PM-GLMB) filter, decomposing the multi-sensor update into simpler single-sensor problems with subsequent Bayes optimal fusion. This paper extends the filter to prior densities with Gaussian mixture spatial distributions, which is an inevitable prerequisite for the Product Multi-sensor Labeled Multi-Bernoulli (PM-LMB) filter we propose afterward. Finally, we evaluate the performance of both in a simulation study. In this context, we address the known performance problems of the PM-GLMB filter in crowded situations and show how the PM-LMB filter overcomes these.
Object-Aware Calibrated Depth-Guided Transformer for RGB-D Co-Salient Object Detection

The key role of RGB-D co-salient object detection is to effectively fuse the common information of RGB and depth signals. Existing works directly mix the information captured from both original depth maps and RGB images, but ignore one critical issue: due to the low contrast of the neighborhood objects in depth, the depth maps’ salient regions may correspond to the interference background regions in the RGB images, thereby leading to unsatisfying performance. To address this issue, we propose an Object-aware Calibrated Depth guided transformer (dubbed as OCDFormer) for RGB-D co-salient object detection. The OCDFormer mainly consists of two key designs: First, we design a depth calibration module via spectral clustering, which yields a group of calibrated depth maps that can highlight the co-object region while suppressing the interference regions. Second, we construct a cross-modal transformer, in which the common information from the RGB and the calibrated depth maps are fully captured by first injecting common tokens into the individual tokens, and then mixing them with an interaction-attention mechanism. Extensive evaluations demonstrate that our OCDFormer sets a new state-of-the-art on two public standard benchmarks including RGB-D CoSall5O and RGB-D CoSegl83.
Compositional Learning in Transformer-Based Human-Object Interaction Detection

Human-object interaction (HOI) detection is an important part of understanding human activities and visual scenes. The long-tailed distribution of labeled instances is a primary challenge in HOI detection, promoting research in few-shot and zero-shot learning. Inspired by the combinatorial nature of HOI triplets, some existing approaches adopt the idea of compositional learning, in which object and action features are learned individually and re-composed as new training samples. However, these methods follow the CNN-based two-stage paradigm with limited feature extraction ability, and often rely on auxiliary information for better performance. Without introducing any additional information, we creatively propose a transformer-based framework for compositional HOI learning. Human-object pair representations and interaction representations are re-composed across different HOI instances, which involves richer contextual information and promotes the generalization of knowledge. Experiments show our simple but effective method achieves state-of-the-art performance, especially on rare HOI classes.
Self-Supervised Depth Estimation Based on the Consistency of Synthetic-real Image Prediction

Learning-based multi-view stereo aims to restore the real scene from multiple images with overlapping areas. The mainstream self-supervised MVS method trains the model based on the assumption that spatial points from different perspectives share the same color information. To further suppress the interference from specular reflection and illumination noise, this work proposes a self-supervised MVS network based on the consistency of synthetic-real image prediction. The network first applies the coarse-to-fine manner to gradually refine the depth map, and the source images are projected to the reference view to generate the synthesized reference image. Then the synthesized image with real source images are re-input into the network to form a cycled network, and the consistency constraint of the prediction results of the two periods is introduced to improve the color anti-interference of the self- supervised MVS model. The comprehensive experiments on the public dataset show that the proposed work can further improve the reconstruction performance of the benchmark model, which verifies the effectiveness of the proposed work.
TextFormer: Component-aware Text Segmentation with Transformer

In recent years, deep learning techniques have made significant advancements in text segmentation. However, most existing methods do not take into account that characters are composed of smaller components, such as strokes and other local patterns. Furthermore, the similarities between text components are crucial for effective text segmentation. With this in mind, we propose a multi-level Transformer-based method for text segmentation that incorporates a recognition module. To enhance the interaction between text components and extract features at different granularities, we introduce Global and Local Self-Attention blocks. Our recognition module is trained jointly with the segmentation module to improve the model’s ability to focus on text details and improve its perception of texts. By aggregating features from multiple granularities, our segmentation module produces accurate pixel-level mask predictions. The experimental results demonstrate the effectiveness of our approach on several text segmentation benchmarks and show that it outperforms existing methods.
Perspective Transformation Layer

Incorporating geometric transformations that reflect the relative position changes between an observer and an object into computer vision and deep learning models has attracted much attention in recent years. However, the existing proposals mainly focus on the affine transformation that is insufficient to reflect such geometric position changes. Furthermore, current solutions often apply a neural network module to learn a single transformation matrix, which not only ignores the importance of multi-view analysis but also includes extra training parameters from the module apart from the transformation matrix parameters that increase the model complexity. In this paper, a perspective transformation layer is proposed in the context of deep learning. The proposed layer can learn homography, therefore reflecting the geometric positions between observers and objects. In addition, by directly training its transformation matrices, a single proposed layer can learn an adjustable number of multiple viewpoints without considering module parameters. The experiments and evaluations confirm the superiority of the proposed layer.
Learning Component-Level and Inter-Class Glyph Representation for few-shot Font Generation

Few-shot font generation (FFG) has received increasing attention recently. However, due to the complex structure of Chinese characters, most existing methods suffer from missing component-level content details and inaccurate capture of content-independent style representations. In this paper, we proposed a novel generative adversarial network for FFG by learning component information and inter-class glyph style representation. Specifically, we proposed a Content-Component Aware Module (CCAM) to help the model learn the accurate content representation by using component images, which is a brand new perspective. In addition, we employed a Glyph Style Contrastive Enhance (GSCE) strategy to help the encoder learn the differences of inter-class glyph style while ignoring the influence of reference character content. Both qualitative and quantitative experiments show our method achieves state-of-the-art performance in terms of the content structure integrity and style accuracy of the glyph.
Blockchain-Based Privacy Preserved Physiological Data Sharing Platform

Wearable devices have significantly impacted precision medicine by providing real-time physiological and behavioral data. This data is crucial for accurate disease diagnosis, monitoring, and improved treatment outcomes. Wearable devices enable personalized health management and prevention programs, particularly being beneficial for chronic disease management. Overall, these devices offer more data and technical support for precision medicine, leading to better-individualized health management and treatment and ultimately improving medical outcomes and quality of life. However, the current device binding requires direct identifiers and grants manufacturers ownership of the generated data, limiting user control and raising privacy concerns. To address this, we propose a blockchain-based platform with two distinct blockchains: decentralized identity and physiological data. Users register decentralized identities on the first blockchain, which are then used for device binding on the second blockchain, enabling de-identified data collection. The platform generates user-specific smart contracts on the Physiological Data blockchain and ensures complete user control over data access. This system enhances the privacy, security, and credibility of users' physiological data, instilling confidence in the use of wearable devices.
Autonomous Path Planning and Realization for Dual Robot Cooperation Based on ROS Framework

To realize dual-robot autonomous path planning and realization, the trajectory planning research of dual-robot is carried out using the open-source robot operating system ROS as the simulation platform. The motion planning-related file configuration is completed by using MoveIt! In trajectory planning, cubic spline interpolation is performed on the trajectory points generated by the RRT-Connect path planning algorithm to complete the planning of the dual-arm assembly task. The communication between ROS nodes and ROS is established in the controller, and the processed trajectory points are communicated with the controller through ROS-Industrial. The experimental results show that the trajectory of the dual robot movement process is smooth and continuous, the stability is strong, and the error is small, which can ensure the completion of basic assembly tasks.
RRT-Forest: An Efficient Multi-UAV Cooperative Path Planning Method for Indoor Rescue

Sending multi unmanned aerial vehicles (UAVs) to extinguish indoor flames is a feasible solution for the difficulty of fire rescuing in high-rise constructions. However, the difficulty of cooperative path planning hinders the application of multi-UAVs in such a dangerous task. Aiming at the difficulty, an improved RRT-Forest algorithm is proposed, which could be operated in high dimensional path planning space with extremely complex and nonconvex environments, to realize efficient multi-UAVs cooperative global path planning. Numerical simulation results indicate that the method is feasible and efficient in obtaining collision-free paths for multi-UAVs.