Reviewerly/exHarmony huggingface dataset:
StyleRes: Transforming the Residuals for Real Image Editing with StyleGAN

We present a novel image inversion framework and a training pipeline to achieve high-fidelity image inversion with high-quality attribute editing. Inverting real images into StyleGAN's latent space is an extensively studied problem, yet the trade-off between the image reconstruction fidelity and image editing quality remains an open challenge. The low-rate latent spaces are limited in their expressiveness power for high-fidelity reconstruction. On the other hand, high-rate latent spaces result in degradation in editing quality. In this work, to achieve high-fidelity inversion, we learn residual features in higher latent codes that lower latent codes were not able to encode. This enables preserving image details in reconstruction. To achieve high-quality editing, we learn how to transform the residual features for adapting to manipulations in latent codes. We train the framework to extract residual features and transform them via a novel architecture pipeline and cycle consistency losses. We run extensive experiments and compare our method with state-of-the-art inversion methods. Qualitative metrics and visual comparisons show significant improvements. Code: https://github.com/hamzapehlivanIStyleRes.
High-Fidelity Generalized Emotional Talking Face Generation with Multi-Modal Emotion Space Learning

Recently, emotional talking face generation has received considerable attention. However, existing methods only adopt one-hot coding, image, or audio as emotion conditions, thus lacking flexible control in practical applications and failing to handle unseen emotion styles due to limited semantics. They either ignore the one-shot setting or the quality of generated faces. In this paper, we propose a more flexible and generalized framework. Specifically, we supplement the emotion style in text prompts and use an Aligned Multi-modal Emotion encoder to embed the text, image, and audio emotion modality into a unified space, which inherits rich semantic prior from CLIP. Consequently, effective multi-modal emotion space learning helps our method support arbitrary emotion modality during testing and could generalize to unseen emotion styles. Besides, an Emotion-aware Audio-to-3DMM Convertor is proposed to connect the emotion condition and the audio sequence to structural representation. A followed style-based High-fidelity Emotional Face generator is designed to generate arbitrary high-resolution realistic identities. Our texture generator hierarchically learns flow fields and animated faces in a residual manner. Extensive experiments demonstrate the flexibility and generalization of our method in emotion control and the effectiveness of high-quality face synthesis.
Annealing-based Label-Transfer Learning for Open World Object Detection

Open world object detection (OWOD) has attracted extensive attention due to its practicability in the real world. Previous OWOD works manually designed unknown-discover strategies to select unknown proposals from the background, suffering from uncertainties without appropriate priors. In this paper, we claim the learning of object detection could be seen as an object-level feature-entanglement process, where unknown traits are propagated to the known proposals through convolutional operations and could be distilled to benefit unknown recognition without manual selection. Therefore, we propose a simple yet effective Annealing-based Label-Transfer framework, which sufficiently explores the known proposals to alleviate the uncertainties. Specifically, a Label-Transfer Learning paradigm is introduced to decouple the known and unknown features, while a Sawtooth Annealing Scheduling strategy is further employed to rebuild the decision boundaries of the known and unknown classes, thus promoting both known and unknown recognition. Moreover, previous OWOD works neglected the trade-off of known and unknown performance, and we thus introduce a metric called Equilibrium Index to comprehensively evaluate the effectiveness of the OWOD models. To the best of our knowledge, this is the first OWOD work without manual unknown selection. Extensive experiments conducted on the common-used benchmark validate that our model achieves superior detection performance (200% unknown mAP improvement with the even higher known detection performance) compared to other state-of-the-art methods. Our code is available at https://github.com/DIG-Beihang/ALLOW.git.
Continual Detection Transformer for Incremental Object Detection

Incremental object detection (IOD) aims to train an object detector in phases, each with annotations for new object categories. As other incremental settings, IOD is subject to catastrophic forgetting, which is often addressed by techniques such as knowledge distillation (KD) and exem-plar replay (ER). However, KD and ER do not work well if applied directly to state-of-the-art transformer-based object detectors such as Deformable DETR [59] and UP-DETR [9]. In this paper, we solve these issues by proposing a ContinuaL DEtection TRansformer (CL-DETR), a new method for transformer-based IOD which enables effective usage of KD and ER in this context. First, we introduce a Detector Knowledge Distillation (DKD) loss, focusing on the most informative and reliable predictions from old versions of the model, ignoring redundant background predictions, and ensuring compatibility with the available ground-truth labels. We also improve ER by proposing a calibration strategy to preserve the label distribution of the training set, therefore better matching training and testing statistics. We conduct extensive experiments on COCO 2017 and demonstrate that CL-DETR achieves state-of-the-art results in the IOD setting. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Code: https://1yy.mpi-inf.mpg.de/CL-DETR/
BAWGNet: Boundary aware wavelet guided network for the nuclei segmentation in histopathology images

Precise cell nucleus segmentation is very critical in many biologically related analyses and disease diagnoses. However, the variability in nuclei structure, color, and modalities of histopathology images make the automatic computer-aided nuclei segmentation task very difficult. Traditional encoder–decoder based deep learning schemes mainly utilize the spatial domain information that may limit the performance of recognizing small nuclei regions in subsequent downsampling operations. In this paper, a boundary aware wavelet guided network (BAWGNet) is proposed by incorporating a boundary aware unit along with an attention mechanism based on a wavelet domain guidance in each stage of the encoder–decoder output. Here the high-frequency 2 Dimensional discrete wavelet transform (2D-DWT) coefficients are utilized in the attention mechanism to guide the spatial information obtained from the encoder–decoder output stages to leverage the nuclei segmentation task. On the other hand, the boundary aware unit (BAU) captures the nuclei’s boundary information, ensuring accurate prediction of the nuclei pixels in the edge region. Furthermore, the preprocessing steps used in our methodology confirm the data’s uniformity by converting it to similar color statistics. Extensive experimentations conducted on three benchmark histopathology datasets (DSB, MoNuSeg and TNBC) exhibit the outstanding segmentation performance of the proposed method (with dice scores 90.82%, 85.74%, and 78.57%, respectively). Implementation of the proposed architecture is available at https://github.com/tamjidimtiaz/BAWGNet.
Scribble-Supervised RGB-T Salient Object Detection

Salient object detection segments attractive objects in scenes. RGB and thermal modalities provide complementary information and scribble annotations alleviate large amounts of human labor. Based on the above facts, we propose a scribble-supervised RGB-T salient object detection model. By a four-step solution (expansion, prediction, aggregation, and supervision), label-sparse challenge of scribble-supervised method is solved. To expand scribble annotations, we collect the superpixels that foreground scribbles pass through in RGB and thermal images, respectively. The expanded multi-modal labels provide the coarse object boundary. To further polish the expanded labels, we propose a prediction module to alleviate the sharpness of boundary. To play the complementary roles of two modalities, we combine the two into aggregated pseudo labels. Supervised by scribble annotations and pseudo labels, our model achieves the state-of-the-art performance on the relabeled RGBT-S dataset. Furthermore, the model is applied to RGB-D and video scribble-supervised applications, achieving consistently excellent performance. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Do We Really Need Temporal Convolutions in Action Segmentation?

Recognizing and segmenting actions from long videos is a challenging problem. Most existing methods focus on designing temporal convolutional models. However, these models are limited in their flexibility and ability to model long-term dependencies. Transformers have recently been used in various tasks. But the lack of inductive bias and the inefficiency of handling long video sequences limit the application of Transformers in action segmentation. In this paper, we present a pure Transformer-based model without temporal convolutions in action segmentation, called Temporal U-Transformer. The U-Transformer architecture not only reduces complexity but also introduces an inductive bias that neighboring frames are more likely to belong to the same class. Besides, we further propose a boundary-aware loss based on the distribution of similarity scores between frames from attention modules to improve the ability to recognize boundaries. Extensive experiments show the effectiveness of our method.
Analyzing the Legal Labyrinth: Current Trends in Genetic Research and Their Legal Perspectives

Genetic research has experienced rapid advancements in recent years, giving rise to a host of legal challenges at the intersection of intellectual property rights, data privacy, and ethical concerns. This article aims to analyze a key legal issue related to genetic research and propose potential solutions. We conducted a literature review, comparative analysis, and policy evaluation to identify the shortcomings of existing legal frameworks and highlight best practices from different jurisdictions. Our findings indicate that current legal frameworks may not adequately address the challenges posed by genetic research. We propose potential solutions, such as developing a comprehensive legal framework for genetic research, encouraging international collaboration and harmonization of laws and regulations, and implementing strong oversight mechanisms. By addressing these legal challenges and implementing the proposed solutions, policymakers and stakeholders can create a stable and predictable environment for genetic research that maximizes its benefits while minimizing potential harms and inequities.
Cooperative Dynamic Motion Planning for Dual Manipulator Arms Based on RRT*Smart-AD Algorithm

Intelligent manufacturing requires robots to adapt to increasingly complex tasks, and dual-arm cooperative operation can provide a more flexible and effective solution. Motion planning serves as a crucial foundation for dual-arm cooperative operation. The rapidly exploring random tree (RRT) algorithm based on random sampling has been widely used in high-dimensional manipulator path planning due to its probability completeness, handling of high-dimensional problems, scalability, and faster exploration speed compared with other planning methods. As a variant of RRT, the RRT*Smart algorithm introduces asymptotic optimality, improved sampling techniques, and better path optimization. However, existing research does not adequately address the cooperative motion planning requirements for dual manipulator arms in terms of sampling methods, path optimization, and dynamic adaptability. It also cannot handle dual-manipulator collaborative motion planning in dynamic scenarios. Therefore, in this paper, a novel motion planner named RRT*Smart-AD is proposed to ensure that the dual-arm robot satisfies obstacle avoidance constraints and dynamic characteristics in dynamic environments. This planner is capable of generating smooth motion trajectories that comply with differential constraints and physical collision constraints for a dual-arm robot. The proposed method includes several key components. First, a dynamic A* cost function sampling method, combined with an intelligent beacon sampling method, is introduced for sampling. A path-pruning strategy is employed to improve the computational efficiency. Strategies for dynamic region path repair and regrowth are also proposed to enhance adaptability in dynamic scenarios. Additionally, practical constraints such as maximum velocity, maximum acceleration, and collision constraints in robotic arm applications are analyzed. Particle swarm optimization (PSO) is utilized to optimize the motion trajectories by optimizing the parameters of quintic non-uniform rational B-splines (NURBSs). Static and dynamic simulation experiments verified that the RRT*Smart-AD algorithm for cooperative dynamic path planning of dual robotic arms outperformed biased RRT* and RRT*Smart. This method not only holds significant practical engineering significance for obstacle avoidance in dual-arm manipulators in intelligent factories but also provides a theoretical reference value for the path planning of other types of robots.
A Comparative Study of Attention-Based Transformer Networks and Traditional Machine Learning Methods for Toxic Comments Classification

With the rapid growth of online communication platforms, the identification and management of toxic comments have become crucial in maintaining a healthy online environment. Various machine learning approaches have been employed to tackle this problem, ranging from traditional models to more recent attention-based transformer networks. This paper aims to compare the performance of attention-based transformer networks with several traditional machine learning methods for toxic comments classification. We present an in-depth analysis and evaluation of these methods using a common benchmark dataset. The experimental results demonstrate the strengths and limitations of each approach, shedding light on the suitability and efficacy of attention-based transformers in this domain.
Intelligent solar photovoltaic power forecasting

This paper presents a day-ahead forecasting method for photovoltaic (PV) power plants in commercial sectors. The method is based on numerical weather prediction (NWP) models from open weather maps and power plant specifications. The output of the model is the predicted power output from the PV power plant, which is incorporated into an optimal control strategy of the PV plant using battery storage. The use of optimal algorithms assists in the PV power plant curtailment in cases of over-generation and reduces the dependence on conventional power sources such as generators in cases of under-generation by the PV plant. It was found that most forecasting methods do not incorporate PV power and storage systems for proper optimization and demand management. This can be seen as a gap for further research of forecasting models integrated with battery storage systems to improve PV power system performance. Results obtained show a good performance of the developed model. A root means square error (RMSE) of 425.79 W and 595.10 W and a mean absolute error (MAE) of 246.26 W and 238 W were achieved for a summer and winter day, respectively. Furthermore, an excellent positive correlation exists between the predicted output power and the observed results, with R2 values over 90%.
An Analysis of Future Prospects of Metaverse

The term “Metaverse” resonates greatly in recent literature, yet many entrepreneurs overlook its potential. Although the Metaverse offers numerous opportunities for present and future businesses, its limitlessness also creates confusion for entrepreneurs. In the future world of the Internet, entrepreneurs must realize that the Metaverse will play a major role in business and other fields. Terms such as NFTs, blockchain, avatars, and Web 3.0 may not be familiar to the present world, but they will shape the future of business. This paper aims to understand the future perspectives, opportunities, and advantages of the Metaverse. As the Metaverse affects the lives and work of common people, they are being involuntarily or voluntarily impacted, limiting the ability to transfer findings from previous scenarios to the top.
Paying attention in metaverse: an experiment on spatial attention allocation in extended reality shopping

Purpose Metaverse, that is extended reality (XR)-based technologies such as augmented reality (AR) and virtual reality (VR), are increasingly believed to facilitate fundamental human practice in the future. One of the vanguards of this development has been the consumption domain, where the multi-modal and multi-sensory technology-mediated immersion is expected to enrich consumers' experience. However, it remains unclear whether these expectations have been warranted in reality and whether, rather than enhancing the experience, metaverse technologies inhibit the functioning and experience, such as cognitive functioning and experience. Design/methodology/approach This study utilizes a 2 (VR: yes vs no) × 2 (AR: yes vs no) between-subjects laboratory experiment. A total of 159 student participants are randomly assigned to one condition — a brick-and-mortar store, a VR store, an AR store and an augmented virtuality (AV) store — to complete a typical shopping task. Four spatial attention indicators — visit shift, duration shift, visit variation and duration variation — are compared based on attention allocation data converted from head movements extracted from recorded videos during the experiments. Findings This study identifies three essential effects of XR technologies on consumers' spatial attention allocation: the inattention effect, acceleration effect and imbalance effect. Specifically, the inattention effect (the attentional visit shift from showcased products to the environmental periphery) appears when VR or AR technology is applied to virtualize the store and disappears when AR and VR are used together. The acceleration effect (the attentional duration shift from showcased products to the environmental periphery) exists in the VR store. Additionally, AR causes an imbalance effect (the attentional duration variation increases horizontally among the showcased products). Originality/value This study provides valuable empirical evidence of how VR and AR influence consumers' spatial bias in attention allocation, filling the research gap on cognitive function in the metaverse. This study also provides practical guidelines for retailers and XR designers and developers.
Comparison of the Diagnostic Accuracy of Mammogram-based Deep Learning and Traditional Breast Cancer Risk Models in Patients Who Underwent Supplemental Screening with MRI

Background Access to supplemental screening breast MRI is determined using traditional risk models, which are limited by modest predictive accuracy. Purpose To compare the diagnostic accuracy of a mammogram-based deep learning (DL) risk assessment model to that of traditional breast cancer risk models in patients who underwent supplemental screening with MRI. Materials and Methods This retrospective study included consecutive patients undergoing breast cancer screening MRI from September 2017 to September 2020 at four facilities. Risk was assessed using the Tyrer-Cuzick (TC) and National Cancer Institute Breast Cancer Risk Assessment Tool (BCRAT) 5-year and lifetime models as well as a DL 5-year model that generated a risk score based on the most recent screening mammogram. A risk score of 1.67% or higher defined increased risk for traditional 5-year models, a risk score of 20% or higher defined high risk for traditional lifetime models, and absolute scores of 2.3 or higher and 6.6 or higher defined increased and high risk, respectively, for the DL model. Model accuracy metrics including cancer detection rate (CDR) and positive predictive values (PPVs) (PPV of abnormal findings at screening [PPV1], PPV of biopsies recommended [PPV2], and PPV of biopsies performed [PPV3]) were compared using logistic regression models. Results This study included 2168 women who underwent 4247 high-risk screening MRI examinations (median age, 54 years [IQR, 48–60 years]). CDR (per 1000 examinations) was higher in patients at high risk according to the DL model (20.6 [95% CI: 11.8, 35.6]) than according to the TC (6.0 [95% CI: 2.9, 12.3]; P < .01) and BCRAT (6.8 [95% CI: 2.9, 15.8]; P = .04) lifetime models. PPV1, PPV2, and PPV3 were higher in patients identified as high risk by the DL model (PPV1, 14.6%; PPV2, 32.4%; PPV3, 36.4%) than those identified as high risk with the TC (PPV1, 5.0%; PPV2, 12.7%; PPV3, 13.5%; P value range, .02–.03) and BCRAT (PPV1, 5.5%; PPV2, 11.1%; PPV3, 12.5%; P value range, .02–.05) lifetime models. Conclusion Patients identified as high risk by a mammogram-based DL risk assessment model showed higher CDR at breast screening MRI than patients identified as high risk with traditional risk models. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Bae in this issue.
Chaining of Maximal Exact Matches in Graphs

We show how to chain maximal exact matches (MEMs) between a query string Q and a labeled directed acyclic graph (DAG) $$G=(V,E)$$ to solve the longest common subsequence (LCS) problem between Q and G. We obtain our result via a new symmetric formulation of chaining in DAGs that we solve in $$O(m+n+k^2|V| + |E| + kN\log N)$$ time, where $$m=|Q|$$ , n is the total length of node labels, k is the minimum number of paths covering the nodes of G and N is the number of MEMs between Q and node labels, which we show encode full MEMs.