Reviewerly/exHarmony huggingface dataset:
High-Fidelity GAN Inversion with Padding Space

Inverting a Generative Adversarial Network (GAN) facilitates a wide range of image editing tasks using pre-trained generators. Existing methods typically employ the latent space of GANs as the inversion space yet observe the insufficient recovery of spatial details. In this work, we propose to involve the padding space of the generator to complement the latent space with spatial information. Concretely, we replace the constant padding (e.g., usually zeros) used in convolution layers with some instance-aware coefficients. In this way, the inductive bias assumed in the pre-trained model can be appropriately adapted to fit each individual image. Through learning a carefully designed encoder, we manage to improve the inversion quality both qualitatively and quantitatively, outperforming existing alternatives. We then demonstrate that such a space extension barely affects the native GAN manifold, hence we can still reuse the prior knowledge learned by GANs for various downstream applications. Beyond the editing tasks explored in prior arts, our approach allows a more flexible image manipulation, such as the separate control of face contour and facial details, and enables a novel editing manner where users can customize their own manipulations highly efficiently. (Project page can be found here .)
Housekeep: Tidying Virtual Households Using Commonsense Reasoning

We introduce Housekeep, a benchmark to evaluate commonsense reasoning in the home for embodied AI. In Housekeep, an embodied agent must tidy a house by rearranging misplaced objects without explicit instructions specifying which objects need to be rearranged. Instead, the agent must learn from and is evaluated against human preferences of which objects belong where in a tidy house. Specifically, we collect a dataset of where humans typically place objects in tidy and untidy houses constituting 1799 objects, 268 object categories, 585 placements, and 105 rooms. Next, we propose a modular baseline approach for Housekeep that integrates planning, exploration, and navigation. It leverages a fine-tuned large language model (LLM) trained on an internet text corpus for effective planning. We find that our baseline planner generalizes to some extent when rearranging objects in unknown environments. See our webpage for code, data and more details: https://yashkant.github.io/housekeep/ .
Hybridizing Euclidean and Hyperbolic Similarities for Attentively Refining Representations in Semantic Segmentation of Remote Sensing Images

Attention mechanisms have revolutionized the semantic segmentation network in interpreting remotely sensed images (RSIs) due to their amazing ability in establishing contextual dependencies.Nevertheless, due to the complex scenes and diverse objects in RSIs, a variety of details and correlations are not available in Euclidean space.Therefore, a similarityhybrid attention module (SHAM) is devised to attentively learn the hyperbolic and Euclidean attention maps between any two positions, followed by a weighted element-wise summation.The hybrid attention maps posses latent geometric properties of both Euclidean and hyperboloid.Taking commonly-used fully convolutional network (FCN) as baseline, HAENet that embeds SHAM, is presented .Experiments on ISPRS Potsdam and DeepGlobe benchmarks reveal its superiority to comparative methods.In addition, the ablation study validates the effectiveness of SHAM compared to other attention modules.
CODER: Coupled Diversity-Sensitive Momentum Contrastive Learning for Image-Text Retrieval

Image-Text Retrieval (ITR) is challenging in bridging visual and lingual modalities. Contrastive learning has been adopted by most prior arts. Except for limited amount of negative image-text pairs, the capability of constrastive learning is restricted by manually weighting negative pairs as well as unawareness of external knowledge. In this paper, we propose our novel Coupled Diversity-Sensitive Momentum Constrastive Learning (CODER) for improving cross-modal representation. Firstly, a novel diversity-sensitive contrastive learning (DCL) architecture is invented. We introduce dynamic dictionaries for both modalities to enlarge the scale of image-text pairs, and diversity-sensitiveness is achieved by adaptive negative pair weighting. Furthermore, two branches are designed in CODER. One learns instance-level embeddings from image/text, and it also generates pseudo online clustering labels for its input image/text based on their embeddings. Meanwhile, the other branch learns to query from commonsense knowledge graph to form concept-level descriptors for both modalities. Afterwards, both branches leverage DCL to align the cross-modal embedding spaces while an extra pseudo clustering label prediction loss is utilized to promote concept-level representation learning for the second branch. Extensive experiments conducted on two popular benchmarks, i.e. MSCOCO and Flicker30K, validate CODER remarkably outperforms the state-of-the-art approaches.
Multi-speed transformer network for neurodegenerative disease assessment and activity recognition

Neurodegenerative diseases are the most frequent age-related diseases. This type of disease, if not discovered in the initial stage, will compromise the quality of life of the affected subject. Thus, a timely diagnosis is of paramount importance. One of the most used tasks from neurologists to detect and determine the severity of the disease is analysing human gait. This work presents the dataset named “Beside Gait” containing timeseries of coordinates of extracted body joints of people with neurodegenerative diseases in various stages of the disease as well as control subjects. In addition, the novel Multi-Speed transformer technique will be presented and benchmarked against several other techniques making use of deep learning and Shallow Learning. The objective is to recognize subjects affected by some form of neurodegenerative disease in early stage using a computer vision technique making use of deep learning that can be integrated into a smartphone app for offline inference with the aim of promptly initiate investigations and treatment to improve the patient's quality of life. The recorded videos were processed, and the skeleton of the person in the video was extracted using pose estimation. The raw time-series coordinates of the joints extracted by the pose estimation algorithm were tested against novel deep neural network architectures and Shallow Learning techniques. In this work, the proposed Multi-Speed Transformer is benchmarked against other deep neural networks such as Temporal Convolutional Neural Networks, Transformers, as well as Shallow Learning techniques making use of feature extraction and different classifiers such as Random Forests, K Nearest Neighbours, Ada Boost, Linear and RBF SVM. The proposed Multi-Speed Transformer architecture has been developed to learn short and long-term patterns to model the various pathological gaits. The Multi-Speed Transformer outperformed all other existing models reaching an accuracy of 96.9%, a sensitivity of 96.9%, a precision of 97.7%, and a specificity of 97.1% in binary classification. The accuracy in multi-class classification for detecting the presence of the disease in various stages is 71.6%, the sensitivity is 67.7%, and the specificity is 71.8%. In addition, tests have also been conducted against two other different activity recognition datasets, namely SHREC and JHMDB, in the exact same conditions. Multi-Speed Transformer has demonstrated to beat always all other tested techniques as well as the techniques reviewed in the state-of-the-art with respectively of accuracy 91.8% and 74%. Having those datasets more than two classes, specificity was not computed. The Multi-Speed Transformer is a valuable technique for neurodegenerative disease assessment through computer vision. In addition, the novel dataset “Beside Gait” here presented is an important starting point for future research work on automatic recognition of neurodegenerative diseases using gait analysis.
Efficient fully-coherent quantum signal processing algorithms for real-time dynamics simulation

Simulating the unitary dynamics of a quantum system is a fundamental problem of quantum mechanics, in which quantum computers are believed to have significant advantage over their classical counterparts. One prominent such instance is the simulation of electronic dynamics, which plays an essential role in chemical reactions, non-equilibrium dynamics, and material design. These systems are time-dependent, which requires that the corresponding simulation algorithm can be successfully concatenated with itself over different time intervals to reproduce the overall coherent quantum dynamics of the system. In this paper, we quantify such simulation algorithms by the property of being fully-coherent: the algorithm succeeds with arbitrarily high success probability 1 - δ while only requiring a single copy of the initial state. We subsequently develop fully-coherent simulation algorithms based on quantum signal processing (QSP), including a novel algorithm that circumvents the use of amplitude amplification while also achieving a query complexity additive in time t, ln(1/δ), and ln(1/ϵ) for error tolerance ϵ: Θ‖H‖|t|+ln(1/ϵ)+ln(1/δ). Furthermore, we numerically analyze these algorithms by applying them to the simulation of the spin dynamics of the Heisenberg model and the correlated electronic dynamics of an H2 molecule. Since any electronic Hamiltonian can be mapped to a spin Hamiltonian, our algorithm can efficiently simulate time-dependent ab initio electronic dynamics in the circuit model of quantum computation. Accordingly, it is also our hope that the present work serves as a bridge between QSP-based quantum algorithms and chemical dynamics, stimulating a cross-fertilization between these exciting fields.
Continual learning strategies for cancer-independent detection of lymph node metastases

Recently, large, high-quality public datasets have led to the development of convolutional neural networks that can detect lymph node metastases of breast cancer at the level of expert pathologists. Many cancers, regardless of the site of origin, can metastasize to lymph nodes. However, collecting and annotating high-volume, high-quality datasets for every cancer type is challenging. In this paper we investigate how to leverage existing high-quality datasets most efficiently in multi-task settings for closely related tasks. Specifically, we will explore different training and domain adaptation strategies, including prevention of catastrophic forgetting, for breast, colon and head-and-neck cancer metastasis detection in lymph nodes. Our results show state-of-the-art performance on colon and head-and-neck cancer metastasis detection tasks. We show the effectiveness of adaptation of networks from one cancer type to another to obtain multi-task metastasis detection networks. Furthermore, we show that leveraging existing high-quality datasets can significantly boost performance on new target tasks and that catastrophic forgetting can be effectively mitigated.Last, we compare different mitigation strategies.
Few-shot link prediction for temporal knowledge graphs based on time-aware translation and attention mechanism

Few-shot knowledge graph completion (KGC) is an important and common task in real applications, which aims to predict unseen facts when only few samples are available for each relation in the knowledge graph (KG). Previous methods on few-shot KGC mainly focus on static KG, however, many KG in real-world applications are dynamic and develop over time. In this work, we consider few-shot KGC in temporal knowledge graphs (TKGs), where the fact may only hold for a specific timestamp. We propose a Few-Shot Completion model in TKG (TFSC), which compare the input query to the given few-shot references to make predictions. Specifically, in order to enhance the representation of entities in the case of few samples, we use the attention mechanism to model the neighbor entities of the task entity with timestamp information, and generate expressive time-aware entity pair representations through the Transformer encoder. A comprehensive set of experiments is finally carried out to demonstrate the effectiveness a of our proposed model TFSC.
Increasing frame rate of echocardiography based on a novel 2D spatio-temporal meshless interpolation

Increasing temporal resolution through numerical methods aids clinicians to evaluate fast moving structures of the heart with more confidence. In this study, a spatio-temporal numerical method is proposed to increase the frame rate based on two-dimensional (2D) interpolation. More specifically, we propose a novel intensity variation time surface (IVTS) strategy to incorporate both temporal and spatial information in the reconstruction. In this regard, we exploit radial basis functions (RBFs) for 2D interpolation. The reason for choosing RBFs for this task is manifold. First, RBFs are able to interpolate on large-scale datasets. Moreover, their mathematical implementation is simple. Another important property of this interpolation technique, which is addressed in this study, is its meshless nature. The meshless property enables higher up-sampling (UpS) rates for echocardiography to improve temporal resolution without noticeably degrading image quality. To evaluate the proposed approach, we tested the RBF interpolation on 2D/3D echocardiography datasets. The reconstructed frames were analyzed using different image quality metrics, and the results were compared with two popular techniques from the literature. The findings demonstrated that, with a down-sampling rate of 3, the proposed technique outperformed the best existing method by 42%, 87%, 8%, and 11%, respectively, in terms of mean square error (MSE), contrast to noise ratio (CNR), peak signal-to-noise ratio (PSNR), and figure of merit (FOM). It should be noted that the proposed method is comparable to the best available method in terms of structural similarity (SSIM) index. Furthermore, when compared to the original images, the results of employing our technique on radio-frequency (RF) level analysis demonstrated that the reconstruction accuracy is satisfactory in terms of image quality criterion. Finally, it is worthwhile noting that the proposed method is better than (or comparable to) the other methods in terms of reconstruction performance and processing time. Therefore, the RBF interpolation can be a promising alternative to the existing methods.
Metaverse office: exploring future teleworking model

Purpose This study aims to explore the application scenarios of metaverse offices in organizations, including immersive work experiences, weak social networks and virtual meetings. Based on the exploration of this application scenario, this study discusses four levels of organizational change that will be brought about by the future metaverse office, such as the inter-enterprise ecological collaboration model, distributed autonomous organization, flexible leadership and individual belongingness. The metaverse office has some special advantages over traditional office models, but the metaverse office still raises some issues, such as privacy, security, addiction, equity and usability. These issues brake the widespread adoption of metaverse technologies. The study recommends that researchers take these issues into account in future metaverse research and development. Design/methodology/approach This study surveys the relevant literature by means of a literature review in order to analyze how metaverse technology can be applied to teleworking. Two databases including Web of Science ( https://www.webofscience.com/ ) and Google Scholar ( http://scholar.google.com ) were selected for this study. Keywords such as “teleworking and metaverse” were used, and 18 publications were found to be relevant to the study. After excluding duplicates, less relevant and older literature, only 14 articles could be used as references for this study. Based on the exploration of this application scenario, this study discusses four levels of organizational change and issues that will be brought about by the metaverse office of the future. Findings It presents the application scenarios of the metaverse office in organizations, including immersive work experience, weak social networking and virtual meetings. Social implications The pandemic triggers a desire for contactless working. The three main applications of the metaverse office also have a practical value which has been proven in some high-tech companies. It is foreseeable that an efficient, electronic and personalized office model will be enthusiastically adopted by society. Metaverse office will gradually step into people's view in the future. Originality/value The concept of the metaverse office has rarely been touched upon by theoretical research, although the technology is gradually becoming known. Even its application to teleworking has only recently been reached by some organizations.
Optimized mitigation of random-telegraph-noise dephasing by spectator-qubit sensing and control

Spectator qubits (SQs) have been proposed as a way to estimate, and thus mitigate, the effect of time-varying noise in hard-to-access data qubits. Using optimal control theory, this Letter shows that a single SQ suffices to suppress the decoherence from a common type of noise in quantum devices by an amount quadratic in the SQ sensitivity.
Forget less, count better: a domain-incremental self-distillation learning benchmark for lifelong crowd counting

Crowd counting has important applications in public safety and pandemic control. A robust and practical crowd counting system has to be capable of continuously learning with the newly incoming domain data in real-world scenarios instead of fitting one domain only. Off-the-shelf methods have some drawbacks when handling multiple domains: (1) the models will achieve limited performance (even drop dramatically) among old domains after training images from new domains due to the discrepancies in intrinsic data distributions from various domains, which is called catastrophic forgetting; (2) the well-trained model in a specific domain achieves imperfect performance among other unseen domains because of domain shift; (3) it leads to linearly increasing storage overhead, either mixing all the data for training or simply training dozens of separate models for different domains when new ones are available. To overcome these issues, we investigate a new crowd counting task in incremental domain training setting called lifelong crowd counting. Its goal is to alleviate catastrophic forgetting and improve the generalization ability using a single model updated by the incremental domains. Specifically, we propose a self-distillation learning framework as a benchmark (forget less, count better, or FLCB) for lifelong crowd counting, which helps the model leverage previous meaningful knowledge in a sustainable manner for better crowd counting to mitigate the forgetting when new data arrive. A new quantitative metric, normalized Backward Transfer (nBwT), is developed to evaluate the forgetting degree of the model in the lifelong learning process. Extensive experimental results demonstrate the superiority of our proposed benchmark in achieving a low catastrophic forgetting degree and strong generalization ability.
Progressive Instance-Aware Feature Learning for Compositional Action Recognition

In order to enable the model to generalize to unseen “action-objects” (compositional action), previous methods encode multiple pieces of information (i.e., the appearance, position, and identity of visual instances) independently and concatenate them for classification. However, these methods ignore the potential supervisory role of instance information (i.e., position and identity) in the process of visual perception. To this end, we present a novel framework, namely Progressive Instance-aware Feature Learning (PIFL), to progressively extract, reason, and predict dynamic cues of moving instances from videos for compositional action recognition. Specifically, this framework extracts features from foreground instances that are likely to be relevant to human actions (Position-aware Appearance Feature Extraction in Section III-B1), performs identity-aware reasoning among instance-centric features with semantic-specific interactions (Identity-aware Feature Interaction in Section III-B2), and finally predicts instances’ position from observed states to force the model into perceiving their movement (Semantic-aware Position Prediction in Section III-B3). We evaluate our approach on two compositional action recognition benchmarks, namely, Something-Else and IKEA-Assembly. Our approach achieves consistent accuracy gain beyond off-the-shelf action recognition algorithms in terms of both ground truth and detected position of instances.
A Novel Transfer Learning-Based Model for Ultrasound Breast Cancer Image Classification

Breast cancer is the second most dangerous disease for women after lung cancer. As in most diseases, an early detection and the corresponding treatment of breast cancer increase the survival rate of patients. An automated system for detection of breast cancer is required as manual techniques are time consuming and expensive. In this study, we have proposed a novel transfer learning model that can classify an ultrasound of the breast as either normal, benign, or malignant with high accuracy. The proposed method uses a modified ResNet50 model trained initially on ImageNet dataset and further on the breast cancer ultrasound dataset (BUSI). We have added custom layers at the head of our model which are able to extract features from ultrasound images. Using the model described in this paper, we have achieved 97.8% accuracy in detecting breast cancer, a recall of 97.68%, precision of 99.21% and 98.44% F1-score. This deep learning model can be implemented as a component of an existing medical diagnosis system or deployed as a stand-alone system. Using our model for breast cancer diagnosis can result in decreased diagnosis time compared to traditional means and hence ensure that patients receive an early treatment for their illness.
Embodiment, immersion, and enjoyment in virtual reality marketing experiences

Abstract This experiential marketing study examines the effects of body position and embodiment in shaping consumer intentions mediated by immersion and enjoyment during a virtual reality experience. After a qualitative study ( n = 13) aimed at understanding what, in a virtual reality experience, motivates individuals to want to consume the promoted offer, we conducted a quantitative study ( n = 300), using a partial least squares structural equation model. Our experimental context placed the users in the role of a kayaker discovering Antarctica via a virtual reality head‐mounted device. The results confirm the positive relationships implied by the conceptual model underpinning the research. The embodiment felt by the users has positive effects on immersion and enjoyment, which in turn positively shape the user‐consumer's intention to discover the destination for real. This study brings theoretical contributions that provide a clearer definition of embodiment in an experiential context. This study also provides professionals with guidance on the choice of virtual reality system contexts and experiential factors to stimulate positive behavioral intentions, in this case to discover a place for real.