Reviewerly/exHarmony huggingface dataset:
Realizing quantum convolutional neural networks on a superconducting quantum processor to recognize quantum phases

Quantum computing crucially relies on the ability to efficiently characterize the quantum states output by quantum hardware. Conventional methods which probe these states through direct measurements and classically computed correlations become computationally expensive when increasing the system size. Quantum neural networks tailored to recognize specific features of quantum states by combining unitary operations, measurements and feedforward promise to require fewer measurements and to tolerate errors. Here, we realize a quantum convolutional neural network (QCNN) on a 7-qubit superconducting quantum processor to identify symmetry-protected topological (SPT) phases of a spin model characterized by a non-zero string order parameter. We benchmark the performance of the QCNN based on approximate ground states of a family of cluster-Ising Hamiltonians which we prepare using a hardware-efficient, low-depth state preparation circuit. We find that, despite being composed of finite-fidelity gates itself, the QCNN recognizes the topological phase with higher fidelity than direct measurements of the string order parameter for the prepared states.
Breast Cancer Calcifications: Identification Using a Novel Segmentation Approach

Breast cancer is a strong risk factor of cancer amongst women. One in eight women suffers from breast cancer. It is a life-threatening illness and is utterly dreadful. The root cause which is the breast cancer agent is still under research. There are, however, certain potentially dangerous factors like age, genetics, obesity, birth control, cigarettes, and tablets. Breast cancer is often a malignant tumor that begins in the breast cells and eventually spreads to the surrounding tissue. If detected early, the illness may be reversible. The probability of preservation diminishes as the number of measurements increases. Numerous imaging techniques are used to identify breast cancer. This research examines different breast cancer detection strategies via the use of imaging techniques, data mining techniques, and various characteristics, as well as a brief comparative analysis of the existing breast cancer detection system. Breast cancer mortality will be significantly reduced if it is identified and treated early. There are technological difficulties linked to scans and people's inconsistency with breast cancer. In this study, we introduced a form of breast cancer diagnosis. There are different methods involved to collect and analyze details. In the preprocessing stage, the input data picture is filtered by using a window or by cropping. Segmentation can be performed using k-means algorithm. This study is aimed at identifying the calcifications found in bosom cancer in the last phase. The suggested approach is already implemented in MATLAB, and it produces reliable performance.
Cloud based secure watermarking using IWT-Schur-RSVD with fuzzy inference system for smart healthcare applications

The healthcare records include personal and sensitive information about the patient. These records are easily distributed via different platform for clinical diagnosis purpose. Presently, large volumes of medical data are stored on cloud platform. However, outsourcing medical data to these popular platforms may introduce security issues. Thus, a secure watermarking algorithm based on integer wavelet transform (IWT)-Schur-Randomized Singular Value Decomposition (RSVD) is put forward for medical records sharing in the cloud environment. To ensure high level of authentication, discrete wavelet transform (DWT) based embedding is conducted to achieve dual watermarking by concealing system MAC address in the logo image. Prior to this, the system address is encoded via turbo code to reduce /eliminate the channel noise, if any. Further, both invisibility and robustness are achieved by fuzzy inference system (FIS), and the marked image is encrypted using chaotic encryption scheme to further improve the security. Furthermore, experimental results is reported and compared with the existing works which demonstrated promising outcomes, indicating its potential towards realization of a value-added tool for smart healthcare applications.
GAN Inversion for Out-of-Range Images with Geometric Transformations

For successful semantic editing of real images, it is critical for a GAN inversion method to find an in-domain latent code that aligns with the domain of a pre-trained GAN model. Unfortunately, such in-domain latent codes can be found only for in-range images that align with the training images of a GAN model. In this paper, we propose BDInvert, a novel GAN inversion approach to semantic editing of out- of-range images that are geometrically unaligned with the training images of a GAN model. To find a latent code that is semantically editable, BDInvert inverts an input out-of-range image into an alternative latent space than the original latent space. We also propose a regularized inversion method to find a solution that supports semantic editing in the alternative space. Our experiments show that BDInvert effectively supports semantic editing of out-of-range images with geometric transformations.
Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-Modal Pretraining

Nowadays, customer’s demands for E-commerce are more diversified, which introduces more complications to the product retrieval industry. Previous methods are either subject to single-modal input or perform supervised image-level product retrieval, thus fail to accommodate real-life scenarios where enormous weakly annotated multi-modal data are present. In this paper, we investigate a more realistic setting that aims to perform weakly-supervised multi-modal instance-level product retrieval among fine-grained product categories. To promote the study of this challenging task, we contribute Product1M, one of the largest multi-modal cosmetic datasets for real-world instance-level retrieval. Notably, Product1M contains over 1 million image-caption pairs and consists of two sample types, i.e., single-product and multi-product samples, which encompass a wide variety of cosmetics brands. In addition to the great diversity, Product1M enjoys several appealing characteristics including fine-grained categories, complex combinations, and fuzzy correspondence that well mimic the real-world scenes. Moreover, we propose a novel model named Cross-modal contrAstive Product Transformer for instance-level prodUct REtrieval (CAPTURE), that excels in capturing the potential synergy between multi-modal inputs via a hybrid-stream transformer in a self-supervised manner. CAPTURE generates discriminative instance features via masked multi-modal learning as well as cross-modal contrastive pretraining and it outperforms several SOTA cross-modal baselines. Extensive ablation studies well demonstrate the effectiveness and the generalization capacity of our model. Dataset and codes are available at https: //github.com/zhanxlin/Product1M.
Modified Artificial Ecosystem-Based Optimization for Multilevel Thresholding Image Segmentation

Multilevel thresholding is one of the most effective image segmentation methods, due to its efficiency and easy implementation. This study presents a new multilevel thresholding method based on a modified artificial ecosystem-based optimization (AEO). The differential evolution (DE) is applied to overcome the shortcomings of the original AEO. The main idea of the proposed method, artificial ecosystem-based optimization differential evolution (AEODE), is to employ the operators of the DE as a local search of the AEO to improve the ecosystem of solutions. We used benchmark images to test the performance of the AEODE, and we compared it to several existing approaches. The proposed AEODE achieved a high performance when evaluated by the structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), and fitness values. Moreover, the AEODE outperformed the basic version of the AEO concerning SSIM and PSNR by 78% and 82%, respectively, which reserves the best features for each of AEO and DE.
Transaction fee economics in the Ethereum blockchain

Abstract We study the economic determinants of transaction fees in the Ethereum blockchain. We estimate an empirical model based on queueing theory and analyze the factors determining the “gas price” (transaction cost per unit of service, “gas”). Using block‐ and transaction‐level data from the Ethereum blockchain, we show that changes in service demand significantly affect the gas price—when there is high block utilization, per‐unit fees increase on average, with strong nonlinear effect above 90% utilization. The transaction type is another important factor—larger fraction of regular transactions (direct transfers between users) is associated with higher gas price.
An Adaptive Robustness Evolution Algorithm With Self-Competition and its 3D Deployment for Internet of Things

Internet of Things (IoT) includes numerous sensing nodes that constitute a large scale-free network. Optimizing the network topology to increase resistance against malicious attacks is a complex problem, especially on 3-dimension (3D) topological deployment. Heuristic algorithms, particularly genetic algorithms, can effectively cope with such problems. However, conventional genetic algorithms are prone to falling into premature convergence owing to the lack of global search ability caused by the loss of population diversity during evolution. Although this can be alleviated by increasing population size, the additional computational overhead will be incurred. Moreover, after crossover and mutation operations, individual changes in the population are mixed, and loss of optimal individuals may occur, which will slow down the population&#x2019;s evolution. Therefore, we combine the population state with the evolutionary process and propose an Adaptive Robustness Evolution Algorithm (AREA) with self-competition for scale-free IoT topologies. In AREA, the crossover and mutation operations are dynamically adjusted according to population diversity to ensure global search ability. A self-competitive mechanism is used to ensure convergence. We construct a 3D IoT topology that is optimized by AREA. The simulation results demonstrate that AREA is more effective in improving the robustness of scale-free IoT networks than several existing methods.
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

We introduce DiscoBox, a novel framework that jointly learns instance segmentation and semantic correspondence using bounding box supervision. Specifically, we propose a self-ensembling framework where instance segmentation and semantic correspondence are jointly guided by a structured teacher in addition to the bounding box supervision. The teacher is a structured energy model incorporating a pairwise potential and a cross-image potential to model the pairwise pixel relationships both within and across the boxes. Minimizing the teacher energy simultaneously yields refined object masks and dense correspondences between intra-class objects, which are taken as pseudo-labels to supervise the task network and provide positive/negative correspondence pairs for dense contrastive learning. We show a symbiotic relationship where the two tasks mutually benefit from each other. Our best model achieves 37.9% AP on COCO instance segmentation, surpassing prior weakly supervised methods and is competitive to supervised methods. We also obtain state of the art weakly supervised results on PASCAL VOC12 and PF-PASCAL with real-time inference.
WENETSPEECH: A 10000+ Hours Multi-Domain Mandarin Corpus for Speech Recognition

In this paper, we present WenetSpeech, a multi-domain Mandarin corpus consisting of 10000+ hours high-quality labeled speech, 2400+ hours weakly labeled speech, and about 10000 hours unlabeled speech, with 22400+ hours in total. We collect the data from YouTube and Podcast, which covers a variety of speaking styles, scenarios, domains, topics and noisy conditions. An optical character recognition (OCR) method is introduced to generate the audio/text segmentation candidates for the YouTube data on the corresponding video subtitles, while a high-quality ASR transcription system is used to generate audio/text pair candidates for the Podcast data. Then we propose a novel end-to-end label error detection approach to further validate and filter the candidates. We also provide three manually labelled high-quality test sets along with WenetSpeech for evaluation – Dev for cross-validation purpose in training, Test_Net, collected from Internet for matched test, and Test_Meeting, recorded from real meetings for more challenging mismatched test. Baseline systems trained with WenetSpeech are provided for three popular speech recognition toolkits, namely Kaldi, ESPnet, and WeNet, and recognition results on the three test sets are also provided as benchmarks. To the best of our knowledge, WenetSpeech is the current largest open-source Mandarin speech corpus with transcriptions, which benefits research on production-level speech recognition.
Face Recognition Smart Attendance System using Deep Transfer Learning

Face identification has been considered an interesting research domain in the past few years as it plays a major biometric authentication role in several applications including attendance management and access control systems. Attendance management systems are very important to all organization though they are complex and time-consuming for managing regular attendance log. There are many automated human identification techniques such as biometrics, RFID, eye tracking, voice recognition. Face is one of the most broadly used biometrics for human identity authentication. This paper presents a facial recognition attendance system based on deep learning convolutional neural networks. We utilize transfer learning by using three pre-trained convolutional neural networks and trained them on our data. The three networks showed very high performance in terms of high prediction accuracy and reasonable training time.
Dual Bipartite Graph Learning: A General Approach for Domain Adaptive Object Detection

Domain Adaptive Object Detection (DAOD) relieves the reliance on large-scale annotated data by transferring the knowledge learned from a labeled source domain to a new unlabeled target domain. Recent DAOD approaches resort to local feature alignment in virtue of domain adversarial training in conjunction with the ad-hoc detection pipelines to achieve feature adaptation. However, these methods are limited to adapt the specific types of object detectors and do not explore the cross-domain topological relations. In this paper, we first formulate DAOD as an open-set domain adaptation problem in which foregrounds (pixel or region) can be seen as the “known class”, while backgrounds (pixel or region) are referred to as the “unknown class”. To this end, we present a new and general perspective for DAOD named Dual Bipartite Graph Learning (DBGL), which captures the cross-domain interactions on both pixel-level and semantic-level via increasing the distinction between foregrounds and backgrounds and modeling the cross-domain dependencies among different semantic categories. Experiments reveal that the proposed DBGL in conjunction with one-stage and two-stage detectors exceeds the state-of-the-art performance on standard DAOD benchmarks.
STVGBert: A Visual-linguistic Transformer based Framework for Spatio-temporal Video Grounding

Spatio-temporal video grounding (STVG) aims to localize a spatio-temporal tube of a target object in an untrimmed video based on a query sentence. In this work, we propose a one-stage visual-linguistic transformer based framework called STVGBert for the STVG task, which can simultaneously localize the target object in both spatial and temporal domains. Specifically, without resorting to pregenerated object proposals, our STVGBert directly takes a video and a query sentence as the input, and then produces the cross-modal features by using the newly introduced cross-modal feature learning module ST-ViLBert. Based on the cross-modal features, our method then generates bounding boxes and predicts the starting and ending frames to produce the predicted object tube. To the best of our knowledge, our STVGBert is the first one-stage method, which can handle the STVG task without relying on any pre-trained object detectors. Comprehensive experiments demonstrate our newly proposed framework outperforms the state-of-the-art multi-stage methods on two benchmark datasets Vid-STG and HC-STVG.
ASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistency

We study self-supervised video representation learning, which is a challenging task due to 1) lack of labels for explicit supervision; 2) unstructured and noisy visual information. Existing methods mainly use contrastive loss with video clips as the instances and learn visual representation by discriminating instances from each other, but they need a careful treatment of negative pairs by either relying on large batch sizes, memory banks, extra modalities or customized mining strategies, which inevitably includes noisy data. In this paper, we observe that the consistency between positive samples is the key to learn robust video representation. Specifically, we propose two tasks to learn appearance and speed consistency, respectively. The appearance consistency task aims to maximize the similarity between two clips of the same video with different playback speeds. The speed consistency task aims to maximize the similarity between two clips with the same playback speed but different appearance information. We show that optimizing the two tasks jointly consistently improves the performance on downstream tasks, e.g., action recognition and video retrieval. Remarkably, for action recognition on the UCF-101 dataset, we achieve 90.8% accuracy without using any extra modalities or negative pairs for unsupervised pretraining, which outperforms the ImageNet supervised pretrained model. Codes and models will be available.
Immersive Virtual Reality Exergames for Persons Living With Dementia: User-Centered Design Study as a Multistakeholder Team During the COVID-19 Pandemic

Advancements in supporting personalized health care and well-being using virtual reality (VR) have created opportunities to use immersive games to support a healthy lifestyle for persons living with dementia and mild cognitive impairment (MCI). Collaboratively designing exercise video games (exergames) as a multistakeholder team is fundamental to creating games that are attractive, effective, and accessible.This research extensively explores the use of human-centered design methods that involve persons living with dementia in long-term care facilitates, exercise professionals, content developers, game designers, and researchers in the creation of VR exergames targeting physical activity promotion for persons living with dementia/MCI.Conceptualization, collaborative design, and playtesting activities were carried out to design VR exergames to engage persons living with dementia in exercises to promote upper limb flexibility, strength, and aerobic endurance. We involved a total of 7 persons living with dementia/MCI, 5 exercise professionals, 5 community-dwelling older adults, a VR company for content creation, and a multidisciplinary research team with game designers, engineers, and kinesiology experts.An immersive VR exergame called Seas the Day was jointly designed and developed and it is freely available to be played in state-of-the-art VR headsets (Oculus Quest 1, 2). A model for the triadic interaction (health care institution, industry partner, academia) is also presented to illustrate how different stakeholders contribute to the design of VR exergames that consider/complement complex needs, preferences, and motivators of an underrepresented group of end users.This study provides evidence that a collaborative multistakeholder design results in more tailored and context-aware VR games for persons living with dementia. The insights and lessons learned from this research can be used by others to co-design games, including remote engagement techniques that were used during the COVID-19 pandemic.
Multi-Task Self-Training for Learning General Representations

Despite the fast progress in training specialized models for various tasks, learning a single general model that works well for many tasks is still challenging for computer vision. Here we introduce multi-task self-training (MuST), which harnesses the knowledge in independent specialized teacher models (e.g., ImageNet model on classification) to train a single general student model. Our approach has three steps. First, we train specialized teachers independently on labeled datasets. We then use the specialized teachers to label an unlabeled dataset to create a multi-task pseudo labeled dataset. Finally, the dataset, which now contains pseudo labels from teacher models trained on different datasets/tasks, is then used to train a student model with multi-task learning. We evaluate the feature representations of the student model on 6 vision tasks including image recognition (classification, detection, segmentation) and 3D geometry estimation (depth and surface normal estimation). MuST is scalable with unlabeled or partially labeled datasets and outperforms both specialized supervised models and self-supervised models when training on large scale datasets. Lastly, we show MuST can improve upon already strong checkpoints [23] trained with billions of examples. The results suggest self-training is a promising direction to aggregate labeled and unlabeled training data for learning general feature representations.
The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation

Vision-and-Language Navigation (VLN) requires an agent to find a path to a remote location on the basis of natural-language instructions and a set of photo-realistic panoramas. Most existing methods take the words in the instructions and the discrete views of each panorama as the minimal unit of encoding. However, this requires a model to match different nouns (e.g., TV, table) against the same input view feature. In this work, we propose an object-informed sequential BERT to encode visual perceptions and linguistic instructions at the same fine-grained level, namely objects and words. Our sequential BERT also enables the visual-textual clues to be interpreted in light of the temporal context, which is crucial to multi-round VLN tasks. Additionally, we enable the model to identify the relative direction (e.g., left/right/front/back) of each navigable location and the room type (e.g., bedroom, kitchen) of its current and final navigation goal, as such information is widely mentioned in instructions implying the desired next and final locations. We thus enable the model to know-where the objects lie in the images, and to know-where they stand in the scene. Extensive experiments demonstrate the effectiveness compared against several state-of-the-art methods on three indoor VLN tasks: REVERIE, NDH, and R2R. Project repository: https://github.com/YuankaiQi/ORIST