Reviewerly/exHarmony huggingface dataset:
FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning

Multimodal tasks in the fashion domain have significant potential for e-commerce, but involve challenging vision-and-language learning problems-e.g., retrieving a fashion item given a reference image plus text feedback from a user.Prior works on multimodal fashion tasks have either been limited by the data in individual benchmarks, or have leveraged generic vision-and-language pre-training but have not taken advantage of the characteristics of fashion data.Additionally, these works have mainly been restricted to multimodal understanding tasks.To address these gaps, we make two key contributions.First, we propose a novel fashion-specific pre-training framework based on weakly-supervised triplets constructed from fashion image-text pairs.We show the tripletbased tasks are an effective addition to standard multimodal pre-training tasks.Second, we propose a flexible decoder-based model architecture capable of both fashion retrieval and captioning tasks.Together, our model design and pre-training approach are competitive on a diverse set of fashion tasks, including crossmodal retrieval, image retrieval with text feedback, image captioning, relative image captioning, and multimodal categorization.
SYGMA: A System for Generalizable and Modular Question Answering Over Knowledge Bases

Knowledge Base Question Answering (KBQA) involving complex reasoning is emerging as an important research direction.However, most KBQA systems struggle with generalizability, particularly on two dimensions: (a) across multiple knowledge bases, where existing KBQA approaches are typically tuned to a single knowledge base, and (b) across multiple reasoning types, where majority of datasets and systems have primarily focused on multi-hop reasoning.In this paper, we present SYGMA, a modular KBQA approach developed with goal of generalization across multiple knowledge bases and multiple reasoning types.To facilitate this, SYGMA is designed as two high level modules: 1) KB-agnostic question understanding module that remain common across KBs, and generates logic representation of the question with high level reasoning constructs that are extensible, and 2) KB-specific question mapping and answering module to address the KB-specific aspects of the answer extraction.We evaluated SYGMA on multiple datasets belonging to distinct knowledge bases (DBpedia and Wikidata) and distinct reasoning types (multi-hop and temporal).State-of-the-art or competitive performances achieved on those datasets demonstrate its generalization capability.
CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision

Scientific extreme summarization (TLDR) aims to form ultra-short summaries of scientific papers.Previous efforts on curating scientific TLDR datasets failed to scale up due to the heavy human annotation and domain expertise required.In this paper, we propose a simple yet effective approach to automatically extracting TLDR summaries for scientific papers from their citation texts.Based on the proposed approach, we create a new benchmark CiteSum without human annotation, which is around 30 times larger than the previous human-curated dataset SciTLDR.We conduct a comprehensive analysis of CiteSum, examining its data characteristics and establishing strong baselines.We further demonstrate the usefulness of CiteSum by adapting models pre-trained on CiteSum (named CITES) to new tasks and domains with limited supervision.For scientific extreme summarization, CITES outperforms most fully-supervised methods on SciTLDR without any fine-tuning and obtains state-of-theart results with only 128 examples.For news extreme summarization, CITES achieves significant gains on XSum over its base model (not pre-trained on CiteSum), e.g., +7.2 ROUGE-1 zero-shot performance and state-of-the-art few-shot performance.For news headline generation, CITES performs the best among unsupervised and zero-shot methods on Gigaword. 1
Disentangling Task Relations for Few-shot Text Classification via Self-Supervised Hierarchical Task Clustering

Few-Shot Text Classification (FSTC) imitates humans to learn a new text classifier efficiently with only few examples, by leveraging prior knowledge from historical tasks.However, most prior works assume that all the tasks are sampled from a single data source, which cannot adapt to real-world scenarios where tasks are heterogeneous and lie in different distributions.As such, existing methods may suffer from their globally knowledge-shared mechanisms to handle the task heterogeneity.On the other hand, inherent task relation are not explicitly captured, making task knowledge unorganized and hard to transfer to new tasks.Thus, we explore a new FSTC setting where tasks can come from a diverse range of data sources.To address the task heterogeneity, we propose a self-supervised hierarchical task clustering (SS-HTC) method.SS-HTC not only customizes cluster-specific knowledge by dynamically organizing heterogeneous tasks into different clusters in hierarchical levels but also disentangles underlying relations between tasks to improve the interpretability.Extensive experiments on five public FSTC benchmark datasets demonstrate the effectiveness of SS-HTC.
Unsupervised Multi-Granularity Summarization

Text summarization is a user-preference based task, i.e., for one document, users often have different priorities for summary.As a key aspect of customization in summarization, granularity is used to measure the semantic coverage between summary and source document.However, developing systems that can generate summaries with customizable semantic coverage is still an under-explored topic.In this paper, we propose the first unsupervised multi-granularity summarization framework, GRANUSUM.We take events as the basic semantic units of the source documents and propose to rank these events by their salience.We also develop a model to summarize input documents with given events as anchors and hints.By inputting different numbers of events, GRANUSUM is capable of producing multi-granular summaries in an unsupervised manner.Meanwhile, we annotate a new benchmark GranuDUC that contains multiple summaries at different granularities for each document cluster.Experimental results confirm the substantial superiority of GRANUSUM on multi-granularity summarization over strong baselines.Furthermore, by exploiting the event information, GRANUSUM also exhibits state-of-the-art performance under conventional unsupervised abstractive setting. 1 Multiple News Articles about Hurricane MitchHonduras braced for potential catastrophe Tuesday as Hurricane Mitch roared through the northwest Caribbean, churning up high waves and intense rain ... (Total 3,358 words) Summary of Coarse Granularity LevelHurricane Mitch, category 5 hurricane, brought widespread death and destruction to Central American, and Honduras was especially hard hit.(Total 19 words) Summary of Medium Granularity LevelHurricane Mitch approached Honduras on Oct. 27, 1998 with winds up to 180mph a Category 5 storm ... The European Union, international relief agencies, Mexico, the U.S., Japan, Taiwan, the U.K. and U.N. sent financial aid, relief workers
WR-One2Set: Towards Well-Calibrated Keyphrase Generation

Keyphrase generation aims to automatically generate short phrases summarizing an input document.The recently emerged ONE2SET paradigm (Ye et al., 2021) generates keyphrases as a set and has achieved competitive performance.Nevertheless, we observe serious calibration errors outputted by ONE2SET, especially in the over-estimation of ∅ token (means "no corresponding keyphrase").In this paper, we deeply analyze this limitation and identify two main reasons behind: 1) the parallel generation has to introduce excessive ∅ as padding tokens into training instances; and 2) the training mechanism assigning target to each slot is unstable and further aggravates the ∅ token over-estimation.To make the model wellcalibrated, we propose WR-ONE2SET which extends ONE2SET with an adaptive instancelevel cost Weighting strategy and a target Reassignment mechanism.The former dynamically penalizes the over-estimated slots for different instances thus smoothing the uneven training distribution.The latter refines the original inappropriate assignment and reduces the supervisory signals of over-estimated slots.Experimental results on commonly-used datasets demonstrate the effectiveness and generality of our proposed paradigm.
Boosting LightWeight Depth Estimation via Knowledge Distillation

Monocular depth estimation (MDE) methods are often either too computationally expensive or not accurate enough due to the trade-off between model complexity and inference performance. In this paper, we propose a lightweight network that can accurately estimate depth maps using minimal computing resources. We achieve this by designing a compact model that maximally reduces model complexity. To improve the performance of our lightweight network, we adopt knowledge distillation (KD) techniques. We consider a large network as an expert teacher that accurately estimates depth maps on the target domain. The student, which is the lightweight network, is then trained to mimic the teacher’s predictions. However, this KD process can be challenging and insufficient due to the large model capacity gap between the teacher and the student. To address this, we propose to use auxiliary unlabeled data to guide KD, enabling the student to better learn from the teacher’s predictions. This approach helps fill the gap between the teacher and the student, resulting in improved data-driven learning. The experiments show that our method achieves comparable performance to state-of-the-art methods while using only 1% of their parameters. Furthermore, our method outperforms previous lightweight methods regarding inference accuracy, computational efficiency, and generalizability.
Prospects of augmented reality and virtual reality for online education: a scientometric view

Purpose Augmented reality (AR) and virtual reality (VR) are becoming increasingly interwoven into people's daily lives, with applications in fields such as science, engineering, medical, the arts, education, finance and business. While much study has been conducted in the field of VR and AR in a variety of areas, less attention has been paid to online education. The purpose of this research is to conduct a systematic review of studies on AR and VR, as well as its applications in many disciplines of education, with a special emphasis on online education. Design/methodology/approach The data were gathered from the SCOPUS database under the subject “Augmented and Virtual Reality Applications in Online Education.” The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) format has been used to summarize the available literature and research in academics throughout the world. First, systematic literature reviews and later bibliometric analysis were conducted for performance and science mapping analysis. Findings The research adds to the body of knowledge about the role of AR and VR in online education and its future prospects. AR and VR have demonstrated a substantial contribution to facilitating teaching in a variety of fields, including engineering, medicine, nursing and chemistry. Additionally, this research indicates that AR and VR can be utilized to teach various areas and specialties. Additionally, it can be used in other aspects of online education, such as admissions, industry visits, training, internationalization and specific courses for students with disabilities. Practical implications This research lays the framework for educational institutions, legislators and researchers to investigate additional applications of VR and AR. Though this research suggests some possible uses for AR and VR in the online education system, more applications can be integrated into the online education system to fully leverage the potential of AR and VR worldwide. AR and VR have the potential to be a very cost-effective method of delivering online education on a wide scale in the future. Originality/value The article explores the applicability of VR and AR in education using bibliometric analysis. This is one of the first studies to do an assessment of VR and AR, evaluating their strengths, drawbacks and current applications. Future application areas are also identified as a result of the investigation.
Reservoir computing using arrayed waveguide grating

We proposed a 32-channel 50G AWG as the core device to implement the on-chip optical reservoir computing. With appropriate iterative mode and output-level training strategy, the chip has achieved good results in predicting Macky-Glass series. For sequences of 1000 length, this type of optical reservoir calculation can obtain an absolute error of about 0.02 and a relative error of 0.018.
Non-uniform Speaker Disentanglement For Depression Detection From Raw Speech Signals

While speech-based depression detection methods that use speaker-identity features, such as speaker embeddings, are popular, they often compromise patient privacy. To address this issue, we propose a speaker disentanglement method that utilizes a non-uniform mechanism of adversarial SID loss maximization. This is achieved by varying the adversarial weight between different layers of a model during training. We find that a greater adversarial weight for the initial layers leads to performance improvement. Our approach using the ECAPA-TDNN model achieves an F1-score of 0.7349 (a 3.7% improvement over audio-only SOTA) on the DAIC-WoZ dataset, while simultaneously reducing the speaker-identification accuracy by 50%. Our findings suggest that identifying depression through speech signals can be accomplished without placing undue reliance on a speaker's identity, paving the way for privacy-preserving approaches of depression detection.
Open-Set Text Recognition via Shape-Awareness Visual Reconstruction

Open-Set Text Recognition (OSTR) is an emerging task that models the constantly evolving char-set in open-world character recognition applications. Compared to conventional counterparts, the OSTR task demands actively spotting and incrementally recognizing novel characters. Existing methods have demonstrated some success, yet confusion among similar characters remains to be a major challenge, potentially due to insufficient shape information preserved in the character features. In this work, we propose to alleviate this problem via visual reconstruction. Specifically, a glyph reconstruction task is adopted to implement shape awareness. Furthermore, cut-and-mixed characters are introduced to alleviate overfitting, by improving the coverage of the glyph space. Finally, a cycle classification task is proposed to prioritize the preservation of classification-critic regions by sending reconstructed images to a classifier network. Extensive experiments show that both tasks yield satisfying improvements on the OSTR task, and the full model demonstrates decent performance in recognizing both seen and novel characters.
Augraphy: A Data Augmentation Library for Document Images

This paper introduces Augraphy, a Python library for constructing data augmentation pipelines which produce distortions commonly seen in real-world document image datasets. Augraphy stands apart from other data augmentation tools by providing many different strategies to produce augmented versions of clean document images that appear as if they have been altered by standard office operations, such as printing, scanning, and faxing through old or dirty machines, degradation of ink over time, and handwritten markings. This paper discusses the Augraphy tool, and shows how it can be used both as a data augmentation tool for producing diverse training data for tasks such as document denoising, and also for generating challenging test data to evaluate model robustness on document image modeling tasks.
A Novel Private Encryption Model in IoT Under Cloud Computing Domain

The presence of the Internet of Things (IoT) works with the assortment and diffusion of metropolitan information data. In any case, it can release clients’ very own security data in brilliant urban areas. Many works have been done and different security systems identifying with the precautions of distributed computing have been implemented from copious points of view. Be that as it may, they don’t propose a quantitative way to deal with dissecting and assessing protection and security in distributed computing frameworks. Accordingly, we propose another private data encryption strategy in IoT under distributed computing climate. Below IoT, as per the properties and securing time, protection data can be isolated into numerous subspaces. Because of the stream figure instrument, we plan an encryption framework model of data assortment. In the subspace, the security data is encoded and moved to the transfer hub. Subsequent to encoding, they are divided and rebuilt. At long last, we use stream figure and dual-key calculation to finish opportunity nondestructive change among plaintext and ciphertext to guarantee the trustworthiness of the encoded private data. Test outcomes illustrate that the presented strategy requires some investment in the encryption and unscrambling measure, which has superior ciphertext transformation yield impact and experiences fewer organization assaults in a similar encryption time. As far as computation cost, the proposed strategy diminishes by roughly 11%. Furthermore, it has greater security and works on the security and integrity of the protection data assortment procedure.
Geometric Visual Similarity Learning in 3D Medical Image Self-Supervised Pre-training

Learning inter-image similarity is crucial for 3D medical images self-supervised pre-training, due to their sharing of numerous same semantic regions. However, the lack of the semantic prior in metrics and the semantic-independent variation in 3D medical images make it challenging to get a reliable measurement for the inter-image similarity, hindering the learning of consistent representation for same semantics. We investigate the challenging problem of this task, i.e., learning a consistent representation between images for a clustering effect of same semantic features. We propose a novel visual similarity learning paradigm, Geometric Visual Similarity Learning, which embeds the prior of topological invariance into the measurement of the inter-image similarity for consistent representation of semantic regions. To drive this paradigm, we further construct a novel geometric matching head, the Z-matching head, to collaboratively learn the global and local similarity of semantic regions, guiding the efficient representation learning for different scale-level inter-image semantic features. Our experiments demonstrate that the pre-training with our learning of inter-image similarity yields more powerful inner-scene, inter-scene, and global-local transferring ability on four challenging 3D medical image tasks. Our codes and pre-trained models will be publicly available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://github.com/YutingHe-list/GVSL.
Task Classification and Scheduling Using Enhanced Coot Optimization in Cloud Computing

Cloud computing benchmarks the dream of rendering computing as a utility, providing high agility and reachability from an existing set of technologies.It facilitates a wider dimension to architect and manage remote resources.Cloud technology with exponential growth is drilling towards issues that tend to lower its explored possibilities.As cloud systems by virtue deal with various virtualized resources, scheduling is opted as an important metric for measuring and leveraging performance.But scheduling efficiency is deteriorated by various parameters that pave scope for our research and projects immense need for improvising the overall makespan of the system.The proposed work aims at projecting a greater drift in the first phase by witnessing a sequence of phases like preprocessing the user tasks for improved accuracy, classifying the tasks with respect to resource demand and execution time using the improved density based clustering method (IDCM).The second phase deals with enhanced coot optimization algorithm for task scheduling (ECOA-TS) that proceeds and proves its novelty by adopting Cauchy mutation overcoming the convergence backdrop for generating an optimal mapping between clustered user tasks and VMs.The overall performance of the proposed work overrides by reduced makespan against existing state-of-the-art optimization algorithms like particle swam optimization (PSO), grey wolf optimization (GWO) and whale optimization algorithm (WOA) by 27.41%, 19.8%, and 15.33% respectively.
Multimodal Image Synthesis and Editing: The Generative AI Era

As information exists in various modalities in real world, effective interaction and fusion among multimodal information plays a key role for the creation and perception of multimodal data in computer vision and deep learning research. With superb power in modeling the interaction among multimodal information, multimodal image synthesis and editing has become a hot research topic in recent years. Instead of providing explicit guidance for network training, multimodal guidance offers intuitive and flexible means for image synthesis and editing. On the other hand, this field is also facing several challenges in alignment of multimodal features, synthesis of high-resolution images, faithful evaluation metrics, etc. In this survey, we comprehensively contextualize the advance of the recent multimodal image synthesis and editing and formulate taxonomies according to data modalities and model types. We start with an introduction to different guidance modalities in image synthesis and editing, and then describe multimodal image synthesis and editing approaches extensively according to their model types. After that, we describe benchmark datasets and evaluation metrics as well as corresponding experimental results. Finally, we provide insights about the current research challenges and possible directions for future research.
Synthesize high-dimensional longitudinal electronic health records via hierarchical autoregressive language model

Synthetic electronic health records (EHRs) that are both realistic and privacy-preserving offer alternatives to real EHRs for machine learning (ML) and statistical analysis. However, generating high-fidelity EHR data in its original, high-dimensional form poses challenges for existing methods. We propose Hierarchical Autoregressive Language mOdel (HALO) for generating longitudinal, high-dimensional EHR, which preserve the statistical properties of real EHRs and can train accurate ML models without privacy concerns. HALO generates a probability density function over medical codes, clinical visits, and patient records, allowing for generating realistic EHR data without requiring variable selection or aggregation. Extensive experiments demonstrated that HALO can generate high-fidelity data with high-dimensional disease code probabilities closely mirroring (above 0.9 R2 correlation) real EHR data. HALO also enhances the accuracy of predictive modeling and enables downstream ML models to attain similar accuracy as models trained on genuine data.