OpenRaiser/Intern-Atlas huggingface dataset:
Unsupervised End-to-End Task-Oriented Dialogue with LLMs: The Power of the Noisy Channel
Venue: arXiv cs.CL 2024
Year: 2024

Training task-oriented dialogue systems typically requires turn-level annotations for interacting with their APIs: e.g. a dialogue state and the system actions taken at each step. These annotations can be costly to produce, error-prone, and require both domain and annotation expertise. With advances in LLMs, we hypothesize that unlabeled data and a schema definition are sufficient for building a working task-oriented dialogue system, completely unsupervised. We consider a novel unsupervised setting of only (1) a well-defined API schema (2) a set of unlabeled dialogues between a user and agent. We propose an innovative approach using expectation-maximization (EM) that infers turn-level annotations as latent variables using a noisy channel model to build an end-to-end dialogue agent. Evaluating our approach on the MultiWOZ benchmark, our method more than doubles the dialogue success rate of a strong GPT-3.5 baseline.
The Architecture of Truth
Venue: arXiv cs.LO 2024
Year: 2024

The theory of institutions is framed as an indexed/fibered duality, where the indexed aspect specifies the fibered aspect. Tarski represented truth in terms of a satisfaction relation. The theory of institutions encodes satisfaction as its core architecture in the indexed aspect. Logical environments enrich this truth architecture by axiomatizing the truth adjunction in the fibered aspect. The truth architecture is preserved by morphisms of logical environments. (Although not every institution is a logical environment, each institution has an associated logical environment defined via the intent of the structures of the institution, and each institution is represented by an indexed functor into the structure category of the classification logical environment $\mathtt{Cls}$.)
Source-free Domain Adaptation for Video Object Detection Under Adverse Image Conditions
Venue: arXiv cs.CV 2024
Year: 2024

When deploying pre-trained video object detectors in real-world scenarios, the domain gap between training and testing data caused by adverse image conditions often leads to performance degradation. Addressing this issue becomes particularly challenging when only the pre-trained model and degraded videos are available. Although various source-free domain adaptation (SFDA) methods have been proposed for single-frame object detectors, SFDA for video object detection (VOD) remains unexplored. Moreover, most unsupervised domain adaptation works for object detection rely on two-stage detectors, while SFDA for one-stage detectors, which are more vulnerable to fine-tuning, is not well addressed in the literature. In this paper, we propose Spatial-Temporal Alternate Refinement with Mean Teacher (STAR-MT), a simple yet effective SFDA method for VOD. Specifically, we aim to improve the performance of the one-stage VOD method, YOLOV, under adverse image conditions, including noise, air turbulence, and haze. Extensive experiments on the ImageNetVOD dataset and its degraded versions demonstrate that our method consistently improves video object detection performance in challenging imaging conditions, showcasing its potential for real-world applications.
From Parts to Whole: A Unified Reference Framework for Controllable Human Image Generation
Venue: arXiv cs.CV 2024
Year: 2024

Recent advancements in controllable human image generation have led to zero-shot generation using structural signals (e.g., pose, depth) or facial appearance. Yet, generating human images conditioned on multiple parts of human appearance remains challenging. Addressing this, we introduce Parts2Whole, a novel framework designed for generating customized portraits from multiple reference images, including pose images and various aspects of human appearance. To achieve this, we first develop a semantic-aware appearance encoder to retain details of different human parts, which processes each image based on its textual label to a series of multi-scale feature maps rather than one image token, preserving the image dimension. Second, our framework supports multi-image conditioned generation through a shared self-attention mechanism that operates across reference and target features during the diffusion process. We enhance the vanilla attention mechanism by incorporating mask information from the reference human images, allowing for the precise selection of any part. Extensive experiments demonstrate the superiority of our approach over existing alternatives, offering advanced capabilities for multi-part controllable human image customization. See our project page at https://huanngzh.github.io/Parts2Whole/.
Automatic Layout Planning for Visually-Rich Documents with Instruction-Following Models
Venue: arXiv cs.CV 2024
Year: 2024

Recent advancements in instruction-following models have made user interactions with models more user-friendly and efficient, broadening their applicability. In graphic design, non-professional users often struggle to create visually appealing layouts due to limited skills and resources. In this work, we introduce a novel multimodal instruction-following framework for layout planning, allowing users to easily arrange visual elements into tailored layouts by specifying canvas size and design purpose, such as for book covers, posters, brochures, or menus. We developed three layout reasoning tasks to train the model in understanding and executing layout instructions. Experiments on two benchmarks show that our method not only simplifies the design process for non-professionals but also surpasses the performance of few-shot GPT-4V models, with mIoU higher by 12% on Crello. This progress highlights the potential of multimodal instruction-following models to automate and simplify the design process, providing an approachable solution for a wide range of design tasks on visually-rich documents.
Expanding Conservation Science through Emerging Interdisciplinary STEM Fields
Venue: arXiv cs.DL 2024
Year: 2024

Conservation science is an interdisciplinary field that primarily draws on knowledge from the natural sciences, social sciences, and humanities to inform policy, planning, and practice. Since its formalization as a discipline, conservation science has also increasingly incorporated tools from integrative biological fields, such as animal behavior, genetics, and, more recently, physiology. Given that the biodiversity crisis constitutes one of the greatest challenges of the 21st century, with tremendous consequences for global sustainability and human health, creating a diverse conservation toolbox is important for addressing complex conservation threats. To assess the integration of three emerging integrative biological disciplines (physiology, biomechanics, and technology) into recent conservation science research, we queried publications from five broad-scope conservation-focused journals from 2010-2022. We found that the proportion of published articles incorporating these integrative biological techniques was low, ranging from 0-4% per year. With only 2.1% of total articles accessing tools or techniques from conservation physiology, conservation technology, and conservation biomechanics, we propose that there is still a substantial opportunity for further integration. We provide a case study for each integrative field to illustrate the capacity for its tools to contribute to positive conservation outcomes. We further outline how each field promotes novel or reimagined opportunities for collaborations. Finally, we discuss the interconnectedness of the three fields and how they can support the continuing expansion of conservation science as an evidence-based, action-oriented discipline through the application of a Challenge-Mechanism-Partnership framework.
Unraveling Retraction Dynamics in COVID-19 Research: Patterns, Reasons, and Implications
Venue: arXiv cs.DL 2024
Year: 2024

Amid the COVID-19 pandemic, while the world sought solutions, some scholars exploited the situation for personal gains through deceptive studies and manipulated data. This paper presents the extent of 400 retracted COVID-19 papers listed by the Retraction Watch database until February 2024. The primary purpose of the research was to analyze journal quality and retraction trends. For all stakeholders involved, such as editors, relevant researchers, and policymakers, evaluating the journal's quality is crucial information since it could help them effectively stop such incidents and their negative effects in the future. The present research results imply that one-fourth of publications were retracted within the first month of their publication, followed by an additional 6\% within six months of publication. One-third of the retractions originated from Q1 journals, with another significant portion coming from Q2 (29.8). A notable percentage of the retracted papers (23.2\%) lacked publishing impact, signifying their publication as conference papers or in journals not indexed by Scopus. An examination of the retraction reasons reveals that one-fourth of retractions were due to numerous causes, mostly in Q2 journals, and another quarter were due to data problems, with the majority happening in Q1 publications. Elsevier retracted 31 of the papers, with the majority published in Q1, followed by Springer (11.5), predominantly in Q2. Retracted papers were mainly associated with the USA, China, and India. In the USA, retractions were primarily from Q1 journals followed by no-impact publications; in China, it was Q1 followed by Q2, and in India, it was Q2 followed by no-impact publications. The study also examined author contributions, revealing that 69.3 were male contributors, with females (30.7) mainly holding middle author positions.
State Space Paradox of Computational Research in Creativity
Venue: arXiv cs.HC 2024
Year: 2024

This paper explores the paradoxical nature of computational creativity, focusing on the inherent limitations of closed digital systems in emulating the open-ended, dynamic process of human creativity. Through a comprehensive analysis, we delve into the concept of the State Space Paradox (SSP) in computational research on creativity, which arises from the attempt to model or replicate creative behaviors within the bounded state spaces of digital systems. Utilizing a combination of procedural and representational paradigms, we examine various computational models and their capabilities to assist or emulate the creative process. Our investigation encompasses rule-based systems, genetic algorithms, case-based reasoning, shape grammars, and data mining, among others, to understand how these methods contribute to or fall short of achieving genuine creativity. The discussion extends to the implications of SSP on the future of creativity-related computer systems, emphasizing the cultural and contextual fluidity of creativity itself and the challenges of producing truly creative outcomes within the constraints of pre-defined algorithmic structures. We argue that while digital systems can provoke sudden mental insights (SMIs) in human observers and potentially support the creative process, their capacity to autonomously break out of their pre-programmed state spaces and achieve originality akin to human creativity remains fundamentally constrained. The paper concludes with reflections on the future directions for research in computational creativity, suggesting that recognizing and embracing the limitations and potentials of digital systems could lead to more nuanced and effective tools for creative assistance.
Two Decades of Scientific Misconduct in India: Retraction Reasons and Journal Quality among Inter-country and Intra-country Institutional Collaboration
Venue: arXiv cs.DL 2024
Year: 2024

Research stands as a pivotal factor in propelling the progress of any nation forward. However, if tainted by misconduct, it poses a significant threat to the nation's development. This study aims to scrutinize various cases of deliberate scientific misconduct by Indian researchers. A comprehensive analysis was conducted on 3,244 retracted publications sourced from the Retraction Watch database. The upward trend in retractions is alarming, although the decreasing duration of retractions indicates proactive measures by journals against misconduct. Approximately 60% of retractions stem from private institutions, with fake peer reviews identified as the primary cause of misconduct. This trend could be attributed to incentivizing publication quantity over quality in private institutions, potentially fostering unfair publishing practices. Retractions due to data integrity issues are predominantly observed in public and medical institutions, while retractions due to plagiarism occur in conference proceedings and non-Scopus-indexed journals. Examining retractions resulting from institutional collaborations reveals that 80% originate from within the country, with the remaining 20% being international collaborations. Among inter-country collaborations, one-third of retractions come from the top two journal quartiles, whereas, in intra-country collaborations, half of the retractions stem from Q1 and Q2 journals. Clinical studies retracted from intra-country collaborations are mostly from Q3 and Q4 journals, whereas in inter-country collaborations, they primarily come from Q1 journals. Regarding top journals by the number of retractions in intra-country collaborations, they belong to the Q2 and Q4 categories, whereas in inter-country collaborations, they are in Q1.
An Optimized Framework for Processing Large-scale Polysomnographic Data Incorporating Expert Human Oversight
Venue: arXiv cs.HC 2024
Year: 2024

Polysomnographic recordings are essential for diagnosing many sleep disorders, yet their detailed analysis presents considerable challenges. With the rise of machine learning methodologies, researchers have created various algorithms to automatically score and extract clinically relevant features from polysomnography, but less research has been devoted to how exactly the algorithms should be incorporated into the workflow of sleep technologists. This paper presents a sophisticated data collection platform developed under the Sleep Revolution project, to harness polysomnographic data from multiple European centers. A tripartite platform is presented: a user-friendly web platform for uploading three-night polysomnographic recordings, a dedicated splitter that segments these into individual one-night recordings, and an advanced processor that enhances the one-night polysomnography with contemporary automatic scoring algorithms. The platform is evaluated using real-life data and human scorers, whereby scoring time, accuracy and trust are quantified. Additionally, the scorers were interviewed about their trust in the platform, along with the impact of its integration into their workflow.
Advances and Open Challenges in Federated Foundation Models
Venue: arXiv cs.LG 2024
Year: 2024

The integration of Foundation Models (FMs) with Federated Learning (FL) presents a transformative paradigm in Artificial Intelligence (AI). This integration offers enhanced capabilities, while addressing concerns of privacy, data decentralization and computational efficiency. This paper provides a comprehensive survey of the emerging field of Federated Foundation Models (FedFM), elucidating their synergistic relationship and exploring novel methodologies, challenges, and future directions that the FL research field needs to focus on in order to thrive in the age of FMs. A systematic multi-tiered taxonomy is proposed, categorizing existing FedFM approaches for model training, aggregation, trustworthiness, and incentivization. Key challenges, including how to enable FL to deal with high complexity of computational demands, privacy considerations, contribution evaluation, and communication efficiency, are thoroughly discussed. Moreover, this paper explores the intricate challenges of communication, scalability and security inherent in training/fine-tuning FMs via FL. It highlights the potential of quantum computing to revolutionize the processes of training, inference, optimization and security. This survey also introduces the implementation requirement of FedFM and some practical FedFM applications. It highlights lessons learned with a clear understanding of our findings for FedFM. Finally, this survey not only provides insights into the current state and challenges of FedFM, but also offers a blueprint for future research directions, emphasizing the need for developing trustworthy solutions. It serves as a foundational guide for researchers and practitioners interested in contributing to this interdisciplinary and rapidly advancing field.
Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMs
Venue: arXiv cs.CV 2024
Year: 2024

Multimodal LLMs are the natural evolution of LLMs, and enlarge their capabilities so as to work beyond the pure textual modality. As research is being carried out to design novel architectures and vision-and-language adapters, in this paper we concentrate on endowing such models with the capability of answering questions that require external knowledge. Our approach, termed Wiki-LLaVA, aims at integrating an external knowledge source of multimodal documents, which is accessed through a hierarchical retrieval pipeline. Relevant passages, using this approach, are retrieved from the external knowledge source and employed as additional context for the LLM, augmenting the effectiveness and precision of generated dialogues. We conduct extensive experiments on datasets tailored for visual question answering with external data and demonstrate the appropriateness of our approach.
Planning the path with Reinforcement Learning: Optimal Robot Motion Planning in RoboCup Small Size League Environments
Venue: arXiv cs.RO 2024
Year: 2024

This work investigates the potential of Reinforcement Learning (RL) to tackle robot motion planning challenges in the dynamic RoboCup Small Size League (SSL). Using a heuristic control approach, we evaluate RL's effectiveness in obstacle-free and single-obstacle path-planning environments. Ablation studies reveal significant performance improvements. Our method achieved a 60% time gain in obstacle-free environments compared to baseline algorithms. Additionally, our findings demonstrated dynamic obstacle avoidance capabilities, adeptly navigating around moving blocks. These findings highlight the potential of RL to enhance robot motion planning in the challenging and unpredictable SSL environment.
OffRAMPS: An FPGA-based Intermediary for Analysis and Modification of Additive Manufacturing Control Systems
Venue: arXiv cs.CR 2024
Year: 2024

Cybersecurity threats in Additive Manufacturing (AM) are an increasing concern as AM adoption continues to grow. AM is now being used for parts in the aerospace, transportation, and medical domains. Threat vectors which allow for part compromise are particularly concerning, as any failure in these domains would have life-threatening consequences. A major challenge to investigation of AM part-compromises comes from the difficulty in evaluating and benchmarking both identified threat vectors as well as methods for detecting adversarial actions. In this work, we introduce a generalized platform for systematic analysis of attacks against and defenses for 3D printers. Our "OFFRAMPS" platform is based on the open-source 3D printer control board "RAMPS." OFFRAMPS allows analysis, recording, and modification of all control signals and I/O for a 3D printer. We show the efficacy of OFFRAMPS by presenting a series of case studies based on several Trojans, including ones identified in the literature, and show that OFFRAMPS can both emulate and detect these attacks, i.e., it can both change and detect arbitrary changes to the g-code print commands.
A Review on Message Complexity of the Algorithms for Clock Synchronization in Distributed Systems
Venue: arXiv cs.DC 2024
Year: 2024

In this work, we present an extensive analysis of clock synchronization algorithms, with a specific focus on message complexity. We begin by introducing fundamental concepts in clock synchronization, such as the Byzantine generals problem and specific concepts like clock accuracy, precision, skew, offset, timestamping, and clock drift estimation. Describing the concept of logical clocks, their implementation in distributed systems is discussed, highlighting their significance and various approaches. The paper then examines four prominent clock synchronization algorithms: Lamport's Algorithm, Ricart-Agrawala Algorithm, Vector Clocks Algorithm, and Christian's Algorithm. Special attention is given to the analysis of message complexity, providing insights into the efficiency of each algorithm. Finally, we compare the message complexities of the discussed algorithms.
IryoNLP at MEDIQA-CORR 2024: Tackling the Medical Error Detection & Correction Task On the Shoulders of Medical Agents
Venue: arXiv cs.CL 2024
Year: 2024

In natural language processing applied to the clinical domain, utilizing large language models has emerged as a promising avenue for error detection and correction on clinical notes, a knowledge-intensive task for which annotated data is scarce. This paper presents MedReAct'N'MedReFlex, which leverages a suite of four LLM-based medical agents. The MedReAct agent initiates the process by observing, analyzing, and taking action, generating trajectories to guide the search to target a potential error in the clinical notes. Subsequently, the MedEval agent employs five evaluators to assess the targeted error and the proposed correction. In cases where MedReAct's actions prove insufficient, the MedReFlex agent intervenes, engaging in reflective analysis and proposing alternative strategies. Finally, the MedFinalParser agent formats the final output, preserving the original style while ensuring the integrity of the error correction process. One core component of our method is our RAG pipeline based on our ClinicalCorp corpora. Among other well-known sources containing clinical guidelines and information, we preprocess and release the open-source MedWiki dataset for clinical RAG application. Our results demonstrate the central role of our RAG approach with ClinicalCorp leveraged through the MedReAct'N'MedReFlex framework. It achieved the ninth rank on the MEDIQA-CORR 2024 final leaderboard.
Understanding Hyperbolic Metric Learning through Hard Negative Sampling
Venue: arXiv cs.CV 2024
Year: 2024

In recent years, there has been a growing trend of incorporating hyperbolic geometry methods into computer vision. While these methods have achieved state-of-the-art performance on various metric learning tasks using hyperbolic distance measurements, the underlying theoretical analysis supporting this superior performance remains under-exploited. In this study, we investigate the effects of integrating hyperbolic space into metric learning, particularly when training with contrastive loss. We identify a need for a comprehensive comparison between Euclidean and hyperbolic spaces regarding the temperature effect in the contrastive loss within the existing literature. To address this gap, we conduct an extensive investigation to benchmark the results of Vision Transformers (ViTs) using a hybrid objective function that combines loss from Euclidean and hyperbolic spaces. Additionally, we provide a theoretical analysis of the observed performance improvement. We also reveal that hyperbolic metric learning is highly related to hard negative sampling, providing insights for future work. This work will provide valuable data points and experience in understanding hyperbolic image embeddings. To shed more light on problem-solving and encourage further investigation into our approach, our code is available online (https://github.com/YunYunY/HypMix).