OpenRaiser/Intern-Atlas huggingface dataset:
Thanos: Enhancing Conversational Agents with Skill-of-Mind-Infused Large Language Model
Venue: arXiv cs.CL 2024
Year: 2024

To increase social bonding with interlocutors, humans naturally acquire the ability to respond appropriately in a given situation by considering which conversational skill is most suitable for the response - a process we call skill-of-mind. For large language model (LLM)-based conversational agents, planning appropriate conversational skills, as humans do, is challenging due to the complexity of social dialogue, especially in interactive scenarios. To address this, we propose a skill-of-mind-annotated conversation dataset, named Multifaceted Skill-of-Mind, which includes multi-turn and multifaceted conversational skills across various interactive scenarios (e.g., long-term, counseling, task-oriented), grounded in diverse social contexts (e.g., demographics, persona, rules of thumb). This dataset consists of roughly 100K conversations. Using this dataset, we introduce a new family of skill-of-mind-infused LLMs, named Thanos, with model sizes of 1B, 3B, and 8B parameters. With extensive experiments, these models successfully demonstrate the skill-of-mind process and exhibit strong generalizability in inferring multifaceted skills across a variety of domains. Moreover, we show that Thanos significantly enhances the quality of responses generated by LLM-based conversational agents and promotes prosocial behavior in human evaluations.
FedDP: Privacy-preserving method based on federated learning for histopathology image segmentation
Venue: arXiv cs.CV 2024
Year: 2024

Hematoxylin and Eosin (H&E) staining of whole slide images (WSIs) is considered the gold standard for pathologists and medical practitioners for tumor diagnosis, surgical planning, and post-operative assessment. With the rapid advancement of deep learning technologies, the development of numerous models based on convolutional neural networks and transformer-based models has been applied to the precise segmentation of WSIs. However, due to privacy regulations and the need to protect patient confidentiality, centralized storage and processing of image data are impractical. Training a centralized model directly is challenging to implement in medical settings due to these privacy concerns.This paper addresses the dispersed nature and privacy sensitivity of medical image data by employing a federated learning framework, allowing medical institutions to collaboratively learn while protecting patient privacy. Additionally, to address the issue of original data reconstruction through gradient inversion during the federated learning training process, differential privacy introduces noise into the model updates, preventing attackers from inferring the contributions of individual samples, thereby protecting the privacy of the training data.Experimental results show that the proposed method, FedDP, minimally impacts model accuracy while effectively safeguarding the privacy of cancer pathology image data, with only a slight decrease in Dice, Jaccard, and Acc indices by 0.55%, 0.63%, and 0.42%, respectively. This approach facilitates cross-institutional collaboration and knowledge sharing while protecting sensitive data privacy, providing a viable solution for further research and application in the medical field.
Emotion Analysis of Social Media Bangla Text and Its Impact on Identifying the Author's Gender
Venue: arXiv cs.HC 2024
Year: 2024

The Gender Identification (GI) problem is concerned with determining the gender of the author from a given text. It has numerous applications in different fields like forensics, literature, security, marketing, trade, etc. Due to its importance, researchers have put extensive efforts into identifying gender from the text for different languages. Unfortunately, the same statement is not true for the Bangla language despite its being the 7th most spoken language in the world. In this work, we explore Gender Identification from Social media Bangla Text. Specially, we consider two approaches for feature extraction. The first one is Bag-Of-Words(BOW) approach and another one is based on computing features from sentiment and emotions. There is a common stereotype that female authors write in a more emotional way than male authors. One goal of this work is to validate this stereotype for the Bangla language.
PASSION for Dermatology: Bridging the Diversity Gap with Pigmented Skin Images from Sub-Saharan Africa
Venue: arXiv cs.CV 2024
Year: 2024

Africa faces a huge shortage of dermatologists, with less than one per million people. This is in stark contrast to the high demand for dermatologic care, with 80% of the paediatric population suffering from largely untreated skin conditions. The integration of AI into healthcare sparks significant hope for treatment accessibility, especially through the development of AI-supported teledermatology. Current AI models are predominantly trained on white-skinned patients and do not generalize well enough to pigmented patients. The PASSION project aims to address this issue by collecting images of skin diseases in Sub-Saharan countries with the aim of open-sourcing this data. This dataset is the first of its kind, consisting of 1,653 patients for a total of 4,901 images. The images are representative of telemedicine settings and encompass the most common paediatric conditions: eczema, fungals, scabies, and impetigo. We also provide a baseline machine learning model trained on the dataset and a detailed performance analysis for the subpopulations represented in the dataset. The project website can be found at https://passionderm.github.io/.
The Impact of Semi-Supervised Learning on Line Segment Detection
Venue: arXiv cs.CV 2024
Year: 2024

In this paper we present a method for line segment detection in images, based on a semi-supervised framework. Leveraging the use of a consistency loss based on differently augmented and perturbed unlabeled images with a small amount of labeled data, we show comparable results to fully supervised methods. This opens up application scenarios where annotation is difficult or expensive, and for domain specific adaptation of models. We are specifically interested in real-time and online applications, and investigate small and efficient learning backbones. Our method is to our knowledge the first to target line detection using modern state-of-the-art methodologies for semi-supervised learning. We test the method on both standard benchmarks and domain specific scenarios for forestry applications, showing the tractability of the proposed method.
Self-Calibrated Listwise Reranking with Large Language Models
Venue: arXiv cs.IR 2024
Year: 2024

Large language models (LLMs), with advanced linguistic capabilities, have been employed in reranking tasks through a sequence-to-sequence approach. In this paradigm, multiple passages are reranked in a listwise manner and a textual reranked permutation is generated. However, due to the limited context window of LLMs, this reranking paradigm requires a sliding window strategy to iteratively handle larger candidate sets. This not only increases computational costs but also restricts the LLM from fully capturing all the comparison information for all candidates. To address these challenges, we propose a novel self-calibrated listwise reranking method, which aims to leverage LLMs to produce global relevance scores for ranking. To achieve it, we first propose the relevance-aware listwise reranking framework, which incorporates explicit list-view relevance scores to improve reranking efficiency and enable global comparison across the entire candidate set. Second, to ensure the comparability of the computed scores, we propose self-calibrated training that uses point-view relevance assessments generated internally by the LLM itself to calibrate the list-view relevance assessments. Extensive experiments and comprehensive analysis on the BEIR benchmark and TREC Deep Learning Tracks demonstrate the effectiveness and efficiency of our proposed method.
Enhancing Trust in Clinically Significant Prostate Cancer Prediction with Multiple Magnetic Resonance Imaging Modalities
Venue: arXiv cs.LG 2024
Year: 2024

In the United States, prostate cancer is the second leading cause of deaths in males with a predicted 35,250 deaths in 2024. However, most diagnoses are non-lethal and deemed clinically insignificant which means that the patient will likely not be impacted by the cancer over their lifetime. As a result, numerous research studies have explored the accuracy of predicting clinical significance of prostate cancer based on magnetic resonance imaging (MRI) modalities and deep neural networks. Despite their high performance, these models are not trusted by most clinical scientists as they are trained solely on a single modality whereas clinical scientists often use multiple magnetic resonance imaging modalities during their diagnosis. In this paper, we investigate combining multiple MRI modalities to train a deep learning model to enhance trust in the models for clinically significant prostate cancer prediction. The promising performance and proposed training pipeline showcase the benefits of incorporating multiple MRI modalities for enhanced trust and accuracy.
DNN-based 3D Cloud Retrieval for Variable Solar Illumination and Multiview Spaceborne Imaging
Venue: arXiv cs.CV 2024
Year: 2024

Climate studies often rely on remotely sensed images to retrieve two-dimensional maps of cloud properties. To advance volumetric analysis, we focus on recovering the three-dimensional (3D) heterogeneous extinction coefficient field of shallow clouds using multiview remote sensing data. Climate research requires large-scale worldwide statistics. To enable scalable data processing, previous deep neural networks (DNNs) can infer at spaceborne remote sensing downlink rates. However, prior methods are limited to a fixed solar illumination direction. In this work, we introduce the first scalable DNN-based system for 3D cloud retrieval that accommodates varying camera poses and solar directions. By integrating multiview cloud intensity images with camera poses and solar direction data, we achieve greater flexibility in recovery. Training of the DNN is performed by a novel two-stage scheme to address the high number of degrees of freedom in this problem. Our approach shows substantial improvements over previous state-of-the-art, particularly in handling variations in the sun's zenith angle.
Sentiment Analysis of Spanish Political Party Tweets Using Pre-trained Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Title: Sentiment Analysis of Spanish Political Party Communications on Twitter Using Pre-trained Language Models Authors: Chuqiao Song, Shunzhang Chen, Xinyi Cai, Hao Chen Comments: 21 pages, 6 figures Abstract: This study investigates sentiment patterns within Spanish political party communications on Twitter by leveraging BETO and RoBERTuito, two pre-trained language models optimized for Spanish text. Using a dataset of tweets from major Spanish political parties: PSOE, PP, Vox, Podemos, and Ciudadanos, spanning 2019 to 2024, this research analyzes sentiment distributions and explores the relationship between sentiment expression and party ideology. The findings indicate that both models consistently identify a predominant Neutral sentiment across all parties, with significant variations in Negative and Positive sentiments that align with ideological distinctions. Specifically, Vox exhibits higher levels of Negative sentiment, while PSOE demonstrates relatively high Positive sentiment, supporting the hypothesis that emotional appeals in political messaging reflect ideological stances. This study underscores the potential of pre-trained language models for non-English sentiment analysis on social media, providing insights into sentiment dynamics that shape public discourse within Spain's multi-party political system. Keywords: Spanish politics, sentiment analysis, pre-trained language models, Twitter, BETO, RoBERTuito, political ideology, multi-party system
SaSR-Net: Source-Aware Semantic Representation Network for Enhancing Audio-Visual Question Answering
Venue: arXiv cs.CV 2024
Year: 2024

Audio-Visual Question Answering (AVQA) is a challenging task that involves answering questions based on both auditory and visual information in videos. A significant challenge is interpreting complex multi-modal scenes, which include both visual objects and sound sources, and connecting them to the given question. In this paper, we introduce the Source-aware Semantic Representation Network (SaSR-Net), a novel model designed for AVQA. SaSR-Net utilizes source-wise learnable tokens to efficiently capture and align audio-visual elements with the corresponding question. It streamlines the fusion of audio and visual information using spatial and temporal attention mechanisms to identify answers in multi-modal scenes. Extensive experiments on the Music-AVQA and AVQA-Yang datasets show that SaSR-Net outperforms state-of-the-art AVQA methods.
Evaluating 21st Century Skills Development through Makerspace Workshops in Computer Science Education
Venue: arXiv cs.CY 2024
Year: 2024

This study evaluates the effectiveness of incorporating makerspace workshops into computer science education by assessing 21st century skills critical thinking, collaboration, communication, and creativity before and after the intervention. Using a pre test and post test approach with the "21st Century Skills Survey Instrument," the study quantifies the impact of makerspace activities on student skill development. Participants included students enrolled in two computer science courses at Cyprus College. Statistical analysis, conducted using Python, revealed significant improvements across all assessed skills, indicating that makerspace workshops enhance essential competencies needed for the modern workforce. These findings provide valuable insights into how experiential learning environments can transform traditional computer science education, promoting a more interactive and engaging learning experience. Future research should focus on larger, more diverse samples and explore specific components of makerspace activities that most effectively contribute to skill development.
Deep Heuristic Learning for Real-Time Urban Pathfinding
Venue: arXiv cs.AI 2024
Year: 2024

This paper introduces a novel approach to urban pathfinding by transforming traditional heuristic-based algorithms into deep learning models that leverage real-time contextual data, such as traffic and weather conditions. We propose two methods: an enhanced A* algorithm that dynamically adjusts routes based on current environmental conditions, and a neural network model that predicts the next optimal path segment using historical and live data. An extensive benchmark was conducted to compare the performance of different deep learning models, including MLP, GRU, LSTM, Autoencoders, and Transformers. Both methods were evaluated in a simulated urban environment in Berlin, with the neural network model outperforming traditional methods, reducing travel times by up to 40%, while the enhanced A* algorithm achieved a 34% improvement. These results demonstrate the potential of deep learning to optimize urban navigation in real time, providing more adaptable and efficient routing solutions.
Leveraging LLMs to Enable Natural Language Search on Go-to-market Platforms
Venue: arXiv cs.CL 2024
Year: 2024

Enterprise searches require users to have complex knowledge of queries, configurations, and metadata, rendering it difficult for them to access information as needed. Most go-to-market (GTM) platforms utilize advanced search, an interface that enables users to filter queries by various fields using categories or keywords, which, historically, however, has proven to be exceedingly cumbersome, as users are faced with seemingly hundreds of options, fields, and buttons. Consequently, querying with natural language has long been ideal, a notion further empowered by Large Language Models (LLMs). In this paper, we implement and evaluate a solution for the Zoominfo product for sellers, which prompts the LLM with natural language, producing search fields through entity extraction that are then converted into a search query. The intermediary search fields offer numerous advantages for each query, including the elimination of syntax errors, simpler ground truths, and an intuitive format for the LLM to interpret. We paired this pipeline with many advanced prompt engineering strategies, featuring an intricate system message, few-shot prompting, chain-of-thought (CoT) reasoning, and execution refinement. Furthermore, we manually created the ground truth for 500+ natural language queries, enabling the supervised fine-tuning of Llama-3-8B-Instruct and the introduction of sophisticated numerical metrics. Comprehensive experiments with closed, open source, and fine-tuned LLM models were conducted through exact, Jaccard, cosine, and semantic similarity on individual search entities to demonstrate the efficacy of our approach. Overall, the most accurate closed model had an average accuracy of 97% per query, with only one field performing under 90%, with comparable results observed from the fine-tuned models.
The Fibonacci Network: A Simple Alternative for Positional Encoding
Venue: arXiv cs.LG 2024
Year: 2024

Coordinate-based Multi-Layer Perceptrons (MLPs) are known to have difficulty reconstructing high frequencies of the training data. A common solution to this problem is Positional Encoding (PE), which has become quite popular. However, PE has drawbacks. It has high-frequency artifacts and adds another hyper-hyperparameter, just like batch normalization and dropout do. We believe that under certain circumstances PE is not necessary, and a smarter construction of the network architecture together with a smart training method is sufficient to achieve similar results. In this paper, we show that very simple MLPs can quite easily output a frequency when given input of the half-frequency and quarter-frequency. Using this, we design a network architecture in blocks, where the input to each block is the output of the two previous blocks along with the original input. We call this a {\it Fibonacci Network}. By training each block on the corresponding frequencies of the signal, we show that Fibonacci Networks can reconstruct arbitrarily high frequencies.
A Brief History of Named Entity Recognition
Venue: arXiv cs.CL 2024
Year: 2024

A large amount of information in today's world is now stored in knowledge bases. Named Entity Recognition (NER) is a process of extracting, disambiguation, and linking an entity from raw text to insightful and structured knowledge bases. More concretely, it is identifying and classifying entities in the text that are crucial for Information Extraction, Semantic Annotation, Question Answering, Ontology Population, and so on. The process of NER has evolved in the last three decades since it first appeared in 1996. In this survey, we study the evolution of techniques employed for NER and compare the results, starting from supervised to the developing unsupervised learning methods.
Precision or Recall? An Analysis of Image Captions for Training Text-to-Image Generation Model
Venue: arXiv cs.CV 2024
Year: 2024

Despite advancements in text-to-image models, generating images that precisely align with textual descriptions remains challenging due to misalignment in training data. In this paper, we analyze the critical role of caption precision and recall in text-to-image model training. Our analysis of human-annotated captions shows that both precision and recall are important for text-image alignment, but precision has a more significant impact. Leveraging these insights, we utilize Large Vision Language Models to generate synthetic captions for training. Models trained with these synthetic captions show similar behavior to those trained on human-annotated captions, underscores the potential for synthetic data in text-to-image training.
Findings of the IWSLT 2024 Evaluation Campaign
Venue: arXiv cs.CL 2024
Year: 2024

This paper reports on the shared tasks organized by the 21st IWSLT Conference. The shared tasks address 7 scientific challenges in spoken language translation: simultaneous and offline translation, automatic subtitling and dubbing, speech-to-speech translation, dialect and low-resource speech translation, and Indic languages. The shared tasks attracted 18 teams whose submissions are documented in 26 system papers. The growing interest towards spoken language translation is also witnessed by the constantly increasing number of shared task organizers and contributors to the overview paper, almost evenly distributed across industry and academia.
MoHeat: A Modular Platform for High-Responsive Non-Contact Thermal Feedback Interactions
Venue: arXiv cs.HC 2024
Year: 2024

MoHeat is a modular hardware and software platform designed for rapid prototyping of highly responsive, non-contact thermal feedback interactions. In our previous work, we developed an intensity-adjustable, highly responsive, non-contact thermal feedback system by integrating the vortex effect and thermal radiation. In this study, we further enhanced the system by developing an authoring tool that allows users to freely adjust the intensity of thermal stimuli, the duration of stimuli, the delay time before stimuli, and the interval between alternating hot and cold stimuli. This modular approach enables countless combinations of non-contact thermal feedback experiences.
Skill Transfer System that Visualizes and Presents Tactile Information in an AR Environment
Venue: arXiv cs.HC 2024
Year: 2024

In recent years, the lack of successors for traditional skills has become an issue. To solve this problem, we propose a skill transfer system that presents tactile information in spatial tasks as a color map on an AR space. We believe that providing the operator with feedback of the force and tactile information during the work is useful for learning skills that require time to master. Furthermore, by following the operator's hand and presenting tactile information, we expect to accelerate the learning of skills by not only presenting tactile information as a physical sensation, but also by making the operator associate tactile information with position.
Friction tunable electrostatic clutch with low driving voltage for kinesthetic haptic feedback
Venue: arXiv cs.HC 2024
Year: 2024

As interest in Virtual Reality (VR) and Augmented Reality (AR) increases, the demand for kinesthetic haptic feedback devices is rapidly rising. Motor based haptic interfaces are heavy and bulky, leading to discomfort for the user. To address this issue, haptic gloves based on electrostatic clutches that offer fast response times and a thin form factor are being researched. However, high operating voltages and variable force control remain challenges to overcome. Electrostatic clutches utilizing functional polymers with charge accumulation properties and dielectric liquid can generate the frictional shear stress over a wide range from 0.35 N/cm$^2$ to 18.9 N/cm$^2$ at low voltages below 100 V. Based on this, the haptic glove generates a high blocking force and is comfortable to wear.
Independent perceptual process of microscopic texture and surface shapes through lateral resistive force cues
Venue: arXiv cs.HC 2024
Year: 2024

Macroscopic surface shapes, such as bumps and dents, as well as microscopic surface features, like texture, can be identified solely through lateral resistive force cues when a stylus moves across them. This perceptual phenomenon has been utilized to advance tactile presentation techniques for surface tactile displays. However, the effects on shape recognition when microscopic textures and macroscopic shapes coexist have not been thoroughly investigated. This study reveals that macroscopic surface shapes can be recognized independently of the presence of microscopic textures. These findings enhance our understanding of human perceptual properties and contribute to the development of tactile displays.