Auscultation-LLM: Bridging Classification and Reasoning in Heart Sound Analysis with a Multimodal Large Language Model
Authors: Zhao Zihan, Pingjie Wang, Liudan Zhao, Ya Zhang, Kun Sun, Xin Sun, Xin Zhou, Yu Wang, Yanfeng Wang


Reviewer paper summaries: This paper introduces Auscultation-LLM, a multimodal large language model capable of analyzing heart sounds and providing diagnostic reasoning. The authors collected a novel dataset with rich annotations including conditions of first (S1), second (S2), third (S3) heart sounds, murmurs, and disease diagnoses. They also proposed different training approach for training such model, including Textbook-driven LLM Pre-training, Enhanced feature alignment method, Logit adjustment for LLM, where the results showed to be effective.
Summary of strengths: 1. The paper is well-written and easy to read.
2. The authors explores an important problem of developing multimodal AI for heart sounds.
3. The Auscultation-LLM incorporates valid techniques to improve performance. Ablation experiments clearly show the benefit of each component.
Summary of weaknesses: 1. The evaluation and training data focus on children heart sounds, so generalizability to adult heart sounds is unknown. If the authors do not plan to evaluate on adult sounds data, I would suggest revising the title accordingly to make it clearer.
2. The mapping between annotations and specific locations of heart sounds is also not provided, which the authors note is important for clinical precision. More details on the data collection process would also be helpful.
3. Using GPT-4 to generate the reasoning content is a bit concerning, as the authors note it could lead to misunderstandings in special cases. More details on how potential inaccuracies were checked and an error analysis of a sample of the reasoning outputs would help validate their reliability.
4. Missing SOTA baselines comparisons. The majority of the tasks mentioned in this work can be formularized as classification task, and there are many models for this [1].
5. The data description lacks sufficient detail. For example, what are the QA pairs look like? What kinds of heart sounds analysis does the dataset cover? Could you provide some examples for this?

[1] Ren, Zhao, et al. "A comprehensive survey on heart sound analysis in the deep learning era." IEEE Computational Intelligence Magazine 19.3 (2024): 42-57.
Comments, suggestions and typos: 1. Why the BLEU score is so poor but ROUGE is relatively high in table1? 
2. Could you provide more detail for the model's responses as qualitative analysis?
3. Are you going to publish the dataset and model checkpoint as these are the main contribution of the work?
Overall assessment scores: 2