Guiding Large Language Models via External Attention Prompting for Scientific Extreme Summarization (2024)

Abstract: Scientific extreme summarization, the task of generating concise one-sentence summaries (TLDRs) for scientific papers, presents significant challenges due to the need for deep domain-specific understanding and the ability to distill salient information. This study identifies the critical role of titles and keywords in enhancing TLDR generation through quantitative analysis. We propose a novel method, External Attention Prompting (EAP), which leverages LLMs by guiding them to focus on the most critical parts of the source text through varying degrees of attention signals. Our method employs Markdown emphasis syntax to annotate attention levels, enabling LLMs to prioritize salient information effectively. Extensive experiments demonstrate that EAP significantly outperforms baseline methods across various LLMs and metrics in both zero-shot and fewshot settings. Further evaluations by GPT-4 demonstrate that EAP can enable LLMs to generate TLDRs of higher human-aligned quality.

heading: 1 Introduction
text: The rapid growth of scientific literature has made it increasingly difficult for researchers to keep up with the research frontiers and quickly identify the most relevant and impactful information. In this context, recent work (Cachola et al., 2020; Lu et al., 2020; Mao et al., 2022; Takeshita et al., 2022; Atri et al., 2023; Stiglic et al., 2023; Syed et al., 2024) has studied the problem of scientific extreme summarization, which involves generating a concise one-sentence summary (TLDR: Too Long; Didn’t Read) that captures the key aspects of a scientific paper. Scientific extreme summarization is a challenging task that requires a deep understanding of domain-specific content to accurately identify and distill the most salient points (Cachola et al., 2020). This task demands advanced methods capable of
*Corresponding Author
leveraging domain knowledge and prioritizing the most important content.
Previous studies have found that key content such as the titles and keywords of source documents are helpful to improve performance in summarization task, as they often contain key information (Chen et al., 2019; Çano and Bojar, 2019; Lee et al., 2020; Cachola et al., 2020; Koto et al., 2022; Mao et al., 2022). Our analysis on the SciTLDR dataset (Cachola et al., 2020) further indicates that their contributions to generating highly compressed summaries differ as they contain diverse density of key information (see Section 2), suggesting that they should be given unequal levels of importance.
In this work, we leverage large language models (LLMs) to tackle the scientific extreme summarization task, as they have demonstrated emergent abilities such as in-context learning and made remarkable advances in solving various NLP problems (Wei et al., 2022; Zhao et al., 2023). We propose a novel method called External Attention Prompting (EAP) that enhances the performance of LLMs by guiding them to pay varying degrees of attention to
226
different segments of the source text. Our method involves constructing prompts that assign different levels of attention signals to the n-grams in the source text based on their importance, using Markdown emphasis syntax to annotate the attention strength. This approach enables the LLMs to prioritize the salient information and generate more accurate and informative TLDRs. Experimental results demonstrate that our proposed prompting method significantly enhances the performance of LLMs in this task, outperforming baseline methods.
The main contributions of this work are as follows:
• We provide insights into the importance of different source input components (title, abstract, and keywords) for scientific extreme summarization, highlighting the role of the title and keywords in capturing key information.
• We propose a novel prompting method, External Attention Prompting, that enhances the performance of LLMs in scientific extreme summarization by guiding them to focus on the most critical parts of the source text.
• We conduct extensive experiments on the SciTLDR dataset, demonstrating the effectiveness of our approach in improving the quality of generated TLDRs.

heading: 2 Source Contribution Analysis
text: Generating high-quality TLDRs requires capturing precisely the salient aspects of the document. In this section, we conduct a preliminary experiment to investigate whether components such as the title, abstract and keywords contribute differently to the formation of a TLDR. By identifying the most influential parts, we aim to enhance the summarization process by directing the model’s attention to the most critical information.
To do this, we firstly introduce the concept of contribution density, which measures the averaged contribution of tokens from a specific source input components (e.g., title, abstract, and keywords) to the gold TLDR. We first tokenize the input components and remove stopwords. Then, the contribution density CD(c) for an input component c is calculated as:
CD(c) =
∑ t∈c TF (t, TLDR)
|c| (1)
where t is a token in the input component c, TF (t, TLDR) is the term frequency of token t in the gold TLDR, and |c| is the total number of tokens in the input component c.
The calculation of contribution density draws inspiration from methods used in previous works (See et al., 2017; Narayan et al., 2018; Cachola et al., 2020) to assess abstractiveness and extractiveness of generated summaries. However, in this context, it serves as an analytical tool rather than a direct evaluation metric for summary quality assessment. The term "contribution" represents how frequently the words from source input appear in the gold TLDR. We use contribution density as a proxy to quantify the relative importance of different source input components in contributing to extreme summarization.
We take the training set of SciTLDR (Cachola et al., 2020) for analysis, and calculate the contribution density for title and abstract, which are 49.2 and 11.9, respectively. The results align with our intuition that the title often captures the most core idea of a paper. While titles may contain less overall information than abstracts, the information they do contain is more densely packed with content relevant to the TLDR. In other words, any given n-gram from the title is more likely to contribute to the TLDR than a random n-gram from the abstract. Abstracts may contain more information that is either less important or not sufficiently crucial to be included in the TLDR.
Additionally, we calculate the contribution densities for the top-k keywords from abstract using TextRank algorithm (Mihalcea and Tarau, 2004), the results are shown in Figure 1. The contribution density decreases as k increases, indicating diminishing returns with more keywords. This highlights the importance of selecting a concise set of highly relevant keywords.
The above findings from our source contribution analysis underscore the necessity of focusing on the most important parts of the input text when generating a TLDR.

heading: 3 External Attention Prompting
text: Based on the insight from section 2, we propose a novel method called External Attention Prompting (EAP). The core idea of our method is to guide LLMs to focus on the most critical parts of the source text when generating TLDRs. As illustrated in Figure 2, our method leverages external signals
derived from the paper’s title and keywords to modulate the attention of the LLM. By emphasizing different segments of the source text with varying degrees of importance, we aim to enhance LLMs’ ability to distill the essence of the scientific paper into a concise and informative TLDR.

heading: 3.1 Attention Signal Acquisition
text: To effectively guide the model’s attention, this method first acquires attention signals. We hypothesize that certain n-grams within the text, particularly those overlapping with the title and keywords, are more indicative of the core content. To identify these n-grams, we perform the following steps: Title Overlap Extraction We extract n-grams from the title and identify their occurrences in the source text. To ensure relevance, we remove stop words and retain only the longest overlapping ngrams. Keyword Extraction Using the TextRank algorithm (Mihalcea and Tarau, 2004), we extract keywords from the source text, removing stop words to focus on meaningful terms. Attention Signal Levels Definition We define attention signals of different levels as follows:
• Strong Attention Signal: N-grams present in both the title and the top-5 keywords.
• Medium Attention Signal: N-grams present in both the title and any keyword.
• Weak Attention Signal: N-grams present in either the title or the top-5 keywords.

heading: 3.2 Prompt Construction
text: To operationalize these attention signals, we employ Markdown emphasis syntax to “visually” highlight the importance of different n-grams within
the source text. Markdown is chosen for its inherent three-level emphasis system (italic, bold, and bold-italic) and because there is a large amount of Markdown-formatted text in the training corpus of LLMs, enabling them to understand the emphasis cues implied by such annotations. Specifically, we mark n-grams of strong attention signal with *** (bold and italic), n-grams of medium attention signal with ** (bold) and n-grams of weak attention signal with * (italic).
By using Markdown’s built-in emphasis features, we avoid the need for custom formatting, allowing the model to leverage its pre-existing understanding of these conventions. This approach ensures that the model can efficiently recognize and prioritize the marked text without additional fine-tuning or instruction.
The marked source text, along with necessary instructions, is then used to construct the prompt, which is fed into the LLMs to generate the TLDR. Furthermore, we explicitly instruct the model to pay special attention to the emphasized parts of the text. The prompt template used for this method can be seen in Appendix B.
The term "External" in this method reflects two key aspects: External attention signals. The attention signals are derived from external information such as the title and keywords, rather than being internally recognized by the model. This external guidance directs the model to follow specific cues. External implementation. The attention mechanism is implemented through external input (i.e., the prompt) without fine-tuning LLMs. This makes the method applicable even when model parameters are inaccessible.
The proposed method is highly generalizable and can be applied to various tasks requiring controlled text generation or tasks where the importance of different parts of the input is known a priori. The attention signals can be provided by domain experts, making the approach adaptable to different contexts. This method is particularly suitable for tasks with long input texts, such as summarization and reading comprehension, where the model needs to focus on key content.
The external attention mechanism makes use of the LLMs’ capabilities in in-context learning and instruction following. By providing clear, externally defined attention signals, LLMs can better understand and prioritize the critical content, leading to improved performance in specific tasks.

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: Dataset. We evaluate our method on SciTLDR (Cachola et al., 2020), a widely used benchmark for scientific extreme summarization, featuring high-quality, human-generated gold TLDRs. Our experiments focus on the abstract-only subset (including title), rather than the AIC (abstract+introduction+conclusion) and full text subsets, to reduce API usage costs and to accommodate the limited context length of most LLMs. Additionally, abstracts are generally more accessible than full texts , making this setting more applicable to real-world scenarios.
Baselines. We compare the proposed EAP method with two baselines:
• Standard Prompting (SP): This baseline only provide necessary instructions and the abstracts to LLMs to generate the TLDRs. No additional guidance or emphasis is included.
• Explicit Guidance Prompting (EGP): In this baseline, the abstract is supplemented with the paper’s title and the top-5 keywords extracted using the TextRank algorithm. The LLMs are explicitly instructed to "pay attention to" the title and keywords, thereby directing its focus towards these critical components. This method provides explicit guidance to the model without altering the abstract.
• Random Attention Prompting (RAP): This baseline is a variant of our proposed EAP
method. It randomly applies different levels of emphasis to n-grams in the source text, maintaining the same number of emphasized n-grams and n-gram lengths as the corresponding EAP method. This baseline is designed to investigate whether the performance changes in EAP are due to the specific attention signals or simply the presence of Markdown emphasis, regardless of the content being emphasized.
All methods are evaluated under both zero-shot and few-shot settings. In the few-shot setting, we use the same two demonstration examples in prompt for all methods, with the input processed according to each respective method. The specific prompts utilized for all methods are detailed in Appendix B.
Large language models. To evaluate the effectiveness and generalizability of our proposed approach, we conduct experiments leveraging a diverse set of LLMs, including GPT-3.5, Llama-3, GLM-4, and DeepSeek-V2, encompassing both open-source and black-box models. The detailed information on the versions of LLMs used in our experiments is summarized in Table 2.
Evaluation metrics. In our evaluation, we use Rouge-1, Rouge-2, and Rouge-L metrics (Lin, 2004), consistent with previous work in summarization (Narayan et al., 2018; Lewis et al., 2020; Cachola et al., 2020). Considering SciTLDR provides multiple target summaries for a given paper, we follow Cachola et al. (2020) to taking the maximum Rouge score calculated over multiple gold TLDRs as the final Rouge score. This can effectively handles the variability in gold TLDRs.
Additionally, we use the Rouge-K metric (Takeshita et al., 2024), a keyword-oriented evaluation metric which specifically assesses the inclusion of essential keywords in the generated summaries, as it aligns with our goal of ensuring that the generated TLDRs capture the most critical content from the source documents.
Implementation details. All LLMs utilized in this work are accessed via API calls and configured with the same generation parameters: temperature of 0.1 and a maximum tokens limit of 128. Evaluation results for each LLM are averaged over three independent inferences for each input to mitigate the impact of sampling randomness of LLMs.

heading: 4.2 Main Results and Analysis
text: The experimental results, as presented in Table 1, demonstrate the effectiveness of the proposed EAP method across various LLMs and metrics in both zero-shot and few-shot settings. This indicates that the external attention signals effectively guide the LLMs to generate more accurate and informative TLDRs.
Zero-shot results. In the zero-shot setting, EAP outperforms all baseline methods (SP, EGP and RAP) across all LLMs on Rouge-1/2, suggesting that the external attention signals provided by EAP are effective even without prior examples. However, in the case of the Rouge-L metric, the EAP method underperforms compared to the baseline methods for GPT-3.5 and Llama-3, possibly due to the additional complexity and noise introduced by this attention mechanism.
Few-shot results. In the few-shot setting, EAP consistently outperforms all baseline methods across all metrics and LLMs. The majority of these improvements are statistically significant, as evidenced by the Wilcoxon signed-rank test
(α = 0.05). This indicates that providing demonstration examples helps LLMs learn how to effectively distill the core content by focusing on segments of varying importance, leading to the generation of more precise and comprehensive TLDRs. This enhanced performance can be attributed to the strong in-context learning capabilities of LLMs.
Explicit vs. implicit guidance. The EGP method, which explicitly provides titles and keywords as additional input, shows mixed results. In some cases, it performs worse than the SP method, which does not include any additional guidance. This suggests that merely adding extra information without fine-grained control can confuse the model, leading to inconsistent results. In contrast, the EAP method implicitly emphasizes different parts of the source text, allowing the model to better understand and prioritize the critical content. This implicit approach appears to be more effective in guiding the model’s attention, resulting in more stable and improved performance.
Random Attention Prompting results. The RAP baseline generally underperforms compared to SP and EAP methods across all LLMs and metrics in both zero-shot and few-shot settings. In the zero-shot setting, RAP often performs worse than SP, suggesting that random emphasis can be detrimental to the LLMs’ understanding of the source text. In the few-shot setting, while RAP shows some improvement over SP in certain cases (e.g., for GLM-4), it still consistently underperforms compared to EAP. These results indicate that the performance gains of EAP are not merely due to the presence of Markdown emphasis, but rather the
strategic placement of attention signals on important content. The effectiveness of EAP over RAP underscores the importance of deriving attention signals from key content such as titles and keywords, rather than applying emphasis randomly.
Results across LLMs. The results indicate that the EAP method consistently enhances performance across different models, demonstrating its generalizability and effectiveness. However, the degree of improvement varies among models, which may be related to the inherent capabilities of each model. For instance, models with stronger incontext learning abilities might benefit more from the EAP method, as they can better utilize the implicit attention signals to generate high-quality summaries.
Rouge-K results. The Rouge-K metric (Takeshita et al., 2024), which assesses the inclusion of essential keywords in the generated summaries, provides additional insights into the performance of the EAP method. As shown in Figure 3, in the zero-shot setting, EAP achieves higher Rouge-K scores compared to the baseline methods on Llama-3 and DeepSeek-V2. However, the EGP method surpasses EAP on GPT-3.5 and GLM-4, suggesting that explicitly providing keywords may be more effective for certain models in the absence of demonstration examples. In the few-shot setting, EAP consistently outperforms all the baseline methods across all models in terms of Rouge-K. This indicates that the combination of implicit attention signals and demonstration examples enables the models to better capture and include the most critical points in the generated TLDRs. The RAP method shows inconsistent performance on Rouge-K across different settings and models. In some cases, RAP outperforms SP or EGP methods, while in others, it underperforms. This variability can be attributed to the random nature of the attention signals in RAP, which may occasionally emphasize important content by chance. It is worth noting that the Rouge-K scores for all models and most methods are significantly lower in the few-shot setting compared to the zero-shot setting. This unexpected result may be attributed to the demonstration examples potentially biasing the models towards generating summaries that prioritize overall content coverage rather than focusing on specific keywords. Further investigation is needed to fully understand this phenomenon and its implications for keyword-oriented
summarization tasks.

heading: 4.3 LLM as Evaluator
text: Setup. In addition to using traditional automatic metrics, we explore the use of LLMs as evaluators to assess the quality of generated TLDRs. This approach aims to address the limitations of automatic metrics, which may not fully capture the nuances of human preferences and the overall quality of the summaries (Zheng et al., 2023). By leveraging LLMs, we seek to provide a more comprehensive and human-aligned evaluation of the generated TLDRs.
To this end, we employ GPT-4 (version gpt-40613), which is currently recognized as one of the most advanced LLMs, to assess the quality of TLDRs generated by our method compared to a baseline method.
We randomly sample 100 data points and collect TLDRs generated by GPT-3.5 using both the baseline SP method and our EAP method under a fewshot setting. The evaluation criteria focus on the TLDR’s ability to concisely capture the key aspects of a scientific paper while maintaining faithfulness to the source. To mitigate position bias (Zheng et al., 2023), each pair is evaluated twice with the positions of the two TLDRs swapped. A method is considered the winner for a sample only if both evaluations favored the same TLDR. In cases of conflicting results or ties, the sample is marked as a tie. The above process is repeated with three different random seeds, resulting in three rounds of evaluation.
Results. The results of GPT-4 evaluation are summarized in Table 3, showing that the proposed EAP method consistently outperforms the baseline SP method, achieving an average win rate of 37.33 % compared to the baseline’s 26.00 %. The average tie rate is 36.67 %, with an average conflict rate of 23.67 %. These results indicate that the EAP method significantly enhances the performance of LLMs in generating TLDRs, as judged by GPT-4.

heading: 4.4 Ablation Study
text: To investigate the effects of different components in EAP method, we conduct an ablation study to address two key research questions: (1) Does the multi-level attention signal mechanism contribute to the performance improvement compared to using a uniform attention signal? (2) Are attention signals derived from the title essential for enhancing the
summarization quality? To investigate the role of multi-level attention signals, we design an ablation variant of our method, in which we remove the differentiation between strong, medium, and weak attention signals. Instead, we uniformly apply a single level of attention signal by marking all relevant n-grams from the title and keywords with bold Markdown syntax.
To assess the importance of attention signals derived from the title, we create another ablation variant in which we exclude attention signals derived from the title and only mark the top-5 keywords extracted using the TextRank algorithm in the source text.
We conduct experiments using the same setup as described in Section 4.1, maintaining consistency in dataset, baselines, and evaluation metrics. The
results are summarized in Table 4.
Multi-Level Attention Signals The variant without multi-level attention signals exhibit a slight decline in performance across vast majority metrics and LLMs compared to the original EAP method. This suggests that the differentiation between strong, medium, and weak attention signals plays a crucial role in guiding the LLMs to prioritize the most salient information effectively.
Title-Derived Attention Signals: The variant without title attention shows a noticeable reduction in performance, indicating that while the multilevel attention mechanism is advantageous, the titlederived signals play a more crucial role in capturing the core idea of scientific papers, thereby contributing significantly to the overall performance improvement.

heading: 5 Related Work
text: Scientific Extreme Summarization Scientific extreme summarization, the task of generating extreme one-sentence summaries (TLDRs) for scientific papers, has gained increasing attention in recent years. Several datasets have been proposed to facilitate research in this area, including SciTLDR (Cachola et al., 2020), a high-quality dataset with human-written summaries, and CiteSum (Mao et al., 2022), which contains automatically generated summaries based on citations. Additionally, multi-document(Lu et al., 2020), multilingual (Takeshita et al., 2022) and multimodal (Atri et al., 2023) datasets have been introduced to explore the task in diverse settings. Existing ap-
proaches to scientific extreme summarization primarily rely on fine-tuned Transformer-based models, such as BART (Lewis et al., 2020) and T5 (Raffel et al., 2023), which have achieved promising results. However, the potential of LLMs for this task remains largely unexplored. While LLMs are trained for general-purpose tasks and may not match the performance of task-specific fine-tuned models (Takeshita et al., 2024), their extensive domain knowledge and strong in-context learning abilities make them a promising avenue for investigation in scientific extreme summarization.
Key Content Utilization for Summarization Previous studies have explored various key content, such as titles, keywords, topics, and key segments, to enhance summarization performance (Chen et al., 2019; Koto et al., 2022; Srivastava et al., 2023; Tang et al., 2023). These methods typically focus on incorporating key content as additional input features or using the prediction of key content as an auxiliary task. For instance, Koto et al. (2022) investigated the impact of incorporating titles and keyphrases as structured input and target outputs in news summarization. Cachola et al. (2020) leveraging titles as auxiliary training signals to improve model performance in the scientific extreme summarization task. In this work, we use key content to compute attention signals, rather than as additional input or target, to guide LLMs to focus on the important parts of the source text.
Prompt Optimization and Engineering Prompt engineering has emerged as a crucial area of research to enhance the capabilities of LLMs on complex tasks. Early approaches focused on tuning soft prompts, which are continuous embedding vectors optimized using gradient descent methods (Li and Liang, 2021; Vu et al., 2022; An et al., 2022; Tam et al., 2023). However, these approaches require access to the Internal parameters of LLMs, limiting their applicability to black-box models. Another direction involves designing task-specific natural language instructions and selecting appropriate incontext demonstrations (Shin et al., 2020; Brown et al., 2020; Deng et al., 2022; Zhou et al., 2023; Sun et al., 2023). Despite the promising results, these methods may struggle to capture fine-grained, instance-specific (Li et al., 2023). proposes a method that generates instance-specific prompts tailored to each input using a small, tunable policy model optimized through supervised fine-tuning and reinforcement learning. In this work, we aim to provide LLMs with instance-specific attention signals derived from key content to enhance their performance in the task of scientific extreme summarization.

heading: 6 Conclusion
text: In this work, we have presented External Attention Prompting (EAP), a novel method for improving the performance of large language models (LLMs) in the task of scientific extreme summarization. By
utilizing external attention signals derived from the title and keywords of scientific papers, EAP effectively guides LLMs to focus on the most salient information, resulting in more accurate and informative TLDRs. Our extensive experiments on the SciTLDR dataset demonstrate the superiority of EAP over standard prompting and explicit guidance methods, particularly in few-shot settings. The results underscore the importance of multi-level attention and the significant role of title emphasis in enhancing summarization quality. Future work could explore the application of EAP to other domains and tasks, further validating its versatility and effectiveness in various text generation scenarios.

heading: 7 Limitations
text: This work has the following limitations: (1) The heuristic-based design and computation of attention signals in EAP may not fully capture salient information in diverse contexts. Future work could explore automated methods. (2) The keyword extraction algorithm used in this work for computing attention signals only considers TextRank. Future research should explore and compare the effectiveness of various keyword extraction methods. (3) Our experiments focus on the abstract-only subset of SciTLDR, which may not fully represent the challenges of summarizing entire scientific papers. Furthermore, SciTLDR is focused on the computer science domain, which may limit the generalizability of our findings.

heading: Acknowledgments
text: The authors thank all the anonymous reviewers for their valuable comments and constructive feedback. The authors acknowledge financial support from the National Social Science Fund of China (No. 23BTQ102). Xiaoqiu Le is the corresponding author.

heading: A Case Study
text: To further illustrate the effectiveness of our proposed EAP method, we present a case study comparing the generated TLDRs from EAP and the baseline SP method in zero-shot setting using GPT3.5.
Table 5 showcases an example from the SciTLDR dataset, where the source paper is about combining differentiable decision trees (DDTs) with supervised variational autoencoders (SVAEs) to enhance the interpretability of classification tasks.
The TLDR generated by the SP method fails to capture the core idea of the paper, instead focusing on individual components such as DDTs and SVAEs without highlighting their synergistic combination. In contrast, the EAP method, guided by the attention signals derived from the title and keywords, successfully identifies the key aspects of the work, even in the absence of demonstration examples. The generated TLDR concisely conveys the main contribution: the integration of DDTs and SVAEs to create an interpretable and competitive classifier+VAE for high-dimensional data.
Moreover, the EAP-generated TLDR achieves a significantly higher Rouge-1 score (46.51) compared to the SP-generated TLDR (27.03), indicating its superior quality and relevance to the gold TLDRs. This case study demonstrates how the EAP method effectively guides the LLM to focus on the most critical information, resulting in more accurate and informative summaries.

heading: B Prompts
text: The prompts employed in this work for the standard prompting, explicit guidance prompting, and external attention prompting methods in the zeroshot and few-shot settings are illustrated in Figures 4, 5, 6, 7 and Figures 8, 9, 10, 11, respectively.
Title Interpretable Classification via Supervised Variational Autoencoders and Differentiable Decision Trees
Source abstract As deep learning-based classifiers are increasingly adopted in real-world applications, the importance of understanding how a particular label is chosen grows. Single decision trees are an example of a simple, interpretable classifier, but are unsuitable for use with complex, high-dimensional data. On the other hand, the variational autoencoder (VAE) is designed to learn a factored, low-dimensional representation of data, but typically encodes high-likelihood data in an intrinsically non-separable way. We introduce the differentiable decision tree (DDT) as a modular component of deep networks and a simple, differentiable loss function that allows for end-to-end optimization of a deep network to compress high-dimensional data for classification by a single decision tree. We also explore the power of labeled data in a supervised VAE (SVAE) with a Gaussian mixture prior, which leverages label information to produce a high-quality generative model with improved bounds on log-likelihood. We combine the SVAE with the DDT to get our classifier+VAE (C+VAE), which is competitive in both classification error and log-likelihood, despite optimizing both simultaneously and using a very simple encoder/decoder architecture.
TLDR-AU We combine differentiable decision trees with supervised variational autoencoders to enhance interpretability of classification.
TLDR-PR1 This paper proposes a hybrid model of a variational autoencoder composed with a differentiable decision tree, and an accompanying training scheme, with experiments demonstrating tree classification performance, neg. log likelihood performance, and latent space interpretability.
TLDR-PR2 The paper tries to build an interpretable and accurate classifier via stacking a supervised VAE and a differentiable decision tree

heading: Future Work
text: Future work could explore automated methods. (2) The keyword extraction algorithm used in this work for computing attention signals only considers TextRank. Future research should explore and compare the effectiveness of various keyword extraction methods. (3) Our experiments focus on the abstract-only subset of SciTLDR, which may not fully represent the challenges of summarizing entire scientific papers. Furthermore, SciTLDR is focused on the computer science domain, which may limit the generalizability of our findings