Spaces:
Runtime error
Runtime error
Update text.py
Browse files
text.py
CHANGED
|
@@ -5,7 +5,8 @@ The goal of text summarization is to condense long documents into summaries, whi
|
|
| 5 |
|
| 6 |
Text summarization methods can either be used as an extractive or abstractive model. An Extractive method does what it sounds like, it concatenates different important sentences or paragraphs without understanding the meaning of those parts. Extractive summarization does not create any new word phrases. For instance, if you presented a page of text to an extractive model, it would just act as a text “highlighter”. However, Abstractive summarization generates text in a fashion that tries to guess the meaning in a summarised way of the page of text it is presented. It would put words together in a meaningful way and add the most important fact found in the text.
|
| 7 |
|
| 8 |
-
|
|
|
|
| 9 |
|
| 10 |
<h3><center> Abstractive Model </center></h3>
|
| 11 |
|
|
@@ -22,7 +23,7 @@ sum_app_text_tab_2= """
|
|
| 22 |
|
| 23 |
The figure below illustrates how the skill level of the model increases at each step:
|
| 24 |
|
| 25 |
-
|
| 26 |
|
| 27 |
The main benefits of transfer learning in general include the saving of resources and improved efficiency when training new models, so feel free to adopt this model for your type of problem!
|
| 28 |
|
|
@@ -30,7 +31,7 @@ The main benefits of transfer learning in general include the saving of resource
|
|
| 30 |
|
| 31 |
The extractive models for this app are using sentence-transformer models, which basically is a bi-encoder that determines how similar two sentences are. This type of models convert texts into vectors (embedding) that capture semantic information. Additionally, LexRank, an unsupervised graph-based algorithm, is used to determine centrality scores as a post-process step to summarise. The main idea is that sentences "recommend" other similar sentences to the reader. Thus, if one sentence is very similar to many others, it will likely be a sentence of great importance. The importance of this sentence also stems from the importance of the sentences "recommending" it. Thus, to get ranked highly and placed in a summary, a sentence must be similar to many sentences that are in turn also similar to many other sentences.
|
| 32 |
|
| 33 |
-
|
| 34 |
|
| 35 |
The figure above showcase how LexRank formats similarity graphs based on all possible sentence combinations sentence similarity from the vector embeddings. Notice that the most "recommended" sentences that are extracted (the right graph) are derived from a threshold value which filters "weaker" connections in the similarity graph.
|
| 36 |
For more information on this topic read: [LexRank](https://www.aaai.org/Papers/JAIR/Vol22/JAIR-2214.pdf)
|
|
|
|
| 5 |
|
| 6 |
Text summarization methods can either be used as an extractive or abstractive model. An Extractive method does what it sounds like, it concatenates different important sentences or paragraphs without understanding the meaning of those parts. Extractive summarization does not create any new word phrases. For instance, if you presented a page of text to an extractive model, it would just act as a text “highlighter”. However, Abstractive summarization generates text in a fashion that tries to guess the meaning in a summarised way of the page of text it is presented. It would put words together in a meaningful way and add the most important fact found in the text.
|
| 7 |
|
| 8 |
+
<center>*image_caption*</center>
|
| 9 |
+
|
| 10 |
|
| 11 |
<h3><center> Abstractive Model </center></h3>
|
| 12 |
|
|
|
|
| 23 |
|
| 24 |
The figure below illustrates how the skill level of the model increases at each step:
|
| 25 |
|
| 26 |
+
<center>*image_caption*</center>
|
| 27 |
|
| 28 |
The main benefits of transfer learning in general include the saving of resources and improved efficiency when training new models, so feel free to adopt this model for your type of problem!
|
| 29 |
|
|
|
|
| 31 |
|
| 32 |
The extractive models for this app are using sentence-transformer models, which basically is a bi-encoder that determines how similar two sentences are. This type of models convert texts into vectors (embedding) that capture semantic information. Additionally, LexRank, an unsupervised graph-based algorithm, is used to determine centrality scores as a post-process step to summarise. The main idea is that sentences "recommend" other similar sentences to the reader. Thus, if one sentence is very similar to many others, it will likely be a sentence of great importance. The importance of this sentence also stems from the importance of the sentences "recommending" it. Thus, to get ranked highly and placed in a summary, a sentence must be similar to many sentences that are in turn also similar to many other sentences.
|
| 33 |
|
| 34 |
+
<center>*image_caption*</center>
|
| 35 |
|
| 36 |
The figure above showcase how LexRank formats similarity graphs based on all possible sentence combinations sentence similarity from the vector embeddings. Notice that the most "recommended" sentences that are extracted (the right graph) are derived from a threshold value which filters "weaker" connections in the similarity graph.
|
| 37 |
For more information on this topic read: [LexRank](https://www.aaai.org/Papers/JAIR/Vol22/JAIR-2214.pdf)
|