Update README.md
Browse files
README.md
CHANGED
@@ -23,4 +23,45 @@ inference:
|
|
23 |
early_stopping: True
|
24 |
---
|
25 |
|
26 |
-
# literary analysis with t5-base
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
23 |
early_stopping: True
|
24 |
---
|
25 |
|
26 |
+
# literary analysis with t5-base
|
27 |
+
|
28 |
+
- t5 sort-of learning to do literary analysis. It was trained on the booksum dataset with `chapter` (original text) as input and `summary_analysis` as the output text, where `summary_analysis` is the sparknotes/cliff notes/etc analysis
|
29 |
+
- It was trained for 8 epochs
|
30 |
+
- Testing may need to be completed in Colab as it seems to be CPU-intensive
|
31 |
+
|
32 |
+
# Example
|
33 |
+
|
34 |
+
```
|
35 |
+
!pip install -U -q transformers
|
36 |
+
!pip install -U -q sentencepiece
|
37 |
+
|
38 |
+
from transformers import pipeline
|
39 |
+
|
40 |
+
analyzer = pipeline("text2text-generation",
|
41 |
+
"pszemraj/t5-v1_1-base-finetuned-booksum")
|
42 |
+
```
|
43 |
+
|
44 |
+
- enter text and compute.
|
45 |
+
|
46 |
+
```
|
47 |
+
text = "text to be analyzed goes here"
|
48 |
+
|
49 |
+
result = analyzer(
|
50 |
+
text,
|
51 |
+
max_length=int(len(text) * 1.2),
|
52 |
+
no_repeat_ngram_size= 2,
|
53 |
+
repetition_penalty= 2.4,
|
54 |
+
num_beams=4,
|
55 |
+
early_stopping= True,
|
56 |
+
)
|
57 |
+
```
|
58 |
+
|
59 |
+
# sample results
|
60 |
+
|
61 |
+
- inputs:
|
62 |
+
|
63 |
+
> The ledge, where I placed my candle, had a few mildewed books piled up in one corner; and it was covered with writing scratched on the paint. This writing, however, was nothing but a name repeated in all kinds of characters, large and small—Catherine Earnshaw, here and there varied to Catherine Heathcliff, and then again to Catherine Linton. In vapid listlessness I leant my head against the window, and continued spelling over Catherine Earnshaw—Heathcliff—Linton, till my eyes closed; but they had not rested five minutes when a glare of white letters started from the dark, as vivid as spectres—the air swarmed with Catherines; and rousing myself to dispel the obtrusive name, I discovered my candle wick reclining on one of the antique volumes, and perfuming the place with an odour of roasted calf-skin.
|
64 |
+
|
65 |
+
- output:
|
66 |
+
|
67 |
+
> In this chapter, Catherine Heathcliff and Linton are the only characters in the novel who have been able to escape from their lives. The candle wick is placed on one of the antique books that were used as a lamp for the first time. It is also important to note that she has no idea what her name means. She does not know how to pronounce it, but she knows that there is something wrong with his name. He cannot understand why he should be called Catherine Earnshaw-Heathcliff; however, I do not want to make him feel comfortable. This is an example of remarkstrayeshadowed by the reader's own mind. As we learn more about the story, we realize that Catherine earnshaw is unable to find out of any kind of anything else. At the end of these chapters, at the beginning of Chapter 1, we see that they are all too much different from each other.
|