DeDeckerThomas
commited on
Commit
β’
ac2ec8c
1
Parent(s):
19fcc00
Update README.md
Browse files
README.md
CHANGED
@@ -55,7 +55,6 @@ from transformers import (
|
|
55 |
AutoModelForSeq2SeqLM,
|
56 |
AutoTokenizer,
|
57 |
)
|
58 |
-
import numpy as np
|
59 |
|
60 |
|
61 |
class KeyphraseGenerationPipeline(Text2TextGenerationPipeline):
|
@@ -72,22 +71,24 @@ class KeyphraseGenerationPipeline(Text2TextGenerationPipeline):
|
|
72 |
results = super().postprocess(
|
73 |
model_outputs=model_outputs
|
74 |
)
|
75 |
-
return [[keyphrase.strip() for keyphrase in result.get("generated_text").split(self.keyphrase_sep_token)] for result in results]
|
|
|
76 |
```
|
77 |
|
78 |
```python
|
79 |
# Load pipeline
|
80 |
-
model_name = "
|
81 |
generator = KeyphraseGenerationPipeline(model=model_name)
|
82 |
|
83 |
```python
|
84 |
text = """
|
85 |
Keyphrase extraction is a technique in text analysis where you extract the important keyphrases from a text.
|
86 |
Since this is a time-consuming process, Artificial Intelligence is used to automate it.
|
87 |
-
Currently, classical machine learning methods, that use statistics and linguistics,
|
88 |
-
The fact that these methods have been widely used in the community
|
89 |
-
|
90 |
-
|
|
|
91 |
""".replace(
|
92 |
"\n", ""
|
93 |
)
|
@@ -100,7 +101,7 @@ print(keyphrases)
|
|
100 |
|
101 |
```
|
102 |
# Output
|
103 |
-
[['keyphrase extraction', 'text analysis', 'artificial intelligence', 'classical machine learning
|
104 |
```
|
105 |
|
106 |
## π Training Dataset
|
@@ -201,4 +202,4 @@ Abstractive keyphrases
|
|
201 |
For more information on the evaluation process, you can take a look at the keyphrase extraction evaluation notebook.
|
202 |
|
203 |
## π¨ Issues
|
204 |
-
Please feel free to
|
|
|
55 |
AutoModelForSeq2SeqLM,
|
56 |
AutoTokenizer,
|
57 |
)
|
|
|
58 |
|
59 |
|
60 |
class KeyphraseGenerationPipeline(Text2TextGenerationPipeline):
|
|
|
71 |
results = super().postprocess(
|
72 |
model_outputs=model_outputs
|
73 |
)
|
74 |
+
return [[keyphrase.strip() for keyphrase in result.get("generated_text").split(self.keyphrase_sep_token) if keyphrase != ""] for result in results]
|
75 |
+
|
76 |
```
|
77 |
|
78 |
```python
|
79 |
# Load pipeline
|
80 |
+
model_name = "ml6team/keyphrase-generation-t5-small-inspec"
|
81 |
generator = KeyphraseGenerationPipeline(model=model_name)
|
82 |
|
83 |
```python
|
84 |
text = """
|
85 |
Keyphrase extraction is a technique in text analysis where you extract the important keyphrases from a text.
|
86 |
Since this is a time-consuming process, Artificial Intelligence is used to automate it.
|
87 |
+
Currently, classical machine learning methods, that use statistics and linguistics,
|
88 |
+
are widely used for the extraction process. The fact that these methods have been widely used in the community
|
89 |
+
has the advantage that there are many easy-to-use libraries. Now with the recent innovations in NLP,
|
90 |
+
transformers can be used to improve keyphrase extraction. Transformers also focus on the semantics
|
91 |
+
and context of a document, which is quite an improvement.
|
92 |
""".replace(
|
93 |
"\n", ""
|
94 |
)
|
|
|
101 |
|
102 |
```
|
103 |
# Output
|
104 |
+
[['keyphrase extraction', 'text analysis', 'artificial intelligence', 'classical machine learning methods']]
|
105 |
```
|
106 |
|
107 |
## π Training Dataset
|
|
|
202 |
For more information on the evaluation process, you can take a look at the keyphrase extraction evaluation notebook.
|
203 |
|
204 |
## π¨ Issues
|
205 |
+
Please feel free to start discussions in the Community Tab.
|