Minerster commited on
Commit
4d3588f
·
1 Parent(s): 1ca6b04

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -1,3 +1,13 @@
1
  ---
2
  license: openrail
3
  ---
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: openrail
3
  ---
4
+ # Initialize the pipeline with the "text-davinci-002" model
5
+ segmenter = pipeline("text-segmentation", model="text-davinci-002", tokenizer='text-davinci-002')
6
+
7
+ # Segment the text
8
+ segmented_text = segmenter("This is a longer text that we want to segment into smaller chunks. Each chunk should correspond to a coherent piece of text.")
9
+
10
+ # Process each segment with ChatGPT
11
+ nlp = pipeline("text-generation", model="text-davinci-002", tokenizer='text-davinci-002')
12
+ for segment in segmented_text:
13
+ print(nlp(segment))