pedro123483 commited on
Commit
30edfc6
1 Parent(s): 6a85c42

Update README.md

Browse files

atualizando o readme com as informações sobre o dataset utilizado para treinamento do modelo, como treinar e como usar o modelo.

Files changed (1) hide show
  1. README.md +83 -0
README.md CHANGED
@@ -11,6 +11,89 @@ model-index:
11
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
  should probably proofread and complete it, then remove this comment. -->
13
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  # results
15
 
16
  This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
 
11
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
  should probably proofread and complete it, then remove this comment. -->
13
 
14
+ # Dataset Utilizado
15
+
16
+ O modelo foi treinado utilizando o dataset IMDB, amplamente utilizado para tarefas de classificação de texto, especialmente para análise de sentimentos. O dataset contém 50.000 revisões de filmes rotuladas, divididas igualmente entre revisões positivas e negativas, com 25.000 exemplos para treinamento e 25.000 para teste.
17
+
18
+ Para carregar o dataset, é preciso utilizar a biblioteca datasets da Hugging Face:
19
+
20
+ from datasets import load_dataset
21
+ dataset = load_dataset("imdb")
22
+
23
+ # Como Treinar o Modelo
24
+
25
+ 1. Carregar o dataset:
26
+
27
+ from datasets import load_dataset
28
+ dataset = load_dataset("imdb")
29
+
30
+ 2. Pré-processamento:
31
+
32
+ from transformers import AutoTokenizer
33
+ tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
34
+ tokenized_datasets = dataset.map(lambda x: tokenizer(x['text'], padding='max_length', truncation=True), batched=True)
35
+
36
+ 3. Definir o Modelo e Argumentos de Treinamento:
37
+
38
+ from transformers import AutoModelForSequenceClassification, TrainingArguments, Trainer
39
+ import numpy as np
40
+
41
+ model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased", num_labels=2)
42
+
43
+ training_args = TrainingArguments(
44
+ output_dir="./results",
45
+ learning_rate=2e-5,
46
+ per_device_train_batch_size=32,
47
+ per_device_eval_batch_size=32,
48
+ num_train_epochs=1,
49
+ weight_decay=0.01,
50
+ evaluation_strategy="epoch",
51
+ push_to_hub=True
52
+ )
53
+
54
+ def compute_metrics(eval_pred):
55
+ logits, labels = eval_pred
56
+ predictions = np.argmax(logits, axis=-1)
57
+ return {"accuracy": (predictions == labels).mean()}
58
+
59
+ 4. Treinamento:
60
+
61
+ small_train_dataset = tokenized_datasets["train"].shuffle(seed=42).select(range(1000))
62
+ small_eval_dataset = tokenized_datasets["test"].shuffle(seed=42).select(range(100))
63
+
64
+ trainer = Trainer(
65
+ model=model,
66
+ args=training_args,
67
+ train_dataset=small_train_dataset,
68
+ eval_dataset=small_eval_dataset,
69
+ compute_metrics=compute_metrics
70
+ )
71
+
72
+ trainer.train()
73
+
74
+ # Como Utilizar o Modelo
75
+
76
+ Usando uma Pipeline:
77
+ from transformers import pipeline
78
+
79
+ pipe = pipeline("text-classification", model="pedro123483/results")
80
+
81
+ result = pipe("I loved this movie! It was fantastic and thrilling.")
82
+ print(result)
83
+
84
+ Carregando o Modelo Diretamente:
85
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
86
+ import numpy as np
87
+
88
+ tokenizer = AutoTokenizer.from_pretrained("pedro123483/results")
89
+ model = AutoModelForSequenceClassification.from_pretrained("pedro123483/results")
90
+
91
+ inputs = tokenizer("I loved this movie! It was fantastic and thrilling.", return_tensors="pt")
92
+ outputs = model(**inputs)
93
+ predictions = np.argmax(outputs.logits.detach().numpy(), axis=-1)
94
+ print(predictions)
95
+
96
+
97
  # results
98
 
99
  This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.