---
license: apache-2.0
---
---
language:
- es
license: apache-2.0
tags:
- "borges"
- "spanish"
- "text-generation"
datasets:
- "borges_works"
widget:
- text: "El modelo del lenguaje GPT es capaz de"
- text: "Las obras de Borges son una fuente rica de conocimiento y creatividad"
---

# GPT2-borges (gpt2-borges)

## Table of Contents
<details>
<summary>Click to expand</summary>

- [Overview](#overview)
- [Model description](#model-description)
- [Intended uses and limitations](#intended-uses-and-limitations)
- [How to Use](#how-to-use)
- [Training](#training)
  - [Training data](#training-data)
  - [Training procedure](#training-procedure)
- [Additional information](#additional-information)
  - [Author](#author)
  - [Contact information](#contact-information)
  - [Copyright](#copyright)
  - [Licensing information](#licensing-information)
  - [Funding](#funding)
  - [Citation Information](#citation-information)
  - [Disclaimer](#disclaimer)

</details>

## Overview

- **Architecture:** gpt2-base
- **Language:** Spanish
- **Task:** text-generation
- **Data:** Borges Works

## Model description

**GPT2-borges** is a transformer-based model for the Spanish language. It is based on the PlanTL-GOB-ES/gpt2-base-bne model and has been pre-trained using a curated dataset consisting of the complete works of Jorge Luis Borges, a renowned Argentine writer.

## Intended uses and limitations
You can use the raw model for text generation or fine-tune it to a downstream task.

## How to Use
Here is how to use this model:

You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we set a seed for reproducibility:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = 'lucasbiagettia/gpt2-base-borges'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
```

```python
input_text = "La arena me recuerda el profundo dolor de la nostalgia"

input_ids = self.tokenizer.encode(input_text, return_tensors="pt")
attention_mask = torch.ones(input_ids.shape, dtype=torch.long)
    

generated_text = self.model.generate(
            input_ids=input_ids,
            attention_mask=attention_mask,
            max_new_tokens = 100,
            num_return_sequences=1,
            no_repeat_ngram_size=6,
            top_k=35,
            top_p=0.95,
            temperature=0.8,
            pad_token_id=50256,
            do_sample=True,
)
```

## Training
was trained with the following dataset:

https://github.com/lucasbiagettia/borges_plain_text_dataset