Instructions to use lzwjava/sec-edgar-gpt-124m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lzwjava/sec-edgar-gpt-124m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lzwjava/sec-edgar-gpt-124m")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lzwjava/sec-edgar-gpt-124m")
model = AutoModelForCausalLM.from_pretrained("lzwjava/sec-edgar-gpt-124m")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lzwjava/sec-edgar-gpt-124m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lzwjava/sec-edgar-gpt-124m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lzwjava/sec-edgar-gpt-124m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/lzwjava/sec-edgar-gpt-124m

SGLang

How to use lzwjava/sec-edgar-gpt-124m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lzwjava/sec-edgar-gpt-124m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lzwjava/sec-edgar-gpt-124m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lzwjava/sec-edgar-gpt-124m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lzwjava/sec-edgar-gpt-124m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use lzwjava/sec-edgar-gpt-124m with Docker Model Runner:
```
docker model run hf.co/lzwjava/sec-edgar-gpt-124m
```

SEC-EDGAR GPT-2 124M

A GPT-2 (124M) language model trained from scratch on SEC EDGAR filings (10-K, 10-Q, 8-K, etc.).

Model Details

Property	Value
Architecture	GPT-2 124M (12 layers, 12 heads, 768 hidden)
Parameters	124,475,904
Context Length	1,024 tokens
Tokenizer	GPT-2 BPE (tiktoken)
Training Tokens	~1.55B (1 epoch)
Training Steps	47,000
Validation Loss	2.28
Training Framework	nanoGPT
Training Hardware	NVIDIA RTX 4070 12GB
Training Time	~8 hours
Bias	No (`bias=False`)

Training Data

SEC EDGAR filings sourced from the SEC-EDGAR corpus on HuggingFace, covering annual reports (10-K), quarterly reports (10-Q), current reports (8-K), and other filing types. Tokenized with GPT-2 BPE into ~1.55B tokens across 16 shards.

Usage

from transformers import GPT2LMHeadModel, GPT2Tokenizer

model = GPT2LMHeadModel.from_pretrained("lzwjava/sec-edgar-gpt-124m")
tokenizer = GPT2Tokenizer.from_pretrained("lzwjava/sec-edgar-gpt-124m")

prompt = "UNITED STATES SECURITIES AND EXCHANGE COMMISSION"
input_ids = tokenizer.encode(prompt, return_tensors="pt")
output = model.generate(input_ids, max_new_tokens=200, temperature=0.8, do_sample=True)
print(tokenizer.decode(output[0]))

Limitations

Trained for only 1 epoch — coherent for ~200-500 tokens before repetitive loops
No instruction tuning or RLHF — raw language model
124M parameters is small; don't expect state-of-the-art quality
GPT-2 tokenizer may not handle all financial notation optimally

Source Code

Training code and development notes: github.com/lzwjava/sec-edgar-gpt

Citation

@misc{sec-edgar-gpt-124m,
  author = {Zhiwei Li},
  title = {SEC-EDGAR GPT-2 124M},
  year = {2026},
  publisher = {GitHub},
  url = {https://github.com/lzwjava/sec-edgar-gpt}
}

Downloads last month: -

Safetensors

Model size

0.1B params

Tensor type

F32