HW-202337458-transformer-encoder

Assignment

Transformer Encoder 구현 코드 업로드 과제입니다.

Student

• Student ID: 202337458
• Hugging Face ID: hf-june

Included Modules

• PositionWiseFeedForward
• EncoderBlock
• TransformerEncoder
• CustomConfig

Files

• transformer_encoder.py
• config.json
• test_transformer_encoder.py
• README.md

How to Use

import torch
from transformer_encoder import CustomConfig, TransformerEncoder

config = CustomConfig()
model = TransformerEncoder(config)

input_ids = torch.randint(0, config.vocab_size, (2, 10))
attention_mask = torch.ones_like(input_ids)

output = model(input_ids, attention_mask)
print(output.shape)

Expected Output

torch.Size([2, 10, 256])