How was the 12 layer geneformer model trained?
#98
by
ricomnl
- opened
Hi there,
Thanks for your amazing work (love the almost daily new commits)! I was curious, how was the 12 layer geneformer model trained? Did you use the same training strategy / hyperparameters or what changed compared to how the 6 layer model was trained?
Cheers!
Thank you for your interest in Geneformer! Yes, it is the same strategy but different hyperparameters. You can load the training_args.bin to see the hyperparameters used:
import torch
training_args = torch.load(path_to_training_args)
training_args
ctheodoris
changed discussion status to
closed