  This paper describes our system for SemEval-2021 Task 5 on Toxic Spans
Detection. We developed ensemble models using BERT-based neural architectures
and post-processing to combine tokens into spans. We evaluated several
pre-trained language models using various ensemble techniques for toxic span
identification and achieved sizable improvements over our baseline fine-tuned
BERT models. Finally, our system obtained a F1-score of 67.55% on test data.
