  Toxic Spans Detection(TSD) task is defined as highlighting spans that make a
text toxic. Many works have been done to classify a given comment or document
as toxic or non-toxic. However, none of those proposed models work at the token
level. In this paper, we propose a self-attention-based bidirectional gated
recurrent unit(BiGRU) with a multi-embedding representation of the tokens. Our
proposed model enriches the representation by a combination of GPT-2, GloVe,
and RoBERTa embeddings, which led to promising results. Experimental results
show that our proposed approach is very effective in detecting span tokens.
