  Hate speech, quite common in the age of social media, at times harmless but
can also cause mental trauma to someone or even riots in communities. Image of
a religious symbol with derogatory comment or video of a man abusing a
particular community, all become hate speech with its every modality (such as
text, image, and audio) contributing towards it. Models based on a particular
modality of hate speech post on social media are not useful, rather, we need
models like multi-modal fusion models that consider both image and text while
classifying hate speech. Text-image fusion models are heavily parameterized,
hence we propose a quaternion neural network-based model having additional
fusion components for each pair of modalities. The model is tested on the
MMHS150K twitter dataset for hate speech classification. The model shows an
almost 75% reduction in parameters and also benefits us in terms of storage
space and training time while being at par in terms of performance as compared
to its real counterpart.
