HUB @ D ravidian L ang T ech- EACL 2021: Meme Classification for T amil Text-Image Fusion (2021)

Abstract: This article describes our system for task DravidianLangTech - EACL2021: Meme classification for Tamil. In recent years, we have witnessed the rapid development of the Internet and social media. Compared with traditional TV and radio media platforms, there are not so many restrictions on the use of online social media for individuals and many functions of online social media platforms are free. Based on this feature of social media, it is difficult for people’s posts/comments on social media to be strictly and effectively controlled like TV and radio content. Therefore, the detection of negative information in social media has attracted attention from academic and industrial fields in recent years. The task of classifying memes is also driven by offensive posts/comments prevalent on social media. The data of the meme classification task is the fusion data of text and image information. To identify the content expressed by the meme, we develop a system that combines BiGRU and CNN. It can fuse visual features and text features to achieve the purpose of using multi-modal information from memetic data. In this article, we discuss our methods, models, experiments, and results.

## introduction and background the meme is a kind of multimedia document based on image level, which is a picture containing text. netizens can express their feelings, opinions, interests, and so on through it @xcite . the meme is an image with some kind of subtitle text embedded in the image pixels. in the past few years, emoticons have become very popular and used in many different contexts, especially young people @xcite . however, this form is also used to produce and spread hate speech in the form of black humor. the meme is a term proposed by biologist richard dawkins. it was used to describe the flow, flow, mutation, and evolution of culture and is a means of cultural resistance to genes @xcite . but the meaning of this term has changed in our public life. in social media, meme is a kind of self-media work, which is produced and spread by the majority of internet users through social media networks @xcite . negative information detection on the internet has become a core social challenge. nowadays, the detection of negative information in social media has become more and more intelligent @xcite . however, due to the multimodal nature of memes, it is difficult to be intelligently identified whether it is aggressive @xcite . especially in india, some recent memes containing hate messages have threatened people's lives several times @xcite . due to the large population and the mixing of multiple languages, a large number of indian memes are difficult to monitor due to the lack of intelligent systems for specific languages. this adds another serious challenge to the problem of memetic classification. in this work, we participated in the shared task of memetic classification in tamil @xcite . the indian state of tamil nadu, as well as two independent countries, singapore and sri lanka, speak tamil as their official language. tamil was the first indian classical language to be listed as such, and it is still one of the world's oldest classical languages. dravidian civilisations are believed to have flourished in the indus valley civilization (3,300-1,900 bce), which was situated in the northwestern indian subcontinent, this period is considered as second sangam period in tamil. tamil is india's oldest language. tamil, pali, and prakrit all added words, texts, and grammar to sanskrit. we are committed to making our contribution to the identification of tamil memetic classification. ## related work in theory, a meme is an example of a large number of humorous words on the internet, copied or modified, and then spread to other users. but sometimes some information in internet memes can bring a negative impact @xcite . compared with the propagation speed of memes on the internet, the research progress of memes is much slower. the work of @xcite showed us that the combination of text and vision can help identify popular memetic descriptions @xcite . the automatic meme generation system implemented by vyalla et al. based on the transformer model can allow users to generate memes of their choice (vyalla and udandarao, 2020). compared with the text field, there is less research on sentiment analysis on memetic data. in recent years, deep learning technology has attracted the attention of researchers, especially in sentiment analysis tasks that are significantly better than traditional methods. zhang et al. used deep learning techniques for sentiment analysis on text datasets @xcite and poria et al. used deep learning models for sentiment analysis on image and video datasets @xcite . the work of sabat found that visual modalities provide more information in hate memetic detection than language modalities @xcite . suryawanshi and others shared with us their data sets and methods for detecting offensive content in multimodal memetic data sets @xcite . @xcite used a multimodal approach to determine the sentiment of a meme. the method of using both text and visual information to improve performance has also proven to be effective @xcite . generally speaking, the fusion strategy of text information and visual information can be divided into two methods @xcite . on the one hand, some models use feature-level fusion methods, where text input sources or image input sources are processed to extract a set of features. then, the feature set is put together for the final decision @xcite . on the other hand, other models perform complete processing of each input source and perform fusion at the decision-making level @xcite . the result we submitted was predicted using the second strategy. ## conclusion this article introduces the method and model system used by our team in the meme classification for the tamil shared task. we use a text and image fusion scheme to detect memetic categories in the tamil language environment. we analyzed the deficiencies of our system. these shortcomings are what we need to improve in our future work. we also have a lot of room for improvement in methods and systems. for example, in image processing, we can try some other network models such as @xcite , resnext @xcite , etc. some pre-trained language models in text processing. considering that meme pictures can quickly spread on internet social media and can express negative information across languages, we determined that dealing with similar issues in social media is very valuable and meaningful. similar issues that exist in social media in some small-language communities should also deserve our attention and study. in addition to improving our models and methods in future work, we will continue to pay attention to the research and development of memetic analysis.