DOI: 10.5281/zenodo.15191021
title: Detecting-Warning-Labels-on-E-Cigarette-Content

description: Detecting-Warning-Labels-on-E-Cigarette-Content




Detecting Warning Labels on E-Cigarette Content Across Social Media Platforms




Introduction


This repository contains scripts for collecting data from TikTok and YouTube, processing them, and feeding them to a rule-based classifier. The pipeline consists of multiple steps, including video downloading, screenshot extraction, OCR processing, language detection, classification, and statistical analysis.



Technical requirements


Before proceeding, ensure that you have the following installed:



Python 3.x

An Oracle cloud instance

A Box account with API access

Basic command-line knowledge




Required Dependencies


Install the following Python libraries before running the scripts:



pip install opencv-python pandas numpy pytesseract langdetect requests boxsdk

 




Warning Label Detection Workflow


The figure illustrates the framework for collecting and extracting text from YouTube and TikTok videos, detailing the language detection process and the development of a rule-based classifier for warning labels.





Box Download




First we need to sign up for a free version of box.

We get to the developer console and then we access the APP console then we click create new app.

Then we click on custom app, and then we create a custom app.

Then we choose the authnetication type which is auth 2 and create the app afterwards.

Now in the configuration tab of the App that we created we click on App access only.

And then we choose Make API calls using using the as-user header and generate user access tokens.

and then we get the client ID, client secret and developer token.

The developer token has to be generated every 60 minutes because it will expire in every 60 minutes.

We now get the folder ID from box.

On the Oracle Instance we write a script to download all the videos. (get_videos.py)




Image Processing


Image processing procedures for each social media platform





Screenshots


The script takes screenshots every one second until the max time which is 79 seconds.



Oracle Vision




Takes the screenshots in form of images and turns them into text. (ocr.py)

Now we write a script called (remove_null.py) that gets rid of the rows containing no text.

We then write a script that gets rid of the duplicate texts inside the output text file called (remove_textdup.py)




Language Detection




We write a program that can take all the text extracted from the images in the csv file and detect their language and output a file called, extracted_lang_output.txt. (lang_detector.py)

In the next step, in the script called (lang_score.py), we parse the json formatted text file (extracted_lang_output.txt) and we set a threshold of 90% and say that if the language detected score is higher than 90% then consider it a valid prediction and write it to a csv file (language_score.csv) alongside its language name and the text.

We then remove the duplicates with a script called (remove_duplicates.py)

We then get the english text rows by performing the command




cat unique_lang_score.csv | grep English > warnings.txt

 




Classifier


We write a classifier script to check if each text row satisfies the classifier conditions. (classifier.py)



Condition Example Images


An example YouTube post which contains a warning label that meets Condition 1



Warning Label from TikTok that fulfills Condition 1 and 2



Example of Warning Label from YouTube that fulfills Condition 1 and 2 (brand names blurred for anonymity)





Conditions




We write a script that checks if the text row belongs to condition 1 or condition 2. (parser.py)

We write a script that counts how many is condition 1 and how many is condition 2. (count.py)

We write a script that checks how many of the conditions overlap. (overlap.py)




Video Length




We write a script that calculates the length of all videos. (All_video_length.py)

We write a script that normalizes (format: minutes and seconds) the durations of the videos. (handle.py)

We write a script that calculates the average of all video lengths. (average.py)

We write a script that claculates the standard deviation of the video lengths. (std.py)