Methodology
The effectiveness of a deep fake detection system largely depends on the
quality and diversity of the dataset. Collecting a comprehensive dataset that
includes a wide variety of real and deep fake images and videos is essential
for training a CNN that can generalize well to unseen data.
3.1 Data Collection
3.1.1 Real Data Collection
Real images and videos can be sourced from various platforms:
• Public Datasets: Leveraging existing public datasets like ImageNet,
CelebA, and YouTube-8M can provide a rich source of authentic content.
• Web Scraping: Custom scripts can be used to scrape real images
and videos from the web, ensuring compliance with legal and ethical
standards.
• Crowdsourcing: Platforms like Amazon Mechanical Turk can be used
to collect real images and videos from diverse contributors.
3.1.2 Deep Fake Data Collection
Generating and collecting deep fake data involves:
• Public Deep Fake Datasets: Utilizing existing datasets like FaceForensics++, DeepFakeDetection, and DFDC (Deep Fake Detection Challenge)
which contain a variety of manipulated content.
• Custom Generation: Using deep fake generation tools like DeepFaceLab, FaceSwap, and FSGAN to create custom deep fake videos. This
ensures the inclusion of the latest manipulation techniques.
10
3.1.3 Dataset Composition
A balanced dataset should include:
• A mix of high and low-quality images and videos.
• A diverse set of subjects to prevent bias.
• Various types of deep fake manipulations (e.g., face swaps, lip-syncing).
3.2 Data Preprocessing
Once the data is collected, preprocessing is required to prepare it for training
the CNN model. This includes cleaning, augmenting, and transforming the
data.
3.2.1 Data Cleaning
Cleaning involves:
• Removing Duplicates: Ensuring no repeated images or videos to avoid
skewing the training process.
• Quality Check: Filtering out corrupted or low-quality files that might
hinder the training process.
3.2.2 Data Annotation
Labeling the dataset accurately is crucial:
• Manual Annotation: Using human annotators to label real and fake
content.
• Automated Tools: Leveraging tools like Labelbox or custom scripts to
streamline the annotation process.
Department of Computer Science and Engineering 11
3.2.3 Data Augmentation
Augmentation techniques are employed to increase the diversity of the
dataset and improve the model’s robustness:
• Geometric Transformations: Including rotations, translations, scaling,
and flips.
• Color Adjustments: Varying brightness, contrast, saturation, and hue.
• Noise Injection: Adding random noise to make the model robust to
noisy inputs.
• Synthetic Data: Generating synthetic data using Generative Adversarial
Networks (GANs) to simulate rare scenarios.
3.2.4 Data Normalization
Normalizing the data helps in standardizing the input, making the training
process more stable and faster:
• Pixel Scaling: Rescaling pixel values to a range of 0 to 1 or -1 to 1.
• Mean Subtraction: Subtracting the dataset mean from each image to
center the data.
3.2.5 Data Splitting
Splitting the dataset into training, validation, and test sets ensures the
model’s performance is evaluated correctly:
• Training Set: Typically 70-80% of the data used for training the model.
• Validation Set: Around 10-15% of the data used for hyperparameter
tuning and validation during training.
• Test Set: Remaining 10-15% of the data used for final evaluation of
the model’s performance.
Department of Computer Science and Engineering 12
3.3 Data Pipeline Implementation
A robust data pipeline ensures seamless data handling from collection to
model training:
• Data Loading: Efficient data loaders using libraries like TensorFlow
Data API or PyTorch DataLoader.
• Data Transformation: Applying preprocessing steps on-the-fly during
training to save memory and improve performance.
• Batch Processing: Processing data in batches to optimize GPU utilization and speed up training.
3.4 Challenges and Considerations
3.4.1 Data Imbalance
Addressing data imbalance is crucial to prevent the model from being biased
towards the majority class. Techniques like oversampling, undersampling, and
synthetic minority over-sampling technique (SMOTE) can be employed.
3.4.2 Privacy and Ethics
Ensuring the collection and use of data comply with privacy laws and
ethical standards:
• Consent: Obtaining consent from individuals whose data is being collected.
• Anonymization: Anonymizing sensitive information to protect privacy.
3.4.3 Computational Resources
Processing and training on large datasets require significant computational
resources. Leveraging cloud services or high-performance computing clusters
can mitigate this issue.
Department of Computer Science


write all sections as the above with size of section texts increased by 3-4 sentences a relative to above text for the project whose code is "from tkinter import messagebox
from tkinter import *
from tkinter import simpledialog
import tkinter
from tkinter import filedialog
from tkinter.filedialog import askopenfilename
import cv2 as cv
import numpy as np

import os
from keras.preprocessing.image import load_img
from keras.utils import to_categorical
from keras.preprocessing.image import img_to_array
# from sklearn.model_selection import train_test_split
import pickle
from keras.models import load_model
from keras.applications import VGG16
from keras.layers import Flatten
from keras.layers import Dropout
from keras.layers import Dense
from keras.layers import Input
from keras.models import Model
from keras.optimizers import Adam
from keras.models import model_from_json


main = tkinter.Tk()
main.title("SSLA Based Traffic Sign and Lane Detection for Autonomous cars")
main.geometry("1300x1200")

global filename
global model

old = None
class_labels = ['Speed limit (20km/h)','Speed limit (30km/h)','Speed limit (50km/h)','Speed limit (60km/h)','Speed limit (70km/h)',
                'Speed limit (80km/h)','End of speed limit (80km/h)','Speed limit (100km/h)','Speed limit (120km/h)','No passing','Stop','No Entry',
                'General caution','Traffic signals']

def cannyDetection(img):
    grayImg = cv.cvtColor(img, cv.COLOR_RGB2GRAY)
    blurImg = cv.GaussianBlur(grayImg, (5, 5), 0)
    cannyImg = cv.Canny(blurImg, 50, 150)
    return cannyImg

def segmentDetection(img):
    height = img.shape[0]
    polygons = np.array([[(0, height), (800, height), (380, 290)]])
    maskImg = np.zeros_like(img)
    cv.fillPoly(maskImg, polygons, 255)
    segmentImg = cv.bitwise_and(img, maskImg)
    return segmentImg

def calculateLines(frame, lines):
    left = []
    right = []
    for line in lines:
        x1, y1, x2, y2 = line.reshape(4)
        parameters = np.polyfit((x1, x2), (y1, y2), 1)
        slope = parameters[0]
        y_intercept = parameters[1]
        if slope < 0:
            left.append((slope, y_intercept))
        else:
            right.append((slope, y_intercept))
    left_avg = np.average(left, axis = 0)
    right_avg = np.average(right, axis = 0)
    left_line = calculateCoordinates(frame, left_avg)
    right_line = calculateCoordinates(frame, right_avg)
    return np.array([left_line, right_line])

def calculateCoordinates(frame, parameters):
    global old
    #print(str(parameters)+" "+str(type(parameters))+" "+str(np.isnan(parameters)))
    if old is None:
        old = parameters        
    if np.isnan(parameters.any()) == False:
        parameters = old
    slope, intercept = parameters
    y1 = frame.shape[0]
    y2 = int(y1 - 150)
    x1 = int((y1 - intercept) / slope)
    x2 = int((y2 - intercept) / slope)
    return np.array([x1, y1, x2, y2])

def visualizeLines(frame, lines):
    lines_visualize = np.zeros_like(frame)
    if lines is not None:
        for x1, y1, x2, y2 in lines:
            cv.line(lines_visualize, (x1, y1), (x2, y2), (0, 255, 0), 5)
    return lines_visualize


def loadModel():
    global model
    model = load_model('model/model.h5')
    pathlabel.config(text="Machine Learning Traffic Sign Detection Model Loaded")
    text.delete('1.0', END)
    text.insert(END,"Machine Learning Traffic Sign Detection Model Loaded\n\n");

def detectSignal():
    global model 
    filename = filedialog.askopenfilename(initialdir="Videos")
    pathlabel.config(text=filename)
    text.delete('1.0', END)
    text.insert(END,filename+" loaded\n\n")
    text.update_idletasks()
    cap = cv.VideoCapture(filename)
    while (cap.isOpened()):
        ret, frame = cap.read()
        if frame is not None:
            canny = cannyDetection(frame)
            cv.imshow("cannyImage", canny)
            segment = segmentDetection(canny)
            hough = cv.HoughLinesP(segment, 2, np.pi / 180, 100, np.array([]), minLineLength = 100, maxLineGap = 50)
            if hough is not None:
                lines = calculateLines(frame, hough)
                linesVisualize = visualizeLines(frame, lines)
                cv.imshow("hough", linesVisualize)
                output = cv.addWeighted(frame, 0.9, linesVisualize, 1, 1)
            cv.imwrite("test.jpg",output)
            temps = cv.imread("test.jpg")
            h, w, c = temps.shape
            image = load_img("test.jpg", target_size=(80, 80))
            image = img_to_array(image) / 255.0
            image = np.expand_dims(image, axis=0)
            (boxPreds, labelPreds) = model.predict(image)
            print(boxPreds)
            boxPreds = boxPreds[0]
            startX = int(boxPreds[0] * w)
            startY = int(boxPreds[1] * h)
            endX = int(boxPreds[2] * w)
            endY = int(boxPreds[3] * h)
            predict= np.argmax(labelPreds, axis=1)
            predict = predict[0]
            accuracy = np.amax(labelPreds, axis=1)
            print(str(class_labels[predict])+" "+str(accuracy))
            if accuracy > 0.97:
                cv.putText(output, "Recognized As "+str(class_labels[predict]), (startX, startY), cv.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0), 2)
                cv.rectangle(output, (startX, startY), (endX, endY), (0, 255, 0), 2)
            cv.imshow("output", output)
            if cv.waitKey(10) & 0xFF == ord('q'):
                break
    cap.release()
    cv.destroyAllWindows()

    

def close():
    main.destroy()
    
    
font = ('times', 16, 'bold')
title = Label(main, text='SSLA Based Traffic Sign and Lane Detection for Autonomous cars',anchor=W, justify=CENTER)
title.config(bg='yellow4', fg='white')  
title.config(font=font)           
title.config(height=3, width=120)       
title.place(x=0,y=5)


font1 = ('times', 14, 'bold')
upload = Button(main, text="Generate & Load Machine Learning Model", command=loadModel)
upload.place(x=50,y=100)
upload.config(font=font1)  

pathlabel = Label(main)
pathlabel.config(bg='yellow4', fg='white')  
pathlabel.config(font=font1)           
pathlabel.place(x=50,y=150)

markovButton = Button(main, text="Upload Video & Detect Hough Lane, Signal", command=detectSignal)
markovButton.place(x=50,y=200)
markovButton.config(font=font1)

predictButton = Button(main, text="Exit", command=close)
predictButton.place(x=50,y=250)
predictButton.config(font=font1)


font1 = ('times', 12, 'bold')
text=Text(main,height=15,width=78)
scroll=Scrollbar(text)
text.configure(yscrollcommand=scroll.set)
text.place(x=450,y=100)
text.config(font=font1)


main.config(bg='magenta3')
main.mainloop()"