Adversarial Machine Learning Mitigation: Adversarial Learning | by Ferhat Ozgur Catak | Towards Data Science

There are several attacks against deep learning models in the literature, including fast-gradient sign method (FGSM), basic iterative method (BIM) or momentum iterative method (MIM) attacks. These attacks are the purest form of the gradient-based evading technique that is used by attackers to evade the classification model.
If you find those results useful please cite this paper :
@PROCEEDINGS{catak-adv-ml-2020,    title = {Deep Neural Network based Malicious Network Activity Detection Under Adversarial Machine Learning Attacks},    booktitle = {Proc.\ 3rd International Conference on Intelligent Technologies and Applications (INTAP 2020)},    volume = 5805,    series = {LNCS},author  = {Ferhat Ozgur Catak},    publisher = {Springer},    year = {2020}    }
In this work, I will present a new approach to protect a malicious activity detection model from the several adversarial machine learning attacks. Hence, we explore the power of applying adversarial training to build a robust model against FGSM attacks. Accordingly, (1) dataset enhanced with the adversarial examples; (2) deep neural network-based detection model is trained using the KDDCUP99 dataset to learn the FGSM based attack patterns. We applied this training model to the benchmark cybersecurity dataset.
The adversarial machine learning has been used to describe the attacks to machine learning models, which tries to mislead models by malicious input instances. The figure shows the typical adversarial machine learning attack.
A typical machine learning model basically consists of two stages as training time and decision time. Thus, the adversarial machine learning attacks occur in either training time or decision time. The techniques used by hackers for adversarial machine learning can be divided into two, according to the time of the attack:
Data Poisoning: The attacker changes some labels of training input instances to mislead the output model.
Model Poisoning: The hacker drives model to produce false labelling using some perturbated instance after the model is created.
Our model is able to respond to the model attacks by hackers who use the adversarial machine learning methods. The figure illustrates the system architecture used to protect the model and to classify correctly.
We import the usual standard libraries plus one cleverhans library to make an adversarial attack to the deep learning model.
from sklearn.datasets import fetch_kddcup99from sklearn.model_selection import train_test_splitfrom sklearn.metrics import confusion_matrixfrom sklearn import preprocessingimport tensorflow as tfimport pandas as pdimport numpy as npfrom keras.utils import np_utilsfrom cleverhans.future.tf2.attacks import fast_gradient_method, \    basic_iterative_method, momentum_iterative_methodnp.random.seed(10)
In this work, we will use standard KDDCUP’99 intrusion detection dataset to show the results. We need to extract the numerical features from the dataset. I created a new method to load and extract the KDDCUP’99 dataset.
COL_NAME = ['duration', 'protocol_type', 'service', 'flag', 'src_bytes',            'dst_bytes', 'land', 'wrong_fragment', 'urgent', 'hot',            'num_failed_logins', 'logged_in', 'num_compromised', 'root_shell',            'su_attempted', 'num_root', 'num_file_creations', 'num_shells',            'num_access_files', 'num_outbound_cmds', 'is_host_login',            'is_guest_login', 'count', 'srv_count', 'serror_rate',            'srv_serror_rate', 'rerror_rate', 'srv_rerror_rate',            'same_srv_rate', 'diff_srv_rate', 'srv_diff_host_rate',            'dst_host_count', 'dst_host_srv_count', 'dst_host_same_srv_rate',            'dst_host_diff_srv_rate', 'dst_host_same_src_port_rate',            'dst_host_srv_diff_host_rate', 'dst_host_serror_rate',            'dst_host_srv_serror_rate', 'dst_host_rerror_rate', 'dst_host_srv_rerror_rate']NUMERIC_COLS = ['duration', 'src_bytes', 'dst_bytes', 'wrong_fragment',                'urgent', 'hot', 'num_failed_logins', 'num_compromised',                'root_shell', 'su_attempted', 'num_root', 'num_file_creations',                'num_shells', 'num_access_files', 'num_outbound_cmds', 'count',                'srv_count', 'serror_rate', 'srv_serror_rate', 'rerror_rate',                'srv_rerror_rate', 'same_srv_rate', 'diff_srv_rate',                'srv_diff_host_rate', 'dst_host_count', 'dst_host_srv_count',                'dst_host_same_srv_rate', 'dst_host_diff_srv_rate',                'dst_host_same_src_port_rate', 'dst_host_srv_diff_host_rate',                'dst_host_serror_rate', 'dst_host_srv_serror_rate',                'dst_host_rerror_rate', 'dst_host_srv_rerror_rate']def get_ds():    """ get_ds: Get the numeric values of the KDDCUP'99 dataset. """    x_kddcup, y_kddcup = fetch_kddcup99(return_X_y=True, shuffle=False)    df_kddcup = pd.DataFrame(x_kddcup, columns=COL_NAME)    df_kddcup['label'] = y_kddcup    df_kddcup.drop_duplicates(keep='first', inplace=True)    df_kddcup['label'] = df_kddcup['label'].apply(lambda d: \                                    str(d).replace('.', '').replace("b'", "").\                                        replace("'", ""))    conversion_dict = {'back':'dos', 'buffer_overflow':'u2r', 'ftp_write':'r2l',                       'guess_passwd':'r2l', 'imap':'r2l', 'ipsweep':'probe',                       'land':'dos', 'loadmodule':'u2r', 'multihop':'r2l',                       'neptune':'dos', 'nmap':'probe', 'perl':'u2r', 'phf':'r2l',                       'pod':'dos', 'portsweep':'probe', 'rootkit':'u2r',                       'satan':'probe', 'smurf':'dos', 'spy':'r2l', 'teardrop':'dos',                       'warezclient':'r2l', 'warezmaster':'r2l'}    df_kddcup['label'] = df_kddcup['label'].replace(conversion_dict)    df_kddcup = df_kddcup.query("label != 'u2r'")    df_y = pd.DataFrame(df_kddcup.label, columns=["label"], dtype="category")    df_kddcup.drop(["label"], inplace=True, axis=1)    x_kddcup = df_kddcup[NUMERIC_COLS].values    x_kddcup = preprocessing.scale(x_kddcup)    y_kddcup = df_y.label.cat.codes.to_numpy()    return x_kddcup, y_kddcup
The tensorflow based classification model is then given for example as exercise here:
def create_tf_model(input_size, num_of_class):    """ This method creates the tensorflow classification model """    model_kddcup = tf.keras.Sequential([        tf.keras.layers.Dense(200, input_dim=input_size, activation=tf.nn.relu),        tf.keras.layers.Dense(500, activation=tf.nn.relu),        tf.keras.layers.Dense(200, activation=tf.nn.relu),        tf.keras.layers.Dense(num_of_class),        # We seperate the activation layer to be able to access        # the logits of the previous layer later        tf.keras.layers.Activation(tf.nn.softmax)        ])    model_kddcup.compile(loss='categorical_crossentropy',                         optimizer='adam',                         metrics=['accuracy'])    return model_kddcup
The next step is to create adversarial machine learning attacks using CleverHans library. I used fast-gradient sign method (FGSM), basic iterative method (BIM) or momentum iterative method (MIM) attacks for the Tensorflow library. I created 3 methods for each attack.
def gen_tf2_fgsm_attack(org_model, x_test):    """ This method creates adversarial examples with fgsm """    logits_model = tf.keras.Model(org_model.input, model.layers[-1].output)    epsilon = 0.1    adv_fgsm_x = fast_gradient_method(logits_model,                                      x_test,                                      epsilon,                                      np.inf,                                      targeted=False)    return adv_fgsm_xdef gen_tf2_bim(org_model, x_test):    """ This method creates adversarial examples with bim """    logits_model = tf.keras.Model(org_model.input, model.layers[-1].output)    epsilon = 0.1    adv_bim_x = basic_iterative_method(logits_model,                                       x_test,                                       epsilon,                                       0.1,                                       nb_iter=10,                                       norm=np.inf,                                       targeted=True)    return adv_bim_xdef gen_tf2_mim(org_model, x_test):    """ This method creates adversarial examples with mim """    logits_model = tf.keras.Model(org_model.input, model.layers[-1].output)    epsilon = 0.1    adv_mim_x = momentum_iterative_method(logits_model,                                          x_test,                                          epsilon,                                          0.1,                                          nb_iter=100,                                          norm=np.inf,                                          targeted=True)    return adv_mim_x
Let’s continue with the training of the attack detection model with the normal (non-manipulated) KDDCUP’99 dataset
EPOCH = 50TEST_RATE = 0.2VALIDATION_RATE = 0.2X, y = get_ds()num_class = len(np.unique(y))attack_functions = [gen_tf2_bim,                    gen_tf2_fgsm_attack,                    gen_tf2_mim]model = create_tf_model(X.shape[1], num_class)X_train, X_test, y_train, y_test = train_test_split(X, y, \                                                    test_size=TEST_RATE)y_train_cat = np_utils.to_categorical(y_train)y_test_cat = np_utils.to_categorical(y_test)history = model.fit(X_train, y_train_cat, epochs=EPOCH,                    batch_size=50000, verbose=0,                    validation_split=VALIDATION_RATE)y_pred = model.predict_classes(X_test)cm_org = confusion_matrix(y_test, y_pred)print("*"*50)print("Original confusion matrix")print(cm_org)C:\Users\ferhatoc\AppData\Roaming\Python\Python37\site-packages\pandas\core\frame.py:3997: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrameSee the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy  errors=errors,WARNING:tensorflow:AutoGraph could not transform <function Model.make_train_function.<locals>.train_function at 0x0000025288139948> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_train_function.<locals>.train_function at 0x0000025288139948> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING:tensorflow:AutoGraph could not transform <function Model.make_test_function.<locals>.test_function at 0x0000025287FF4318> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_test_function.<locals>.test_function at 0x0000025287FF4318> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING:tensorflow:From <ipython-input-7-6c756bab0648>:24: Sequential.predict_classes (from tensorflow.python.keras.engine.sequential) is deprecated and will be removed after 2021-01-01.Instructions for updating:Please use instead:* `np.argmax(model.predict(x), axis=-1)`,   if your model does multi-class classification   (e.g. if it uses a `softmax` last-layer activation).* `(model.predict(x) > 0.5).astype("int32")`,   if your model does binary classification   (e.g. if it uses a `sigmoid` last-layer activation).WARNING:tensorflow:AutoGraph could not transform <function Model.make_predict_function.<locals>.predict_function at 0x0000025283D0E798> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_predict_function.<locals>.predict_function at 0x0000025283D0E798> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert**************************************************Original confusion matrix[[10873    16     0     0] [   16 17528     6    18] [    7    20   403    11] [    3    23     4   179]]
The original model’s confusion matrix is shown here. According to the confusion matrix, the model’s classification performance quite good.
Let’s continue with the attacks, attacked model’s confusion matrices and adversarial trained model’s confusion matrices
for attack_function in attack_functions:        print("*"*20)        print("Attack function is ", attack_function)                model = create_tf_model(X.shape[1], num_class)        history = model.fit(X_train, y_train_cat, epochs=EPOCH,                            batch_size=50000, verbose=0,                            validation_split=VALIDATION_RATE)                X_adv_list = []        y_adv_list = []        adv_x = attack_function(model, X_test)        y_pred = model.predict_classes(adv_x)        cm_adv = confusion_matrix(y_test, y_pred)        print("*"*20)        print("Attacked confusion matrix")        print(cm_adv)        print("Adversarial training")        # define the checkpoint        adv_x = attack_function(model, X_train)        adv_x_test = attack_function(model, X_test)        concat_adv_x = np.concatenate([X_train, adv_x])        concat_y_train = np.concatenate([y_train_cat, y_train_cat])        history = model.fit(concat_adv_x, concat_y_train, epochs=EPOCH,                            batch_size=50000, verbose=0,                            validation_data=(adv_x_test, y_test_cat))        y_pred = model.predict_classes(adv_x_test)        cm_adv = confusion_matrix(y_test, y_pred)        print("*"*20)        print("Attacked confusion matrix - adv training")        print(cm_adv)********************Attack function is  <function gen_tf2_bim at 0x00000252FCF84A68>WARNING:tensorflow:AutoGraph could not transform <function Model.make_train_function.<locals>.train_function at 0x00000252FD05FDC8> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_train_function.<locals>.train_function at 0x00000252FD05FDC8> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING:tensorflow:AutoGraph could not transform <function Model.make_test_function.<locals>.test_function at 0x00000252877B80D8> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_test_function.<locals>.test_function at 0x00000252877B80D8> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING:tensorflow:AutoGraph could not transform <function Model.make_predict_function.<locals>.predict_function at 0x0000025281E8B168> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_predict_function.<locals>.predict_function at 0x0000025281E8B168> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert********************Attacked confusion matrix[[10874    15     0     0] [   14 17532     7    15] [    6    19   404    12] [    3    23     5   178]]Adversarial training********************Attacked confusion matrix - adv training[[10877    12     0     0] [   12 17535     6    15] [    1    13   425     2] [    0    22     3   184]]********************Attack function is  <function gen_tf2_fgsm_attack at 0x00000252FCF84B88>WARNING:tensorflow:AutoGraph could not transform <function Model.make_train_function.<locals>.train_function at 0x0000025281E8B438> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_train_function.<locals>.train_function at 0x0000025281E8B438> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING:tensorflow:AutoGraph could not transform <function Model.make_test_function.<locals>.test_function at 0x0000025288BB38B8> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_test_function.<locals>.test_function at 0x0000025288BB38B8> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING:tensorflow:AutoGraph could not transform <function Model.make_predict_function.<locals>.predict_function at 0x0000025287EF2558> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_predict_function.<locals>.predict_function at 0x0000025287EF2558> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert********************Attacked confusion matrix[[10702   180     6     1] [   79 17353    31   105] [    9    47   376     9] [    3    88     8   110]]Adversarial training********************Attacked confusion matrix - adv training[[10877    11     0     1] [    9 17543     4    12] [    1    15   422     3] [    2    25     2   180]]********************Attack function is  <function gen_tf2_mim at 0x00000252FCF84D38>WARNING:tensorflow:AutoGraph could not transform <function Model.make_train_function.<locals>.train_function at 0x00000252875459D8> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_train_function.<locals>.train_function at 0x00000252875459D8> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING:tensorflow:AutoGraph could not transform <function Model.make_test_function.<locals>.test_function at 0x00000252F9990048> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_test_function.<locals>.test_function at 0x00000252F9990048> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING:tensorflow:5 out of the last 14 calls to <function compute_gradient at 0x00000252FCF6A318> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings is likely due to passing python objects instead of tensors. Also, tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. Please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for more details.WARNING:tensorflow:AutoGraph could not transform <function Model.make_predict_function.<locals>.predict_function at 0x0000025280355A68> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convertWARNING: AutoGraph could not transform <function Model.make_predict_function.<locals>.predict_function at 0x0000025280355A68> and will run it as-is.Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.Cause: Bad argument number for Name: 4, expecting 3To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert********************Attacked confusion matrix[[10874    15     0     0] [   16 17530     5    17] [    6    24   400    11] [    3    23     5   178]]Adversarial training********************Attacked confusion matrix - adv training[[10878    11     0     0] [   12 17537     4    15] [    1    16   420     4] [    0    21     2   186]]
Attacked model’s confusion matrix
Adversarial trained model’s confusion matrix
Fast Gradient Sign Method (FGSM)
Attacked model’s confusion matrix
Adversarial trained model’s confusion matrix
Momentum Iterative Method (MIM)
Attacked model’s confusion matrix
Adversarial trained model’s confusion matrix