Pre-processing on NSL_KDD dataset

username: arezoo_moradi
cooked: <p>I want to load the NSL_KDD dataset contained in this link with using the Python programming.<br>
</p><aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/smellslikeml/deepIDS/blob/master/deep_IDS.ipynb" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/smellslikeml/deepIDS/blob/master/deep_IDS.ipynb" target="_blank" rel="nofollow noopener">smellslikeml/deepIDS/blob/master/deep_IDS.ipynb</a></h4>
<pre><code class="lang-ipynb">{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Network Intrusion Detection with Deep Learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# For a broad introduction to the problem and dataset: https://arxiv.org/pdf/1701.02145.pdf\n",
    "# For modern results using deep learning: http://ieeexplore.ieee.org/document/7777224/"
   ]
</code></pre>

  This file has been truncated. <a href="https://github.com/smellslikeml/deepIDS/blob/master/deep_IDS.ipynb" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
In this database, 22 features for training and testing data are classified into 5 separate classes(Normal, DOS, U2R, R2L, Probe)<br>
But when I run this line of code y_test = pd.get_dummies(y_test), instead of being categorized into 5 classes, it shows me the same 22 features, while I did the same thing train data (target = pd.get_dummies(target) and crrocet result), it using for the test data.<br>
with open(‘G:/RUN_PYTHON/kddcup.names.txt’, ‘r’) as infile:<br>
kdd_names = infile.readlines()<br>
kdd_cols = [x.split(’:’)[0] for x in kdd_names[1:]]
<h1>The Train+/Test+ datasets include sample difficulty rating and the attack class</h1>
<p>kdd_cols += [‘class’, ‘difficulty’]</p>
<p>kdd = pd.read_csv(‘G:/RUN_PYTHON/KDDTrain+.txt’, names=kdd_cols)<br>
kdd_t = pd.read_csv(‘G:/RUN_PYTHON/KDDTest+.txt’, names=kdd_cols)<br>
<span class="hashtag">#kdd</span> = pd.read_csv(‘G:/RUN_PYTHON/kddcup.txt.data_10_percent_corrected’, names=kdd_cols)<br>
<span class="hashtag">#kdd_t</span> = pd.read_csv(‘G:/RUN_PYTHON/kddcup.testdata.unlabeled_10_percent’, names=kdd_cols)</p>
<h1>Consult the linked references for attack categories:</h1>
<h1><a href="https://www.researchgate.net/post/What_are_the_attack_types_in_the_NSL-KDD_TEST_set_For_example_processtable_is_a_attack_type_in_test_set_Im_wondering_is_it_prob_DoS_R2L_U2R" rel="nofollow noopener">https://www.researchgate.net/post/What_are_the_attack_types_in_the_NSL-KDD_TEST_set_For_example_processtable_is_a_attack_type_in_test_set_Im_wondering_is_it_prob_DoS_R2L_U2R</a></h1>
<h1>The traffic can be grouped into 5 categories: Normal, DOS, U2R, R2L, Probe</h1>
<h1>or more coarsely into Normal vs Anomalous for the binary classification task</h1>
<p>kdd_cols = [kdd.columns[0]] + sorted(list(set(kdd.protocol_type.values))) + sorted(list(set(kdd.service.values))) + sorted(list(set(kdd.flag.values))) + kdd.columns[4:].tolist()<br>
attack_map = [x.strip().split() for x in open(‘G:/RUN_PYTHON/training_attack_types.txt’, ‘r’)]<br>
attack_map = {x[0]: x[1] for x in attack_map if x}</p>
<h1>Here we opt for the 5-class problem</h1>
<p>kdd[‘class’] = kdd[‘class’].replace(attack_map)<br>
kdd_t[‘class’] = kdd_t[‘class’].replace(attack_map)</p>
<p>def cat_encode(df, col):<br>
return pd.concat([df.drop(col, axis=1), pd.get_dummies(df[col].values)], axis=1)</p>
<p>def log_trns(df, col):<br>
return df[col].apply(np.log1p)</p>
<p>cat_lst = [‘protocol_type’, ‘service’, ‘flag’]<br>
for col in cat_lst:<br>
kdd = cat_encode(kdd, col)<br>
kdd_t = cat_encode(kdd_t, col)</p>
<p>log_lst = [‘duration’, ‘src_bytes’, ‘dst_bytes’]<br>
for col in log_lst:<br>
kdd[col] = log_trns(kdd, col)<br>
kdd_t[col] = log_trns(kdd_t, col)</p>
<p>kdd = kdd[kdd_cols]<br>
for col in kdd_cols:<br>
if col not in kdd_t.columns:<br>
kdd_t[col] = 0<br>
kdd_t = kdd_t[kdd_cols]</p>
<h1>Now we have used one-hot encoding and log scaling</h1>
<p>difficulty = kdd.pop(‘difficulty’)<br>
target = kdd.pop(‘class’)<br>
y_diff = kdd_t.pop(‘difficulty’)<br>
y_test = kdd_t.pop(‘class’)</p>
<p>target = pd.get_dummies(target)<br>
print(target)<br>
y_test = pd.get_dummies(y_test)<br>
print(y_test)</p>
<p>the output of  target:<br>
Out[27]:<br>
dos  normal  probe  r2l  u2r<br>
0         0       1      0    0    0<br>
1         0       1      0    0    0<br>
2         1       0      0    0    0<br>
3         0       1      0    0    0<br>
4         0       1      0    0    0<br>
5         1       0      0    0    0<br>
6         1       0      0    0    0<br>
7         1       0      0    0    0<br>
8         1       0      0    0    0<br>
9         1       0      0    0    0<br>
10        1       0      0    0    0<br>
11        1       0      0    0    0<br>
12        0       1      0    0    0<br>
13        0       0      0    1    0<br>
14        1       0      0    0    0<br>
15        1       0      0    0    0<br>
16        0       1      0    0    0<br>
17        0       0      1    0    0<br>
18        0       1      0    0    0<br>
19        0       1      0    0    0<br>
20        1       0      0    0    0<br>
21        1       0      0    0    0<br>
22        0       1      0    0    0<br>
23        0       1      0    0    0<br>
24        1       0      0    0    0<br>
25        0       1      0    0    0<br>
26        1       0      0    0    0<br>
27        0       1      0    0    0<br>
28        0       1      0    0    0<br>
29        0       1      0    0    0<br>
…     …    …  …  …<br>
125943    0       1      0    0    0<br>
125944    0       1      0    0    0<br>
125945    0       1      0    0    0<br>
125946    1       0      0    0    0<br>
125947    0       0      1    0    0<br>
125948    1       0      0    0    0<br>
125949    0       1      0    0    0<br>
125950    1       0      0    0    0<br>
125951    0       1      0    0    0<br>
125952    0       1      0    0    0<br>
125953    1       0      0    0    0<br>
125954    0       1      0    0    0<br>
125955    0       1      0    0    0<br>
125956    0       1      0    0    0<br>
125957    0       1      0    0    0<br>
125958    1       0      0    0    0<br>
125959    0       1      0    0    0<br>
125960    0       1      0    0    0<br>
125961    0       1      0    0    0<br>
125962    0       1      0    0    0<br>
125963    0       1      0    0    0<br>
125964    1       0      0    0    0<br>
125965    0       1      0    0    0<br>
125966    1       0      0    0    0<br>
125967    0       1      0    0    0<br>
125968    1       0      0    0    0<br>
125969    0       1      0    0    0<br>
125970    0       1      0    0    0<br>
125971    1       0      0    0    0<br>
125972    0       1      0    0    0</p>
<p>[125973 rows x 5 columns]</p>
<p>the output of y_test:<br>
print(y_test)<br>
apache2  dos  httptunnel  mailbomb  mscan  named  normal  probe  <br>
0            0    1           0         0      0      0       0      0<br>
1            0    1           0         0      0      0       0      0<br>
2            0    0           0         0      0      0       1      0<br>
3            0    0           0         0      0      0       0      0<br>
4            0    0           0         0      1      0       0      0<br>
5            0    0           0         0      0      0       1      0<br>
6            0    0           0         0      0      0       1      0<br>
7            0    0           0         0      0      0       0      0<br>
8            0    0           0         0      0      0       1      0<br>
9            0    0           0         0      0      0       0      0<br>
10           0    0           0         0      1      0       0      0<br>
11           0    0           0         0      0      0       1      0<br>
12           0    1           0         0      0      0       0      0<br>
13           0    1           0         0      0      0       0      0<br>
14           0    0           0         0      0      0       1      0<br>
15           0    0           0         0      0      0       1      0<br>
16           0    0           0         0      0      0       1      0<br>
17           0    0           0         0      0      0       1      0<br>
18           0    0           0         0      0      0       1      0<br>
19           0    1           0         0      0      0       0      0<br>
20           0    1           0         0      0      0       0      0<br>
21           0    0           0         0      1      0       0      0<br>
22           0    0           0         0      0      0       1      0<br>
23           0    0           0         0      0      0       1      0<br>
24           0    1           0         0      0      0       0      0<br>
25           0    1           0         0      0      0       0      0<br>
26           0    0           0         0      0      0       1      0<br>
27           0    0           0         0      0      0       1      0<br>
28           0    1           0         0      0      0       0      0<br>
29           0    0           0         0      0      0       1      0<br>
…  …         …       …    …    …     …    …<br>
22514        0    0           0         0      0      0       1      0<br>
22515        1    0           0         0      0      0       0      0<br>
22516        0    0           0         0      0      0       1      0<br>
22517        0    0           0         0      0      0       0      0<br>
22518        0    0           0         0      0      0       1      0<br>
22519        0    0           0         0      0      0       0      0<br>
22520        0    0           0         0      0      0       0      1<br>
22521        0    0           0         0      0      0       0      1<br>
22522        0    1           0         0      0      0       0      0<br>
22523        0    0           0         0      0      0       1      0<br>
22524        0    0           0         0      0      0       0      0<br>
22525        1    0           0         0      0      0       0      0<br>
22526        0    0           0         0      0      0       1      0<br>
22527        0    0           0         0      0      0       1      0<br>
22528        0    1           0         0      0      0       0      0<br>
22529        0    0           0         0      0      0       1      0<br>
22530        0    1           0         0      0      0       0      0<br>
22531        0    1           0         0      0      0       0      0<br>
22532        0    0           0         0      0      0       1      0<br>
22533        0    0           0         0      0      0       1      0<br>
22534        0    1           0         0      0      0       0      0<br>
22535        0    0           0         0      0      0       1      0<br>
22536        0    1           0         0      0      0       0      0<br>
22537        0    0           0         1      0      0       0      0<br>
22538        0    1           0         0      0      0       0      0<br>
22539        0    0           0         0      0      0       1      0<br>
22540        0    0           0         0      0      0       1      0<br>
22541        0    1           0         0      0      0       0      0<br>
22542        0    0           0         0      0      0       1      0<br>
22543        0    0           0         0      1      0       0      0</p>
<pre><code>   processtable  ps  ...    sendmail  snmpgetattack  snmpguess  sqlattack  \
</code></pre>
<p>0                 0   0  …           0              0          0          0<br>
1                 0   0  …           0              0          0          0<br>
2                 0   0  …           0              0          0          0<br>
3                 0   0  …           0              0          0          0<br>
4                 0   0  …           0              0          0          0<br>
5                 0   0  …           0              0          0          0<br>
6                 0   0  …           0              0          0          0<br>
7                 0   0  …           0              0          0          0<br>
8                 0   0  …           0              0          0          0<br>
9                 0   0  …           0              0          0          0<br>
10                0   0  …           0              0          0          0<br>
11                0   0  …           0              0          0          0<br>
12                0   0  …           0              0          0          0<br>
13                0   0  …           0              0          0          0<br>
14                0   0  …           0              0          0          0<br>
15                0   0  …           0              0          0          0<br>
16                0   0  …           0              0          0          0<br>
17                0   0  …           0              0          0          0<br>
18                0   0  …           0              0          0          0<br>
19                0   0  …           0              0          0          0<br>
20                0   0  …           0              0          0          0<br>
21                0   0  …           0              0          0          0<br>
22                0   0  …           0              0          0          0<br>
23                0   0  …           0              0          0          0<br>
24                0   0  …           0              0          0          0<br>
25                0   0  …           0              0          0          0<br>
26                0   0  …           0              0          0          0<br>
27                0   0  …           0              0          0          0<br>
28                0   0  …           0              0          0          0<br>
29                0   0  …           0              0          0          0<br>
…  …  …         …            …        …        …<br>
22514             0   0  …           0              0          0          0<br>
22515             0   0  …           0              0          0          0<br>
22516             0   0  …           0              0          0          0<br>
22517             1   0  …           0              0          0          0<br>
22518             0   0  …           0              0          0          0<br>
22519             1   0  …           0              0          0          0<br>
22520             0   0  …           0              0          0          0<br>
22521             0   0  …           0              0          0          0<br>
22522             0   0  …           0              0          0          0<br>
22523             0   0  …           0              0          0          0<br>
22524             0   0  …           0              0          0          0<br>
22525             0   0  …           0              0          0          0<br>
22526             0   0  …           0              0          0          0<br>
22527             0   0  …           0              0          0          0<br>
22528             0   0  …           0              0          0          0<br>
22529             0   0  …           0              0          0          0<br>
22530             0   0  …           0              0          0          0<br>
22531             0   0  …           0              0          0          0<br>
22532             0   0  …           0              0          0          0<br>
22533             0   0  …           0              0          0          0<br>
22534             0   0  …           0              0          0          0<br>
22535             0   0  …           0              0          0          0<br>
22536             0   0  …           0              0          0          0<br>
22537             0   0  …           0              0          0          0<br>
22538             0   0  …           0              0          0          0<br>
22539             0   0  …           0              0          0          0<br>
22540             0   0  …           0              0          0          0<br>
22541             0   0  …           0              0          0          0<br>
22542             0   0  …           0              0          0          0<br>
22543             0   0  …           0              0          0          0</p>
<pre><code>   u2r  udpstorm  worm  xlock  xsnoop  xterm  
</code></pre>
<p>0        0         0     0      0       0      0<br>
1        0         0     0      0       0      0<br>
2        0         0     0      0       0      0<br>
3        0         0     0      0       0      0<br>
4        0         0     0      0       0      0<br>
5        0         0     0      0       0      0<br>
6        0         0     0      0       0      0<br>
7        0         0     0      0       0      0<br>
8        0         0     0      0       0      0<br>
9        0         0     0      0       0      0<br>
10       0         0     0      0       0      0<br>
11       0         0     0      0       0      0<br>
12       0         0     0      0       0      0<br>
13       0         0     0      0       0      0<br>
14       0         0     0      0       0      0<br>
15       0         0     0      0       0      0<br>
16       0         0     0      0       0      0<br>
17       0         0     0      0       0      0<br>
18       0         0     0      0       0      0<br>
19       0         0     0      0       0      0<br>
20       0         0     0      0       0      0<br>
21       0         0     0      0       0      0<br>
22       0         0     0      0       0      0<br>
23       0         0     0      0       0      0<br>
24       0         0     0      0       0      0<br>
25       0         0     0      0       0      0<br>
26       0         0     0      0       0      0<br>
27       0         0     0      0       0      0<br>
28       0         0     0      0       0      0<br>
29       0         0     0      0       0      0<br>
…       …   …    …     …    …<br>
22514    0         0     0      0       0      0<br>
22515    0         0     0      0       0      0<br>
22516    0         0     0      0       0      0<br>
22517    0         0     0      0       0      0<br>
22518    0         0     0      0       0      0<br>
22519    0         0     0      0       0      0<br>
22520    0         0     0      0       0      0<br>
22521    0         0     0      0       0      0<br>
22522    0         0     0      0       0      0<br>
22523    0         0     0      0       0      0<br>
22524    1         0     0      0       0      0<br>
22525    0         0     0      0       0      0<br>
22526    0         0     0      0       0      0<br>
22527    0         0     0      0       0      0<br>
22528    0         0     0      0       0      0<br>
22529    0         0     0      0       0      0<br>
22530    0         0     0      0       0      0<br>
22531    0         0     0      0       0      0<br>
22532    0         0     0      0       0      0<br>
22533    0         0     0      0       0      0<br>
22534    0         0     0      0       0      0<br>
22535    0         0     0      0       0      0<br>
22536    0         0     0      0       0      0<br>
22537    0         0     0      0       0      0<br>
22538    0         0     0      0       0      0<br>
22539    0         0     0      0       0      0<br>
22540    0         0     0      0       0      0<br>
22541    0         0     0      0       0      0<br>
22542    0         0     0      0       0      0<br>
22543    0         0     0      0       0      0</p>
<p>[22544 rows x 22 columns]<br>
best regards</p>

username: ptrblck
cooked: <p>It seems your <code>attack_map</code> doesn’t contain all necessary classes to replace all targets from the test set.<br>
The <code>attack_map</code> contains the replacement map for:</p>
<pre><code class="lang-python">{'back': 'dos',
 'buffer_overflow': 'u2r',
 'ftp_write': 'r2l',
 'guess_passwd': 'r2l',
 'imap': 'r2l',
 'ipsweep': 'probe',
 'land': 'dos',
 'loadmodule': 'u2r',
 'multihop': 'r2l',
 'neptune': 'dos',
 'nmap': 'probe',
 'perl': 'u2r',
 'phf': 'r2l',
 'pod': 'dos',
 'portsweep': 'probe',
 'rootkit': 'u2r',
 'satan': 'probe',
 'smurf': 'dos',
 'spy': 'r2l',
 'teardrop': 'dos',
 'warezclient': 'r2l',
 'warezmaster': 'r2l'}
</code></pre>
<p>which should work for the training set, which contains <code>kdd['class'].unique()</code>:</p>
<pre><code class="lang-python"> array(['normal', 'neptune', 'warezclient', 'ipsweep', 'portsweep',
       'teardrop', 'nmap', 'satan', 'smurf', 'pod', 'back',
       'guess_passwd', 'ftp_write', 'multihop', 'rootkit',
       'buffer_overflow', 'imap', 'warezmaster', 'phf', 'land',
       'loadmodule', 'spy', 'perl'], dtype=object)
</code></pre>
<p>such that after calling <code>kdd['class'] = kdd['class'].replace(attack_map)</code> it will contain:</p>
<pre><code class="lang-python">array(['normal', 'dos', 'r2l', 'probe', 'u2r'], dtype=object)
</code></pre>
<p>However, the test set seems to contain more unique classes:</p>
<pre><code class="lang-python">array(['neptune', 'normal', 'saint', 'mscan', 'guess_passwd', 'smurf',
       'apache2', 'satan', 'buffer_overflow', 'back', 'warezmaster',
       'snmpgetattack', 'processtable', 'pod', 'httptunnel', 'nmap', 'ps',
       'snmpguess', 'ipsweep', 'mailbomb', 'portsweep', 'multihop',
       'named', 'sendmail', 'loadmodule', 'xterm', 'worm', 'teardrop',
       'rootkit', 'xlock', 'perl', 'land', 'xsnoop', 'sqlattack',
       'ftp_write', 'imap', 'udpstorm', 'phf'], dtype=object)
</code></pre>
<p>so you end up with:</p>
<pre><code class="lang-python">kdd_t['class'] = kdd_t['class'].replace(attack_map)
print(kdd_y['class'].unique())
&gt; ['dos' 'normal' 'saint' 'mscan' 'r2l' 'apache2' 'probe' 'u2r'
 'snmpgetattack' 'processtable' 'httptunnel' 'ps' 'snmpguess' 'mailbomb'
 'named' 'sendmail' 'xterm' 'worm' 'xlock' 'xsnoop' 'sqlattack' 'udpstorm']
</code></pre>
<p>PS: It’s easier to debug, if you wrap the code into three backticks ``` <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: arezoo_moradi
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>, yes you say right. Thanks for your response.</p>
<p>when I am a newbie in python programming and I want to load the data according to the table of the article but I don’t know how to can do categorical training and testing the NSL_KDD dataset into(‘normal’, ‘dos’, ‘r2l’, ‘probe’, ‘u2r’).</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/f/af4fc0d9cca11117d0af3f9d1f9af397a90daab9.png" alt="image" data-base62-sha1="p0SqxmV8PaLTpmdvUClcpxdVjGp" width="476" height="491"></p>
<p>I changed this line of code kdd_t[‘class’] = kdd_t[‘class’].replace(attack_map) to kdd_t[‘class’] = kdd[‘class’].replace(attack_map)  but I don’t know this changing is a right work for my target that shows in the above pictures.</p>

username: ptrblck
cooked: <p>Oh no, you shouldn’t use this replacement, since you are assigning the training set to the test set and will see a major data leakage in your experiments.</p>
<p>I would suggest to extend the <code>attack_map</code> table with all missing mappings.<br>
I.e. since the test set contains more unique classes, you would have to add e.g. the mapping <code>{'worm': 'U2R'}</code>. Note that I don’t know how the additional classes should be mapped so this is just an example, since I’m not familiar in this domain. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: arezoo_moradi
cooked: <p>although you say right but in the above table just used 22 features for training and testing.<br>
I don’t know how to can classification according to the article(A Deep Learning Approach to Network Intrusion Detection)<br>
could anyone help me? I desperately need help.</p>

username: ptrblck
cooked: <p>As already said, I’m not familiar with the dataset and paper, but since the test set contains more attack types, which are neither in the <code>attack_map</code>, nor in the table, you could just drop these types.</p>

username: arezoo_moradi
cooked: <p>thanks for your response but I can’t solve this problem that said in the question.<br>
(But when I run this line of code y_test = pd.get_dummies(y_test), instead of being categorized into 5 classes, it shows me the same 22 features, while I did the same thing train data (target = pd.get_dummies(target) and correct result), it using for the test data.)<br>
I replaced this line kdd_t[‘class’] = kdd[‘class’].replace(attack_map) but  you said this work is incrocet.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="arezoo_moradi" data-post="7" data-topic="80995">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/arezoo_moradi/48/23983_2.png" class="avatar"> arezoo_moradi:</div>
<blockquote>
<p>(But when I run this line of code y_test = pd.get_dummies(y_test), instead of being categorized into 5 classes, it shows me the same 22 features, while I did the same thing train data (target = pd.get_dummies(target) and correct result), it using for the test data.)</p>
</blockquote>
</aside>
<p>As explained, you test data frame contains more unique classes, which are not in the attack map and won’t be replaced.</p>
<aside class="quote no-group" data-username="arezoo_moradi" data-post="7" data-topic="80995">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/arezoo_moradi/48/23983_2.png" class="avatar"> arezoo_moradi:</div>
<blockquote>
<p>I replaced this line kdd_t[‘class’] = kdd[‘class’].replace(attack_map) but you said this work is incrocet.</p>
</blockquote>
</aside>
<p>Don’t do this, as you are reusing the training dataset and all your experiments will be invalid.</p>
<p>If you want to remove the additional classes from the test <code>DataFrame</code>, this code should work:</p>
<pre data-code-wrap="python"><code class="lang-python">kdd['class'] = kdd['class'].replace(attack_map)
unique_classes = kdd['class'].unique()
kdd_t['class'] = kdd_t['class'].replace(attack_map)

# Drop unwanted classes
unique_classes_test = kdd_t['class'].unique()
drop_class_indices = (~np.in1d(unique_classes_test, unique_classes)).nonzero()[0]
for drop_class_idx in drop_class_indices:
    kdd_t = kdd_t[kdd_t['class'] != unique_classes_test[drop_class_idx]]
</code></pre>

username: arezoo_moradi
cooked: <p>Thank you very much for your reply, but when I replace these commands and execute this line of code,</p>
<h1>We rescale features to [0, 1]</h1>
<p>min_max_scaler = MinMaxScaler ()<br>
train = min_max_scaler.fit_transform (train)<br>
test = min_max_scaler.transform (test)<br>
I get this error,<br>
File “C: \ Python \ Anaconda3 \ lib \ site-packages \ sklearn \ utils \ validation.py”, line 43, in _assert_all_finite<br>
“or a value too large for% r.” % X.dtype)<br>
ValueError: Input contains NaN, infinity or a value too large for dtype (‘float64’).<br>
Which goes back to line 43, which contains this order,<br>
kdd [‘class’] = kdd [‘class’]. replace (attack_map)<br>
unique_classes = kdd [‘class’]. unique ()</p>