Encoding names for ML

MagaMakhauri: <p>Hi!<br>
This may be a stupid question, but if I am manually encoding string data (such as city names, airports, class of the aircraft)<br>
I later use them for ML.</p>
<p>Does it matter what number I chose for any given ‘city’ of no?</p>
<p>Model should calculate coefficient from a city to city<br>
While I have dates, most of my input data isnt numerical, so I encoded it.</p>
<p>Now running this model I am coming back to the question if I should do some sort of sorting before encoding?<br>
Major cities such as Dubai, Moscow and London are ‘responsible’ for higher coefficients. Does it mean that their encoding number must be higher than of, lets say, Wellington or Dublin?</p>

ptrblck: <p>I don’t know what “coefficients” refer to, but the encoding and sorting depends on the use case and how the loss is calculated.<br>
E.g. if you are working on a multi-class classification use case the order wouldn’t matter as each class corresponds to an integer label without any relation to each other.<br>
E.g. while “Dubai - 0” and “Moscow - 1” would be close to each other, the actual “distance” between these labels doesn’t matter in e.g. <code>nn.CrossEntropyLoss</code> (only how wrong/right your model predictions are for the current class index).<br>
However, if you are using e.g. <code>nn.MSELoss</code> and are encoding the cities also with labels, then of course the distance would matter, but I wouldn’t know if this would fit your use case.</p>

MagaMakhauri: <p>Thank you! This is very informative and helpful.</p>
<blockquote>
<p>and are encoding the cities also with labels</p>
</blockquote>
<p>What does that mean?</p>
<p>In my calculation there are almost a 1000 cities. Is it too much for multi-class?</p>

ptrblck: <p>This sentence refers to the usage of <code>nn.MSELoss</code> which would show a larger loss between a prediction of <code>2 - Dublin</code> while the target is <code>0 - Dubai</code> compared to an also wrong prediction of <code>1 - Moscow</code> to <code>0 - Dubai</code>. However, I would not recommend to use <code>nn.MSELoss</code> for a classification use case and also don’t know what you are working on.</p>

MagaMakhauri: <p>I did try to use <code>nn.CrossEntropyLoss</code>.</p>
<p>I get error<br>
<code>*IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)*</code></p>
<p>I tried reading other posts regarding this error. But cant figure out where I went wrong</p>
<pre><code class="lang-auto">criterion = torch.nn.CrossEntropyLoss()
before_train = criterion(y_pred.squeeze(), test_output)
</code></pre>
<p>Now for the variables:</p>
<pre><code class="lang-auto">y_pred:
tensor([0.5098, 0.5024, 0.5043,  ..., 0.5023, 0.5060, 0.5058], device='cuda:0',
       grad_fn=&lt;SqueezeBackward0&gt;)
torch.Size([4608])

test_output:
tensor([1., 0., 1.,  ..., 0., 1., 1.], device='cuda:0')
torch.Size([4608])
</code></pre>
<p>They are the same size and both tensors, so what is causing the error?</p>

ptrblck: <p>Your prediction shape is wrong as <code>[batch_size, nb_classes]</code> is expected.</p>

MagaMakhauri: <p>Sorry ones again.</p>
<p>My inputs are:</p>
<pre><code class="lang-auto">'month', 'day', 'week_day', 'classname_ml', 'departure_country', 'departure_city',
                     'departure', 'arrival', 'arrival_city', 'arrival_country'
</code></pre>
<p>month, day and week_day are int type and pretty straightforward.<br>
I understand that I need to encode the cities as int too and that’s fine.</p>
<p>But since I have same cities in departure <strong>and</strong> arrivals.<br>
I did it by simply numbering each city (and country and airport).<br>
Is it a good solution? (apparently not according to article I have read) and they suggest using OneHotEncoder, but my issue with it is that it will encode departure Moscow differently to arrival Moscow (or it doesnt matter?)<br>
Also since it doesnt rember the values how can I convert values back for testing? (imagine I want to get the output for 7-11-2022 Dubai-Moscow)</p>