Train accuracy rises, train loss falls, valid test loss rises, valid test accuracy fluctuates

Gorgen: <p>Hello everyone:</p>
<p>my task is multimodal emotion recognition.</p>
<p>The baseline is only text information, and the current experiment is to fuse the voice information with the text and put it into the original model for training. The basic architecture of the original model is a GRU model with an attention mechanism, the loss function is nn.NLLLoss, and the optimizer is optimizer = optim.Adam(model.parameters(), lr=args.lr, weight_decay=args.l2). But the experimental result is Train accuracy rises, train loss falls, valid test loss rises, valid test accuracy fluctuates.<br>
Here is baseline orignal coding: <a href="https://github.com/declare-lab/conv-emotion/blob/master/COSMIC/erc-training/train_meld.py" class="inline-onebox" rel="noopener nofollow ugc">conv-emotion/COSMIC/erc-training/train_meld.py at master · declare-lab/conv-emotion · GitHub</a></p>
<p>Could you please give me some details to find the ture error? some suggestion to fix it? Thanks</p>
<p>Here are more detailed informations.<br>
train valid test accuracy loss<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/0/90a890cd6794780461d4b9317de5a336e1e1bc2c.png" data-download-href="https://discuss.pytorch.org/uploads/default/90a890cd6794780461d4b9317de5a336e1e1bc2c" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/0/90a890cd6794780461d4b9317de5a336e1e1bc2c_2_400x500.png" alt="image" data-base62-sha1="kDHUatlhWP6wxDbNie2e3FwqCfy" width="400" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/0/90a890cd6794780461d4b9317de5a336e1e1bc2c_2_400x500.png, https://discuss.pytorch.org/uploads/default/original/3X/9/0/90a890cd6794780461d4b9317de5a336e1e1bc2c.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/9/0/90a890cd6794780461d4b9317de5a336e1e1bc2c.png 2x" data-dominant-color="FAFAFA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">500×624 24.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/0/60559372529bb64519e1b1daaea588f4c47d8b02.png" data-download-href="https://discuss.pytorch.org/uploads/default/60559372529bb64519e1b1daaea588f4c47d8b02" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/0/60559372529bb64519e1b1daaea588f4c47d8b02_2_690x323.png" alt="image" data-base62-sha1="dKdbkKNxb9lzfQ9XUCig1j9Tosy" width="690" height="323" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/0/60559372529bb64519e1b1daaea588f4c47d8b02_2_690x323.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/0/60559372529bb64519e1b1daaea588f4c47d8b02_2_1035x484.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/0/60559372529bb64519e1b1daaea588f4c47d8b02.png 2x" data-dominant-color="FCFBFB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1101×516 65.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>loss function<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/2/c/2c921fe688f3555e2542f20aa8db0db0ea3944df.png" alt="E3%HPN@VM5`HY" data-base62-sha1="6mi4UA88TB9VDyF8u2ypK1ilsxp" width="658" height="488"></p>
<p>model summary<br>
CommonsenseGRUModel_bimodal(<br>
(linear_in): Linear(in_features=1024, out_features=300, bias=True)<br>
(norm1a): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)<br>
(norm1b): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)<br>
(norm1c): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)<br>
(norm1d): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)<br>
(norm3a): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(norm3b): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(norm3c): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(norm3d): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(dropout): Dropout(p=0.5, inplace=False)<br>
(dropout_rec): Dropout(p=False, inplace=False)<br>
(cs_rnn_f): CommonsenseRNN(<br>
(dropout): Dropout(p=False, inplace=False)<br>
(dialogue_cell): CommonsenseRNNCell(<br>
(g_cell): GRUCell(600, 150)<br>
(p_cell): GRUCell(918, 150)<br>
(r_cell): GRUCell(1218, 150)<br>
(i_cell): GRUCell(918, 150)<br>
(e_cell): GRUCell(750, 450)<br>
(dropout): Dropout(p=False, inplace=False)<br>
(dropout1): Dropout(p=False, inplace=False)<br>
(dropout2): Dropout(p=False, inplace=False)<br>
(dropout3): Dropout(p=False, inplace=False)<br>
(dropout4): Dropout(p=False, inplace=False)<br>
(dropout5): Dropout(p=False, inplace=False)<br>
(attention): SimpleAttention(<br>
(scalar): Linear(in_features=150, out_features=1, bias=False)<br>
)<br>
)<br>
)<br>
(cs_rnn_r): CommonsenseRNN(<br>
(dropout): Dropout(p=False, inplace=False)<br>
(dialogue_cell): CommonsenseRNNCell(<br>
(g_cell): GRUCell(600, 150)<br>
(p_cell): GRUCell(918, 150)<br>
(r_cell): GRUCell(1218, 150)<br>
(i_cell): GRUCell(918, 150)<br>
(e_cell): GRUCell(750, 450)<br>
(dropout): Dropout(p=False, inplace=False)<br>
(dropout1): Dropout(p=False, inplace=False)<br>
(dropout2): Dropout(p=False, inplace=False)<br>
(dropout3): Dropout(p=False, inplace=False)<br>
(dropout4): Dropout(p=False, inplace=False)<br>
(dropout5): Dropout(p=False, inplace=False)<br>
(attention): SimpleAttention(<br>
(scalar): Linear(in_features=150, out_features=1, bias=False)<br>
)<br>
)<br>
)<br>
(sense_gru): GRU(768, 384, bidirectional=True)<br>
(matchatt): MatchingAttention(<br>
(transform): Linear(in_features=900, out_features=900, bias=True)<br>
)<br>
(linear): Linear(in_features=900, out_features=300, bias=True)<br>
(smax_fc): Linear(in_features=300, out_features=7, bias=True)<br>
(linear_in_mixed_feature): Linear(in_features=1408, out_features=300, bias=True)<br>
)</p>