Trained on Kurier [Simple Noise Decoder-Dropout 0.1] (2 epochs)

Trained on Kurier [dropout (0.1/0.1)] (2 epochs)

train_on_best_data_05p

train_on_best_data_100p

train_on_best_data_95p

train_on_best_data_25p

train_on_best_data_50p

train_on_best_data_75p

Trained on Kurier seed 42 [low learning rate , 1000 steps]

Trained on Kurier seed 42 + bt [fixmatch]

Trained on Kurier seed 42 [consistency bt]

Trained on Kurier seed 100 [consistency bt]

Trained on Kurier seed 42 [consistency dropout]

Trained on Kurier seed 100 [no dropout, consistency bt]

Trained on Kurier seed 42 [no dropout, consistency bt]

Trained on Kurier seed 100 [ft bart decoder]

Trained on Kurier seed 42 [ft bart decoder]

Trained on KURIER seed 100 [no dropout]

Trained on KURIER seed 42 [no dropout]

Trained on 20min seed 100 [no dropout]

Trained on 20min seed 42 [no dropout]

Trained on Kurier seed 100 [ft decoder + gaussian noise]

Trained on Kurier seed 42 [ft decoder + gaussian noise]

Trained on Kurier seed 42 [ft decoder]

Trained on Kurier seed 100

Trained on Kurier seed 42

Retrained cross-attention (distilled layers: [r r -1] 0.75 rel) seed 100

Retrained cross-attention (distilled layers: [r r -1] 0.75 rel) seed 100

Retrained cross-attention (distilled layers: [r r -1] 1.0) seed 100

Retrained cross-attention (distilled layers: [r r -1] 1.0) seed 42

Retrained cross-attention (distilled layers: [r r -1] 0.75) seed 100

Retrained cross-attention (distilled layers: [r r -1] 0.75) seed 42

Retrained cross-attention (distilled layers: [r r r]) seed 100

Retrained cross-attention (distilled layers: [r r r]) seed 42

Retrained cross-attention (distilled layers: [0 -2 -1]) seed 100

Retrained cross-attention (distilled layers: [0 -2 -1]) seed 42

Retrained cross-attention (distilled layers: [-1]) seed 100

Retrained cross-attention (distilled layers: [-1]) seed 42

Random + pretrained cross-attention seed 100

Random + pretrained cross-attention seed 42

Pretrained cross-attention with normal text replication (+ bart noise)

Regular cross-attention seed 100 [75% Data]

Regular cross-attention seed 100 [75% Data]

Regular cross-attention seed 42 [75% Data]

Regular cross-attention seed 100 [50% Data]

Regular cross-attention seed 42 [50% Data]

Regular cross-attention seed 100 [25% Data]

Regular cross-attention seed 42 [25% Data]

Regular cross-attention seed 100

Regular cross-attention seed 42