Cannot perform test on GLUE example
🐛 Bug
I tried to run the GLUE example in the README.md file to see what does the model return when running test.
However, when I create a new cell to run trainer.test(), I got the following error
/usr/local/lib/python3.6/dist-packages/zmq/backend/cython/socket.cpython-36m-x86_64-linux-gnu.so in zmq.backend.cython.socket.Socket.__reduce_cython__()

TypeError: no default __reduce__ due to non-trivial __cinit__

After a quick Google search, I suspect it has something to do with the setup() method in GLUEDataModule. To verify it, I compare the data module before and after I run trainer.test() with the code
for iter in dm.dataset["test"]:
    print(iter)

The result obtained before the trainer.test() is as expected, but the result after is changed back to original data from datasets library.
To Reproduce
In the Colab GLUE notebook, create new cells with the following code
mocked_args = """
    --model_name_or_path distilbert-base-cased
    --task_name mrpc
    --max_epochs 1
    --gpus 1""".split()

args = parse_args(mocked_args)
dm, model, trainer = main(args)

for iter in dm.dataset["test"]:
    print(iter)

trainer.fit(model, datamodule=dm)

trainer.test(verbose=True) # Error

for iter in dm.dataset["test"]:
    print(iter)

Expected behavior
The output of the 2nd cell and 5th cell should be the same.
The 4th cell should perform test on finetuned model.
Environment

Google Colab
CUDA:

GPU:

Tesla T4


available:         True
version:           10.1


Packages:

numpy:             1.18.5
pyTorch_debug:     True
pyTorch_version:   1.7.0+cu101
pytorch-lightning: 1.0.8
tqdm:              4.41.1


System:

OS:                Linux
architecture:

64bit



processor:         x86_64
python:            3.6.9
version:           #1 SMP Thu Jul 23 08:00:38 PDT 2020