Repository: pytorch/text

Issue:
Error running unit tests when building with setup.py install
## 🐛 Bug

**Describe the bug** A clear and concise description of what the bug is.
When building with python setup.py install, running pytest from either the project root directory or the test/ directory causes the error `ImportError: torchtext C++ Extension is not found`. This can be worked-around by renaming the torchtext subdirectory, or by instead using python setup.py develop like the CI does (see .circleci/unittest/linux/scripts/install.sh#L36).

**To Reproduce** Steps to reproduce the behavior:

1. Follow the build steps like normal, running python setup.py install
2. Run pytest
3. Every test fails with the error `ImportError: torchtext C++ Extension is not found`.

**Expected behavior** A clear and concise description of what you expected to happen.

The tests should succeed even when installing with setup.py install, either running pytest from the project root or the test/ directory (this is the case in pytorch) without having to rename the torchtext subdirectory.

**Screenshots** If applicable, add screenshots to help explain your problem.

**Environment**

Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py) (or
fill out the checklist below manually).

You can get the script and run it with:

```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
python -c "import torchtext; print(\"torchtext version is \", torchtext.__version__)"
```

- PyTorch Version (e.g., 1.0): 1.12
- OS (e.g., Linux): Linux
- How you installed PyTorch (`conda`, `pip`, source): Compiled from source
- Build command you used (if compiling from source): python3 ./setup.py install
- Python version: 3.7.13
- CUDA/cuDNN version: ROCm version 5.2
- GPU models and configuration: N/A
- Any other relevant information:

**Additional context** Add any other context about the problem here.



Discussion:
https://github.com/pytorch/text/pull/1874 resolves this.
Hi @parmeet. While installing torchdata does satisfy one of the testing requirements, I'm still running into the problem where the tests will error out with `ImportError: torchtext C++ Extension is not found` if I build with `setup.py install` instead of `setup.py develop`. Is there any way to fix this issue?
> Hi @parmeet. While installing torchdata does satisfy one of the testing requirements, I'm still running into the problem where the tests will error out with `ImportError: torchtext C++ Extension is not found` if I build with `setup.py install` instead of `setup.py develop`. Is there any way to fix this issue?

This happens because the source code directory shadows the installed package. One proper fix is to introduce intermediate directory and put all the unit test code in there. In TorchAudio, unit test codes are placed in `torchaudio_unittest` directory.

There was a PR to do this clean up but it got staled. https://github.com/pytorch/text/pull/1223

cc @Nayef211 

Resolving patch:
diff --git a/tools/setup_helpers/extension.py b/tools/setup_helpers/extension.py
--- a/tools/setup_helpers/extension.py
+++ b/tools/setup_helpers/extension.py
@@ -21,6 +21,10 @@
 _ROOT_DIR = _THIS_DIR.parent.parent.resolve()
 
 
+def _get_cxx11_abi():
+    return "-D_GLIBCXX_USE_CXX11_ABI=" + str(int(torch.compiled_with_cxx11_abi()))
+
+
 def get_ext_modules():
     modules = [
         Extension(name=_LIBTORCHTEXT_NAME, sources=[]),
@@ -72,6 +76,7 @@ def build_extension(self, ext):
             "-DBUILD_SHARED_LIBS=OFF",
             "-DCMAKE_POLICY_DEFAULT_CMP0063=NEW",
             "-DSPM_ENABLE_SHARED=OFF",
+            f"-DTORCH_COMPILED_WITH_CXX_ABI={_get_cxx11_abi()}",
         ]
         build_args = ["--target", "install"]
 


Test patch:
diff --git a/.circleci/unittest/linux/scripts/run_test.sh b/.circleci/unittest/linux/scripts/run_test.sh
--- a/.circleci/unittest/linux/scripts/run_test.sh
+++ b/.circleci/unittest/linux/scripts/run_test.sh
@@ -6,4 +6,5 @@ eval "$(./conda/bin/conda shell.bash hook)"
 conda activate ./env
 
 python -m torch.utils.collect_env
-pytest --cov=torchtext --junitxml=test-results/junit.xml -v --durations 20 test
+cd test
+pytest --cov=torchtext --junitxml=test-results/junit.xml -v --durations 20 torchtext_unittest
diff --git a/.circleci/unittest/windows/scripts/run_test.sh b/.circleci/unittest/windows/scripts/run_test.sh
--- a/.circleci/unittest/windows/scripts/run_test.sh
+++ b/.circleci/unittest/windows/scripts/run_test.sh
@@ -6,4 +6,5 @@ eval "$(./conda/Scripts/conda.exe 'shell.bash' 'hook')"
 conda activate ./env
 
 python -m torch.utils.collect_env
-pytest --cov=torchtext --junitxml=test-results/junit.xml -v --durations 20 test
+cd test
+pytest --cov=torchtext --junitxml=test-results/junit.xml -v --durations 20 torchtext_unittest
diff --git a/test/.gitignore b/test/.gitignore
deleted file mode 100644
diff --git a/test/integration_tests/conftest.py b/test/integration_tests/conftest.py
--- a/test/integration_tests/conftest.py
+++ b/test/integration_tests/conftest.py
@@ -15,9 +15,9 @@ def pytest_addoption(parser):
     )
 
 
-@pytest.fixture(scope="class")
+@pytest.fixture(autouse=True, scope="class")
 def temp_hub_dir(tmp_path_factory, pytestconfig):
-    if not pytestconfig.getoption("--use-tmp-hub-dir"):
+    if not pytestconfig.getoption("use_tmp_hub_dir"):
         yield
     else:
         tmp_dir = tmp_path_factory.mktemp("hub", numbered=True).resolve()
diff --git a/test/prototype/integration_tests/test_models.py b/test/integration_tests/prototype/test_models.py
similarity index 95%
rename from test/prototype/integration_tests/test_models.py
rename to test/integration_tests/prototype/test_models.py
--- a/test/prototype/integration_tests/test_models.py
+++ b/test/integration_tests/prototype/test_models.py
@@ -1,9 +1,6 @@
 import pytest  # noqa: F401
 import torch
 from parameterized import parameterized, parameterized_class
-from test.common.assets import get_asset_path
-from test.common.parameterized_utils import nested_params
-from test.common.torchtext_test_case import TorchtextTestCase
 from torchtext.prototype.models import (
     T5_BASE_ENCODER,
     T5_BASE,
@@ -18,6 +15,9 @@
     T5Transform,
 )
 from torchtext.prototype.models.t5.wrapper import T5Wrapper
+from torchtext_unittest.common.assets import get_asset_path
+from torchtext_unittest.common.parameterized_utils import nested_params
+from torchtext_unittest.common.torchtext_test_case import TorchtextTestCase
 
 
 BUNDLERS = {
diff --git a/test/integration_tests/test_models.py b/test/integration_tests/test_models.py
--- a/test/integration_tests/test_models.py
+++ b/test/integration_tests/test_models.py
@@ -7,9 +7,15 @@
     XLMR_BASE_ENCODER,
     XLMR_LARGE_ENCODER,
 )
+from torchtext_unittest.common.assets import get_asset_path
+from torchtext_unittest.common.torchtext_test_case import TorchtextTestCase
 
-from ..common.assets import get_asset_path
-from ..common.torchtext_test_case import TorchtextTestCase
+BUNDLERS = {
+    "xlmr_base": XLMR_BASE_ENCODER,
+    "xlmr_large": XLMR_LARGE_ENCODER,
+    "roberta_base": ROBERTA_BASE_ENCODER,
+    "roberta_large": ROBERTA_LARGE_ENCODER,
+}
 
 BUNDLERS = {
     "xlmr_base": XLMR_BASE_ENCODER,
diff --git a/test/prototype/models/__init__.py b/test/prototype/models/__init__.py
deleted file mode 100644
diff --git a/test/__init__.py b/test/torchtext_unittest/__init__.py
similarity index 100%
rename from test/__init__.py
rename to test/torchtext_unittest/__init__.py
diff --git a/test/asset/SST2/SST-2.zip b/test/torchtext_unittest/asset/SST2/SST-2.zip
similarity index 100%
rename from test/asset/SST2/SST-2.zip
rename to test/torchtext_unittest/asset/SST2/SST-2.zip
diff --git a/test/asset/bert_base_cased_vocab.txt b/test/torchtext_unittest/asset/bert_base_cased_vocab.txt
similarity index 100%
rename from test/asset/bert_base_cased_vocab.txt
rename to test/torchtext_unittest/asset/bert_base_cased_vocab.txt
diff --git a/test/asset/bert_base_uncased_vocab.txt b/test/torchtext_unittest/asset/bert_base_uncased_vocab.txt
similarity index 100%
rename from test/asset/bert_base_uncased_vocab.txt
rename to test/torchtext_unittest/asset/bert_base_uncased_vocab.txt
diff --git a/test/asset/clip_encoder.json b/test/torchtext_unittest/asset/clip_encoder.json
similarity index 100%
rename from test/asset/clip_encoder.json
rename to test/torchtext_unittest/asset/clip_encoder.json
diff --git a/test/asset/clip_vocab.bpe b/test/torchtext_unittest/asset/clip_vocab.bpe
similarity index 100%
rename from test/asset/clip_vocab.bpe
rename to test/torchtext_unittest/asset/clip_vocab.bpe
diff --git a/test/asset/glove.6B.zip b/test/torchtext_unittest/asset/glove.6B.zip
similarity index 100%
rename from test/asset/glove.6B.zip
rename to test/torchtext_unittest/asset/glove.6B.zip
diff --git a/test/asset/glove.840B.300d.zip b/test/torchtext_unittest/asset/glove.840B.300d.zip
similarity index 100%
rename from test/asset/glove.840B.300d.zip
rename to test/torchtext_unittest/asset/glove.840B.300d.zip
diff --git a/test/asset/gpt2_bpe_encoder.json b/test/torchtext_unittest/asset/gpt2_bpe_encoder.json
similarity index 100%
rename from test/asset/gpt2_bpe_encoder.json
rename to test/torchtext_unittest/asset/gpt2_bpe_encoder.json
diff --git a/test/asset/gpt2_bpe_vocab.bpe b/test/torchtext_unittest/asset/gpt2_bpe_vocab.bpe
similarity index 100%
rename from test/asset/gpt2_bpe_vocab.bpe
rename to test/torchtext_unittest/asset/gpt2_bpe_vocab.bpe
diff --git a/test/asset/label_names.txt b/test/torchtext_unittest/asset/label_names.txt
similarity index 100%
rename from test/asset/label_names.txt
rename to test/torchtext_unittest/asset/label_names.txt
diff --git a/test/asset/raw_datasets.jsonl b/test/torchtext_unittest/asset/raw_datasets.jsonl
similarity index 100%
rename from test/asset/raw_datasets.jsonl
rename to test/torchtext_unittest/asset/raw_datasets.jsonl
diff --git a/test/asset/roberta.base.output.pt b/test/torchtext_unittest/asset/roberta.base.output.pt
similarity index 100%
rename from test/asset/roberta.base.output.pt
rename to test/torchtext_unittest/asset/roberta.base.output.pt
diff --git a/test/asset/roberta.large.output.pt b/test/torchtext_unittest/asset/roberta.large.output.pt
similarity index 100%
rename from test/asset/roberta.large.output.pt
rename to test/torchtext_unittest/asset/roberta.large.output.pt
diff --git a/test/asset/spm_example.model b/test/torchtext_unittest/asset/spm_example.model
similarity index 100%
rename from test/asset/spm_example.model
rename to test/torchtext_unittest/asset/spm_example.model
diff --git a/test/asset/t5.base.encoder.output.pt b/test/torchtext_unittest/asset/t5.base.encoder.output.pt
similarity index 100%
rename from test/asset/t5.base.encoder.output.pt
rename to test/torchtext_unittest/asset/t5.base.encoder.output.pt
diff --git a/test/asset/t5.base.generation.output.pt b/test/torchtext_unittest/asset/t5.base.generation.output.pt
similarity index 100%
rename from test/asset/t5.base.generation.output.pt
rename to test/torchtext_unittest/asset/t5.base.generation.output.pt
diff --git a/test/asset/t5.base.model.output.pt b/test/torchtext_unittest/asset/t5.base.model.output.pt
similarity index 100%
rename from test/asset/t5.base.model.output.pt
rename to test/torchtext_unittest/asset/t5.base.model.output.pt
diff --git a/test/asset/t5.large.encoder.output.pt b/test/torchtext_unittest/asset/t5.large.encoder.output.pt
similarity index 100%
rename from test/asset/t5.large.encoder.output.pt
rename to test/torchtext_unittest/asset/t5.large.encoder.output.pt
diff --git a/test/asset/t5.large.generation.output.pt b/test/torchtext_unittest/asset/t5.large.generation.output.pt
similarity index 100%
rename from test/asset/t5.large.generation.output.pt
rename to test/torchtext_unittest/asset/t5.large.generation.output.pt
diff --git a/test/asset/t5.large.model.output.pt b/test/torchtext_unittest/asset/t5.large.model.output.pt
similarity index 100%
rename from test/asset/t5.large.model.output.pt
rename to test/torchtext_unittest/asset/t5.large.model.output.pt
diff --git a/test/asset/t5.small.encoder.output.pt b/test/torchtext_unittest/asset/t5.small.encoder.output.pt
similarity index 100%
rename from test/asset/t5.small.encoder.output.pt
rename to test/torchtext_unittest/asset/t5.small.encoder.output.pt
diff --git a/test/asset/t5.small.generation.output.pt b/test/torchtext_unittest/asset/t5.small.generation.output.pt
similarity index 100%
rename from test/asset/t5.small.generation.output.pt
rename to test/torchtext_unittest/asset/t5.small.generation.output.pt
diff --git a/test/asset/t5.small.model.output.pt b/test/torchtext_unittest/asset/t5.small.model.output.pt
similarity index 100%
rename from test/asset/t5.small.model.output.pt
rename to test/torchtext_unittest/asset/t5.small.model.output.pt
diff --git a/test/asset/t5_tokenizer_base.model b/test/torchtext_unittest/asset/t5_tokenizer_base.model
similarity index 100%
rename from test/asset/t5_tokenizer_base.model
rename to test/torchtext_unittest/asset/t5_tokenizer_base.model
diff --git a/test/asset/text_normalization_ag_news_ref_results.test b/test/torchtext_unittest/asset/text_normalization_ag_news_ref_results.test
similarity index 100%
rename from test/asset/text_normalization_ag_news_ref_results.test
rename to test/torchtext_unittest/asset/text_normalization_ag_news_ref_results.test
diff --git a/test/asset/text_normalization_ag_news_test.csv b/test/torchtext_unittest/asset/text_normalization_ag_news_test.csv
similarity index 100%
rename from test/asset/text_normalization_ag_news_test.csv
rename to test/torchtext_unittest/asset/text_normalization_ag_news_test.csv
diff --git a/test/asset/vectors_test.csv b/test/torchtext_unittest/asset/vectors_test.csv
similarity index 100%
rename from test/asset/vectors_test.csv
rename to test/torchtext_unittest/asset/vectors_test.csv
diff --git a/test/asset/vocab_raw_text_test.txt b/test/torchtext_unittest/asset/vocab_raw_text_test.txt
similarity index 100%
rename from test/asset/vocab_raw_text_test.txt
rename to test/torchtext_unittest/asset/vocab_raw_text_test.txt
diff --git a/test/asset/vocab_test.txt b/test/torchtext_unittest/asset/vocab_test.txt
similarity index 100%
rename from test/asset/vocab_test.txt
rename to test/torchtext_unittest/asset/vocab_test.txt
diff --git a/test/asset/vocab_test2.txt b/test/torchtext_unittest/asset/vocab_test2.txt
similarity index 100%
rename from test/asset/vocab_test2.txt
rename to test/torchtext_unittest/asset/vocab_test2.txt
diff --git a/test/asset/wiki.en.vec b/test/torchtext_unittest/asset/wiki.en.vec
similarity index 100%
rename from test/asset/wiki.en.vec
rename to test/torchtext_unittest/asset/wiki.en.vec
diff --git a/test/asset/xlmr.base.output.pt b/test/torchtext_unittest/asset/xlmr.base.output.pt
similarity index 100%
rename from test/asset/xlmr.base.output.pt
rename to test/torchtext_unittest/asset/xlmr.base.output.pt
diff --git a/test/asset/xlmr.large.output.pt b/test/torchtext_unittest/asset/xlmr.large.output.pt
similarity index 100%
rename from test/asset/xlmr.large.output.pt
rename to test/torchtext_unittest/asset/xlmr.large.output.pt
diff --git a/test/common/__init__.py b/test/torchtext_unittest/common/__init__.py
similarity index 100%
rename from test/common/__init__.py
rename to test/torchtext_unittest/common/__init__.py
diff --git a/test/common/assets.py b/test/torchtext_unittest/common/assets.py
similarity index 100%
rename from test/common/assets.py
rename to test/torchtext_unittest/common/assets.py
diff --git a/test/common/case_utils.py b/test/torchtext_unittest/common/case_utils.py
similarity index 100%
rename from test/common/case_utils.py
rename to test/torchtext_unittest/common/case_utils.py
diff --git a/test/common/parameterized_utils.py b/test/torchtext_unittest/common/parameterized_utils.py
similarity index 100%
rename from test/common/parameterized_utils.py
rename to test/torchtext_unittest/common/parameterized_utils.py
diff --git a/test/common/torchtext_test_case.py b/test/torchtext_unittest/common/torchtext_test_case.py
similarity index 100%
rename from test/common/torchtext_test_case.py
rename to test/torchtext_unittest/common/torchtext_test_case.py
diff --git a/test/csrc/__init__.py b/test/torchtext_unittest/csrc/__init__.py
similarity index 100%
rename from test/csrc/__init__.py
rename to test/torchtext_unittest/csrc/__init__.py
diff --git a/test/csrc/test_gpt2_bpe_tokenizer.py b/test/torchtext_unittest/csrc/test_gpt2_bpe_tokenizer.py
similarity index 100%
rename from test/csrc/test_gpt2_bpe_tokenizer.py
rename to test/torchtext_unittest/csrc/test_gpt2_bpe_tokenizer.py
diff --git a/test/data/__init__.py b/test/torchtext_unittest/data/__init__.py
similarity index 100%
rename from test/data/__init__.py
rename to test/torchtext_unittest/data/__init__.py
diff --git a/test/data/test_dataset_utils.py b/test/torchtext_unittest/data/test_dataset_utils.py
similarity index 100%
rename from test/data/test_dataset_utils.py
rename to test/torchtext_unittest/data/test_dataset_utils.py
diff --git a/test/data/test_functional.py b/test/torchtext_unittest/data/test_functional.py
similarity index 100%
rename from test/data/test_functional.py
rename to test/torchtext_unittest/data/test_functional.py
diff --git a/test/data/test_jit.py b/test/torchtext_unittest/data/test_jit.py
similarity index 100%
rename from test/data/test_jit.py
rename to test/torchtext_unittest/data/test_jit.py
diff --git a/test/data/test_metrics.py b/test/torchtext_unittest/data/test_metrics.py
similarity index 100%
rename from test/data/test_metrics.py
rename to test/torchtext_unittest/data/test_metrics.py
diff --git a/test/data/test_modules.py b/test/torchtext_unittest/data/test_modules.py
similarity index 100%
rename from test/data/test_modules.py
rename to test/torchtext_unittest/data/test_modules.py
diff --git a/test/data/test_utils.py b/test/torchtext_unittest/data/test_utils.py
similarity index 100%
rename from test/data/test_utils.py
rename to test/torchtext_unittest/data/test_utils.py
diff --git a/test/datasets/__init__.py b/test/torchtext_unittest/datasets/__init__.py
similarity index 100%
rename from test/datasets/__init__.py
rename to test/torchtext_unittest/datasets/__init__.py
diff --git a/test/datasets/common.py b/test/torchtext_unittest/datasets/common.py
similarity index 100%
rename from test/datasets/common.py
rename to test/torchtext_unittest/datasets/common.py
diff --git a/test/datasets/test_agnews.py b/test/torchtext_unittest/datasets/test_agnews.py
similarity index 100%
rename from test/datasets/test_agnews.py
rename to test/torchtext_unittest/datasets/test_agnews.py
diff --git a/test/datasets/test_amazonreviews.py b/test/torchtext_unittest/datasets/test_amazonreviews.py
similarity index 100%
rename from test/datasets/test_amazonreviews.py
rename to test/torchtext_unittest/datasets/test_amazonreviews.py
diff --git a/test/datasets/test_cc100.py b/test/torchtext_unittest/datasets/test_cc100.py
similarity index 100%
rename from test/datasets/test_cc100.py
rename to test/torchtext_unittest/datasets/test_cc100.py
diff --git a/test/datasets/test_cnndm.py b/test/torchtext_unittest/datasets/test_cnndm.py
similarity index 100%
rename from test/datasets/test_cnndm.py
rename to test/torchtext_unittest/datasets/test_cnndm.py
diff --git a/test/datasets/test_cola.py b/test/torchtext_unittest/datasets/test_cola.py
similarity index 100%
rename from test/datasets/test_cola.py
rename to test/torchtext_unittest/datasets/test_cola.py
diff --git a/test/datasets/test_conll2000chunking.py b/test/torchtext_unittest/datasets/test_conll2000chunking.py
similarity index 100%
rename from test/datasets/test_conll2000chunking.py
rename to test/torchtext_unittest/datasets/test_conll2000chunking.py
diff --git a/test/datasets/test_dbpedia.py b/test/torchtext_unittest/datasets/test_dbpedia.py
similarity index 100%
rename from test/datasets/test_dbpedia.py
rename to test/torchtext_unittest/datasets/test_dbpedia.py
diff --git a/test/datasets/test_enwik9.py b/test/torchtext_unittest/datasets/test_enwik9.py
similarity index 100%
rename from test/datasets/test_enwik9.py
rename to test/torchtext_unittest/datasets/test_enwik9.py
diff --git a/test/datasets/test_imdb.py b/test/torchtext_unittest/datasets/test_imdb.py
similarity index 100%
rename from test/datasets/test_imdb.py
rename to test/torchtext_unittest/datasets/test_imdb.py
diff --git a/test/datasets/test_iwslt2016.py b/test/torchtext_unittest/datasets/test_iwslt2016.py
similarity index 100%
rename from test/datasets/test_iwslt2016.py
rename to test/torchtext_unittest/datasets/test_iwslt2016.py
diff --git a/test/datasets/test_iwslt2017.py b/test/torchtext_unittest/datasets/test_iwslt2017.py
similarity index 100%
rename from test/datasets/test_iwslt2017.py
rename to test/torchtext_unittest/datasets/test_iwslt2017.py
diff --git a/test/datasets/test_mnli.py b/test/torchtext_unittest/datasets/test_mnli.py
similarity index 100%
rename from test/datasets/test_mnli.py
rename to test/torchtext_unittest/datasets/test_mnli.py
diff --git a/test/datasets/test_mrpc.py b/test/torchtext_unittest/datasets/test_mrpc.py
similarity index 100%
rename from test/datasets/test_mrpc.py
rename to test/torchtext_unittest/datasets/test_mrpc.py
diff --git a/test/datasets/test_multi30k.py b/test/torchtext_unittest/datasets/test_multi30k.py
similarity index 100%
rename from test/datasets/test_multi30k.py
rename to test/torchtext_unittest/datasets/test_multi30k.py
diff --git a/test/datasets/test_penntreebank.py b/test/torchtext_unittest/datasets/test_penntreebank.py
similarity index 100%
rename from test/datasets/test_penntreebank.py
rename to test/torchtext_unittest/datasets/test_penntreebank.py
diff --git a/test/datasets/test_qnli.py b/test/torchtext_unittest/datasets/test_qnli.py
similarity index 100%
rename from test/datasets/test_qnli.py
rename to test/torchtext_unittest/datasets/test_qnli.py
diff --git a/test/datasets/test_qqp.py b/test/torchtext_unittest/datasets/test_qqp.py
similarity index 100%
rename from test/datasets/test_qqp.py
rename to test/torchtext_unittest/datasets/test_qqp.py
diff --git a/test/datasets/test_rte.py b/test/torchtext_unittest/datasets/test_rte.py
similarity index 100%
rename from test/datasets/test_rte.py
rename to test/torchtext_unittest/datasets/test_rte.py
diff --git a/test/datasets/test_sogounews.py b/test/torchtext_unittest/datasets/test_sogounews.py
similarity index 100%
rename from test/datasets/test_sogounews.py
rename to test/torchtext_unittest/datasets/test_sogounews.py
diff --git a/test/datasets/test_squads.py b/test/torchtext_unittest/datasets/test_squads.py
similarity index 100%
rename from test/datasets/test_squads.py
rename to test/torchtext_unittest/datasets/test_squads.py
diff --git a/test/datasets/test_sst2.py b/test/torchtext_unittest/datasets/test_sst2.py
similarity index 100%
rename from test/datasets/test_sst2.py
rename to test/torchtext_unittest/datasets/test_sst2.py
diff --git a/test/datasets/test_stsb.py b/test/torchtext_unittest/datasets/test_stsb.py
similarity index 100%
rename from test/datasets/test_stsb.py
rename to test/torchtext_unittest/datasets/test_stsb.py
diff --git a/test/datasets/test_udpos.py b/test/torchtext_unittest/datasets/test_udpos.py
similarity index 100%
rename from test/datasets/test_udpos.py
rename to test/torchtext_unittest/datasets/test_udpos.py
diff --git a/test/datasets/test_wikitexts.py b/test/torchtext_unittest/datasets/test_wikitexts.py
similarity index 100%
rename from test/datasets/test_wikitexts.py
rename to test/torchtext_unittest/datasets/test_wikitexts.py
diff --git a/test/datasets/test_wnli.py b/test/torchtext_unittest/datasets/test_wnli.py
similarity index 100%
rename from test/datasets/test_wnli.py
rename to test/torchtext_unittest/datasets/test_wnli.py
diff --git a/test/datasets/test_yahooanswers.py b/test/torchtext_unittest/datasets/test_yahooanswers.py
similarity index 100%
rename from test/datasets/test_yahooanswers.py
rename to test/torchtext_unittest/datasets/test_yahooanswers.py
diff --git a/test/datasets/test_yelpreviews.py b/test/torchtext_unittest/datasets/test_yelpreviews.py
similarity index 100%
rename from test/datasets/test_yelpreviews.py
rename to test/torchtext_unittest/datasets/test_yelpreviews.py
diff --git a/test/models/__init__.py b/test/torchtext_unittest/models/__init__.py
similarity index 100%
rename from test/models/__init__.py
rename to test/torchtext_unittest/models/__init__.py
diff --git a/test/models/test_models.py b/test/torchtext_unittest/models/test_models.py
similarity index 100%
rename from test/models/test_models.py
rename to test/torchtext_unittest/models/test_models.py
diff --git a/test/models/test_transformers.py b/test/torchtext_unittest/models/test_transformers.py
similarity index 100%
rename from test/models/test_transformers.py
rename to test/torchtext_unittest/models/test_transformers.py
diff --git a/test/prototype/__init__.py b/test/torchtext_unittest/prototype/__init__.py
similarity index 100%
rename from test/prototype/__init__.py
rename to test/torchtext_unittest/prototype/__init__.py
diff --git a/test/prototype/integration_tests/__init__.py b/test/torchtext_unittest/prototype/models/__init__.py
similarity index 100%
rename from test/prototype/integration_tests/__init__.py
rename to test/torchtext_unittest/prototype/models/__init__.py
diff --git a/test/prototype/models/test_models.py b/test/torchtext_unittest/prototype/models/test_models.py
similarity index 98%
rename from test/prototype/models/test_models.py
rename to test/torchtext_unittest/prototype/models/test_models.py
--- a/test/prototype/models/test_models.py
+++ b/test/torchtext_unittest/prototype/models/test_models.py
@@ -2,8 +2,8 @@
 from unittest.mock import patch
 
 import torch
-from test.common.torchtext_test_case import TorchtextTestCase
 from torch.nn import functional as F
+from torchtext_unittest.common.torchtext_test_case import TorchtextTestCase
 
 
 class TestModels(TorchtextTestCase):
diff --git a/test/prototype/models/test_transforms.py b/test/torchtext_unittest/prototype/models/test_transforms.py
similarity index 93%
rename from test/prototype/models/test_transforms.py
rename to test/torchtext_unittest/prototype/models/test_transforms.py
--- a/test/prototype/models/test_transforms.py
+++ b/test/torchtext_unittest/prototype/models/test_transforms.py
@@ -1,7 +1,7 @@
 import torch
-from test.common.assets import get_asset_path
-from test.common.torchtext_test_case import TorchtextTestCase
 from torchtext.prototype.models import T5Transform
+from torchtext_unittest.common.assets import get_asset_path
+from torchtext_unittest.common.torchtext_test_case import TorchtextTestCase
 
 
 class TestTransforms(TorchtextTestCase):
diff --git a/test/prototype/test_functional.py b/test/torchtext_unittest/prototype/test_functional.py
similarity index 100%
rename from test/prototype/test_functional.py
rename to test/torchtext_unittest/prototype/test_functional.py
diff --git a/test/prototype/test_transforms.py b/test/torchtext_unittest/prototype/test_transforms.py
similarity index 97%
rename from test/prototype/test_transforms.py
rename to test/torchtext_unittest/prototype/test_transforms.py
--- a/test/prototype/test_transforms.py
+++ b/test/torchtext_unittest/prototype/test_transforms.py
@@ -3,14 +3,14 @@
 import tempfile
 
 import torch
-from test.common.assets import get_asset_path
-from test.common.torchtext_test_case import TorchtextTestCase
 from torchtext.prototype.transforms import (
     sentencepiece_processor,
     sentencepiece_tokenizer,
     VectorTransform,
 )
 from torchtext.prototype.vectors import FastText
+from torchtext_unittest.common.assets import get_asset_path
+from torchtext_unittest.common.torchtext_test_case import TorchtextTestCase
 
 
 class TestTransforms(TorchtextTestCase):
diff --git a/test/prototype/test_vectors.py b/test/torchtext_unittest/prototype/test_vectors.py
similarity index 98%
rename from test/prototype/test_vectors.py
rename to test/torchtext_unittest/prototype/test_vectors.py
--- a/test/prototype/test_vectors.py
+++ b/test/torchtext_unittest/prototype/test_vectors.py
@@ -4,8 +4,8 @@
 import unittest
 
 import torch
-from test.common.torchtext_test_case import TorchtextTestCase
 from torchtext.prototype.vectors import build_vectors
+from torchtext_unittest.common.torchtext_test_case import TorchtextTestCase
 
 
 class TestVectors(TorchtextTestCase):
diff --git a/test/prototype/test_with_asset.py b/test/torchtext_unittest/prototype/test_with_asset.py
similarity index 99%
rename from test/prototype/test_with_asset.py
rename to test/torchtext_unittest/prototype/test_with_asset.py
--- a/test/prototype/test_with_asset.py
+++ b/test/torchtext_unittest/prototype/test_with_asset.py
@@ -6,7 +6,6 @@
 from functools import partial
 
 import torch
-from test.common.torchtext_test_case import TorchtextTestCase
 from torch.utils.data import DataLoader
 from torchtext.data.functional import custom_replace
 from torchtext.prototype.transforms import (
@@ -19,6 +18,7 @@
 from torchtext.prototype.vectors import build_vectors, FastText, GloVe, load_vectors_from_file_path
 from torchtext.prototype.vocab_factory import build_vocab_from_text_file, load_vocab_from_file
 from torchtext.utils import download_from_url
+from torchtext_unittest.common.torchtext_test_case import TorchtextTestCase
 
 from ..common.assets import get_asset_path
 
diff --git a/test/test_build.py b/test/torchtext_unittest/test_build.py
similarity index 100%
rename from test/test_build.py
rename to test/torchtext_unittest/test_build.py
diff --git a/test/test_functional.py b/test/torchtext_unittest/test_functional.py
similarity index 100%
rename from test/test_functional.py
rename to test/torchtext_unittest/test_functional.py
diff --git a/test/test_transforms.py b/test/torchtext_unittest/test_transforms.py
similarity index 100%
rename from test/test_transforms.py
rename to test/torchtext_unittest/test_transforms.py
diff --git a/test/test_utils.py b/test/torchtext_unittest/test_utils.py
similarity index 98%
rename from test/test_utils.py
rename to test/torchtext_unittest/test_utils.py
--- a/test/test_utils.py
+++ b/test/torchtext_unittest/test_utils.py
@@ -5,9 +5,9 @@
 import unittest
 from urllib.parse import urljoin
 
-from test.common.assets import conditional_remove, get_asset_path
 from torchtext import _TEXT_BUCKET
 from torchtext import utils
+from torchtext_unittest.common.assets import conditional_remove, get_asset_path
 
 from .common.torchtext_test_case import TorchtextTestCase
 
diff --git a/test/test_vocab.py b/test/torchtext_unittest/test_vocab.py
similarity index 99%
rename from test/test_vocab.py
rename to test/torchtext_unittest/test_vocab.py
--- a/test/test_vocab.py
+++ b/test/torchtext_unittest/test_vocab.py
@@ -4,8 +4,8 @@
 
 import pytest
 import torch
-from test.common.torchtext_test_case import TorchtextTestCase
 from torchtext.vocab import build_vocab_from_iterator, vocab
+from torchtext_unittest.common.torchtext_test_case import TorchtextTestCase
 
 
 class TestVocab(TorchtextTestCase):