Repository: huggingface/accelerate
Pull request: Guard stateful objects
# What does this PR do?

This PR is the second half of solving https://github.com/huggingface/accelerate/issues/2564

The issue boils down to:

1. The user defines a `PartialState`/creates an `Accelerator`
2. The user then calls `_reset_state()` themselves
3. The user then calls subsequent functions in `Accelerator`.

That instance will still point to our reset/broken `PartialState`, and so a better error needs to be raised that contextualizes what's going on early:

```python
Traceback (most recent call last):
  File "/home/zach/test.py", line 11, in <module>
    _ = accelerator.prepare(model)
  File "/home/zach/accelerate/accelerate/src/accelerate/accelerator.py", line 1230, in prepare
    if self.distributed_type == DistributedType.DEEPSPEED:
  File "/home/zach/accelerate/accelerate/src/accelerate/accelerator.py", line 515, in distributed_type
    return self.state.distributed_type
  File "/home/zach/accelerate/accelerate/src/accelerate/state.py", line 1082, in __getattr__
    raise AttributeError(
AttributeError: `AcceleratorState` object has no attribute `distributed_type`. This can happen if the state was reset and using an old reference. If you are using an `Accelerator`, make sure to re-initialize it.
```

Fixes https://github.com/huggingface/accelerate/issues/2564


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/accelerate/blob/main/CONTRIBUTING.md#submitting-a-pull-request-pr),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/accelerate/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/accelerate/tree/main/docs#writing-documentation---specification).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@BenjaminBossan 

Problem statement:
'AcceleratorState' object has no attribute 'distributed_type'
### System Info

```Shell
accelerate == 0.28.0
python == 3.8.18
transformers == 4.37.2
torch == 2.2.1
```


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] One of the scripts in the examples/ folder of Accelerate or an officially supported `no_trainer` script in the `examples` folder of the `transformers` repo (such as `run_no_trainer_glue.py`)
- [X] My own task or dataset (give details below)

### Reproduction

I was fine-tuning a bert-base model using codes from huggingface's example.I just copied the code from the website and then the bug occured. Code and bug are listed below. By the way,the pycharm told me that accelerator's arguments should be list but I give it Dataloader. 
Bug:  
AttributeError: 'AcceleratorState' object has no attribute 'distributed_type'

Code:   
```python
model = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2) 
train_dataloader = DataLoader(datasets["train"], shuffle=True, batch_size=8, collate_fn=data_collator) 
eval_dataloader = DataLoader(datasets["validation"], batch_size=8, collate_fn=data_collator) 
optimizer = torch.optim.AdamW(model.parameters(), lr=3e-5)
model, optimizer, train_dataloader, eval_dataloader = accelerator.prepare(model, optimizer, train_dataloader, eval_dataloader)
```

### Expected behavior

the code works properly.

Hints from issue discussion:
Please give us the full stack trace
full stack trace:
```python
Traceback (most recent call last):
  File "C:\Users\Noah\PycharmProjects\pythonProject\venv\fine-tune.py", line 44, in <module>
    model, optimizer, train_dataloader, eval_dataloader = accelerator.prepare(model, optimizer, train_dataloader, eval_dataloader)
  File "E:\Anaconda\envs\pytorch\lib\site-packages\accelerate\accelerator.py", line 1219, in prepare
    if self.distributed_type == DistributedType.DEEPSPEED:
  File "E:\Anaconda\envs\pytorch\lib\site-packages\accelerate\accelerator.py", line 509, in distributed_type
    return self.state.distributed_type
AttributeError: 'AcceleratorState' object has no attribute 'distributed_type'
```
What is your full, entire code please. 
Entire code:
```python
import torch
import numpy as np
import evaluate
from datasets import load_dataset
from torch.optim import AdamW
from transformers import AutoTokenizer, AutoModelForSequenceClassification, DataCollatorWithPadding, TrainingArguments
from transformers import Trainer, AdamW, get_scheduler
from torch.utils.data import DataLoader
from tqdm.auto import tqdm
from accelerate import Accelerator

def tokenize_function(example):
    return tokenizer(example['sentence1'], example["sentence2"], truncation=True)



checkpoint = r"E:\model\bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2)
accelerator = Accelerator()
device = accelerator.device
training_args = TrainingArguments("test-trainer", evaluation_strategy="epoch")
raw_dataset = load_dataset(r"E:\date\glue-mrpc")
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)
datasets = raw_dataset.map(tokenize_function, batched=True)
datasets = datasets.remove_columns(["sentence1", "sentence2", "idx"])
datasets = datasets.rename_column("label", "labels")
datasets.set_format("torch")
print(datasets["train"].column_names)
train_dataloader = DataLoader(datasets["train"], shuffle=True, batch_size=8, collate_fn=data_collator)
eval_dataloader = DataLoader(datasets["validation"], batch_size=8, collate_fn=data_collator)
for batch in train_dataloader:
    break
print({k: v.shape for k, v in batch.items()})
outputs = model(**batch)
print(outputs["loss"], outputs.logits.shape)
optimizer = AdamW(model.parameters(), lr=3e-5)
model, optimizer, train_dataloader, eval_dataloader = accelerator.prepare(model, optimizer, train_dataloader, eval_dataloader)
num_epoch = 3
num_train_steps = len(train_dataloader)*num_epoch
lr_scheduler = get_scheduler(
    "linear", optimizer=optimizer, num_warmup_steps=0, num_training_steps=num_train_steps
)
print(num_train_steps)

progress_bar = tqdm(range(num_train_steps))
model.train()
for epoch in range(num_epoch):
    for batch in train_dataloader:
       # batch = {k: v.to(device) for k, v in batch.items()}
        outputs = model(**batch)
        loss = outputs.loss
      #  loss.backward()
        accelerator.backward(loss)
        optimizer.step()
        lr_scheduler.step()
        optimizer.zero_grad()
        progress_bar.update(1)
metric = evaluate.load(r"E:\微调\测试\metrics\glue","mrpc")
model.eval()
for batch in eval_dataloader:
    batch = {k: v.to(device) for k, v in batch.items()}
    with torch.no_grad():
        outputs = model(**batch)
    logits = outputs.logits
    predictions = torch.argmax(logits, dim=-1)
    metric.add_batch(predictions=predictions, references=batch["labels"])
temp = metric.compute()
```


Thanks, one more bit we need (this is great, as this issue has been reported before with none of us being able to reproduce, so this is very helpful!)

How are you:
1. Launching the code (`python`, `accelerate launch` or `torchrun`?)
2. If you are using `accelerate launch`, what is the output of `accelerate env`

1. I lauch the code by python in pycharm
2. sorry,I am afraid I can not answer you this question right now, but I will answer you tomorrow.
Thanks for helping me solving this bug!

Just running in `python` I don't see this:

command: `python test.py`
file:
```python
from datasets import load_dataset
from torch.optim import AdamW
from transformers import AutoTokenizer, AutoModelForSequenceClassification, DataCollatorWithPadding
from transformers import AdamW
from torch.utils.data import DataLoader
from tqdm.auto import tqdm
from accelerate import Accelerator

def tokenize_function(example):
    return tokenizer(example['sentence1'], example["sentence2"], truncation=True)

checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2)
accelerator = Accelerator()
device = accelerator.device
raw_dataset = load_dataset("glue", "mrpc")

data_collator = DataCollatorWithPadding(tokenizer=tokenizer)
datasets = raw_dataset.map(tokenize_function, batched=True)
datasets = datasets.remove_columns(["sentence1", "sentence2", "idx"])
datasets = datasets.rename_column("label", "labels")
datasets.set_format("torch")
train_dataloader = DataLoader(datasets["train"], shuffle=True, batch_size=8, collate_fn=data_collator)
eval_dataloader = DataLoader(datasets["validation"], batch_size=8, collate_fn=data_collator)
optimizer = AdamW(model.parameters(), lr=3e-5)
model, optimizer, train_dataloader, eval_dataloader = accelerator.prepare(model, optimizer, train_dataloader, eval_dataloader)
```
Can you verify if that fails to run for you? Thanks!
Also the output of `pip freeze` too
accelerate env:
- `Accelerate` version: 0.28.0
- Platform: Windows-10-10.0.19045-SP0
- Python version: 3.8.18
- Numpy version: 1.24.3
- PyTorch version (GPU?): 2.2.1 (True)
- PyTorch XPU available: False
- PyTorch NPU available: False
- System RAM: 31.84 GB
- GPU type: NVIDIA GeForce RTX 4070
- `Accelerate` default config:
        Not found

pip freeze:
```
accelerate==0.28.0
aiohttp @ file:///C:/b/abs_27h_1rpxgd/croot/aiohttp_1707342354614/work
aiosignal @ file:///tmp/build/80754af9/aiosignal_1637843061372/work
antlr4-python3-runtime==4.9.3
appdirs==1.4.4
arrow @ file:///C:/b/abs_cal7u12ktb/croot/arrow_1676588147908/work
asttokens @ file:///opt/conda/conda-bld/asttokens_1646925590279/work
async-timeout @ file:///C:/b/abs_c8fgiuixkq/croot/async-timeout_1703097556097/work
attrs @ file:///C:/b/abs_35n0jusce8/croot/attrs_1695717880170/work
backcall @ file:///home/ktietz/src/ci/backcall_1611930011877/work
binaryornot @ file:///tmp/build/80754af9/binaryornot_1617751525010/work
bitarray==2.9.2
Bottleneck @ file:///C:/b/abs_f05kqh7yvj/croot/bottleneck_1707864273291/work
Brotli @ file:///C:/Windows/Temp/abs_63l7912z0e/croots/recipe/brotli-split_1659616056886/work
certifi @ file:///C:/b/abs_35d7n66oz9/croot/certifi_1707229248467/work/certifi
cffi==1.16.0
chardet @ file:///C:/ci/chardet_1607690654534/work
charset-normalizer @ file:///tmp/build/80754af9/charset-normalizer_1630003229654/work
click @ file:///C:/b/abs_f9ihnt72pu/croot/click_1698129847492/work
colorama @ file:///C:/b/abs_a9ozq0l032/croot/colorama_1672387194846/work
comm @ file:///C:/b/abs_67a8058udb/croot/comm_1709322909844/work
contourpy @ file:///C:/b/abs_d5rpy288vc/croots/recipe/contourpy_1663827418189/work
cookiecutter @ file:///C:/b/abs_3d1730toam/croot/cookiecutter_1700677089156/work
cycler @ file:///tmp/build/80754af9/cycler_1637851556182/work
Cython==3.0.9
datasets==2.18.0
debugpy @ file:///C:/b/abs_c0y1fjipt2/croot/debugpy_1690906864587/work
decorator @ file:///opt/conda/conda-bld/decorator_1643638310831/work
dill @ file:///C:/b/abs_42h_07z1yj/croot/dill_1667919550096/work
evaluate @ file:///C:/b/abs_b0u82qzwka/croot/evaluate_1679573623633/work
executing @ file:///opt/conda/conda-bld/executing_1646925071911/work
-e git+https://github.com/pytorch/fairseq@34973a94d09ecc12092a5ecc8afece5e536b7692#egg=fairseq
filelock @ file:///C:/b/abs_f2gie28u58/croot/filelock_1700591233643/work
fonttools==4.25.0
frozenlist @ file:///C:/b/abs_d8e__s1ys3/croot/frozenlist_1698702612014/work
fsspec @ file:///C:/b/abs_97mpfsesn0/croot/fsspec_1701286534629/work
gmpy2 @ file:///C:/ci/gmpy2_1645456279018/work
huggingface-hub @ file:///C:/b/abs_2f65ujfg51/croot/huggingface_hub_1708635586197/work
hydra-core==1.0.7
idna @ file:///C:/b/abs_bdhbebrioa/croot/idna_1666125572046/work
importlib-metadata @ file:///C:/b/abs_c1egths604/croot/importlib_metadata-suite_1704813568388/work
importlib-resources @ file:///C:/b/abs_d0dmp77t95/croot/importlib_resources-suite_1704281892795/work
ipykernel @ file:///C:/b/abs_c2u94kxcy6/croot/ipykernel_1705933907920/work
ipython @ file:///C:/b/abs_254uk73z5b/croot/ipython_1691532131313/work
jedi @ file:///C:/ci/jedi_1644315425835/work
Jinja2 @ file:///C:/b/abs_f7x5a8op2h/croot/jinja2_1706733672594/work
joblib @ file:///C:/b/abs_1anqjntpan/croot/joblib_1685113317150/work
jupyter_client @ file:///C:/b/abs_a6h3c8hfdq/croot/jupyter_client_1699455939372/work
jupyter_core @ file:///C:/b/abs_c769pbqg9b/croot/jupyter_core_1698937367513/work
kiwisolver @ file:///C:/b/abs_88mdhvtahm/croot/kiwisolver_1672387921783/work
lightgbm @ file:///C:/b/abs_40p8j029wh/croot/lightgbm_1700267980215/work
lxml==5.1.0
markdown-it-py @ file:///C:/b/abs_a5bfngz6fu/croot/markdown-it-py_1684279915556/work
MarkupSafe @ file:///C:/b/abs_ecfdqh67b_/croot/markupsafe_1704206030535/work
matplotlib @ file:///C:/b/abs_085jhivdha/croot/matplotlib-suite_1693812524572/work
matplotlib-inline @ file:///C:/ci/matplotlib-inline_1661934035815/work
mdurl @ file:///C:/Windows/TEMP/abs_3197pzpjbi/croots/recipe/mdurl_1659716032440/work
mkl-fft @ file:///C:/b/abs_19i1y8ykas/croot/mkl_fft_1695058226480/work
mkl-random @ file:///C:/b/abs_edwkj1_o69/croot/mkl_random_1695059866750/work
mkl-service==2.4.0
mlxtend @ file:///C:/b/abs_abdi0b9anv/croot/mlxtend_1708368176260/work
mpmath @ file:///C:/b/abs_7833jrbiox/croot/mpmath_1690848321154/work
multidict @ file:///C:/b/abs_44ido987fv/croot/multidict_1701097803486/work
multiprocess @ file:///C:/b/abs_ca0rg6wl6_/croot/multiprocess_1668006439310/work
munkres==1.1.4
nest-asyncio @ file:///C:/b/abs_65d6lblmoi/croot/nest-asyncio_1708532721305/work
networkx @ file:///C:/b/abs_e6gi1go5op/croot/networkx_1690562046966/work
numexpr @ file:///C:/b/abs_afm0oewmmt/croot/numexpr_1683221839116/work
numpy @ file:///C:/Users/dev-admin/mkl/numpy_and_numpy_base_1682982345978/work
omegaconf==2.0.6
packaging @ file:///C:/b/abs_28t5mcoltc/croot/packaging_1693575224052/work
pandas @ file:///C:/miniconda3/conda-bld/pandas_1692299636855/work
parso @ file:///opt/conda/conda-bld/parso_1641458642106/work
pickleshare @ file:///tmp/build/80754af9/pickleshare_1606932040724/work
pillow @ file:///C:/b/abs_e22m71t0cb/croot/pillow_1707233126420/work
platformdirs @ file:///C:/b/abs_b6z_yqw_ii/croot/platformdirs_1692205479426/work
pooch @ file:///tmp/build/80754af9/pooch_1623324770023/work
portalocker==2.8.2
prompt-toolkit @ file:///C:/b/abs_68uwr58ed1/croot/prompt-toolkit_1704404394082/work
protobuf==3.20.3
psutil @ file:///C:/Windows/Temp/abs_b2c2fd7f-9fd5-4756-95ea-8aed74d0039flsd9qufz/croots/recipe/psutil_1656431277748/work
pure-eval @ file:///opt/conda/conda-bld/pure_eval_1646925070566/work
pyarrow @ file:///C:/b/abs_93i_y2dub4/croot/pyarrow_1707330894046/work/python
pyarrow-hotfix==0.6
pycparser==2.21
Pygments @ file:///C:/b/abs_fay9dpq4n_/croot/pygments_1684279990574/work
pyparsing @ file:///C:/Users/BUILDE~1/AppData/Local/Temp/abs_7f_7lba6rl/croots/recipe/pyparsing_1661452540662/work
PySocks @ file:///C:/ci/pysocks_1605287845585/work
python-dateutil @ file:///tmp/build/80754af9/python-dateutil_1626374649649/work
python-slugify @ file:///tmp/build/80754af9/python-slugify_1620405669636/work
pytz @ file:///C:/b/abs_19q3ljkez4/croot/pytz_1695131651401/work
pywin32==305.1
PyYAML @ file:///C:/b/abs_782o3mbw7z/croot/pyyaml_1698096085010/work
pyzmq @ file:///C:/b/abs_89aq69t0up/croot/pyzmq_1705605705281/work
regex @ file:///C:/b/abs_d5e2e5uqmr/croot/regex_1696515472506/work
requests @ file:///C:/b/abs_474vaa3x9e/croot/requests_1707355619957/work
responses @ file:///tmp/build/80754af9/responses_1619800270522/work
rich @ file:///C:/b/abs_09j2g5qnu8/croot/rich_1684282185530/work
sacrebleu==2.4.0
safetensors @ file:///C:/b/abs_88nwhm1qj3/croot/safetensors_1708633899663/work
scikit-learn @ file:///C:/b/abs_daon7wm2p4/croot/scikit-learn_1694788586973/work
scipy==1.10.1
seaborn @ file:///C:/b/abs_68ltdkoyoo/croot/seaborn_1673479199997/work
six @ file:///tmp/build/80754af9/six_1644875935023/work
stack-data @ file:///opt/conda/conda-bld/stack_data_1646927590127/work
sympy @ file:///C:/b/abs_82njkonm7f/croot/sympy_1701397685028/work
tabulate==0.9.0
tensorboardX @ file:///tmp/build/80754af9/tensorboardx_1621440489103/work
text-unidecode @ file:///Users/ktietz/demo/mc3/conda-bld/text-unidecode_1629401354553/work
threadpoolctl @ file:///Users/ktietz/demo/mc3/conda-bld/threadpoolctl_1629802263681/work
tokenizers @ file:///C:/b/abs_ba1xnwbavr/croot/tokenizers_1708633883417/work
torch==2.2.1
torchaudio==2.2.1
torchvision==0.17.1
tornado @ file:///C:/b/abs_0cbrstidzg/croot/tornado_1696937003724/work
tqdm @ file:///C:/b/abs_f76j9hg7pv/croot/tqdm_1679561871187/work
traitlets @ file:///C:/b/abs_e5m_xjjl94/croot/traitlets_1671143896266/work
transformers @ file:///C:/b/abs_495035xqf4/croot/transformers_1708700635762/work
typing_extensions @ file:///C:/b/abs_72cdotwc_6/croot/typing_extensions_1705599364138/work
tzdata @ file:///croot/python-tzdata_1690578112552/work
Unidecode @ file:///tmp/build/80754af9/unidecode_1614712377438/work
urllib3 @ file:///C:/b/abs_4etpfrkumr/croot/urllib3_1707770616184/work
wcwidth @ file:///Users/ktietz/demo/mc3/conda-bld/wcwidth_1629357192024/work
win-inet-pton @ file:///C:/ci/win_inet_pton_1605306167264/work
xxhash @ file:///C:/b/abs_e59ry5bslh/croot/python-xxhash_1667919514787/work
yarl @ file:///C:/b/abs_8bxwdyhjvp/croot/yarl_1701105248152/work
zipp @ file:///C:/b/abs_b0beoc27oa/croot/zipp_1704206963359/work
```

When I use your code above, it works properly. And after trying a lot times , I deem I found the cause of the bug: 
training_args = TrainingArguments("test-trainer", evaluation_strategy="epoch")
Everytime I add this to my code, then the same bug occurs. When I delete this sentence, the code works properly.

Commit message: Add test

Patch to review:
diff --git a/src/accelerate/state.py b/src/accelerate/state.py
index 3e3a66d5d18..9ec38714c8f 100644
--- a/src/accelerate/state.py
+++ b/src/accelerate/state.py
@@ -797,6 +797,18 @@ def default_device(self) -> torch.device:
         else:
             return torch.device("cpu")
 
+    def __getattr__(self, name: str):
+        try:
+            # First try the normal way
+            return object.__getattribute__(self, name)
+        except AttributeError:
+            # If not found, give a more contextualized answer
+            raise AttributeError(
+                f"`PartialState` object has no attribute `{name}`. "
+                "This can happen if the state was reset and using an old reference. "
+                "If you are using an `Accelerator`, make sure to re-initialize it."
+            )
+
 
 class AcceleratorState:
     """
@@ -1060,6 +1072,19 @@ def local_main_process_first(self):
     def print(self, *args, **kwargs):
         PartialState().print(*args, **kwargs)
 
+    def __getattr__(self, name: str):
+        try:
+            # First try the normal way
+            return object.__getattribute__(self, name)
+        except AttributeError:
+            # If not found, give a more contextualized answer
+            # adjust err message
+            raise AttributeError(
+                f"`AcceleratorState` object has no attribute `{name}`. "
+                "This can happen if the state was reset and using an old reference. "
+                "If you are using an `Accelerator`, make sure to re-initialize it."
+            )
+
 
 class GradientState:
     """
diff --git a/src/accelerate/test_utils/scripts/test_script.py b/src/accelerate/test_utils/scripts/test_script.py
index d8298a5c667..b28d5825619 100644
--- a/src/accelerate/test_utils/scripts/test_script.py
+++ b/src/accelerate/test_utils/scripts/test_script.py
@@ -22,6 +22,7 @@
 from pathlib import Path
 
 import numpy as np
+import pytest
 import torch
 from torch.utils.data import DataLoader, Dataset
 
@@ -690,6 +691,18 @@ def test_trigger():
     assert accelerator.check_trigger() is False
 
 
+def test_reinstantiated_state():
+    AcceleratorState._reset_state()
+    simple_model = torch.nn.Linear(1, 1)
+    # First define an accelerator
+    accelerator = Accelerator()
+    # Then call `reset_state`, breaking the state existing in the accelerator
+    AcceleratorState._reset_state()
+    # Now try and prepare a simple model, should raise the custom error early
+    with pytest.raises(AttributeError, match="This can happen if the state was reset and using an old reference."):
+        accelerator.prepare(simple_model)
+
+
 def main():
     accelerator = Accelerator()
     state = accelerator.state
@@ -755,6 +768,10 @@ def main():
         print("\n**Breakpoint trigger test**")
     test_trigger()
 
+    if state.local_process_index == 0:
+        print("\n**Test reinstantiated state**")
+    test_reinstantiated_state()
+
 
 if __name__ == "__main__":
     main()


Code review comments:
diff_hunk: @@ -797,6 +797,18 @@ def default_device(self) -> torch.device:
         else:
             return torch.device("cpu")
 
+    def __getattr__(self, name: str):
+        try:
+            # First try the normal way
+            return object.__getattribute__(self, name)
path: src/accelerate/state.py
text: @user1:
Trying to think this through: When we hit `__getattr__`, we already know the attribute wasn't found, right? So can we ever successfully `return object.__getattribute__(self, name)`? Or can we raise the below error right away?

@author:
Can confirm that yes we can, `getattr` does not know by this point, it's what is immediately hit when doing `getattr()`. This then is what does the actual getattr check. 

@user1:
Hmm, are you sure? I modified the method like this:

```python
    def __getattr__(self, name: str):
        try:
            # First try the normal way
            attr = object.__getattribute__(self, name)
        except AttributeError as e:
            if name in self._known_attrs:
                # If not found, give a more contextualized answer
                raise AttributeError(
                    f"`PartialState` object has no attribute `{name}`. "
                    "This happens if `PartialState._reset_state()` was called and "
                    "an `Accelerator` or `PartialState` was not reinitialized."
                )
            else:
                raise e
        breakpoint()
        return attr
```

and never hit the breakpoint when running `test_state_after_reset` or any other test. Attribute access goes via `__getattribute__` and only calls `__getattr__` when the former fails with an `AttributeError`. Therefore, I don't think this `__getattribute__` can succeed.

I guess one reason to still leave it there is so that we can re-trigger the `AttributeError` and re-raise it instead of directly raising `AttributeError`, though I can't come up with a scenario where this is useful.

@author:
So this is because the implementation needs to be under `__getattribute__` via `__getattr__`, as you said. Thanks :)