Repository: jazzband/django-oauth-toolkit

Issue:
fix(tasks): fix error caused by relative import
## Description of the Change

Running `oauth2_provider.tasks.clear_tokens` results in an error e.g.:
```python
>>> from oauth2_provider.tasks import clear_tokens
>>> clear_tokens()
Traceback (most recent call last):
  File "[python3.9]/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
  File "[site-packages]/celery/local.py", line 188, in __call__
    return self._get_current_object()(*a, **kw)
  File "[site-packages]/celery/app/task.py", line 392, in __call__
    return self.run(*args, **kwargs)
  File "[site-packages]/oauth2_provider/tasks.py", line 6, in clear_tokens
    from ...models import clear_expired  # noqa
ImportError: attempted relative import beyond top-level package
```

This update fixes the import path.

## Checklist

<!-- Replace '[ ]' with '[x]' to indicate that the checklist item is completed. -->
<!-- You can check the boxes now or later by just clicking on them. -->

- [x] PR only contains one change (considered splitting up PR)
- [ ] unit-test added
- [ ] documentation updated
- [ ] `CHANGELOG.md` updated (only for user relevant changes)
- [ ] author name in `AUTHORS`



Discussion:
Closing as not applicable.

I first saw this failing in production running inside celery. From Sentry:

<img width="491" alt="Screenshot 2022-03-18 at 20 28 56" src="https://user-images.githubusercontent.com/96283/159062351-273b1e40-56b0-420c-a6c9-e1ca19ef9810.png">

My example was to illustrate what's going on. But `clear_tokens` too is a regular function, so I see no reason why it should fail when called inside the shell.

Regardless, the reason why the import is failing is because `clear_expired` is imported from `...models`, but `tasks` and `models` are on the same level. At worst, the path would be `.models` i.e. `from .models import clear_expired`.
clear_tokens works as designed for use with celery-beat. I've tested it as such even though I don't use celery. It is not "regular" function as it is preceded by the celery `@shared_task` decorator. It is not meant to be invoked directlyl.
@n2ygk why are you arguing it works as indended when I just offered proof that it doesn't, and even upon reviewing the code it becomes clear the import can't possibly work since it looks up the function outside of the package?
> @n2ygk why are you arguing it works as indended when I just offered proof that it doesn't, and even upon reviewing the code it becomes clear the import can't possibly work since it looks up the function outside of the package?

Good question. So how come it worked for me and the original contributor @Natureshadow? Can you please help me out here and document how this is supposed to be configured with celery? I had asked @Natureshadow to do so but he just provided a link so I configured celery in a test app as best I could and didn't see any errors and that the task showed up in the django admin intervface. I suspect that the [@shared_task](https://github.com/celery/celery/blob/25ca389039f667cdf940c9efe44bdefd1cf70f30/celery/app/__init__.py#L24) decorator is changing the relative path location somehow but it's a bit over my head when reading it.

Perhaps you can share a full sample app configuration that demonstrates the problem since it is not reproducible given this report alone and I frankly don't have unlimited time or knowledge to debug this. Cutting and pasting an error message without context is not enough.

Given the other conflict with Huey #1114 I'm seriously considering reverting #1070 and leaving it up to someone to document how to add a Celery or Huey autoloaded tasks rather than including it in this package.


# [Codecov](https://codecov.io/gh/jazzband/django-oauth-toolkit/pull/1123?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband) Report
> Merging [#1123](https://codecov.io/gh/jazzband/django-oauth-toolkit/pull/1123?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband) (559478e) into [master](https://codecov.io/gh/jazzband/django-oauth-toolkit/commit/c48b7516fe6aebf89c5964e4855ecf129274c3c3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband) (c48b751) will **not change** coverage.
> The diff coverage is `0.00%`.

```diff
@@           Coverage Diff           @@
##           master    #1123   +/-   ##
=======================================
  Coverage   96.62%   96.62%           
=======================================
  Files          32       32           
  Lines        1806     1806           
=======================================
  Hits         1745     1745           
  Misses         61       61           
```


| [Impacted Files](https://codecov.io/gh/jazzband/django-oauth-toolkit/pull/1123?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband) | Coverage Δ | |
|---|---|---|
| [oauth2\_provider/tasks.py](https://codecov.io/gh/jazzband/django-oauth-toolkit/pull/1123/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband#diff-b2F1dGgyX3Byb3ZpZGVyL3Rhc2tzLnB5) | `0.00% <0.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/jazzband/django-oauth-toolkit/pull/1123?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/jazzband/django-oauth-toolkit/pull/1123?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband). Last update [c48b751...559478e](https://codecov.io/gh/jazzband/django-oauth-toolkit/pull/1123?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband).

> Good question. So how come it worked for me and the original contributor @Natureshadow?

I don't think it did, really. I don't think you're liars either, but I checked https://github.com/jazzband/django-oauth-toolkit/pull/1070 and I see no proof it ever worked. Both you and @Natureshadow confirm the task is discovered, but none of you attested it actually ran. For example, at one point you're [saying](https://github.com/jazzband/django-oauth-toolkit/pull/1070#issuecomment-1015804567):

> And see the admin interface and was able to autodiscover the task. I didn't get it completely working (econnrefused) but close enough to understand what I needed to.

---

Detective work aside, in my experience with Celery I'd say simply importing the task and running it inside the shell ought to do it. [unbound] celery tasks are regular functions and can be called directly like any other function.

Initially I started writing a smoke test, but I was faced with the choice of adding celery to the requirements or manipulating `sys.modules`, so I abandoned the idea. I agree to the point made by @Natureshadow in https://github.com/jazzband/django-oauth-toolkit/pull/1070#issuecomment-1006073286:

>  I don't think it is worth adding that complexity to the test suite

But an alternative that would work as a smoke test would be to move the import statement at the top of the file. If there was an issue it would fail a lot sooner.

---

I don't have experience with Huey. Regardless, personally I'm ok with reverting the original commit if it's causing problems.
> > Good question. So how come it worked for me and the original contributor @Natureshadow?
> 
> I don't think it did, really. I don't think you're liars either, but I checked #1070 and I see no proof it ever worked. Both you and @Natureshadow confirm the task is discovered, but none of you attested it actually ran. For example, at one point you're [saying](https://github.com/jazzband/django-oauth-toolkit/pull/1070#issuecomment-1015804567):

You will note that the import statement is *inside* the function definition not top-level in the tasks.py file. As such, it is not executed in the context of importing tasks.py but in the context of the decorator function for which the function definition is an argument. 
@n2ygk I reported a bug, showed proof of it and offered instructions to reproduce it, showed there was no proof that it ever worked, and offered to fix it. But rather than listening and possibly executing the code, it seems to me you're looking to argue and trying to be right or something like that. Whatever it is, it's not helping advance this conversation and not making the celery task run either. As such, I'll take a step back until we can have a reasonable conversation. Do let me know if you’re willing to do that.

Luckily, in my project I'm not using celery’s autodiscover feature, so I'll set up a custom task that imports `clear_expired` from `oauth2_provider.models`, where it actually lives, not the root of the virtual env like the builtin `clear_tokens` does.
@alexei I asked you to provide an example that reproduces the bug in context. You did not.
@n2ygk I showed you a screenshot with the error in production running in celery beat, and offered simple steps to reproduce it in the shell. Calling an unbound celery task directly is reasonable use. That said, I think I did my part of the deal. Besides, the error literally says you're importing from the outside of the package and the extra dots confirm it. If you don't connect the dots, there's nothing I can do about it.
@alexei With **tremendous amounts of egg on my face**, I have connected the dots and apologize profusely. It seems @Natureshadow supplied an untested PR #1070.  I was sticking by his word over yours. That was a mistake.

I've configured Celery and reproduced the error and applied your fix and have documented how to use it [here](https://github.com/n2ygk/dot-tutorial/blob/a3cb8ae62772b11e078b82e21c05edfe31021e95/README.md#add-celery-automated-task-discovery).

I'll be reverting #1070 and documenting how to do instead add the `clear_tokens` task into an app that uses DOT.


Resolving patch:
diff --git a/oauth2_provider/tasks.py b/oauth2_provider/tasks.py
deleted file mode 100644
--- a/oauth2_provider/tasks.py
+++ /dev/null
@@ -1,8 +0,0 @@
-from celery import shared_task
-
-
-@shared_task
-def clear_tokens():
-    from ...models import clear_expired  # noqa
-
-    clear_expired()


Test patch: