Issue 1:
Checklist

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

Task invocation via .delay() not working, in our case is with django_templated_email

Possible Duplicates

None

Environment & Settings

Celery version:

celery==4.3.0

django-celery-beat==1.5.0

celery report Output:

/usr/local/lib/python3.6/site-packages/celery/platforms.py:801: RuntimeWarning: You're running the worker with superuser privileges: this is
absolutely not recommended!

Please specify a different user using the --uid option.

User information: uid=0 euid=0 gid=0 egid=0

uid=uid, euid=euid, gid=gid, egid=egid,

-------------- celery@worker-deployment-695fbd498-rmln6 v4.3.0 (rhubarb)
---- **** -----
--- * *** * -- Linux-4.14.138+-x86_64-with-debian-10.1 2019-11-12 17:00:00
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app: app:0x7f694a5ba160
- ** ---------- .> transport: redis://10.0.0.3:6379/0
- ** ---------- .> results: disabled://
- *** --- * --- .> concurrency: 1 (prefork)
-- ******* ---- .> task events: ON
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery

[tasks]
. accounting.tasks.generate_payment_receipt_by_collection_id
. loans.tasks.send_loan_approval_email
. loans.tasks.send_loan_approval_sms
. loans.tasks.send_loan_disbursement_sms
. loans.tasks.send_loan_rejection_email
. loans.tasks.send_loan_rejection_sms
. loans.tasks.send_loan_revision_sms
. loans.tasks.update_repayments_by_loan_id

[2019-11-12 17:00:01,232: INFO/MainProcess] Connected to redis://10.0.0.3:6379/0
[2019-11-12 17:00:01,313: INFO/MainProcess] mingle: searching for neighbors
[2019-11-12 17:00:02,423: INFO/MainProcess] mingle: sync with 1 nodes
[2019-11-12 17:00:02,424: INFO/MainProcess] mingle: sync complete
[2019-11-12 17:00:02,510: WARNING/MainProcess] /usr/local/lib/python3.6/site-packages/celery/fixups/django.py:202: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
warnings.warn('Using settings.DEBUG leads to a memory leak, never '
[2019-11-12 17:00:02,511: INFO/MainProcess] celery@worker-deployment-695fbd498-rmln6 ready.
[2019-11-12 17:00:27,433: INFO/MainProcess] missed heartbeat from celery@worker-deployment-594c7656c-llbh5
[2019-11-12 17:57:57,857: INFO/MainProcess] Received task: loans.tasks.send_loan_approval_sms[fbc8eefe-2155-4754-983a-2409ef2e544a]
[2019-11-12 17:57:58,008: INFO/ForkPoolWorker-1] Running loans.tasks task
[2019-11-12 17:57:59,486: INFO/ForkPoolWorker-1] Successfully sent SMS
[2019-11-12 17:57:59,487: INFO/ForkPoolWorker-1] <Response [200]>
[2019-11-12 17:58:00,761: ERROR/ForkPoolWorker-1] An error occured when sending an SMS
[2019-11-12 17:58:00,762: ERROR/ForkPoolWorker-1] Traceback
[2019-11-12 17:58:00,765: ERROR/ForkPoolWorker-1] 400 Client Error: for url: https://endpoint.api.infobip.com/sms/2/text/single
[2019-11-12 17:58:00,766: INFO/ForkPoolWorker-1] Task loans.tasks.send_loan_approval_sms[fbc8eefe-2155-4754-983a-2409ef2e544a] succeeded in 2.8980900809401646s: None

Steps to Reproduce

I encountered this issue when doing the following

I have a task definition like this

@shared_task
def send_loan_approval_email(loan_id):
from loans.models import Loan
from templated_email import get_templated_mail
loan = Loan.objects.get(pk=loan_id)

# some code here

email_borrower.send()
email_guarantor.send()

@shared_task
def send_loan_approval_sms(loan_id):
from loans.models import Loan
from app.services import sms_service
loan = Loan.objects.get(pk=loan_id)
log.info(f"Running {__name__} task")
sms_service.send(
# some code here
)

I have code invocation here

class Loan:
def approve(self, by=None):
send_loan_approval_sms.delay(self.id)
send_loan_approval_email.delay(self.id)

When I run approve() somehow the sms run but the email doesn't, but sometime it works too, sms consistently running, but the email sometime work sometime broken. Any idea why?

Required Dependencies

Minimal Python Version: N/A or Unknown

Minimal Celery Version: N/A or Unknown

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

asn1crypto==0.24.0
backports.ssl-match-hostname==3.7.0.1
bcrypt==3.1.6
cached-property==1.5.1
certifi==2019.3.9
cffi==1.12.3
chardet==3.0.4
cryptography==2.6.1
Django==1.11.26
django-storages==1.7.2
docker==3.7.2
docker-compose==1.24.0
docker-pycreds==0.4.0
dockerpty==0.4.1
docopt==0.6.2
enum34==1.1.6
functools32==3.2.3.post2
idna==2.7
ipaddress==1.0.22
jsonschema==2.6.0
paramiko==2.4.2
pyasn1==0.4.5
pycparser==2.19
PyNaCl==1.3.0
pytz==2019.3
PyYAML==3.13
requests==2.20.1
six==1.12.0
texttable==0.9.1
urllib3==1.24.2
websocket-client==0.56.0

Other Dependencies

N/A

Minimally Reproducible Test Case

Expected Behavior

Both send_loan_approval_sms and send_loan_approval_email should emitted to worker.

Actual Behavior

Only send_loan_approval_sms emitted to worker.

Issue 2:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Environment & Settings

Celery version: 4.4.2

celery report Output:

(base) C:\Users\headmaster\Desktop\WEB>celery -A tasks report

software -> celery:4.4.2 (cliffs) kombu:4.6.8 py:3.7.4
billiard:3.6.3.0 redis:3.4.1
platform -> system:Windows arch:64bit, WindowsPE
kernel version:10 imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:redis results:redis://localhost:6379/1

broker_url: 'redis://localhost:6379/0'
result_backend: 'redis://localhost:6379/1'

Steps to Reproduce

Required Dependencies

Minimal Python Version: 3.7

Minimal Celery Version: N/A or Unknown

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: win10

Minimal Broker Client Version: Redis 3.2.100

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

Other Dependencies

N/A

Minimally Reproducible Test Case

Expected Behavior

I was trying to add Celery asychronized feature to my simple flask web project as a flask beginner. After managing to install and launching celery, redis and related python pip thing, I got struggling with the delay() in the very beginning. It seemed I could not even get into the delay(). I am so frustrated, could you help me please?

I have app.py and tasks.py under my flask project. (At first I wrote everything in app.py, then I split the celery object out, which didn't help)

tasks.py:

from celery import Celery

celery = Celery('app', broker='redis://localhost:6379/0', backend='redis://localhost:6379/1')

@celery.task
def modeling_task():
print('yes')

app.py:

@app.route('/train', methods=['GET', 'POST'])
def train():
if request.method == "GET":
return render_template('train.html')
else: # when request POST
# check if the post request has the file part
if 'file' not in request.files:
return jsonify({"code": 500, "status": 'No file is uploaded!'})

if file and allowed_file(file.filename):
print(request.files['file'].filename)
print('before task')
task = modeling_task.delay()
print('after task')

return jsonify({"code": 200, "status": 'model training', "task_id": task.id})

return 'Uploading file is wrong'

If things work well, after the server receives a uploading file, the print results on console should be

Car_TEST.csv
before task
yes
after task

Actual Behavior

However, the console result is simply this

* Detected change in 'C:\\Users\\headmaster\\Desktop\\WEB\\tasks.py', reloading
* Restarting with stat
* Debugger is active!
* Debugger PIN: 308-608-393
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [22/Apr/2020 11:39:22] "GET /train HTTP/1.1" 200 -
Car_TEST.csv
before task

and the server got stuck in here, keeping enlessly running.

So it seems the server don't even run the "task = modeling_task.delay()", otherwise it should've print "yes" on the console, right?

Would it print "yes" in another place? Why did it get stuck?

Down below is the celery info:

(base) C:\Users\headmaster\Desktop\WEB>celery worker -A tasks.celery --loglevel=info

-------------- celery@LAPTOP-KLKJCK2F v4.4.2 (cliffs)
--- ***** -----
-- ******* ---- Windows-10-10.0.18362-SP0 2020-04-22 10:55:13
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: app:0x2526977ac08
- ** ---------- .> transport: redis://localhost:6379/0
- ** ---------- .> results: redis://localhost:6379/1
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery

[tasks]
. tasks.modeling_task

[2020-04-22 10:55:15,501: INFO/SpawnPoolWorker-1] child process 6692 calling self.run()
[2020-04-22 10:55:15,511: INFO/SpawnPoolWorker-2] child process 17736 calling self.run()
[2020-04-22 10:55:15,518: INFO/SpawnPoolWorker-3] child process 16320 calling self.run()
[2020-04-22 10:55:15,571: INFO/SpawnPoolWorker-4] child process 5608 calling self.run()
[2020-04-22 10:55:15,972: INFO/MainProcess] Connected to redis://localhost:6379/0
[2020-04-22 10:55:20,332: INFO/MainProcess] mingle: searching for neighbors
[2020-04-22 10:55:27,369: INFO/MainProcess] mingle: all alone
[2020-04-22 10:55:35,402: INFO/MainProcess] celery@LAPTOP-KLKJCK2F ready.

redis info:

PS D:\Applications\Redis> redis-server redis.conf
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.100 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
| `-._ `._ / _.-' | PID: 21420
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'

[21420] 22 Apr 10:38:22.506 # Server started, Redis version 3.2.100
[21420] 22 Apr 10:38:22.508 * DB loaded from disk: 0.000 seconds
[21420] 22 Apr 10:38:22.508 * The server is now ready to accept connections on port 6379
[21420] 22 Apr 10:43:23.025 * 10 changes in 300 seconds. Saving...
[21420] 22 Apr 10:43:23.030 * Background saving started by pid 6204
[21420] 22 Apr 10:43:23.231 # fork operation complete
[21420] 22 Apr 10:43:23.232 * Background saving terminated with success
[21420] 22 Apr 10:51:18.099 * 10 changes in 300 seconds. Saving...
[21420] 22 Apr 10:51:18.103 * Background saving started by pid 10116
[21420] 22 Apr 10:51:18.305 # fork operation complete
[21420] 22 Apr 10:51:18.306 * Background saving terminated with success
[21420] 22 Apr 10:56:19.022 * 10 changes in 300 seconds. Saving...
[21420] 22 Apr 10:56:19.026 * Background saving started by pid 11748
[21420] 22 Apr 10:56:19.227 # fork operation complete
[21420] 22 Apr 10:56:19.227 * Background saving terminated with success

wish you a best day!