Issue 1:
Hey Guys, I just noticed an upgrade to 4.2.0 from 4.1.1 causes breaking changes. Any ideas where this was intoduced?

System:

OSX 10.13.5

Anaconda Python 3.7

Steps to reproduce

Install:

$ pip install Celery=4.2.0

Code:

celery_amqp_url=os.getenv("CELERY_AMQP_URL")

app = Celery('magic_worker',broker=celery_amqp_url)

@task
def parse(uid):
print('parsing: %s' % uid)

Run:

$ celery -A magic_worker worker

Expected behavior

Celery task is consumed.

Actual behavior

Error throw:

[2018-07-16 21:04:52,098: CRITICAL/MainProcess] Unrecoverable error: TypeError("'NoneType' object is not iterable",)
Traceback (most recent call last):
File "python3.5/site-packages/celery/worker/worker.py", line 205, in start
self.blueprint.start(self)
File "python3.5/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "python3.5/site-packages/celery/bootsteps.py", line 369, in start
return self.obj.start()
File "python3.5/site-packages/celery/worker/consumer/consumer.py", line 322, in start
blueprint.start(self)
File "python3.5/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "python3.5/site-packages/celery/worker/consumer/consumer.py", line 598, in start
c.loop(*c.loop_args())
File "python3.5/site-packages/celery/worker/loops.py", line 91, in asynloop
next(loop)
File "python3.5/site-packages/kombu/asynchronous/hub.py", line 354, in create_loop
cb(*cbargs)
File "python3.5/site-packages/kombu/transport/base.py", line 236, in on_readable
reader(loop)
File "python3.5/site-packages/kombu/transport/base.py", line 218, in _read
drain_events(timeout=0)
File "python3.5/site-packages/amqp/connection.py", line 491, in drain_events
while not self.blocking_read(timeout):
File "python3.5/site-packages/amqp/connection.py", line 497, in blocking_read
return self.on_inbound_frame(frame)
File "python3.5/site-packages/amqp/method_framing.py", line 77, in on_frame
callback(channel, msg.frame_method, msg.frame_args, msg)
File "python3.5/site-packages/amqp/connection.py", line 501, in on_inbound_method
method_sig, payload, content,
File "python3.5/site-packages/amqp/abstract_channel.py", line 128, in dispatch_method
listener(*args)
File "python3.5/site-packages/amqp/channel.py", line 1597, in _on_basic_deliver
fun(msg)
File "python3.5/site-packages/kombu/messaging.py", line 624, in _receive_callback
return on_m(message) if on_m else self.receive(decoded, message)
File "python3.5/site-packages/celery/worker/consumer/consumer.py", line 572, in on_task_received
callbacks,
File "python3.5/site-packages/celery/worker/strategy.py", line 200, in task_message_handler
handle(req)
File "python3.5/site-packages/celery/worker/worker.py", line 223, in _process_task_sem
return self._quick_acquire(self._process_task, req)
File "python3.5/site-packages/kombu/asynchronous/semaphore.py", line 62, in acquire
callback(*partial_args, **partial_kwargs)
File "python3.5/site-packages/celery/worker/worker.py", line 228, in _process_task
req.execute_using_pool(self.pool)
File "python3.5/site-packages/celery/worker/request.py", line 520, in execute_using_pool
time_limit, soft_time_limit = self.time_limits
TypeError: 'NoneType' object is not iterable

Work Around

Downgrade to celery 4.1.1

pip uninstall Celery
pip install Celery=4.1.1

Issue 2:
I am getting an error in the stack trace below. It looks like this line is referencing the wrong field name: https://github.com/celery/celery/blob/master/celery/worker/request.py#L520. I don't think time_limits exists for this Request. The only place in celery codebase that I see using create_request_cls is here

celery/celery/worker/strategy.py

Line 130
in
47ca2b4

Req = create_request_cls(Request, task, consumer.pool, hostname, eventer)

. It seems to be creating the Request object from task.Request base. According to the docs http://docs.celeryproject.org/en/latest/userguide/tasks.html#task-request task.Request has a field timelimit, not time_limits, hence the TypeError. worker.Request, on the other hand, does have time_limits http://docs.celeryproject.org/en/latest/reference/celery.worker.request.html.

The workaround I did to get my code working is basically to check if the variable is None here https://github.com/celery/celery/blob/master/celery/worker/request.py#L520 :

if not self.time_limits:
time_limit = default_time_limit
soft_time_limit = default_soft_time_limit
else:
time_limit, soft_time_limit = self.time_limits

I am not sure if I did something stupid and don't understand how the code is structured. Please point me at why it's not an error, if it is by design.

Stack trace:

[2018-07-16 06:09:46,229: CRITICAL/MainProcess] Unrecoverable error: TypeError("'NoneType' object is not iterable",)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line 205, in start
self.blueprint.start(self)
File "/usr/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/lib/python2.7/site-packages/celery/bootsteps.py", line 369, in start
return self.obj.start()
File "/usr/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 322, in start
blueprint.start(self)
File "/usr/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 598, in start
c.loop(*c.loop_args())
File "/usr/lib/python2.7/site-packages/celery/worker/loops.py", line 118, in synloop
qos.update()
File "/usr/lib/python2.7/site-packages/kombu/common.py", line 417, in update
return self.set(self.value)
File "/usr/lib/python2.7/site-packages/kombu/common.py", line 410, in set
self.callback(prefetch_count=new_value)
File "/usr/lib/python2.7/site-packages/celery/worker/consumer/tasks.py", line 47, in set_prefetch_count
apply_global=qos_global,
File "/usr/lib/python2.7/site-packages/kombu/messaging.py", line 558, in qos
apply_global)
File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1812, in basic_qos
wait=spec.Basic.QosOk,
File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 59, in send_method
return self.wait(wait, returns_tuple=returns_tuple)
File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 79, in wait
self.connection.drain_events(timeout=timeout)
File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 491, in drain_events
while not self.blocking_read(timeout):
File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 497, in blocking_read
return self.on_inbound_frame(frame)
File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 77, in on_frame
callback(channel, msg.frame_method, msg.frame_args, msg)
File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 501, in on_inbound_method
method_sig, payload, content,
File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 128, in dispatch_method
listener(*args)
File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1597, in _on_basic_deliver
fun(msg)
File "/usr/lib/python2.7/site-packages/kombu/messaging.py", line 624, in _receive_callback
return on_m(message) if on_m else self.receive(decoded, message)
File "/usr/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 572, in on_task_received
callbacks,
File "/usr/lib/python2.7/site-packages/celery/worker/strategy.py", line 200, in task_message_handler
handle(req)
File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line 228, in _process_task
req.execute_using_pool(self.pool)
File "/usr/lib/python2.7/site-packages/celery/worker/request.py", line 520, in execute_using_pool
time_limit, soft_time_limit = self.time_limits
TypeError: 'NoneType' object is not iterable

Checklist

celery v 4.2.0

Steps to reproduce

It's part of a much larger project, so I don't have a self-contained example yet.

Expected behavior

return a tuple of (None, None)

Actual behavior

TypeError