Issue 1:
It seems that the local cache feature introduced in v3.1.10 breaks the propagation of exceptions via celery.result.AsyncResult.get if the result is serialized with JSON (and possibly others, except pickle).

To reproduce:

# test.py

import sys
import traceback

from celery import Celery

app = Celery('test')
class Config:
BROKER_URL = 'amqp://guest:guest@127.0.0.1:5672/celery'
CELERY_RESULT_BACKEND = 'amqp'
CELERY_RESULT_SERIALIZER = 'json'
app.config_from_object(Config)

@app.task
def exception_raising_task(to_raise=None):
if to_raise is not None:
raise RuntimeError(to_raise)
return True

if __name__ == '__main__':
result = exception_raising_task.delay()
result.get() # returns True

result = exception_raising_task.delay('test')
try:
result.get() # raises TypeError at celery.backends.base:exception_to_python
# displayed by print_exception() below
except Exception as e:
traceback.print_exception(*sys.exc_info())
finally:
result.get() # correctly raises RuntimeError propagated from the task

The exact traceback of the broken propagation is:

>>> result = exception_raising_task.delay('test')
>>> result.get()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/celery-dev/local/lib/python2.7/site-packages/celery-3.1.10-py2.7.egg/celery/result.py", line 171, in get
self._get_task_meta() # update self._cache
File "/opt/celery-dev/local/lib/python2.7/site-packages/celery-3.1.10-py2.7.egg/celery/result.py", line 332, in _get_task_meta
return self._set_cache(meta)
File "/opt/celery-dev/local/lib/python2.7/site-packages/celery-3.1.10-py2.7.egg/celery/result.py", line 339, in _set_cache
d['result'] = self.backend.exception_to_python(d['result'])
File "/opt/celery-dev/local/lib/python2.7/site-packages/celery-3.1.10-py2.7.egg/celery/backends/base.py", line 171, in exception_to_python
from_utf8(exc['exc_type']), __name__)(exc['exc_message'])
TypeError: sequence index must be integer, not 'str'

Upon inspection, this seems to be caused by a duplicate call of celery.result.AsyncResult._set_cache in celery.result.AsyncResult._get_task_meta, which could be left there by a mistake:

# celery/result.py

def _get_task_meta(self):
if self._cache is None:
meta = self.backend.get_task_meta(self.id)
if meta:
state = meta['status']
if state == states.SUCCESS or state in states.PROPAGATE_STATES:
self._set_cache(meta) # <--- this call substitutes meta['result'] field, so the following will fail
return self._set_cache(meta)
return meta
return self._cache

So, the problem should be fixed by removing the line highlighted above (works for my local environment).

Issue 2:
celery inspect conf command always returns empty BROKER_TRANSPORT_OPTIONS even when it seems set properly with celery report command.

I'm not sure this setting is really working on each active workers. But I see some 1h+ tasks are running on duplicately on two workers.

Output of celery report

redash@02f67972c8fb:/app$ celery report -A redash.worker

software -> celery:3.1.23 (Cipater) kombu:3.0.37 py:2.7.12
billiard:3.3.0.23 redis:2.10.5
platform -> system:Linux arch:64bit, ELF imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:redis results:redis://***

CELERY_ACKS_LATE: True
CELERYBEAT_SCHEDULE: {
'cleanup_query_results': { 'schedule': datetime.timedelta(0, 300),
'task': 'redash.tasks.cleanup_query_results'},
'cleanup_tasks': { 'schedule': datetime.timedelta(0, 300),
'task': 'redash.tasks.cleanup_tasks'},
'refresh_queries': { 'schedule': datetime.timedelta(0, 30),
'task': 'redash.tasks.refresh_queries'},
'refresh_schemas': { 'schedule': datetime.timedelta(0, 1800),
'task': 'redash.tasks.refresh_schemas'},
'version_check': { 'schedule': <crontab: 14 1 * * * (m/h/d/dM/MY)>,
'task': 'redash.tasks.version_check'}}
BROKER_TRANSPORT_OPTIONS: {
'visibility_timeout': 16200}
CELERY_TASK_RESULT_EXPIRES: 43200
CELERY_TIMEZONE: 'UTC'
CELERYD_PREFETCH_MULTIPLIER: 1
CELERY_RESULT_BACKEND: 'redis://***'

Output of celery inspect conf

redash@02f67972c8fb:/app$ celery inspect conf -A redash.worker
-> celery@6d971ca8001a: OK
{
"ADMINS": [],
"BROKER_CONNECTION_MAX_RETRIES": 100,
"BROKER_CONNECTION_RETRY": true,
"BROKER_CONNECTION_TIMEOUT": 4,
"BROKER_FAILOVER_STRATEGY": null,
"BROKER_HEARTBEAT": null,
"BROKER_HEARTBEAT_CHECKRATE": 3.0,
"BROKER_HOST": null,
"BROKER_LOGIN_METHOD": null,
"BROKER_PASSWORD": "********",
"BROKER_POOL_LIMIT": 10,
"BROKER_PORT": null,
"BROKER_TRANSPORT": null,
"BROKER_TRANSPORT_OPTIONS": {},

Steps to reproduce

Set BROKER_TRANSPORT_OPTIONS with config file or in your code.

Run celery inspect conf

Expected behavior

Returns proper not empty BROKER_TRANSPORT_OPTIONS

Actual behavior

Returns empty BROKER_TRANSPORT_OPTIONS