When the PythonException is thrown inside the Java process on the worker,
won't that get logged? Isn't that just like any other exception that occurs
inside of an RDD compute(). ?


On Fri, Feb 1, 2013 at 11:01 AM, Josh Rosen <notifications@github.com>wrote:

> In python/pyspark/worker.py:
>
> > @@ -40,8 +41,13 @@ def main():
> >      else:
> >          dumps = dump_pickle
> >      iterator = read_from_pickle_file(sys.stdin)
> > -    for obj in func(split_index, iterator):
> > -        write_with_length(dumps(obj), old_stdout)
> > +    try:
> > +        for obj in func(split_index, iterator):
> > +           write_with_length(dumps(obj), old_stdout)
> > +    except Exception as e:
>
> Do you think that we should also log the inspection on the worker? It
> might help during debugging.
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/mesos/spark/pull/434/files#r2863481>.
>
>