Repository: nalepae/pandarallel

Issue:
parallel_apply on empty dataframe fails with confusing error
I occasionally have code that should fail earlier because I've accidentally created an empty dataframe, however I've found that `parallel_apply` fails when `apply` does not on these, with an error from `multiprocessing`. I'm not sure if the desired behavior for `parallel_apply` on an empty dataframe is for it to fail, but if it is then it would be nice to have that explicitly handled as it can be rather confusing to figure out what's going on.

Here's a [gist](https://gist.github.com/zachary-jablons-okcupid/9c0e48b0eae286d585443e5030a77f5f) that reproduces this issue, and the error that I get is below:
```
INFO: Pandarallel will run on 48 workers.
INFO: Pandarallel will use Memory file system to transfer data between the main process and workers.
Traceback (most recent call last):
  File "empty_error.py", line 13, in <module>
    pandarallel_apply = empty_df.a.parallel_apply(lambda x: x + 1)
  File "/home/u1/zach/.pyenv/versions/py37/lib/python3.7/site-packages/pandarallel/pandarallel.py", line 447, in closure
    nb_workers, worker_init, (prepare_worker(use_memory_fs)(worker),),
  File "/home/u1/zach/.pyenv/versions/3.7.5/lib/python3.7/multiprocessing/context.py", line 119, in Pool
    context=self.get_context())
  File "/home/u1/zach/.pyenv/versions/3.7.5/lib/python3.7/multiprocessing/pool.py", line 169, in __init__
    raise ValueError("Number of processes must be at least 1")
ValueError: Number of processes must be at least 1
```

It's not obvious that this error is due to an empty dataframe. I think what happens is that `nb_workers` is set to `0` [here](https://github.com/nalepae/pandarallel/blob/master/pandarallel/pandarallel.py#L443).

This is on Python 3.7.5 with
```
pandas==1.1.3
pandarallel==1.5.1
```

Thanks,
Zach

Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/nalepae/pandarallel/pull/245?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Manu+NALEPA) Report
> Merging [#245](https://app.codecov.io/gh/nalepae/pandarallel/pull/245?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Manu+NALEPA) (3bfbc1f) into [master](https://app.codecov.io/gh/nalepae/pandarallel/commit/46fc0e542d91e3508c5ccb63bdcea0827a760744?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Manu+NALEPA) (46fc0e5) will **decrease** coverage by `12.69%`.
> The diff coverage is `50.00%`.

```diff
@@             Coverage Diff             @@
##           master     #245       +/-   ##
===========================================
- Coverage   91.37%   78.69%   -12.69%     
===========================================
  Files          12       12               
  Lines         580      582        +2     
===========================================
- Hits          530      458       -72     
- Misses         50      124       +74     
```


| [Impacted Files](https://app.codecov.io/gh/nalepae/pandarallel/pull/245?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Manu+NALEPA) | Coverage Δ | |
|---|---|---|
| [pandarallel/utils.py](https://app.codecov.io/gh/nalepae/pandarallel/pull/245?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Manu+NALEPA#diff-cGFuZGFyYWxsZWwvdXRpbHMucHk=) | `88.88% <50.00%> (-11.12%)` | :arrow_down: |

... and [8 files with indirect coverage changes](https://app.codecov.io/gh/nalepae/pandarallel/pull/245/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Manu+NALEPA)

:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Manu+NALEPA)

till-m: Hi @Mithil467,

thanks for the PR. Could you please add a test for this case?
Mithil467: Sure. I noticed that when progress_bar is True, we get ZeroDivisionError. In order to fix that, I would need to know what we expect UI wise. I personally like [2] more than [1] as it gives a sense of success, but would like to know your opinions or if you want to do things differently.

Consoles:

1. ![image](https://github.com/nalepae/pandarallel/assets/41439841/8d5d8f98-9636-4875-aa3b-b366bbc5d438)
2. ![image](https://github.com/nalepae/pandarallel/assets/41439841/3e4fb15d-a563-4124-9ca0-f19803ed6629)

Notebook:

1. ![image](https://github.com/nalepae/pandarallel/assets/41439841/66d3f0b7-4f7c-4497-b13c-f612d5555f0e)
2. ![image](https://github.com/nalepae/pandarallel/assets/41439841/6297f304-3388-4b98-b5fd-5a3b4dec07e6)


till-m: Personally, I prefer option 1.: The grey bar is neither failure (red) nor success (green), similarly, the processing of the empty DataFrame didn't really succeed or fail.
@nalepae do you have an opinion on the matter?
Mithil467: @till-m @nalepae If you say so, I can make the necessary changes to implement option [1]. Should I go ahead?
SiRumCz: Hi dev team, any updates on this one?
till-m: My apologies, I am not maintaining this project anymore, hence me not responding. But I can make an exception and see this PR through.

I would suggest going with [1]. @Mithil467 could you kindly ping me when/if you've implemented that?
Mithil467: @till-m @nalepae I have implemented the progress bar change to type [1]. Please review.
nalepae: **Pandaral·lel** is looking for a maintainer!
If you are interested, please open an GitHub issue.

Patch:
diff --git a/pandarallel/progress_bars.py b/pandarallel/progress_bars.py
index ffaa738..ec6602c 100644
--- a/pandarallel/progress_bars.py
+++ b/pandarallel/progress_bars.py
@@ -90,7 +90,10 @@ class ProgressBarsConsole(ProgressBars):
         self.__lines = []
 
     def __update_line(self, done: int, total: int) -> str:
-        percent = done / total
+        if total == 0:
+            percent = 0
+        else:
+            percent = done / total
         bar = (":" * int(percent * 40)).ljust(40, " ")
         percent = round(percent * 100, 2)
         format = " {percent:6.2f}% {bar:s} | {done:8d} / {total:8d} |"
@@ -155,13 +158,15 @@ class ProgressBarsNotebookLab(ProgressBars):
         for index, value in enumerate(values):
             bar, label = self.__bars[index].children
 
+            label.value = "{} / {}".format(value, bar.max)
+            
             bar.value = value
-            bar.description = "{:.2f}%".format(value / bar.max * 100)
 
             if value >= bar.max:
                 bar.bar_style = "success"
 
-            label.value = "{} / {}".format(value, bar.max)
+            if bar.max != 0:
+                bar.description = "{:.2f}%".format(bar.value / bar.max * 100)
 
     def set_error(self, index: int) -> None:
         """Set a bar on error"""
diff --git a/pandarallel/utils.py b/pandarallel/utils.py
index 1f7739c..1784ad8 100644
--- a/pandarallel/utils.py
+++ b/pandarallel/utils.py
@@ -31,6 +31,9 @@ def chunk(nb_item: int, nb_chunks: int, start_offset=0) -> List[slice]:
     >>> chunks
     [slice(0, 26, None), slice(26, 52, None), slice(52, 78, None), slice(78, 103, None)]
     """
+    if nb_item == 0:
+        return [slice(0)]
+    
     if nb_item <= nb_chunks:
         return [slice(max(0, idx - start_offset), idx + 1) for idx in range(nb_item)]
 


Test patch:
diff --git a/tests/test_pandarallel.py b/tests/test_pandarallel.py
index 0f66fcf..0f91c32 100644
--- a/tests/test_pandarallel.py
+++ b/tests/test_pandarallel.py
@@ -210,6 +210,20 @@ def test_dataframe_apply_invalid_axis(pandarallel_init):
 
     with pytest.raises(ValueError):
         df.parallel_apply(lambda x: x, axis="invalid")
+    
+def test_empty_dataframe_apply_axis_0(pandarallel_init, func_dataframe_apply_axis_0):
+    df = pd.DataFrame()
+
+    res = df.apply(func_dataframe_apply_axis_0)
+    res_parallel = df.parallel_apply(func_dataframe_apply_axis_0)
+    assert res.equals(res_parallel)
+
+def test_empty_dataframe_apply_axis_1(pandarallel_init, func_dataframe_apply_axis_1):
+    df = pd.DataFrame()
+
+    res = df.apply(func_dataframe_apply_axis_1)
+    res_parallel = df.parallel_apply(func_dataframe_apply_axis_1)
+    assert res.equals(res_parallel)
 
 
 def test_dataframe_applymap(pandarallel_init, func_dataframe_applymap, df_size):
@@ -238,6 +252,13 @@ def test_series_apply(pandarallel_init, func_series_apply, df_size):
     res_parallel = df.a.parallel_apply(func_series_apply, args=(2,), bias=3)
     assert res.equals(res_parallel)
 
+def test_empty_series_apply(pandarallel_init, func_series_apply):
+    df = pd.DataFrame(dict(a=[]))
+
+    res = df.a.apply(func_series_apply, args=(2,), bias=3)
+    res_parallel = df.a.parallel_apply(func_series_apply, args=(2,), bias=3)
+    assert res.equals(res_parallel)
+
 
 def test_series_rolling_apply(pandarallel_init, func_series_rolling_apply, df_size):
     df = pd.DataFrame(dict(a=np.random.randint(1, 8, df_size), b=list(range(df_size))))