Repository: spotify/luigi

Issue:
Feature Request: binary write mode for LocalTarget
I'd like to use `pickle.dump` to write to a LocalTarget after calling `open` in write mode, kind of like this: 

```
def run(self):
    some_object = object()
    local_target = self.output()  # An instance of LocalTarget
    with local_target.open(mode="w") as f:  # <-- binary write not supported
        pickle.dump(some_object, f)
```

Unfortunately this causes an error: `TypeError: write() argument must be str, not bytes` because the LocalTarget open method doesn't allow binary write mode. This appears to be a deliberate decision, as the [source code](https://luigi.readthedocs.io/en/stable/_modules/luigi/local_target.html#LocalTarget) for the `open` method just strips the `b` out of the mode argument. 

It's easy enough to work around this by using `self.output().path` as suggested by [this kind stranger](https://stackoverflow.com/a/44418914). Nonetheless I think it would be nice either to include this capability as part of the LocalTarget `open` method, or to throw a more informative error instead of just stripping out the `b`.
Feature Request: binary write mode for LocalTarget
I'd like to use `pickle.dump` to write to a LocalTarget after calling `open` in write mode, kind of like this: 

```
def run(self):
    some_object = object()
    local_target = self.output()  # An instance of LocalTarget
    with local_target.open(mode="w") as f:  # <-- binary write not supported
        pickle.dump(some_object, f)
```

Unfortunately this causes an error: `TypeError: write() argument must be str, not bytes` because the LocalTarget open method doesn't allow binary write mode. This appears to be a deliberate decision, as the [source code](https://luigi.readthedocs.io/en/stable/_modules/luigi/local_target.html#LocalTarget) for the `open` method just strips the `b` out of the mode argument. 

It's easy enough to work around this by using `self.output().path` as suggested by [this kind stranger](https://stackoverflow.com/a/44418914). Nonetheless I think it would be nice either to include this capability as part of the LocalTarget `open` method, or to throw a more informative error instead of just stripping out the `b`.


Discussion:
Hi,

I am not a maintainer, but I'm currently using these parts of Luigi heavily, and familiar with the code. If you create a PR, I am happy to review.
Hey @mwheeler-hdai , iirc this is something of a design decision due to how the atomic file writes work. Instead of using `format=UTF8` or whatever, use `format=Nop` - this fixes your problem. I'll try and dig up the issue that mentions this.

Edit: Here's an issue: #1647 and here's a PR: #2495 which unfortunately did not get merged. Maybe we could revisit this?
Or perhaps for now we just include a word or two about the option `format=Nop` in the docs for `LocalTarget`, or again in an error message at that point in the code. I feel like that's an acceptable middle ground, especially if this issue has been unsuccessfully raised before.
Would love any kind of improvement (perhaps just docs and code comment), the "silently remove the b" way is not ideal.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions.

Ah, crap. Forgot to comment on here. Could we re-open this? I'll try and write a PR for docs within the next week or so.
Hi,

I am not a maintainer, but I'm currently using these parts of Luigi heavily, and familiar with the code. If you create a PR, I am happy to review.
Hey @mwheeler-hdai , iirc this is something of a design decision due to how the atomic file writes work. Instead of using `format=UTF8` or whatever, use `format=Nop` - this fixes your problem. I'll try and dig up the issue that mentions this.

Edit: Here's an issue: #1647 and here's a PR: #2495 which unfortunately did not get merged. Maybe we could revisit this?
Or perhaps for now we just include a word or two about the option `format=Nop` in the docs for `LocalTarget`, or again in an error message at that point in the code. I feel like that's an acceptable middle ground, especially if this issue has been unsuccessfully raised before.
Would love any kind of improvement (perhaps just docs and code comment), the "silently remove the b" way is not ideal.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions.

Ah, crap. Forgot to comment on here. Could we re-open this? I'll try and write a PR for docs within the next week or so.

Resolving patch:
diff --git a/luigi/target.py b/luigi/target.py
--- a/luigi/target.py
+++ b/luigi/target.py
@@ -235,7 +235,8 @@ def open(self, mode):
 
         :param str mode: the mode `r` opens the FileSystemTarget in read-only mode, whereas `w` will
                          open the FileSystemTarget in write mode. Subclasses can implement
-                         additional options.
+                         additional options. Using `b` is not supported; initialize with
+                         `format=Nop` instead.
         """
         pass
 


Test patch: