Repository: qutebrowser/qutebrowser

Issue:
Close stdin for processes called from qutebrowser
When running this as userscript:

```bash
#!/bin/bash
buku --debug --add "$QUTE_URL"
echo "message-info 'Bookmark added to Buku!'" >> "$QUTE_FIFO"
```

[buku](https://github.com/jarun/Buku) just hangs, without producing any output (not even when adding `&> /tmp/buku.log` to the `buku` call).

Running `QUTE_URL=https://www.example.com/ ./buku.sh` works fine though, and every other python application I've seen so far works...

@jarun - do you have any idea what could be going wrong here?

cc @Chinggis6


Discussion:
Please provide the buku debug logs.
As said, there's no output at all.
OK. I think I would be able to reproduce this easily. Thanks for the issue report.
Thanks for having a look! :+1: Once you get qutebrowser to run, you should be able to just dump the above into a script, make it executable, and then run `:spawn -u ~/path/to/script.sh` in qutebrowser.
Oh, I need to install the browser too?
If you just define "$QUTE_URL" and run your script what happens?
How about this one?

```
#!/bin/bash
buku --debug --add "$QUTE_URL" >> /dev/null 2>&1
echo "message-info 'Bookmark added to Buku!'" >> "$QUTE_FIFO"
```
This seems to be related to buku being spawned from PyQt/Qt somehow.

With this minimal PyQt script:

```python
from PyQt5.QtCore import QProcess

p = QProcess()
p.setProcessChannelMode(QProcess.ForwardedChannels)
# p.start('echo', ['foo'])  # works as expected
p.start('buku', ['--debug', '--add', 'http://www.example.com/'])
ok = p.waitForFinished()
print(ok)
```

I get a hang until `waitForFinished` times out, and *then* get some output:

```
python3 ~/tmp/proc.py
False
[DEBUG] Version 2.9                                                                                      
─[florian@hooch]──[~]──[17-03-31]──[8:14]────┄
[DEBUG] netloc: www.example.com
[DEBUG] Starting new HTTP connection (1): www.example.com
[DEBUG] http://www.example.com:80 "GET / HTTP/1.1" 200 606
[DEBUG] Title: [Example Domain]
1. http://www.example.com/
   > Example Domain
```

When using the asynchronous code like inside qutebrowser, that'd mean it "waits" for buku to finish forever, without a timeout.

i.e. somehow Qt doesn't realize buku has finished, or buku is blocked by Qt somehow?!
We are using `sys.exit(exitval)`. Can you simply run `buku` in the script without any args and see what happens?
Ah, looks like buku waits until its input is closed when not started in a tty:

```python
# Handle piped input
def piped_input(argv, pipeargs=None):
    if not sys.stdin.isatty():
        pipeargs.extend(argv)
        for s in sys.stdin.readlines():
            pipeargs.extend(s.split())
```

Commenting this out in `main` makes things work:

```python
    try:
        piped_input(sys.argv, pipeargs)
    except KeyboardInterrupt:
        pass
```

Maybe qutebrowser should call [closeWriteChannel()](http://doc.qt.io/qt-5/qprocess.html#closeWriteChannel) right after starting processes...
I can't try it out right now but does the following work?

```
# Handle piped input
def piped_input(argv, pipeargs=None):
    if not sys.stdin.isatty():
        pipeargs.extend(argv)
        for s in sys.stdin.readlines():
            pipeargs.extend(s.split())
            return
```
No - I'm guessing it hangs on the `.readlines()` because there is nothing to read.

It's basically the same than doing `cat | buku`. `buku` will wait until `ctrl-d` is pressed (end-of-file for `cat`) and do nothing before that.

I don't think there's anything buku can do about this, but I wonder if other stuff calling buku outside a shell might be affected by this as well.

Either way, this is quite surprising behavior - I think it should at least print something ("waiting for arguments on stdin" or so?). I also didn't find it documented anywhere from a quick look through the manpage. Note if someone wanted to do this, they could just do `buku $(cat)` instead.
There are ways to handle this e.g. a non-blocking read() or threading etc. However, I would prefer documenting this.
That won't help, you'll still need your arguments before continuing to run :wink:
Nope. Without args we show help and exit.
Ahh OK. Got it!
Hi, I was quite busy the last time we talked on this and couldn't spend more than a few minutes. So yes, readlines() blocks if the input stream in not closed. I have added an informative log and documented it in README.

Are you trying to integrate qutebrowser with buku?
Not directly, but it's possible to spawn processes and shell scripts from qutebrowser, so that's what some people seem to do.
Cool!

Resolving patch:
diff --git a/qutebrowser/misc/guiprocess.py b/qutebrowser/misc/guiprocess.py
--- a/qutebrowser/misc/guiprocess.py
+++ b/qutebrowser/misc/guiprocess.py
@@ -142,6 +142,7 @@ def start(self, cmd, args, mode=None):
             self._proc.start(cmd, args)
         else:
             self._proc.start(cmd, args, mode)
+        self._proc.closeWriteChannel()
 
     def start_detached(self, cmd, args, cwd=None):
         """Convenience wrapper around QProcess::startDetached."""


Test patch:
diff --git a/tests/end2end/data/userscripts/stdinclose.py b/tests/end2end/data/userscripts/stdinclose.py
new file mode 100755
--- /dev/null
+++ b/tests/end2end/data/userscripts/stdinclose.py
@@ -0,0 +1,27 @@
+#!/usr/bin/python3
+# vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
+
+# Copyright 2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+#
+# This file is part of qutebrowser.
+#
+# qutebrowser is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# qutebrowser is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with qutebrowser.  If not, see <http://www.gnu.org/licenses/>.
+
+"""A userscript to check if the stdin gets closed."""
+
+import sys
+import os
+sys.stdin.read()
+with open(os.environ['QUTE_FIFO'], 'wb') as fifo:
+    fifo.write(b':message-info "stdin closed"\n')
diff --git a/tests/end2end/features/spawn.feature b/tests/end2end/features/spawn.feature
--- a/tests/end2end/features/spawn.feature
+++ b/tests/end2end/features/spawn.feature
@@ -55,3 +55,13 @@ Feature: :spawn
         Then the following tabs should be open:
             - about:blank
             - about:blank (active)
+
+    @posix
+    Scenario: Running :spawn with userscript that expects the stdin getting closed
+        When I run :spawn -u (testdata)/userscripts/stdinclose.py
+        Then the message "stdin closed" should be shown
+
+    @posix
+    Scenario: Running :spawn -d with userscript that expects the stdin getting closed
+        When I run :spawn -d -u (testdata)/userscripts/stdinclose.py
+        Then the message "stdin closed" should be shown