Repository: ytdl-org/youtube-dl

Issue:
Launching Youtube-dl as a background command
Hi, I just noticed when i try to launch $ youtube-dl http://www.ZZzzzZZ.com &
It fails, program cant terminate.

Any clue ? Thanks.



Discussion:
Can you elaborate what you mean by "cant terminate"? Also, what do you mean by "fails"? Do you get an error message?

It works fine for me:

``` sh
$ youtube-dl http://www.ZZzzzZZ.com &
[1] 9302
$ Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)                                                                                                                                                                             
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code                                                                                                                                                          
    exec code in run_globals                                                                                                                                                                                         
  File "/home/phihag/projects/youtube-dl/youtube_dl/__main__.py", line 18, in <module>                                                                                                                               
    youtube_dl.main()                                                                                                                                                                                                
  File "/home/phihag/projects/youtube-dl/youtube_dl/__init__.py", line 613, in main
    _real_main(argv)
  File "/home/phihag/projects/youtube-dl/youtube_dl/__init__.py", line 597, in _real_main
    retcode = ydl.download(all_urls)
  File "/home/phihag/projects/youtube-dl/youtube_dl/YoutubeDL.py", line 567, in download
    videos = self.extract_info(url)
  File "/home/phihag/projects/youtube-dl/youtube_dl/YoutubeDL.py", line 305, in extract_info
    ie_result = ie.extract(url)
  File "/home/phihag/projects/youtube-dl/youtube_dl/extractor/common.py", line 91, in extract
    return self._real_extract(url)
  File "/home/phihag/projects/youtube-dl/youtube_dl/extractor/generic.py", line 95, in _real_extract
    new_url = self._test_redirect(url)
  File "/home/phihag/projects/youtube-dl/youtube_dl/extractor/generic.py", line 83, in _test_redirect
    response = opener.open(HeadRequest(url))
  File "/usr/lib/python2.7/urllib2.py", line 410, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 523, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 448, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 502: Bad Gateway

[1]+  Exit 1                  youtube-dl http://www.ZZzzzZZ.com
```

@vanilla38 Can you give us some more info so that we can investigate the issue ? It is working for phihag.

I've got (probably) the same problem, using python 2.7 on IRIX.

$ ./youtube-dl http://vimeo.com/79695097&
[2] 2859
$ 

[2]+  Stopped                 ./youtube-dl http://vimeo.com/79695097
$ 

However, it continues as normal if I let it run in the foreground again (e.g. with 'fg'), which would be consistent with youtube-dl trying to read stdin. Backgrounding youtube-dl works as well when stdin is redirected from somewhere else:

$ ./youtube-dl http://player.vimeo.com/video/79140726/ < /dev/null &
[2] 3098
$ [vimeo] 79140726: Downloading webpage
[vimeo] 79140726: Extracting information
[download] Verruckt - KC 2014-79140726.mp4 has already been downloaded

[2]+  Done                    ./youtube-dl http://player.vimeo.com/video/79140726/ < /dev/null

The last bits I can see from python -m trace -t 
[...]
**init**.py(91): def parseOpts(overrideArguments=None):
**init**.py(446): def _real_main(argv=None):
__init__.py(742): def _setup_opener(jar=None, opts=None, timeout=300):
__init__.py(772): def main(argv=None):
**main**.py(17): if **name** == '**main**':
**main**.py(18):     youtube_dl.main()
 --- modulename: **init**, funcname: main
**init**.py(773):     try:
**init**.py(774):         _real_main(argv)
 --- modulename: __init__, funcname: _real_main
__init__.py(448):     if sys.platform == 'win32':
**init**.py(452):     parser, opts, args = parseOpts(argv)
 --- modulename: **init**, funcname: parseOpts
**init**.py(92):     def _readOptions(filename_bytes):
__init__.py(105):     def _format_option_string(option):
__init__.py(121):     def _comma_separated_values_options_callback(option, opt_str, value, parser):
__init__.py(124):     def _find_term_columns():
__init__.py(137):     def _hide_login_info(opts):
__init__.py(147):     max_width = 80
**init**.py(148):     max_help_position = 80
**init**.py(151):     columns = _find_term_columns()
 --- modulename: __init__, funcname: _find_term_columns
__init__.py(125):         columns = os.environ.get('COLUMNS', None)
 --- modulename: UserDict, funcname: get
UserDict.py(59):         if key not in self:
 --- modulename: UserDict, funcname: **contains**
UserDict.py(71):         return key in self.data
UserDict.py(60):             return failobj
**init**.py(126):         if columns:
**init**.py(129):         try:
**init**.py(130):             sp = subprocess.Popen(['stty', 'size'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
 --- modulename: subprocess, funcname: **init**
subprocess.py(625):         _cleanup()
 --- modulename: subprocess, funcname: _cleanup
subprocess.py(461):     for inst in _active[:]:
subprocess.py(627):         self._child_created = False
subprocess.py(628):         if not isinstance(bufsize, (int, long)):
subprocess.py(631):         if mswindows:
subprocess.py(641):             if startupinfo is not None:
subprocess.py(644):             if creationflags != 0:
subprocess.py(648):         self.stdin = None
subprocess.py(649):         self.stdout = None
subprocess.py(650):         self.stderr = None
subprocess.py(651):         self.pid = None
subprocess.py(652):         self.returncode = None
subprocess.py(653):         self.universal_newlines = universal_newlines
subprocess.py(672):          errread, errwrite) = self._get_handles(stdin, stdout, stderr)
 --- modulename: subprocess, funcname: _get_handles
subprocess.py(1031):             p2cread, p2cwrite = None, None
subprocess.py(1032):             c2pread, c2pwrite = None, None
subprocess.py(1033):             errread, errwrite = None, None
subprocess.py(1035):             if stdin is None:
subprocess.py(1036):                 pass
subprocess.py(1045):             if stdout is None:
subprocess.py(1047):             elif stdout == PIPE:
subprocess.py(1048):                 c2pread, c2pwrite = self.pipe_cloexec()
 --- modulename: subprocess, funcname: pipe_cloexec
subprocess.py(1091):             r, w = os.pipe()
subprocess.py(1092):             self._set_cloexec_flag(r)
 --- modulename: subprocess, funcname: _set_cloexec_flag
subprocess.py(1073):             try:
subprocess.py(1074):                 cloexec_flag = fcntl.FD_CLOEXEC
subprocess.py(1078):             old = fcntl.fcntl(fd, fcntl.F_GETFD)
subprocess.py(1079):             if cloexec:
subprocess.py(1080):                 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
subprocess.py(1093):             self._set_cloexec_flag(w)
 --- modulename: subprocess, funcname: _set_cloexec_flag
subprocess.py(1073):             try:
subprocess.py(1074):                 cloexec_flag = fcntl.FD_CLOEXEC
subprocess.py(1078):             old = fcntl.fcntl(fd, fcntl.F_GETFD)
subprocess.py(1079):             if cloexec:
subprocess.py(1080):                 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
subprocess.py(1094):             return r, w
subprocess.py(1055):             if stderr is None:
subprocess.py(1057):             elif stderr == PIPE:
subprocess.py(1058):                 errread, errwrite = self.pipe_cloexec()
 --- modulename: subprocess, funcname: pipe_cloexec
subprocess.py(1091):             r, w = os.pipe()
subprocess.py(1092):             self._set_cloexec_flag(r)
 --- modulename: subprocess, funcname: _set_cloexec_flag
subprocess.py(1073):             try:
subprocess.py(1074):                 cloexec_flag = fcntl.FD_CLOEXEC
subprocess.py(1078):             old = fcntl.fcntl(fd, fcntl.F_GETFD)
subprocess.py(1079):             if cloexec:
subprocess.py(1080):                 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
subprocess.py(1093):             self._set_cloexec_flag(w)
 --- modulename: subprocess, funcname: _set_cloexec_flag
subprocess.py(1073):             try:
subprocess.py(1074):                 cloexec_flag = fcntl.FD_CLOEXEC
subprocess.py(1078):             old = fcntl.fcntl(fd, fcntl.F_GETFD)
subprocess.py(1079):             if cloexec:
subprocess.py(1080):                 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
subprocess.py(1094):             return r, w
subprocess.py(1067):             return (p2cread, p2cwrite,
subprocess.py(1068):                     c2pread, c2pwrite,
subprocess.py(1069):                     errread, errwrite)
subprocess.py(674):         self._execute_child(args, executable, preexec_fn, close_fds,
subprocess.py(675):                             cwd, env, universal_newlines,
subprocess.py(676):                             startupinfo, creationflags, shell,
subprocess.py(677):                             p2cread, p2cwrite,
subprocess.py(678):                             c2pread, c2pwrite,
subprocess.py(679):                             errread, errwrite)
 --- modulename: subprocess, funcname: _execute_child
subprocess.py(1119):             if isinstance(args, types.StringTypes):
subprocess.py(1122):                 args = list(args)
subprocess.py(1124):             if shell:
subprocess.py(1129):             if executable is None:
subprocess.py(1130):                 executable = args[0]
subprocess.py(1135):             errpipe_read, errpipe_write = self.pipe_cloexec()
 --- modulename: subprocess, funcname: pipe_cloexec
subprocess.py(1091):             r, w = os.pipe()
subprocess.py(1092):             self._set_cloexec_flag(r)
 --- modulename: subprocess, funcname: _set_cloexec_flag
subprocess.py(1073):             try:
subprocess.py(1074):                 cloexec_flag = fcntl.FD_CLOEXEC
subprocess.py(1078):             old = fcntl.fcntl(fd, fcntl.F_GETFD)
subprocess.py(1079):             if cloexec:
subprocess.py(1080):                 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
subprocess.py(1093):             self._set_cloexec_flag(w)
 --- modulename: subprocess, funcname: _set_cloexec_flag
subprocess.py(1073):             try:
subprocess.py(1074):                 cloexec_flag = fcntl.FD_CLOEXEC
subprocess.py(1078):             old = fcntl.fcntl(fd, fcntl.F_GETFD)
subprocess.py(1079):             if cloexec:
subprocess.py(1080):                 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
subprocess.py(1094):             return r, w
subprocess.py(1136):             try:
subprocess.py(1137):                 try:
subprocess.py(1138):                     gc_was_enabled = gc.isenabled()
subprocess.py(1141):                     gc.disable()
subprocess.py(1142):                     try:
subprocess.py(1143):                         self.pid = os.fork()
subprocess.py(1148):                     self._child_created = True
subprocess.py(1149):                     if self.pid == 0:
subprocess.py(1151):                         try:
subprocess.py(1153):                             if p2cwrite is not None:
subprocess.py(1155):                             if c2pread is not None:
subprocess.py(1156):                                 os.close(c2pread)
subprocess.py(1157):                             if errread is not None:
subprocess.py(1158):                                 os.close(errread)
subprocess.py(1159):                             os.close(errpipe_read)
subprocess.py(1164):                             if c2pwrite == 0:
subprocess.py(1166):                             if errwrite == 0 or errwrite == 1:
subprocess.py(1170):                             def _dup2(a, b):
subprocess.py(1178):                             _dup2(p2cread, 0)
 --- modulename: subprocess, funcname: _dup2
subprocess.py(1174):                                 if a == b:
subprocess.py(1176):                                 elif a is not None:
subprocess.py(1179):                             _dup2(c2pwrite, 1)
 --- modulename: subprocess, funcname: _dup2
subprocess.py(1174):                                 if a == b:
subprocess.py(1176):                                 elif a is not None:
subprocess.py(1177):                                     os.dup2(a, b)
subprocess.py(1148):                     self._child_created = True
subprocess.py(1149):                     if self.pid == 0:
subprocess.py(1219):                     if gc_was_enabled:
subprocess.py(1220):                         gc.enable()
subprocess.py(1223):                     os.close(errpipe_write)
subprocess.py(1225):                 if p2cread is not None and p2cwrite is not None:
subprocess.py(1227):                 if c2pwrite is not None and c2pread is not None:
subprocess.py(1228):                     os.close(c2pwrite)
subprocess.py(1229):                 if errwrite is not None and errread is not None:
subprocess.py(1230):                     os.close(errwrite)
subprocess.py(1234):                 data = _eintr_retry_call(os.read, errpipe_read, 1048576)
 --- modulename: subprocess, funcname: _eintr_retry_call
subprocess.py(476):     while True:
subprocess.py(477):         try:
subprocess.py(478):             return func(*args)
subprocess.py(1237):                 os.close(errpipe_read)
subprocess.py(1239):             if data != "":
subprocess.py(681):         if mswindows:
subprocess.py(689):         if p2cwrite is not None:
subprocess.py(691):         if c2pread is not None:
subprocess.py(692):             if universal_newlines:
subprocess.py(695):                 self.stdout = os.fdopen(c2pread, 'rb', bufsize)
subprocess.py(696):         if errread is not None:
subprocess.py(697):             if universal_newlines:
subprocess.py(700):                 self.stderr = os.fdopen(errread, 'rb', bufsize)
__init__.py(131):             out,err = sp.communicate()
 --- modulename: subprocess, funcname: communicate
subprocess.py(734):         if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
subprocess.py(754):         return self._communicate(input)
 --- modulename: subprocess, funcname: _communicate
subprocess.py(1304):             if self.stdin:
subprocess.py(1311):             if _has_poll:
subprocess.py(1312):                 stdout, stderr = self._communicate_with_poll(input)
 --- modulename: subprocess, funcname: _communicate_with_poll
 --- modulename: subprocess, funcname: communicate
subprocess.py(734):         if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
subprocess.py(754):         return self._communicate(input)
 --- modulename: subprocess, funcname: _communicate
subprocess.py(1304):             if self.stdin:
subprocess.py(1311):             if _has_poll:
subprocess.py(1312):                 stdout, stderr = self._communicate_with_poll(input)
 --- modulename: subprocess, funcname: _communicate_with_poll
subprocess.py(1337):             stdout = None # Return
subprocess.py(1338):             stderr = None # Return
subprocess.py(1339):             fd2file = {}
subprocess.py(1340):             fd2output = {}

par (like strace) continues with the stuff below just after 'fg':

```
0mS[  1]                ( 2908): was sent signal SIGCLD by the kernel
0mS[  1]                ( 2908): was sent signal SIGCONT by pid 2534
0mS[  0]                ( 2908): was sent signal SIGCLD by the kernel
0mS[  2]                 : END-poll() = 2
0mS[  2]                 : close(3) OK
0mS[  2]                 : read(5, "unknown mode: size\n", 4096) = 19
1mS[  2]                 : poll(0x105546f8, 1, -1)
1mS[  2]                 : END-poll() = 1
1mS[  2]                 : close(5) OK
```

+1
Running `youtube-dl` in background stops the download. Bringing it back with `fg` starts the download.


Resolving patch:
diff --git a/youtube_dl/utils.py b/youtube_dl/utils.py
--- a/youtube_dl/utils.py
+++ b/youtube_dl/utils.py
@@ -1820,6 +1820,7 @@ def get_exe_version(exe, args=['--version'],
     try:
         out, _ = subprocess.Popen(
             [encodeArgument(exe)] + args,
+            stdin=subprocess.PIPE,
             stdout=subprocess.PIPE, stderr=subprocess.STDOUT).communicate()
     except OSError:
         return False


Test patch: