opened
cdhunt
username_0
I'm testing this pipeline.

`powershell -c "ps | convertto-json" | from-json`

The PowerShell PS command returns a lot more process information, but the shell just hangs. There's no process activity for `nu.exe` or `Windows Command Processor`. The shell is unresponsive.

Just the external command, `powershell -c "ps | convertto-json"` works as expected and spits out json to the screen.
Piping external to internal hangs
issue
created
wycats
username_1
Yikes! Seems bad.
comment
created
jonathandturner
username_2
I've seen this with some invocations of the Azure `az` command and keyvault combinations. I wonder if there's a timeout we could introduce and/or tune so that at least something comes back to the user with a warning that the operation couldn't be finished successfully
comment
created
cdhunt
username_0
I've encountered this again while using `from-json`.

`docker images --format "{{json .}}" | from-json`
comment
created
cdhunt
username_0
I tried `docker images | split-column "   "` and it hung until I hit `^C`. Then rendered as expected.
comment
created
jonathandturner
username_2
@username_0 - out of curiosity, are the `docker` hangs on Windows too?
comment
created
cdhunt
username_0
Yes. All on Windows 10.
comment
created
androbtech
username_3
@username_0 can you try `docker images --format "{{json .}}" | from-json --objects` ?
comment
created
androbtech
username_3
Possibly related to #489
comment
created
cdhunt
username_0
I get `error: Unimplemented: passing flags is not supported yet.`
comment
created
rrichardson
username_4
^C
thread 'main' panicked at 'index out of bounds: the len is 2 but the index is 2', /rustc/9eae1fc0ea9b00341b8fe191582c4decb5cb86a3/src/libcore/slice/mod.rs:2715:10
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:47
   3: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:36
   4: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:200
   5: std::panicking::default_hook
             at src/libstd/panicking.rs:214
   6: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:477
   7: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:384
   8: rust_begin_unwind
             at src/libstd/panicking.rs:311
   9: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  10: core::panicking::panic_bounds_check
             at src/libcore/panicking.rs:61
  11: nu::format::table::TableView::from_list
  12: <futures_async_stream::stream::GenStream<G,T> as futures_core::stream::Stream>::poll_next
  13: <futures_util::stream::collect::Collect<St,C> as core::future::future::Future>::poll
  14: <futures_async_stream::stream::GenStream<G,T> as futures_core::stream::Stream>::poll_next
  15: std::future::poll_with_tls_context
  16: nu::cli::cli::{{closure}}
  17: std::thread::local::LocalKey<T>::with
  18: nu::main
  19: std::rt::lang_start::{{closure}}
  20: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:49
  21: std::panicking::try::do_call
             at src/libstd/panicking.rs:296
  22: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  23: std::panicking::try
             at src/libstd/panicking.rs:275
  24: std::panic::catch_unwind
             at src/libstd/panic.rs:394
  25: std::rt::lang_start_internal
             at src/libstd/rt.rs:48
  26: main
  27: __libc_start_main
  28: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
```
comment
created
androbtech
username_3
`--objects` was introduced after the release you are using. Try building from master and it should work.
comment
created
androbtech
username_3
Thanks!

Can you try `kubectl logs mysvc | from-json --objects | debug` and paste the output? It looks like it does the parsing correctly but it's failing when displaying the table (the pipeline ends implicitly with `autoview`)
comment
created
rrichardson
username_4
``` 
from-json --objects | debug
```

```
^CTagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("> mm-server@0.0.1 start /home/node/app")) }
Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("> npm run migrate up && node src/index.js")) }
Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("> mm-server@0.0.1 migrate /home/node/app")) }
Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("> node-pg-migrate --config-file database.json \"up\"")) }
Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Object(Dictionary { entries: {"WARNING": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("NODE_ENV value of \
'production\' did not match any deployment config file names.")) }} }) }
Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Object(Dictionary { entries: {"WARNING": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("See https://github.
com/lorenwest/node-config/wiki/Strict-Mode")) }} }) }
Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("No migrations to run!")) }
Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("Migrations complete!")) }
Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Object(Dictionary { entries: {"level": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data
: BigUint { data: [30] } })) }, "time": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data: BigUint { data: [478974102, 365] } })) }, "pid": Tagged { tag: Tag { origi
n: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data: BigUint { data: [43] } })) }, "hostname": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("me
tal-manager-6f8756975-fnq7t")) }, "msg": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("Server listening at http://0.0.0.0:7654")) }, "v": Tagged { tag: Tag { origin: None, span: Span
{ start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data: BigUint { data: [1] } })) }} }) }
Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Object(Dictionary { entries: {"level": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data
: BigUint { data: [30] } })) }, "time": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data: BigUint { data: [478990977, 365] } })) }, "pid": Tagged { tag: Tag { origi
n: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data: BigUint { data: [43] } })) }, "hostname": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("me
tal-manager-6f8756975-fnq7t")) }, "reqId": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data: BigUint { data: [1] } })) }, "req": Tagged { tag: Tag { origin: None, s
pan: Span { start: 45, end: 54 } }, item: Object(Dictionary { entries: {"method": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("OPTIONS")) }, "url": Tagged { tag: Tag { origin: None,
span: Span { start: 45, end: 54 } }, item: Primitive(String("/api/v1/cluster/4")) }, "hostname": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("api.app.rick0i.dev.superhub.io")) }, "re
moteAddress": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("10.2.0.27")) }, "remotePort": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(Bi
gInt { sign: Plus, data: BigUint { data: [45236] } })) }} }) }, "msg": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("incoming request")) }, "v": Tagged { tag: Tag { origin: None, span
: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data: BigUint { data: [1] } })) }} }) }
Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Object(Dictionary { entries: {"level": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data
: BigUint { data: [30] } })) }, "time": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data: BigUint { data: [478990982, 365] } })) }, "pid": Tagged { tag: Tag { origi
n: None, span: Span { start: 45, end: 54 } }, item: Primitive(Int(BigInt { sign: Plus, data: BigUint { data: [43] } })) }, "hostname": Tagged { tag: Tag { origin: None, span: Span { start: 45, end: 54 } }, item: Primitive(String("me
...
item: Primitive(Int(BigInt { sign: Plus, data: BigUint { data: [1] } })) }} }) }
thread 'main' panicked at 'index out of bounds: the len is 2 but the index is 2', /rustc/9eae1fc0ea9b00341b8fe191582c4decb5cb86a3/src/libcore/slice/mod.rs:2715:10
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:47
   3: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:36
   4: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:200
   5: std::panicking::default_hook
             at src/libstd/panicking.rs:214
   6: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:477
   7: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:384
   8: rust_begin_unwind
             at src/libstd/panicking.rs:311
   9: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  10: core::panicking::panic_bounds_check
             at src/libcore/panicking.rs:61
  11: nu::format::table::TableView::from_list
  12: <futures_async_stream::stream::GenStream<G,T> as futures_core::stream::Stream>::poll_next
  13: <futures_util::stream::collect::Collect<St,C> as core::future::future::Future>::poll
  14: <futures_async_stream::stream::GenStream<G,T> as futures_core::stream::Stream>::poll_next
  15: std::future::poll_with_tls_context
  16: nu::cli::cli::{{closure}}
  17: std::thread::local::LocalKey<T>::with
  18: nu::main
  19: std::rt::lang_start::{{closure}}
  20: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:49
  21: std::panicking::try::do_call
             at src/libstd/panicking.rs:296
  22: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  23: std::panicking::try
             at src/libstd/panicking.rs:275
  24: std::panic::catch_unwind
             at src/libstd/panic.rs:394
  25: std::rt::lang_start_internal
             at src/libstd/rt.rs:48
  26: main
  27: __libc_start_main
  28: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
```
comment
created
rrichardson
username_4
interestingly... 
```
kubectl logs mysvc | from-json --objects | save foo.json
```

It hangs, but it doesn't error..  I still hit ctrl-c... but there is no panic or backtrace, and foo.json is populated with what appears to be the expected data, including strings containing the invalid json
comment
created
androbtech
username_3
@username_4  How about now `> open foo.json` ?
comment
created
rrichardson
username_4
thread 'main' panicked at 'index out of bounds: the len is 2 but the index is 2', /rustc/9eae1fc0ea9b00341b8fe191582c4decb5cb86a3/src/libcore/slice/mod.rs:2715:10
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:47
   3: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:36
   4: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:200
   5: std::panicking::default_hook
             at src/libstd/panicking.rs:214
   6: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:477
   7: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:384
   8: rust_begin_unwind
             at src/libstd/panicking.rs:311
   9: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  10: core::panicking::panic_bounds_check
             at src/libcore/panicking.rs:61
  11: nu::format::table::TableView::from_list
  12: <futures_async_stream::stream::GenStream<G,T> as futures_core::stream::Stream>::poll_next
  13: <futures_util::stream::collect::Collect<St,C> as core::future::future::Future>::poll
  14: <futures_async_stream::stream::GenStream<G,T> as futures_core::stream::Stream>::poll_next
  15: std::future::poll_with_tls_context
  16: nu::cli::cli::{{closure}}
  17: std::thread::local::LocalKey<T>::with
  18: nu::main
  19: std::rt::lang_start::{{closure}}
  20: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:49
  21: std::panicking::try::do_call
             at src/libstd/panicking.rs:296
  22: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  23: std::panicking::try
             at src/libstd/panicking.rs:275
  24: std::panic::catch_unwind
             at src/libstd/panic.rs:394
  25: std::rt::lang_start_internal
             at src/libstd/rt.rs:48
  26: main
  27: __libc_start_main
  28: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
```
comment
created
androbtech
username_3
@username_4 Thanks. The problem could be in the parsing of json in that case (and potentially the culprit of the hanging issue)
comment
created
jonathandturner
username_2
@username_0 - I just landed a fix that should help with the hangs you were seeing. I've tested it on Linux, and I'm hoping the same fix helps Windows also.

If you can, would you be able to checkout the latest Nu from master, build it, and try your example again?
comment
created
cdhunt
username_0
I'm working on a Rustlang Windows container to run the build.
comment
created
cdhunt
username_0
I give up. I can't find a way to get `vs-buildtools` installed to provide `link.exe`.
comment
created
jonathandturner
username_2
@username_0 thanks for trying. I've never built a Windows container.

We're you using a container for your original report?
comment
created
nickolay
username_5
FWIW, I experienced the same problem on Windows, and the fix did help: the command from the first comment no longer hands (though the powershell command itself takes a while to finish).
comment
created
cdhunt
username_0
@username_2 No. I was originally using the 0.2.0 release on my workstation. I just don't want to pollute my workstation with the Rust toolchain to run a single build and from what I've read several people have a problem getting `link.exe` without containers being involved. However, that's all really a Rust issue and not Nu. Sounds like @username_5 verified it so I think you can close this out.
comment
created
jonathandturner
username_2
Sounds good. (and thanks @username_5 for verifying)

We should be rolling out the next release in a couple weeks, which will include this fix.
comment
closed
jonathandturner
username_2
issue