mono/mono — pull request #707 by LHCGreg (2013-07-19 01:11:53+00:00)
Title: Improve detecting if a handle is a console on Windows.

Use GetConsoleMode to detect if a handle is a console on Windows. If it succeeds (returns non-zero) then the handle is a console. If it fails (returns zero) then the handle is not a console.

The old check, (GetFileType (handle) == FILE_TYPE_CHAR) assumes that a handle of type FILE_TYPE_CHAR is a console. This falsely reports a handle directed to NUL as a console.

Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=12752

The Windows build is currently broken but I tested it applied against e0f26cc2, the last commit that built successfully on Windows according to MonkeyWrench. Using the test program in the bug report:

C:\Users\Greg\Documents\Visual Studio 2012\Projects\IsRedirectedTest\IsRedirecte
dTest\bin\Debug>C:\Users\Greg\Documents\Programs\cygwin\usr\local\bin\mono IsRed
irectedTest.exe
Input redirected: False
Output redirected: False
Error redirected: False

C:\Users\Greg\Documents\Visual Studio 2012\Projects\IsRedirectedTest\IsRedirecte
dTest\bin\Debug>C:\Users\Greg\Documents\Programs\cygwin\usr\local\bin\mono IsRed
irectedTest.exe < NUL
Input redirected: True
Output redirected: False
Error redirected: False

C:\Users\Greg\Documents\Visual Studio 2012\Projects\IsRedirectedTest\IsRedirecte
dTest\bin\Debug>C:\Users\Greg\Documents\Programs\cygwin\usr\local\bin\mono IsRed
irectedTest.exe > NUL

(output.txt contents)
Input redirected: False
Output redirected: True
Error redirected: False

C:\Users\Greg\Documents\Visual Studio 2012\Projects\IsRedirectedTest\IsRedirecte
dTest\bin\Debug>C:\Users\Greg\Documents\Programs\cygwin\usr\local\bin\mono IsRed
irectedTest.exe 2> NUL
Input redirected: False
Output redirected: False
Error redirected: True

C:\Users\Greg\Documents\Visual Studio 2012\Projects\IsRedirectedTest\IsRedirecte
dTest\bin\Debug>C:\Users\Greg\Documents\Programs\cygwin\usr\local\bin\mono IsRed
irectedTest.exe > outputx.txt

(output.txt contents)
Input redirected: False
Output redirected: True
Error redirected: False