Virtual DOS machine

A Virtual DOS machine (VDM) permits the users of computers to run 16-bit or 32-bit DOS and 16-bit Windows programs on 32-bit computers when there already is a different operating system using the hardware.
Overview
Virtual DOS machines normally use the virtual 8086 mode of the Intel 80386 (or later) processor, allowing many 8086 applications to function by translating operating system commands to make applications think that they are running on an 8086. The operating system can then emulate the DOS software.
VDMs like Windows 3.x 386 Enhanced Mode can add support for using 16-/32-bit protected mode software if they use the DOS Protected Mode Interface (DPMI).
If a DOS program in a VDM needs to use a peripheral, Windows will give the DOS program a virtual device driver (VDD) which emulates the hardware using commands that the operating system it is running on can understand.
DOS VDMs
DOS VDMs appeared on Windows/386 2.01 in 1987 and were also in Windows 3.x, if you were using 386 Enhanced Mode as well as in 95, 98, 98 SE and Me.
Similar to Windows 3.x, VDMs can let you run many DOS applications at one time if you use the EMM386 /MULTI option. Digital Research / Novell was working on a similar system since 1991 under the codename "Vladivar" (originally a device driver KRNL386.SYS and not part of EMM386). It was made for the next version of DR DOS, released as Novell DOS 7 in 1994, but it was also used in their "Star Trek" project, which they stopped working on before finishing.
OS/2 MVDM
VDMs called MVDM (Multiple Virtual DOS Machine) are used in OS/2 2.0 and later. OS/2 MVDMs are better than NTVDM in many ways, for example with block devices. While the OS/2 1.x emulated DOS 3.0, 2.x emulated DOS 5.0.
Running Windows 3.1 applications in OS/2 looks similar to XP Mode in Windows 7, but the technologies used are very different. A "guest" VDM permits programs to use the disks of the OS/2 or NT operating system that is running the VDM. Applications in a VDM can use named pipes to talk to their "host".
Windows NTVDM
NTVDM is included in all x86 versions of Windows made on NT, and let 16-bit Windows and 16-/32-bit DOS applications run. It is not included with 64-bit versions (see Limitations). ntvdm.exe helps emulate the APIs needed to run 16-bit DOS and Windows applications.
NTVDM uses NTIO.SYS and NTDOS.SYS, which runs a changed version of COMMAND.COM. The 16-bit system files are smaller versions of their MS-DOS 5.0 counterparts, including IO.SYS, MSDOS.SYS and COMMAND.COM, which remove biases with the FAT file system and can also call the 32-bit NTVDM if needed. NTDOS originally gave a DOS version of 30.00 to programs, but this soon was changed to 5.00 to let more programs to run without being changed However, there are many newer MS-DOS functions and commands added to MS-DOS versions 6.x and in Windows 9x which are not there.
16-bit programs run separately within one 32-bit NTVDM process. Checking "Run in separate memory space" in the Run dialog box or shortcut of the program creates a new process instead. NTVDM emulates BIOS functions, the Windows 3.1 kernel, and its 16-bit API.
32-bit DOS emulation is used for DPMI and using 32-bit memory. wowexec.exe is the emulator process for 16-bit Windows. Windows 2000 added Sound Blaster 2.0 emulation to 32-bit NTVDM. 16-bit and DOS virtual device drivers can't be run. Win16 programs can communicate with other parts of Windows using OLE, DDE and named pipes.
As virtual 8086 mode isn't available on non-x86 processors, NTVDM was a full emulator for NT on these architectures. NT 3.51 could only 80286, but since NT 4.0, 486 emulation was added.
Security issues
In January 2010, Google security researcher Tavis Ormandy discovered a large security problem in NTVDM system letting non-Administrators to increase their permissions to the SYSTEM level. Ormandy claimed it was a problem for all x86 versions of Windows NT after 1993. 32-bit versions of NT 3.X/4.0, 2000, XP, 2003, Vista, 2008, and 7 were affected. Ormandy published how the problem works. Before Microsoft could fix it, you could turn off 16-bit application support to solve the problem, but it made so you could not run 16-bit programs. x64 Windows never had this problem because it doesn't use NTVDM. After applying the security fixes Microsoft made available, NTVDM could be turned on safely.
Limitations
NTVDM and wowexec had limitations on Windows XP and later because of a GDI object limit per session, which might cause GDI handles to be moved two bits to the right when changing 32 to 16 bits. Handles thus could not be larger than 14 bits, so 16-bit programs with a GDI handle larger than 16384 would crash.
On x86-64 CPUs, virtual 8086 mode is only available as a legacy sub-mode (if running a 32-bit operating system), but not in 64-bit long mode, which is why NTVDM is not included with 64-bit versions of Windows, so 16-bit applications cannot be run on 64-bit Windows. You'll need Windows XP Mode or similar virtualization software to run 16-bit programs within a 64-bit operating system.
NTVDM cannot run DOS games on computers of today. Emulation is only provided for simple peripherals. For example, sound in NTVDM can be limited. Windows NT only updates the screen a few times per second when DOS programs write on it, and they can't emulate high-resolution graphics options. As software runs at the speed of the CPU, all timing loops stop too early. This either makes games run too fast, or causes the software to not see some emulated hardware peripherals, because it can not wait that long for an answer.
Related pages
Virtualization
Windows on Windows
Notes
References
Other websites
Virtual DOS Machine Structure
Troubleshooting MS-DOS-based programs in Windows XP
Troubleshooting an MS-DOS application which hangs the NTVDM subsystem in Windows XP and Windows Server 2003
Troubleshooting MS-DOS-based serial communication programs in Windows 2000 and later
Microsoft Windows software