Mystery of VT100 keyboard codes - Page 1
Forum section: electronics / general

author: peter-h
author_rank: Super Contributor
timestamp: September 09, 2022, 03:20:44 pm
content: I am running Teraterm (a "normal" VT100 emulator, on a COM port) to my target.I am getting funny key codes being received.For example, cursor up should be ESC [ A i.e. 0x1b,0x5b,0x41. But I am seeing just ESC A i.e. 0x1b,0x41Googling around on key codes, this doesn't compute.OTOH I recall testing this a few months ago, where I exercised all the keys to produce this tableCode:static const char KEYS[] = {0x0d,0x00,// ENTER0x0a,0x00,// LF0x08,0x00,// BS0x1b,0x00,// ESC0x1b,0x5b,0x31,0x7e,0x00,// HOME0x1b,0x5b,0x34,0x7e,0x00,// END0x7f,0x00,// DEL0x1b,0x5b,0x35,0x7e,0x00,// PGUP0x1b,0x5b,0x36,0x7e,0x00,// PGDN0x1b,0x5b,0x41,0x00,// CUR_UP0x1b,0x5b,0x43,0x00,// CUR_RT0x1b,0x5b,0x42,0x00,// CUR_DN0x1b,0x5b,0x44,0x00,// CUR_LF0x09,0x00,// TAB0x1b,0x5b,0x31,0x31,0x7e,0x00,// F10x1b,0x5b,0x31,0x32,0x7e,0x00,// F20x1b,0x5b,0x31,0x33,0x7e,0x00,// F30x1b,0x5b,0x31,0x34,0x7e,0x00,// F40x1b,0x5b,0x31,0x35,0x7e,0x00,// F50x1b,0x5b,0x31,0x37,0x7e,0x00,// F60x1b,0x5b,0x31,0x38,0x7e,0x00,// F70x1b,0x5b,0x31,0x39,0x7e,0x00,// F80x1b,0x5b,0x32,0x30,0x7e,0x00,// F90x1b,0x5b,0x32,0x31,0x7e,0x00,// F100x1b,0x5b,0x32,0x33,0x7e,0x00,// F110x1b,0x5b,0x32,0x34,0x7e,0x00,// F120x00,0x00,0x00,0x00,0x00,0x00// prevent spurious matches with data past table end};and they matched the codes herehttps://www2.ccs.neu.edu/research/gpc/VonaUtils/vona/terminal/vtansi.htmalthough nowhere could I find the PC function keys F1-F12; I had to catch these empirically, but even those are different now e.g. F1 should be 0x1b,0x5b,0x31,0x31,0x7e but I am getting just 0x1b,0x31,0x7e.Due to the passage of time since a VT100 was a real thing, the more obscure key codes have become fluid, while old stuff like DEL, BS, TAB, BELL, have remained.Teraterm doesn't have a config for a different terminal.Can anyone spot what is going on?I am running this under win7-64, to a VCP (USB CDC function in my target).Everything else is working fine, and if I echo the bytes back to the monitor window, cursor UP actually moves the cursor up, so Teraterm is clearly happy with what it is sendingI am getting on two machines and two targets, and one of them is the one originally used to determine the key codes.

author: wek
author_rank: Frequent Contributor
timestamp: September 09, 2022, 06:29:59 pm
content: Maybe you should've explained your exact setup in more detail.You're gulping every even character somewhere where you observe it (but not where you echo), probably when they arrive in one packet, as otherwise you'd notice when typing plain characters.JW

author: RoGeorge
author_rank: Super Contributor
timestamp: September 09, 2022, 07:00:50 pm
content: No need to reverse engineer the VT100 codes, they are published online in many places, random link:http://www.braun-home.net/michael/info/misc/VT100_commands.htmI don't think the Fn keys were a thing back then, during the time VT100 standard was made, but I'm not very sure.Teraterm has many modes, I would check its setting to be sure Teraterm is indeed in VT100 mode.https://ttssh2.osdn.jp/manual/4/en/about/emulations.htmlhttps://ttssh2.osdn.jp/manual/4/en/reference/keycode.txt

author: peter-h
author_rank: Super Contributor
timestamp: September 09, 2022, 08:18:31 pm
content: Interesting!I have my 32F417 board, which among other stuff has a USB port implementing a VCP (USB CDC device profile). This is accessible to the PC like any other VCP e.g. COM3. It is also accessible to internal software as a UART port.And this all works. I can output long chunks of text via the output to the PC (Teraterm), and I can read stuff from the PC (Teraterm keyboard). I have done a simple loopback routine in the target, which echoes keystrokes nicely.The immediate purpose of this feature is to run a factory test screen over it. Later it will be documented for general use; it is pretty fast at > 100kbytes/sec.Now, would you expect, if running a VT100 terminal on the PC, for the cursor keys to move the cursor on the terminal? I am not actually sure, but during a test a while ago this was working just like that. It isn't now. The terminal is doing VT100 cursor codes (it evidently supports "cursor home" etc, but the keyboard isn't generating them.There was a problem with the VCP code back then, corrupting a few % of the data, which was never noticed on keystrokes, but when I got around to testing the VCP feature properly for not just output (which has been working for years, for debugs) but also input, I found this issue, and fixed it (various forum posts which WEK will recall).I went looking for config options, but according to the above links, Teraterm is basically aVT100emulator.It does support keyboard mappings via a key mapping file, under Setup / Load key map. But among the files provided, none generate VT100 cursor codes (readily found on google e.g. ESC [ A (3 bytes) is cursor UP).To be fair, I would not expect F keys to be specially supported, because they didn't exist on a VT100.Some DEC terminals did have F keys but mostly these sent out whole text strings.But I would expect TT to emulate "something". How do PC applications deal with F keys? Do they look for ESC sequences, or is there a specific API? If the former then I would expect to find a table somewhere, but I found nothing. It seems to be dealt with using scan codes or some such.And only one of the .cnf files, function.cnf, which come with TT, makes the F keys do anything. Some others make all F keys generate the same code. Using the function.cnf keyboard mapping file, I get these F key codes again; F1= 1B 50, F2= 1B 51... That's OK; I can use that, but do these relate to anything whatsoever?The cursor keys are 1B 41 to 1B 44, consistently across the .cnf files, but again I cannot see these codes relating to anything. They are not VT100 cursor codes for sure.There is no other config I can find in TT.Possibly I was using some keyboard mapping file which I can't find now, but I doubt it because my Teraterm installations (I use it everywhere) are straight out of the box, no config.I cannot see how TT ever generated the VT100 cursor codes.I uninstalled TT, deleting previous settings, and reinstalled it, and get the same thing.I have a usable terminal now, but with keyboard F key and cursor codes which don't relate to anything I find online.The TT "terminal ID" does not do anything in terms of changing key codes and suchThe keyboard cursor codes I am seeing are e.g. herehttps://docs.microsoft.com/en-us/windows/console/console-virtual-terminal-sequencesbut without the '[' i.e. ESC [ A is just ESC A. But my VCP passes through the '[' fine.I am printing the hex codes to the SWV ITM data console. I am quite confident about this part.

author: westfw
author_rank: Super Contributor
timestamp: September 09, 2022, 08:45:58 pm
content: Iirc, vt100 had an escape sequence that would change the “mode” of the cursor keys…Umm,[size=78%]https://terminalguide.namepad.de/mode/p1/?

author: Bobson
author_rank: Regular Contributor
timestamp: September 09, 2022, 09:12:05 pm
content: Your VT-100 is in VT-52 mode. Need to switch to VT-100 mode. ESC <, if I remember correctly.

author: peter-h
author_rank: Super Contributor
timestamp: September 09, 2022, 09:33:22 pm
content: QuoteThat does not explain the cursor codes I am seeing.QuoteYou are damn right!!!http://toshyp.atari.org/en/VT_52_terminal.htmlHowever that string doesn't appear to work. But now that I know where this ESC A etc comes from, I will just create a new keyboard mapping file. Actually the encoding there is damn complicated. Maybe I should ditch TT and get a proper VT100 emulator? I had a look at a load and most seem to either be crap, or need site licenses.It seems this "keyboard is VT52 while screen is VT100" thing is not uncommon. Just tried Realterm and it does the same.I found the ESC < here:http://www.braun-home.net/michael/info/misc/VT100_commands.htmbut it doesn't do anything on TT. The keyboard is firmly VT52.I've concluded TT is useless. One can config a different key mapping but there is no way to save it! The teraterm.ini file has no parameter for the .cnf file. There is only a command line parameter, which is a long term nightmare for setting up a factory test procedure (the command line applies only to a specific shortcut file). Also it seems that a lot of the functionality works only on TCP/IP connections, not on serial connections.I would be happy to buy something that actually works as-spec, but don't want to pay for each installed program, and absolutely don't want online license checking because the test laptops are not online.

author: DiTBho
author_rank: Super Contributor
timestamp: September 11, 2022, 07:16:15 pm
content: Tcp/ip, try or buy mobaterm

author: peter-h
author_rank: Super Contributor
timestamp: September 11, 2022, 07:48:09 pm
content: Gosh that does everything including the kitchen sink. All except serial portsNice tool, with RDP as well.

author: bsdphk
author_rank: Regular Contributor
timestamp: September 12, 2022, 01:44:29 pm
content: Check if your serial port is correctly configured.ANSI escape codes also have a 8-bit mode:0x1b [ Acan also be sent as0x9b AIf your serial port strips the top bit...

author: m k
author_rank: Super Contributor
timestamp: September 12, 2022, 05:27:11 pm
content: VT100 is 7-bit machine.0x9b is Control Sequence Introduce, it came later.VT132 is defining CSI but the actual code is still ESC [.Can't remember how 8th bit is treated but guess is that the whole character is dropped.Emulation can obviously do as it likes.Correct general operation would be so that only VT52 or VT100 codes are supported, not both nor mix.Can't remember either how VT101 was but VT102 was 8-bit machine, though can't remember what it actually meant since multinational character set came later, maybe alternative character set of some sort was available.Operation modes were also ANSI and VT52, name VT100 was not used.Maybe alternative keypad mode can be used for testing.ESC = on and ESC > off.Off is keypad numbers and on is something else, VT52 and VT100 responds differently.Reading DEC manual pages without knowing it also can be misleading since character numbers are usually octal.

author: peter-h
author_rank: Super Contributor
timestamp: September 12, 2022, 08:02:39 pm
content: My code is definitely not eating bit 7.I think the issue is that I am testing with Tera Term and most people just use that as a dumb terminal, so they never get to care about which ESCape sequences the keyboard generates.There is no doubt that TT can generate the VT100 cursor codes but I didn't spend enough time with it to work out how. The keyboard.cnf file has various sections in it and it isn't exactly obvious which section is in use in a given context.The files which come with TT just do the VT52 codes.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: September 12, 2022, 09:34:46 pm
content: PuTTY sends ESC [ A/B/C/D for arrow keys correctly

author: peter-h
author_rank: Super Contributor
timestamp: September 16, 2022, 05:38:07 am
content: I could not find where Putty is configured to use a COM port. I use it only for remote server work. It's a weird program - too weird to use in production test.I am sure somebody has done a keyboard map file for Teraterm; I just haven't found it, and have not managed to work out its file format. I can remap the function keys but not the cursor keys. One also has to do it in the keyboard.cnf file, not a different-name file.

author: oPossum
author_rank: Super Contributor
timestamp: September 16, 2022, 08:38:33 am
content: Quote from: peter-h on September 16, 2022, 05:38:07 amConnection type: Serial

author: peter-h
author_rank: Super Contributor
timestamp: September 16, 2022, 12:39:54 pm
content: Ah yes I did actually find it before.Unfortunately it doesn't work. Just beeps once. Maybe COM9 is too high a number?

author: MarkF
author_rank: Super Contributor
timestamp: September 16, 2022, 01:36:52 pm
content: Sorry, I haven't been following this topic.However to answer your PuTTY question, I have used PuTTY to connect to a PIC32 chip with a USB_to_Serial cable without an problems. Using Win7 and serial port 10 and the default PuTTY settings. I also was using a 38400 baud rate.

author: MarkF
author_rank: Super Contributor
timestamp: September 16, 2022, 01:42:53 pm
content: Also, the following ESC sequences worked with PuTTY:Code:// VT100 control macros#define backspace 0x7f // make sure your backspace matches this!// PuTTY serial terminal control codes// from https://en.wikipedia.org/wiki/ANSI_escape_code// ANSI CSI (Control Sequence Introducer) sequences#define clrscr() printf("\x1b[2J")#define clr_right() printf("\x1b[K")#define home() printf("\x1b[H")#define cursor_pos(line,column) printf("\x1b[%02d;%02dH",line,column)// ANSI SGR (Select Graphic Rendition) parameters#define normal_text() printf("\x1b[0m")#define inverse_text() printf("\x1b[7m")#define red_text() printf("\x1b[31m")#define green_text() printf("\x1b[32m")#define yellow_text() printf("\x1b[33m")#define blue_text() printf("\x1b[34m")#define magenta_text() printf("\x1b[35m")#define cyan_text() printf("\x1b[36m")

author: abyrvalg
author_rank: Frequent Contributor
timestamp: September 16, 2022, 02:03:08 pm
content: Try \\.\COM9 instead of COM9

author: T3sl4co1l
author_rank: Super Contributor
timestamp: September 16, 2022, 02:21:26 pm
content: Can't speak for Teraterm, but can also confirm PuTTY works as expected. Has been some years now I've had a console system built around it, and works fine with PuTTY, and also HyperTerm (if you're old fashioned). Only quirk really, I think, is force CRLF mode (my code is full of "\r" instead of "\n" or "\r\n").Have never had to enter fully qualified / DOS path for extended COM ports in PuTTY, though have for other (less featureful?*) tools.*Including my own; specifically, opening the device as file, requires the DOS path specifier ("\\.\") when it's a two-digit COM port. I don't know how other programs do it (that doesn't involve OpenFile(), or not as naively).Tim

author: oPossum
author_rank: Super Contributor
timestamp: September 16, 2022, 03:27:25 pm
content: Quote from: peter-h on September 16, 2022, 12:39:54 pmNo, it will work with any COM port number.You must specify the port in the session settings.

author: peter-h
author_rank: Super Contributor
timestamp: September 26, 2022, 11:08:00 am
content: Indeed, Putty does work on serial.However, if you disconnect the USB VCP device (unplug the USB cable) Putty renders that COM port unusable for ever. One has to reboot the PCTeraterm doesn't do that; it merely needs the COM port reconnected in the menu.I need to figure out how to use the TT keyboard remapping file to modify the keyboard codes. I have managed to do some keys but the file has specific named sections and you have to get this right. I am damn sure somebody has done this before...Maybe somebody can help. I am trying to change the codes emitted by the cursor keys. This is the start of the keyboard.cnf fileCode:; Sample of KEYBOARD.CNF for the IBM-PC/AT 101-key keyboard.; See KEYCODE.TXT for the description of the format.;; This file based on IBMKEYB.CNF; 1. send the symbol of each key top "as-is" at editor keypad.; 2. send the symbol of each key top "as-is" at *, /, +, -, Ctrl+"+" on a Ten-Key pad.; 3. enables using the function key from F1 to F10.; See [User keys] section.;; 10/9/2022 PH Copied from function.cnf. These are VT52 keyboard codes.;;[VT editor keypad];Up arrow keyUp=328;Down arrow keyDown=336;Right arrow keyRight=333;Left arrow keyLeft=331;Insert keyInsert=338;Home keyFind=327;PageUp keyPrev=329;Delete keyRemove=339;End keySelect=335;PageDown keyNext=337But nothing edited seems to make any difference. For example I am trying to edit the Down key, scan code 336. I have tried all kinds of stuff e.g.Down=336,0,$1BABCDto generate ESC ABCDbut it just outputs the same 1B 42 (ESC B).I found one can remap the cursor keys to User keys e.g.User11=336,0,$1B$1C where 336 is the scan code, and this was supposed to emit 1B 1C but it emits only 1B. If one emits 3 chars, only the 1st and 3rd come out. It is weird, and contrary to the TT's author's documentation e.g.https://github.com/Testato/TeraTerm/blob/master/doc/en/html/reference/keycode.txtBasically every 2nd char comes out. So a string like aabbccddee generates abcde. I was wondering if there is a "unicode" config somewhere but can't see it.But you can tell that everybody who was involved with TT is now about 120 years old

author: wek
author_rank: Frequent Contributor
timestamp: September 26, 2022, 02:39:52 pm
content: QuoteAs I've said in my post above, IMO this is misinterpretation in your program, not TeraTerm's fault.JW

author: peter-h
author_rank: Super Contributor
timestamp: September 26, 2022, 03:10:06 pm
content: You were right, of course