I'd be curious as to what game the Gameboy Color was tested on, because that 80ms figure seems off to me. I don't know much about the joypad hardware so I'm not sure how long it would take from pressing the button to delivering the updated joypad state to the CPU, but I can tell you how long it should take once it hits the CPU:

In most games, the joypad input is scanned once per game loop, which is generally the same thing as a graphical refresh, which is set by the hardware at 60Hz. So depending on at what point in the cycle you press the button, the worst case latency for the software to detect the change is ~16ms.

Supposing this button press resulted in some in-game action, it would then take visible effect at the next screen refresh.

Now, it's common that a game loop looks like this:

Update graphics from current game state
Check inputs
Update game state
Wait for next frame

This is because the graphics ram (VRAM) can only be updated during a ~1ms window after the start of a frame. An interrupt tells you when this period begins. So the most common way to code it is to rush to do all your graphics work quickly, then spend the rest of the frame on other things while you wait for the next one.

As a result, it wouldn't be surprising if our input was delayed an extra 16ms while we wait for the frame AFTER the one where our input is detected.

So I would have expected a latency around 16-32ms - to get 80ms, either the button hardware would have to have added 50ms of latency, or the game they were using was very strangely coded, or the action the game took simply did not have any visible change on screen despite internal state changing - for example, an animation whose first few frames are the same as the standing still animation.

As an interesting additional note: If they were testing the Gameboy the same way as other mobile devices - scrolling the screen - it would be even faster. In the Gameboy scrolling is implemented in hardware - you draw tiles to a 32x32 tile memory region, but the screen is only 18x20 tiles. You can update a scroll X and Y register to pan the visible screen over the 32x32 area (this is how everything from sidescrollers to Pokemon do smooth scrolling, since tiles can only be placed on an 8x8 pixel grid). This register, if updated immediately after detecting the scroll input, would take effect immediately, even if it was midway through drawing a frame (resulting in a tearing effect which in most cases wouldn't be noticed but could be abused to produce interesting special effects). So the latency then would be 0-16ms + whatever the hardware adds.

Of course, you could pare this down further by writing specialised code that checks the joypad input rapidly instead of once per frame, or use the joypad interrupt to be immediately notified upon a button press (note this interrupt suffered from significant limitations, so no-one used it, and as far as I can tell the Gameboy Color doesn't have it at all, much to my chagrin since I'd figured out how to use it in a good way). That would likely violate the spirit of the comparison though, since you could equally program any of the other contenders to be a dedicated "fast scroll" machine.