Upgrade the MCU to ATmega32U4. Disable the JTAGEN flag to be able to drive the rows and columns of the keyboard matrix. Set efuse to the default value for this MCU.