selftest: add tests for 4096PSK and 4096QAM