[G4 Android Development] ilovecm:
I can confirm this bug. I've encountered the problem multiple times with recent nightlies, currently I'm on 20160417.

Steps to reproduce:

Flash a fresh CM13 nightly (no gapps) -> OK.

Set a lockscreen pin -> OK.

Encrypt the phone -> OK.

Reboot and enter the pin for decryption -> OK.

Enter the pin for screen unlock -> Failed. The pin is always reported as wrong.

I found a pretty similar bugreport and a description for a workaround in the cyanogennmod-forums (since I'm new I can't post a link - sorry). The workaround is to delete the gatekeeper-keyfiles, which are stored in /data/system/gatekeeper.password.key and gatekeeper.pattern.key, which triggers gatekeeper to fall back to swipe-to-unlock. This gives access to the phone again, but defeats the purpose of the encryption - I don't want anyone to gain access to my mail/accounts/etc. in case my phone should be lost or stolen. The content of those files did not change during the encryption - their md5sums remained identical.

Logcat shows the following error while confirming the lockscreen-pin:

05-19 15:11:10.671 3551 3551 E gatekeeper_device: Verify send cmd failed

05-19 15:11:10.671 3551 3551 E gatekeeper_device: ret: 0

05-19 15:11:10.671 3551 3551 E gatekeeper_device: resp->status: -30

The result is the same for valid or invalid pins.

Additionally I strace'd gatekeeperd (right when confirming the pin to unlock the screen).

strace without encryption disabled:

root@h815:/ # strace -p 443
Process 443 attached
ioctl(5, BINDER_WRITE_READ, 0x7fc1abbf60) = 0
futex(0x7fa50e3040, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(9, "443", 3) = 3
prctl(PR_SET_THP_DISABLE, 50000, 0x1bb, 0x7fc1abbf85, 0) = 0
getpriority(PRIO_PROCESS, 443) = 20
ioctl(6, _IOC(_IOC_READ|_IOC_WRITE, 0x97, 0x03, 0x20), 0x7fc1abbbd0) = 0
ioctl(5, BINDER_WRITE_READ, 0x7fc1abb940) = 0
ioctl(5, BINDER_WRITE_READ, 0x7fc1abb940) = 0
ioctl(5, BINDER_WRITE_READ, 0x7fc1abb970) = 0
ioctl(5, BINDER_WRITE_READ, 0x7fc1abb970) = 0
faccessat(AT_FDCWD, "0", F_OK) = 0
ioctl(5, BINDER_WRITE_READ, 0x7fc1abbd00) = 0
futex(0x7fa50e3040, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(9, "443", 3) = 3
prctl(PR_SET_THP_DISABLE, 50000, 0x1bb, 0x7fc1abbf85, 0) = 0
ioctl(5, BINDER_WRITE_READ

Strace with encryption enabled:

root@h815:/ # strace -p 3551
Process 3551 attached
ioctl(5, BINDER_WRITE_READ, 0x7fcdcc5c90) = 0
futex(0x7f844e3040, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(9, "3551", 4) = 4
prctl(PR_SET_THP_DISABLE, 50000, 0xddf, 0x7fcdcc5cb5, 0) = 0
getpriority(PRIO_PROCESS, 3551) = 20
ioctl(6, _IOC(_IOC_READ|_IOC_WRITE, 0x97, 0x03, 0x20), 0x7fcdcc5900) = 0
writev(3, [{"l<\0\350\3\337\r", 7}, {"\0\337\r\226Y\23W{\231\276'", 11}, {"\6", 1}, {"gatekeeper_device\0", 18}, {"Verify send cmd failed\0", 23}], 5) = 60
writev(4, [{"\0\337\r\226Y\23W{\231\276'", 11}, {"\6", 1}, {"gatekeeper_device\0", 18}, {"Verify send cmd failed\0", 23}], 4) = 53
writev(3, [{"l,\0\350\3\337\r", 7}, {"\0\337\r\226Y\23W\336s\331'", 11}, {"\6", 1}, {"gatekeeper_device\0", 18}, {"ret: 0\0", 7}], 5) = 44
writev(4, [{"\0\337\r\226Y\23W\336s\331'", 11}, {"\6", 1}, {"gatekeeper_device\0", 18}, {"ret: 0\0", 7}], 4) = 37
writev(3, [{"l7\0\350\3\337\r", 7}, {"\0\337\r\226Y\23W\311\374\370'", 11}, {"\6", 1}, {"gatekeeper_device\0", 18}, {"resp->status: -30\0", 18}], 5) = 55
writev(4, [{"\0\337\r\226Y\23W\311\374\370'", 11}, {"\6", 1}, {"gatekeeper_device\0", 18}, {"resp->status: -30\0", 18}], 4) = 48
ioctl(5, BINDER_WRITE_READ, 0x7fcdcc5a30) = 0
futex(0x7f844e3040, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(9, "3551", 4) = 4
prctl(PR_SET_THP_DISABLE, 50000, 0xddf, 0x7fcdcc5cb5, 0) = 0
ioctl(5, BINDER_WRITE_READ

I also added full logcat outputs to this post.

It seems to me that the problem originates in the communication with QSSE (ioctl on fd 6, which points to /dev/qssecom).

Before going to CM13, I initially flashed the stock 20A rom to my G4, which provided working encryption, so I suppose it's not an issue of missing binary firmware. Recently I flashed stock 20D boot, bootloader and modem before flashing the latest CM, and I wiped system, data, cache, dalvik cache beforehand.

I'm using the latest TWRP (3.0.2.0), which also can't decrypt the data partition.

Does anyone have an idea where the problem might be and what I can do to fix the problem?