AE: fix AE_FMT_S24NE3 format

see title
@anssih could you have a look at this. User reports there is still a problem with this but I don't see where
http://forum.xbmc.org/showthread.php?tid=200867