Bug fixes and optimization

Note that under laggy conditions, some keys fail to work. Also some
other random bugs to look into.