Fixing illegal use of cache when it's turned off

Thanks to Vegard Nossum for reporting this one

fixes #55