General perfomance optimizations.

After some profiling with callgrind I was able to find a few bottlenecks which may be easily fixed.