Made some perf investigations and based on that committed some smart changes