further profiling, lots of hacky code added to the end to use cProfiler