this is a working tryp but ugly code and hard to maintain and no testing and hard to debug unreadable code due to insane deadline and which is my fault because my mandays estimation sucks and also I am new to pandas. I need to refactor this asap