Using DONE_SET method to make algorithm faster

What we do is like to maintain a dict {DONE_SET: memory peak}, then Each time when we execute Node X, we make a check if the peak in dict and the current peak bigger than that peak. If yes(or equal), break and start a new branch. if no, update the peak in dict and continue.
@nouiz 
I see, I will fix this place. Thanks for reviewing, Fred :)