Use pytest_(un)configure to setup cache_directory

This is to finish #168.  Without this change, test fails in my local machine because jedi uses  some cache from my ~/.cache/jedi.