add directory middleware, fixes and improvements