Do not handle directory if no index and passthrough

Request is accidentally handled if `if-modified-since` header is sent.