What are the attributes of this lfn_open() that make it better than the other one?
You name "FDCache stuff as that was serializing the requests and also not gaining much performance wise."
It's obviously calling the faster underlying functions.
And it's got a different interface to export the FD upwards.

What's the relative importance of those? For instance, if we just make the FDCache not serialize things (perhaps partitioning the locking by collection or something) and use the faster underlying functions, does that get us the same performance advantages as this implementation without destroying our external interfaces?