fruux/sabre-dav — pull request #347 by bc-brendan-roche (2013-07-10 06:05:35+00:00)
Title: Several optimizations to reduce peak memory usage when a directory has a large number of files

Continuation from the following PR now targeting master:

'https://github.com/fruux/sabre-dav/pull/346

I havent manually tested this version but unit tests pass ok

https://travis-ci.org/bc-brendan-roche/sabre-dav/builds/8914274 :green_heart:

I had an issue where if I try to access a folder that has over 5000 files using a webdav client like cyberduck our app will crash due to exceeding php's memory limit (currently set at 48 MB). Now as the app we develop has very strict requirements for its memory footprint I decided I would try to optimise the sabre-dav library and if need be use a forked version of it for our app.

These updates now allow me to access a folder with just over 20000 files before it will hit this limit.

The way it does this is it splits the method getPropertiesForPath into two separate methods

getNodesForPath (renamed original method getPropertiesForPath)
getPathProperties

Such that rather than returning one big multidimensional array of properties, it returns an array of paths to nodes. All areas where getPropertiesForPath is called are updated to loop through this array of nodes and then retrieve the path properties during the loop. This way we are only accessing these properties when we need them and not storing them in a gigantic array.

I havent benchmarked this but i expect it to be faster as its not having to copy these path properties in a array.

There is a couple of areas that are a little messy that could be cleaned up further due to my changes. The method generateMultiStatus can now receive fileProperties as a array of path to nodes as returned from getNodesForPath or as it originally did as a list of file paths. I'm sure there are other changes that can be made to clean up my changes further.

Anyway I hope you like these improvements and consider merging them, then we wont have to maintain a separate fork of this project. Thanks for all your work in a maintaining a great OS library!