I can see a benefit to doing this lazily if the application is not using second-level caching and there happens to be no need for auto-flushing during the Session in question.  However, if either of those 2 cases occur then lazily building the list would actually be worse performance.  

We obviously cannot know ahead of time if the Session will need to be auto-flushed later.  However, we certainly know whether second level caching is enabled, in which case it seems to me it would be better to collect these query/property spaces as actions are added.