Refactored to go back to sub selects

This switches back to sub selects, and also has an optimization where it doesn't do the sub select if it already knows the exact object id.