When rendering a subject, indicate whether or not it's linked to anything in a 'is_linked' property.  Index that property for easy filtering.
Use a loaded hook to set up reciprocal relationships for subjects and agents.

Saves having to duplicate the knowledge of which models link to agents and subjects.