First, I renamed NetworkTopology to SwitchLink.  I think this more accurately describes the table's role.
Also, I figured out how to properly use the SQLAlchemy relationships to have the behavior we're looking for.  Now the NetworkSwitch.neighbors member is now a query object that queries the switch_link table for entries where the srcswitch_id is equal to the owner switch's id.  The new getneighbors() method has the same behavior as the neighbors() method.