Remember which site a BuddyPress group was created on and limit access to that group to that site only.