Libraries should only be split off when they are stable and no longer require frequent changes alongside the product, ensuring they serve as proper abstractions.