Inherent in the idea of shared libraries is the fact that they save memory only when more than one program uses them. For elements of this second type, the installing program will usually have available copies of its own to install and configure. It may also replace existing libraries with updated versions. So far, so good. When the uninstall utility runs, it has no problem in removing components that are used exclusively by the target program.
Shared resources are another matter.
Even if it remembers which elements it has installed, other programs which were installed later may use these shared resources. At the current time, the uninstall program has no certain way of knowing what it is safe to remove. When the counter reaches zero, it can be safely freed for real. Such a scheme would require that every install and uninstall program obey the rules and that there be a safe recovery from such events as an interrupted uninstall. The decision is usually to play it safe and leave any possible shared elements behind.