Make memory ports part of specials

This is needed to handle cases where a single memory has ports
in two different modules, and one of these modules is subject
to clock domain remapping. The clock domain of the port in that
module only must be remapped.