You can use firewalls to restrict access to Remote Desktop listening ports. Consider using an RDP Gateway to control access to desktops and servers.