You can instruct search engine crawlers using the rel=canonical to allow them to include the webpage in the index or not. You can also create these instructions via the Robots meta tag in your href HTML pages section or the X-Robots-Tag in the HTTP header.