INPUT ARTICLE: Article: A robots.txt file is a plain or ASCII text file that informs search engine spiders what they are allowed to access on your site. Files and folders listed in a robots.txt file may not be crawled and indexed by a search engine spiders. You may need a robots.txt file if: You want to block specific content from search engine spiders. You are developing a live site and are not prepared to have search engine spiders crawl and index the site You want to limit access to reputable bots. To create the file, launch a plain text editor or a code editor. Save the file as: robots.txt. The file name must be all lowercase. Do not forget the "s." When you save the file, choose the extension "'.txt"'. If you are using Word, select the "Plain Text" option. It is possible to block every reputable search engine spider from crawling and indexing your site with a "full-disallow" robots.txt. Write the following lines in your text file: User-agent: * Disallow: / Using a "full-disallow" robots.txt file is not strongly recommended. When a bot, such as Bingbot, reads this file, it will not index your site and the search engine will not display your website. User-agents: this is another term for search engine spiders, or robots *: the asterisk signifies that the code applies to all user-agents Disallow: /: the forward slash indicates that the entire site is off-limits to bots Instead of blocking all bots, consider blocking specific spiders from certain areas of your site. Common conditional-allow commands include: Block a specific bot: replace the asterisks next to User-agent with googlebot, googlebot-news, googlebot-image, bingbot, or teoma. Block a directory and its contents: User-agent: * Disallow: /sample-directory/ Block a webpage: User-agent: * Disallow: /private_file.html Block an image: User-agent: googlebot-image Disallow: /images_mypicture.jpg Block all images: User-agent: googlebot-image Disallow: / Block a specific file format: User-agent: * Disallow: /p*.gif$ Many people want to welcome, instead of block, search engine spiders because they want their entire site indexed. To accomplish this, you have three options. First, you can opt out of creating a robots.txt filewhen the robot does not find a robots.txt file, it will continue to crawl and index your entire site. Second, you can create an empty robots.txt filethe robot will find the robots.txt file, recognize that it is empty, and continue to crawl and index your site. Lastly, you can write a full-allow robots.txt file. Use the code: User-agent: * Disallow: When a bot, such as googlebot, reads this file, it will feel free to visit your entire site. User-agents: this is another term for search engine spiders, or robots *: the asterisk signifies that the code applies to all user-agents Disallow: the blank disallow command indicates that all files and folders are accessible After you have written the robots.txt file, save the changes. Upload the file to your site's root directory. For example, if your domain is www.yourdomain.com, place the robots.txt file at www.yourdomain.com/robots.txt.

SUMMARY: Understand robots.txt files. Create and save and robots.txt file. Write a full-disallow robots.txt file. Write a conditional-allow robots.txt file. Encourage bots to index and crawl your site. Save the txt file to the root of your domain.

INPUT ARTICLE: Article: Before you start looking at hosting plans, you'll need a good idea of your basic needs. Use this list of questions as a guide for determining the type of service that will best suit your website: Do you have, need, or want your own domain name? Do you have (or want) a blog? Do you need a specific blogging platform (e.g., Wordpress)? Are you tech-savvy or a beginner? If you're not a developer, you might consider a host like Squarespace, Wix, or Weebly. These hosts make it easy to create a stylish personal or professional site without any technical know-how. Do you really need a paid webhost or could you get by with a free option? If you plan to have a personal blog without your own domain name, consider free options like Tumblr, Blogger, or Wordpress.com. Do you rely on (or want) a visual panel like cPanel to install software and updates? Do you need to migrate an existing website to a new host? If so, does your site require specific tools (a certain blogging platform, support for certain types of apps, etc.) to run? Do you need to be able to sell products or services and take payments? Web hosts often charge different rates depending on how much traffic your site gets and how much space your content takes up. Is your business expanding? Do you foresee a significant increase in visitors? There's a chance that what works for you now will need to be scaled up in the near future. When choosing a web host, keep the future in mind and plan accordingly. If you have access to your current website's statistics, use that information to gauge growth. There are three general types of plans offered by web hosts: Shared hosting: Shared hosting is great for personal websites, bloggers whose sites are mostly text-based, and small businesses who use external ecommerce sites like Shopify or Etsy to handle their sales. Your website will be hosted on a server that also hosts other peoples' websites, which means you'll be sharing resources, such as bandwidth, hard-drive space, CPU power, and RAM. As the most affordable and user-friendly type of web hosting, shared hosting plans typically offer website-building tools, one-click management panels, and limited support. Many shared plans also come with a free domain name. Bandwidth and hard drive space are limited. If a shared server becomes overcrowded or hosts resource-intensive websites, your site's performance may suffer. Dedicated hosting: Dedicated hosting is ideal for large businesses with server administrators, as well as high-traffic/resource-intensive websites with lots of customization needs. With this type of hosting, you'll have an entire server dedicated to your website. You'll control every aspect of your website, including maintaining all applications and databases relied on by your website. You'll also be responsible for ensuring that your server is secure, patched, and up-to-date. If you don't have a server admin and aren't technical yourself, you'll need a "managed" service plan, which can be pricey. Virtual Private Servers: This plan is recommended for businesses whose sites rely on more customization than what's available on a shared server, but don't require as much bandwidth or storage space as what comes with a dedicated server. VPS service is similar to dedicated hosting in that you'll have your own operating system and the ability to manage all aspects of your server's backend tools. It's also similar to a shared server because you'll be sharing the physical hardware with other customers. Like a dedicated server, you'll need advanced technical know-how (or a managed support plan) to build and maintain your site.

SUMMARY:
Determine what your website (or website idea) needs to function. Consider size and potential growth. Learn the difference between different types of web hosting.