# Presentation: 125353

## Search Engine Optimization Basics

- Bob Keating
- Search Program Manager
- FirstGov Technologies, GSA

## Search Visibility Factors
Basics of Search Friendly Design
Site Maintenance
Resources 

- Basics of Search Friendly Design
- Site Maintenance
- Resources

- SEO Topics

## Search Visibility Factors

## Basic Page Components

**Text (Keyword) Component**
- Words and phrases that match what your target audience types into search engines
**Link Component**
- Site navigation and URL structure that search crawlers can easily follow
**Popularity Component**
- Are sites link to you?

## Text: Keyword Density

- Choose one or two keywords or phases to optimize for each page
- Do not over use - avoid keyword stuffing
- Over use of keywords can result in being penalized or ignored
- Incorporate other complimentary words and phrases
- Check Keyword Density:
- _[http://www.webjectives.com/keyword.htm](http://www.webjectives.com/keyword.htm)_
- _[http://](http://www.keyworddensity.com/)__[www.keyworddensity.com](http://www.keyworddensity.com/)__[/](http://www.keyworddensity.com/)_

## Text: Keyword Prominence

- Search engines place “weight” on terms according to where they are used
**Place Keywords in ...**
- Title tags
- Headings and emphasized text
- Visible body text
- Description meta tags
- Alt text in images
- Title and body tags are most important
- Keyword in the URL is helpful, but not a significant factor in ranking

## The Metadata Myth

**Quote:** “Metadata improves search relevancy”
**False.** Except for title and description tags, Web search engines ignore other metadata. However, Metadata does matter to enterprise search.
**Quote:** “Using standard metadata is a best practice.”
**False. **It is only a best practice if the metadata is used by your agency for specific applications (e.g. enterprise search or content syndication).  It is not a best practice among professional web designers.

## The Metadata Myth

**Quote:** “If more agencies were diligent about adding metadata, Google would pay attention”
**False.** Industry is focused on developing algorithms that determine relevancy based on content rather than what an author or metadata creator says about the content
- _Key points_
- Metadata is not the key to high rankings
- Only use metadata if it is important to your agency’s mission
- Paying more attention to keyword usage in other areas will have a greater impact on relevancy

## Pages will not rank well if your site does not have a navigation scheme
Navigation scheme must please users and search engines
Create a site map, but also plan how pages link to each other
Avoid dangling pages 

- Navigation scheme must please users and search engines
- Create a site map, but also plan how pages link to each other
- Avoid dangling pages

- Link Component

##  Poor HTML coding
 Image maps
 Frames
 JavaScript
 Dynamic Pages
 Flash

-  Image maps
-  Frames
-  JavaScript
-  Dynamic Pages
-  Flash

- Problem Navigation Schemes

## Keywords in links tell crawlers about the pages to which you are linking
Keywords in links influence relevancy of the page to which you are linking
Avoid “click here” links, instead create links like: 
Bad: Click here for more info on famous admirals.
Good: Visit our naval history site for more info on famous admirals.

- Keywords in links influence relevancy of the page to which you are linking
- Avoid “click here” links, instead create links like:
**Bad: **Click _here_ for more info on famous admirals.
**Good: **Visit our _naval history site_ for more info on _famous admirals_.

- Creating Links

## Based on the number pages that link to you 
The more popular pages that link to you, the higher your popularity
All search engines have different popularity algorithms
Google’s algorithm is called Page Rank
Every page on the Web is given a calculation of it is popularity based on inbound links

- The more popular pages that link to you, the higher your popularity
- All search engines have different popularity algorithms
- Google’s algorithm is called Page Rank
- Every page on the Web is given a calculation of it is popularity based on inbound links

- Popularity Component

## Number and Popularity of Inbound Links
Get listed in Yahoo!, DMOZ
Network with other agency and industry sites
Make your site a link magnet
Anchor Text
Others’ use of keywords in a link to your site
Popularity is assigned per page, not for the entire site
Popularity is not inherited
Need to deliberately link internal pages to pass on PR

- Get listed in Yahoo!, DMOZ
- Network with other agency and industry sites
- Make your site a link magnet
- Anchor Text
- Others’ use of keywords in a link to your site
- Popularity is assigned per page, not for the entire site
- Popularity is not inherited
- Need to deliberately link internal pages to pass on PR

- Popularity Factors

## Your popularity is not determined by the  sites you link to
Your outbound links affect the popularity of the sites you’re linking to
Internal links and inbound links have the most impact on your popularity
Outbound links help identify you with a hub

- Your outbound links affect the popularity of the sites you’re linking to
- Internal links and inbound links have the most impact on your popularity
- Outbound links help identify you with a hub

- Outbound Links

## Robots Exclusion

- Meta-Tag Robots Exclusion
- <head><meta name=“robots” content=“no index, nofollow”></head>
- Robots.txt File
- Place in server’s root directory
- Two elements: User-agent, Disallow
- Example:
- User-agent: *
- Disallow: /cgi-bin/
- Disallow: /scripts/
- Disallow: /images/

## Not all search engines pay attention to robots.txt instructions
MSN and Yahoo! obey robots exclusion more often than Google
Never exclude msnbot ... you won’t have site search
Blocking bots is contrary to OMB’s guidance 

- MSN and Yahoo! obey robots exclusion more often than Google
- Never exclude msnbot ... you won’t have site search
- _Blocking bots is contrary to OMB’s guidance_

- Robots Exclusion

## Case Study: NOAA Fisheries

- _Term_: Fisheries
- #1 on FirstGov
- #1 on MSN
- #2 on Google
- #3 on Yahoo

- _Term_: Fish
- Doesn’t show up on the first page or subsequent pages

## Slide 18

## Case Study: NOAA Fisheries

**Text Component**: page is well-optimized for “fisheries” but not “fish”
**Link Component**: Fly-out navigation may be a problem for some crawlers. Add search friendly navigation (e.g., site map)
**Popularity Component**: Lots of inbound links using “fisheries” (not “fish”) in anchor text

## Basics of Search Friendly Design

## Page Content: using content and text that target your audiences; and attract search engines and links from other sites
Navigation: giving users and crawlers easy access to content
Design Considerations: make sure bells and whistles don’t undermine SEM efforts
Page Rank: link popularity

**Navigation:** giving users and crawlers easy access to content
**Design Considerations:** make sure bells and whistles don’t undermine SEM efforts
**Page Rank: **link popularity

- Basic Concepts

## Managing Page Content

## Types of Pages

- Home Page
- About
- Contact
- Site Map
- News
- Forms

- Galleries
- FAQs
- Catalogs
- Product pages
- Shopping cart
- Search Results

**How you write, design and optimize a page depends on the type of page**

##  Keywords
Use language of your audience
Use keyword selection tools
Yahoo!/Overture: http://inventory.overture.com/
Google: https://adwords.google.com/select/KeywordSandbox
Page Content
 Make content appear focused
Title tag, headings, contextual links, cross-links
Body text should be visible, i.e., should not have to do any action to view main page text

- Use language of your audience
- Use keyword selection tools
- Yahoo!/Overture: _[http://inventory.overture.com/](http://inventory.overture.com/)_
- Google: _[https://adwords.google.com/select/KeywordSandbox](https://adwords.google.com/select/KeywordSandbox)_
- Page Content
-  Make content appear focused
- Title tag, headings, contextual links, cross-links
- Body text should be visible, i.e., should not have to do any action to view main page text

- Text to Include

## Primary vs. Secondary Text

**Primary Text**
- Title tag
- Body text
- Text near the top of the page
- Text in and around links (e.g. anchor text)

**Secondary Text**
- Alt text
- Description tag
- Domain name and URL elements

## Write your own
Pros: Original, unique content
Cons: Time consuming, bureaucratic process
Use someone else’s  
Pros: Easy way to bulk up content
Cons: Still need to worry about how to add value in order to attract links

- Pros: Original, unique content
- Cons: Time consuming, bureaucratic process
**Use someone else’s  **
- Pros: Easy way to bulk up content
- Cons: Still need to worry about how to add value in order to attract links

- What Kind of Content to Include?

## Tends to update frequently
Crawlers visit frequently-updated pages more often
Combine syndicated content to create a unique resource
Syndicate your own content (e.g. RSS)
Increase site visibility and attracts traffic
Another opportunity for inbound links

- Crawlers visit frequently-updated pages more often
- Combine syndicated content to create a unique resource
- Syndicate your own content (e.g. RSS)
- Increase site visibility and attracts traffic
- Another opportunity for inbound links

- Use Syndicated Content

## Managing Site Architecture

## Text Links
 Very search engine friendly
 Use for primary or secondary navigation
Problems with Text Links
 Can negatively skew keyword density
 Crawlers tend to read text links first

-  Very search engine friendly
-  Use for primary or secondary navigation
- Problems with Text Links
-  Can negatively skew keyword density
-  Crawlers tend to read text links first

- Site Navigation Scheme

## Navigation buttons
Okay as long as you include alt text
Avoid JavaScript, unless you can provide navigation crawlers can follow
Recommendation: use alt text and text navigation at the bottom of the page – allows you put keywords in multiple places

- Okay as long as you include alt text
- Avoid JavaScript, unless you can provide navigation crawlers can follow
- Recommendation: use alt text and text navigation at the bottom of the page – allows you put keywords in multiple places

- Site Navigation Scheme

## Image Maps
Crawlers ignore links inside image maps
User text links or navigation buttons elsewhere
Pull Down Menus
Generally not crawler friendly because they need JavaScript or a CGI program
Always provide two forms of navigation: one for your users, and one for your crawlers

- Crawlers ignore links inside image maps
- User text links or navigation buttons elsewhere
- Pull Down Menus
- Generally not crawler friendly because they need JavaScript or a CGI program
- Always provide two forms of navigation: one for your users, and one for your crawlers

- Site Navigation Scheme

## Create a site map
Subscribe to Google Site Maps
A crawler enabling tool to assist the Google crawler
Analyzes information about your site’s architecture to improve crawling

- Subscribe to Google Site Maps
- A crawler enabling tool to assist the Google crawler
- Analyzes information about your site’s architecture to improve crawling

- Help Crawlers Navigate

## Design Considerations

## Bells and whistles (Flash, JavaScript, animation) enhance user experience but can hurt search visibility
Implement features carefully in order to keep search engine ranking
Ensure design team understands SEM concepts

- Implement features carefully in order to keep search engine ranking
- Ensure design team understands SEM concepts

- Design Considerations

## Use External JavaScript and CSS

- Using JavaScript on site navigation can greatly decrease site crawlability
- Crawlers do not follow links embedded inside JavaScript code; or they limit the types of embedded links crawled
- JavaScript in <body> can decrease page load time
- Long download times may indicate the site is spam, and crawlers could ignore

## Use External JavaScript and CSS

- External files decrease page load time for visitors
- External files decrease download time for crawlers
- Remember to disallow crawling scripts in the robots.txt
- External scripts are easier to re-use

## Do not design in frames
Crawlers have trouble getting from the frameset page to the actual web page
Frameset does not provide crawler with keyword rich text and links
Each page is indexed separately, so pages that only make sense as a frameset will be indexed individually in search engines
<no frames> tag is ignored due to spam

- Crawlers have trouble getting from the frameset page to the actual web page
- Frameset does not provide crawler with keyword rich text and links
- Each page is indexed separately, so pages that only make sense as a frameset will be indexed individually in search engines
- <no frames> tag is ignored due to spam

- Frames

## Add Navigation
Give all pages unique title and description tags
Put navigation links on your pages
Add Java Script in your <head> tag
<script language=“javascript”>
<! –
if (top == self) self.location.href = “index.html”;
// 
</script>
This will force the browser to always load the frameset
However, browser will always load the home page, not the indexed page
Back button will be disabled

- Give all pages unique title and description tags
- Put navigation links on your pages
- Add Java Script in your <head> tag
- <script language=“javascript”>
- <! –
- if (top == self) self.location.href = “index.html”;
- // 
- </script>
- This will force the browser to always load the frameset
- However, browser will always load the home page, not the indexed page
- Back button will be disabled

- Frames Workarounds

## Few search engines crawl links embedded inside a Flash navigation scheme
Flash sites contain little text
If you include Flash ...
Include a “Skip” link so both the user and crawler can go to the real homepage
Include title and description meta tags

- Flash sites contain little text
- If you include Flash ...
- Include a “Skip” link so both the user and crawler can go to the real homepage
- Include title and description meta tags

- Flash

## Database-driven, created on the fly by asp, cfm, php, jsp or cgi scripts
Dynamic sites are comprised of templates, but usually without original content
When a page is viewed, the template loads the content from the database
Parameters are added to the URL, which tells the template to load specific content

- Dynamic sites are comprised of templates, but usually without original content
- When a page is viewed, the template loads the content from the database
- Parameters are added to the URL, which tells the template to load specific content

- Dynamic Pages

## Example: http://smithsonianstore.com/catalog/product.jsp?productId=14273&parentCategoryId=3151&categoryId=3152
URLs such as this are difficult for search engines to index because they do not know the parameters that define a unique page
The more parameters, the less likely pages will be indexed
A database may continually feed data, crashing your server and scaring off the crawler

- URLs such as this are difficult for search engines to index because they do not know the parameters that define a unique page
- The more parameters, the less likely pages will be indexed
- A database may continually feed data, crashing your server and scaring off the crawler

- Dynamic Pages

## Create static HTML pages 
Modify URLs so they don’t look like dynamic pages, fewer parameters
Use URL re-write trick using mod_rewrite

- Modify URLs so they don’t look like dynamic pages, fewer parameters
- Use URL re-write trick using mod_rewrite

- Search Friendly Dynamic Pages

## The kiss of death if left unmanaged
Same content is delivered to the crawler but as unique URLs
Crawlers will ignore web pages with session IDs
Omit session IDs if the requestor is a crawler ... but no cloaking!

- Same content is delivered to the crawler but as unique URLs
- Crawlers will ignore web pages with session IDs
- Omit session IDs if the requestor is a crawler ... but no cloaking!

- Session IDs

## Make sure PDFs contain actual text, not images of text
Same rules for use of keywords and phrases apply
Put the most important text in the title, headlines
Minimize document size (> 100K)
Create optimized HTML pages for PDFs

- Same rules for use of keywords and phrases apply
- Put the most important text in the title, headlines
- Minimize document size (> 100K)
- Create optimized HTML pages for PDFs

- Optimizing PDFs

## Managing Page Rank

## All search engines assign a value to your site based on inbound links
Google calls this relevancy factor “Page Rank” (PR) - synonymous with popularity ranking
An inbound link is a vote for your page
An outbound link is a vote for the page you’re linking to

- Google calls this relevancy factor “Page Rank” (PR) - synonymous with popularity ranking
- An inbound link is a vote for your page
- An outbound link is a vote for the page you’re linking to

- Understanding Page Rank

## A page is assigned PR as soon as it’s indexed
Make sure all your pages have at least one link back into your site
The page receiving the most inbound links gets the highest page rank
You can pass PR around to pages on your site

- Make sure all your pages have at least one link back into your site
- The page receiving the most inbound links gets the highest page rank
- You can pass PR around to pages on your site

- Understanding Page Rank

## Controversial topic among SEOs
Supporting View: PR leaks in the sense that a page’s outbound links will decrease that page’s available PR for redistribution throughout the site
Opposing View: Search engines analyze inbound and outbound links to determine your authority as hub. No site is an island

- _Supporting View_: PR leaks in the sense that a page’s outbound links will decrease that page’s available PR for redistribution throughout the site
- _Opposing View_: Search engines analyze inbound and outbound links to determine your authority as hub. No site is an island

- Can You Leak Page Rank?

## Do not create pages with mostly outbound links
Link to quality, related sites – helps to establish you as an authority or hub
If a page contains several outgoing links, also include links to other pages on your site
Don’t be afraid to link to sites with low page rank, but quality content

- Link to quality, related sites – helps to establish you as an authority or hub
- If a page contains several outgoing links, also include links to other pages on your site
- Don’t be afraid to link to sites with low page rank, but quality content

- Outbound Link Strategy

## Links from Yahoo! and DMOZ
Impact on popularity may vary
Helps to get noticed by crawlers
Establish reciprocal link arrangements with agencies covering similar topics
Reach out to state and local agencies
Syndicate your content
Reach out to professional communities of interest

- Impact on popularity may vary
- Helps to get noticed by crawlers
- Establish reciprocal link arrangements with agencies covering similar topics
- Reach out to state and local agencies
- Syndicate your content
- Reach out to professional communities of interest

- Getting Links to Your Site

## Links that Aren’t Links

- Link has been created in a manner that search engines can’t understand
- Link actually points elsewhere, and not directly to your site
- Example: FirstGov.gov
-  _[http://www.firstgov.gov/external/external.jsp?url=http://www.noaa.gov/](http://www.firstgov.gov/external/external.jsp?url=http://www.noaa.gov/)_
- Link passes through a JSP program that logs the click and re-directs the browser to noaa.gov
- Search engines read this as an internal link to FirstGov, not an outbound link to NOAA
- A listing on FirstGov.gov will not help boost your search ranking

## Site Maintenance

## Plan BEFORE you embark on a re-design
Will the site architecture change?
Static to Dynamic?
How is the content changing?
Add SEO review as an activity to your project plan
Make sure your contractor has SEO skills
Poor planning and execution can kill your search rankings 
Do not wait until after you re-designed!
Do not ask the FirstGov Search Team to re-crawl your site!

- Will the site architecture change?
- Static to Dynamic?
- How is the content changing?
- Add SEO review as an activity to your project plan
- Make sure your contractor has SEO skills
- Poor planning and execution can kill your search rankings
- Do not wait until after you re-designed!
- Do not ask the FirstGov Search Team to re-crawl your site!

- Re-Designing Your Site

## Try to keep the same filenames and directory structure when redesigning
Follow MSN’s “What to do when your site moves”: http://search.msn.com/docs/siteowner.aspx?t=SEARCH_WEBMASTER_REF_Redirectcode.htm
Recommended: Set up HTTP 301 redirects that point to the new site or pages
Not Recommended: Add a meta-refresh tag to your page header.  This won’t remove your original page from the MSN index, and thus your site’s search engine.

- Follow MSN’s “What to do when your site moves”: _[http://search.msn.com/docs/siteowner.aspx?t=SEARCH_WEBMASTER_REF_Redirectcode.htm](http://search.msn.com/docs/siteowner.aspx?t=SEARCH_WEBMASTER_REF_Redirectcode.htm)_
- _Recommended_: Set up HTTP 301 redirects that point to the new site or pages
- _Not Recommended_: Add a meta-refresh tag to your page header.  This won’t remove your original page from the MSN index, and thus your site’s search engine.

- Changes to Site Architecture

## Avoid using default 404 pages
Custom 404 pages are more user-friendly
Submit all 404 URLs to the search engines using their Add URL form – quickest way to get the 404 pages out of the index
Remember to change the HTTP status code on custom error pages from 200 to 404
Do not ask the FirstGov Search Team to re-crawl your site!

- Custom 404 pages are more user-friendly
- Submit all 404 URLs to the search engines using their Add URL form – quickest way to get the 404 pages out of the index
- Remember to change the HTTP status code on custom error pages from 200 to 404
- Do not ask the FirstGov Search Team to re-crawl your site!

- Removing Content

## Resources and Recommendations

## Read Industry Forums
Webmaster World: http://www.webmasterworld.com/
Search Engine Watch: http://www.searchenginewatch.com/
Digital Point Forums: http://forums.digitalpoint.com/
High Rankings: http://www.highrankings.com/forum/
Go to Industry Events
Search Engine Strategies: http://www.searchenginestrategies.com/

- Webmaster World: _[http://www.webmasterworld.com/](http://www.webmasterworld.com/)_
- Search Engine Watch: _[http://www.searchenginewatch.com/](http://www.searchenginewatch.com/)_
- Digital Point Forums: _[http://forums.digitalpoint.com/](http://forums.digitalpoint.com/)_
- High Rankings: _[http://www.highrankings.com/forum/](http://www.highrankings.com/forum/)_
**Go to Industry Events**
- Search Engine Strategies: _[http://www.searchenginestrategies.com/](http://www.searchenginestrategies.com/)_

- Recommendations

## Battelle, John. The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed our Culture. New York. Penguin, 2005.
Kent, Peter. Search Engine Optimization for Dummies. Hoboken, NJ. Wiley, 2004.
Moran, Mike, Hunt, Bill. Search Engine Marketing, Inc. Upper Saddle River, NJ. IBM Press, 2006.
Thurow, Shari. Search Engine Visibility. Indianapolis, IN: New Riders, 2003.

- Kent, Peter. _Search Engine Optimization for Dummies._ Hoboken, NJ. Wiley, 2004.
- Moran, Mike, Hunt, Bill. _Search Engine Marketing, Inc._ Upper Saddle River, NJ. IBM Press, 2006.
- Thurow, Shari. _Search Engine Visibility._* *Indianapolis, IN: New Riders, 2003.

- Recommended Reading

## Contact InformationBob KeatingSearch Program Managerrobert.keating@gsa.gov(202) 208-1047Susan FarissSearch Analystsusan.fariss@gsa.gov(202) 219-1437