Search Engine Optimization (SEO) for your website

286

Your SEO Plan, Week 4, Monday: Robots Visiting Your Site

In Chapter 7, “Month One: Kick It into Gear,” we discussed search engine indexing of

your landing pages. With a little sleuthing, you can determine which search engine

robots have visited your site and when.

To scope out robot visits to your site, do the following:

• First, find the name of the robot that you are interested in monitoring. Table 10.1

lists the robot identifiers for the four search engines we’ve featured in this book,

but be prepared for some variations.

Table 10.1

Robot Identifiers for the Four Major Search Engines

Search Engine

Robot Name

Ask

Teoma

Google

Googlebot

MSN

msnbot

Yahoo!

Slurp

• Then, review your website stats program. Look for a section called “User

Agents,” “Robots,” or “Browsers” (servers interpret a search engine robot as a

type of browser).

• Depending on the sophistication of your stats program, you may be able to spec-

ify a date range and view robot visits. You might even be able to see exactly

which pages within your site were visited.

If this all feels a bit too tedious, you may wish to look into software that pro-

vides simple yet detailed reports on robot visits. One such program is Robot Manager,

available here:

www.websitemanagementtools.com

Your SEO Plan, Week 5, Tuesday: Google Sitemaps

In Chapter 8, “Month Two: Establish the Habit,” we talked about building a site map

page for your website. The Google Sitemap is a specially formatted “map” made just

for Google’s search engine spider to use as its very own key to navigating your site.

Google Sitemaps (in beta as of this writing) is especially helpful if Google has a

hard time indexing your site; for example, if your site has a very large number of pages

or dynamically generated pages. Here are the basics:

• You’ll first need to create a specially formatted file—a Sitemap—using Google’s

Sitemap generator tool. Google’s help page at

www.google.com/webmasters/

sitemaps/docs/en/faq.html

has the scoop on how to generate a Sitemap and

c10. 8:10 286