SEO for Beginners, Part 2: Spiders are People, too

OK, arachnids are not people, and search engine spiders are not really people, either. But what I'm driving at is that search engine spiders "think" like people. How do I know this? Because human beings wrote them. Any software that does analysis of any kind does so with the intelligence and analytical rules programmed into it by its human developers.

How Humans Comprehend

Imagine that I handed you an article to read that I had printed out. It's about 20 pages long, but I didn't tell you anything about it. Worse, you're going to have to pass a test on the content. Get out your highlighter!

The best reading instructors teach their students to get the context before seriously studying a text like a chapter of a history book or an article. This is because educational research has shown that knowing the general scope of a selection helps human readers comprehend and retain more of the subject matter. By the way, this example is real, not contrived: I used to be a high school teacher and I took a good deal of post-graduate coursework in what is known as "content area reading".

The easiest way to get the general context is to find the title. The next step is to look at section headings. The combination of the two frames the range or scope of the document or chapter. You can also pick up some hints from picture and diagram captions.

Once you know what the document is about, you can begin to dig into the actual text. As you read, you can often discern important or significant points by repetition. That is, content that is paraphrased and rephrased is often very important: The author obviously thought it worth the effort to rephrase the explanation a couple times (or more). [This paragraph contains a clever example, if I don't say so myself: Did you see it?]

How Search Engine Spiders Comprehend

Not too surprisingly, search engine spiders take the exact same approach as humans.

Like human readers, spiders start to get a clue by looking at titles and section headings. On a web page the spiders all look for your title tag, so it certainly pays to create a title for your web page. Furthermore, it pays even more if that title contains keywords.

An Aside: What are "keywords", anyway?

Most search engines ignore the keyword metatag in the HTML of your web pages, so why do web marketers constantly talk about keywords? The reason is that even if you do not use the keyword metatag on your page, you should select several keywords that describe the content of your page, and then make sure those keywords are sprinkled throughout the page multiple times and in a natural-sounding manner.

The places to use keywords include the title (in the h1 HTML tag), heading text (e.g. in h2 tags), alt and title tags for images, and in body text.

These keywords give the search engine spiders a stronger sense of what your page is really about.

Here's an important observation: Ultimately, our goal is not to "trick" the search engine, but rather to help it see clearly what our content is about. Google and the other engines are getting very adept at figuring out artificial keyword sprinkling. Never string keywords or keyword phrases many times in succession. Google, in particular, sees this is a keyword spamming and will exact painful revenge.

There is also speculation that if your keyword-to-content ratio is unnaturally high, the search engines also discount that. I believe that to be quite likely, but "unnaturally high" is a closely guarded secret. The bottom line for me is that if the body text sounds funny because of all the keywords, you've moved into the danger zone. I'll recommend to you the same thing I did to my Advanced Composition students years ago: Read your page aloud. If it sounds odd in any way, rewrite the odd part.

The search engine spider looks for your section heading in--surprise--the HTML heading tags, especially the h1 tag. You should use these tags on your pages, and make sure they contain keywords.

One often overlooked place to insert keywords is in the alt and title tags of images. Although not quite analogous to a caption that a human reader would examine, the spiders nevertheless figure that these tags probably give a hint as to what the image is about (and in normal circumstances, they do). The image, in turn, gives another clue as to what the page is about.

Once the spider provides a pretty good idea what your page is about, the search engine's next job is to figure out whether your site has high quality content.

More Software and Human Similarities

If somebody claims to be an authority on a subject, it is usually prudent to check out the claim--especially if we're going to be hiring them or our money is otherwise involved. We humans generally do this by checking references.

The search engine spiders operate in fundamentally the same fashion. They assume that a site has high quality content on a given subject if other web sites have links pointing to it. In other words, the site is likely good if the references check out.

Google, in particular, has become very sophisticated in its link analysis. For example, if you trade links, the value of the link is diminished somewhat (nobody knows how much for sure). The reason is that if you really have content that I value and I think my site visitors need, I'll link to you without a trade. An unreciprocated link is one of the surest signs of high quality material on the referenced web site.

The search engines have also gotten very careful about paid links. If your link shows up on a link "farm" (a page or site that only exists for the purpose of creating links to fool the search engines), you may find your site banned altogether. Remember the old butter commercial where the lady says, "It's not nice to fool Mother Nature!" and then she conjures up a storm? Well, the same is true for the search engines. Get caught trying to fool the spider and she'll bite you.

Forgive the mixed metaphors and just remember the point.

The search engines are also known to discount run-of-site links. Let's say you have a link to your home page on every single page of your monstrously large web site. Even though it might add up to thousands of links, it will have almost zero credibility in terms of boosting your page ranking.

The moral to the story is this: Don't use run-of-site links to boost your ranking. If it is crucial to your site navigation, fine. Just don't do the work creating all those links and expect them to help your page rank at all.

There is one remaining question, namely "Do internal links help? That is, does a link from page "A" of my web site to page "B" boost page B's page rank on Google or any other search engine?

According to my sources, the answer is yes, but only just a little. External links are given a lot more credibility. Unfortunately, external links are substantially more difficult to acquire.

Next time: The importance of links and how to get them.