Spiders crawl

Free clip art spider crawling the World Wide Web

Installment 6 in an opinion series by Linda Bradshaw about blogging and SEO

As I learn more and more about blogging, the Internet, and the all-mighty Google, there have been a lot of terms used by people who I had never heard in relation to the Internet. Most specifically, the terms spider, crawl, index, page rank, and sweep. What does all of that mean?

Well, if I go to the source, Google itself, and type in those terms I get a lot of different crazy answers. Seems that Google uses little action codes that they like to call spiders (after all, the Internet IS the World Wide Web!) that look at (crawl) the Internet every month, looking for new web pages, and updated pages. When they find the new pages, and the updated pages, the spiders report back to their headquarters details of what they found while they were out crawling!

Those details are compared to the current formula (algorithm) that Google has in place to determine what Page Rank to give the individual web pages that were crawled. And Google creates a method of keeping track of all of the different pages that were crawled – that method is called “indexing.” When the spiders are crawling and reporting back, Google is said to be “indexing” or “re-indexing.”

At the end of this indexing, Google assigns each web page that was crawled a page rank. Page rank, quite simply put, is the measure of value that Google has determined that the web page actually has. Page ranks range from 0 to 10, 0 having no value, and 10 having the most value. Some pages are not ranked at all (a different topic entirely) and those pages are marked “n/a.” Those “n/a” web pages have somehow been removed from the indexing process, and their value is essentially 0. Some webmasters feel that these page ranks are vitally important, and some webmasters feel that these page ranks have no value. It is not my intention to debate the value of Google Page Rank.

As for the term “sweep,” when I try to Google that term in relation to Google, and page rank, I can’t find anything. In fact I can’t find that Google “sweeps” the Internet at all, ever, but among my group of friends, when we see that our Page Rank has changed one way or the other we call each other up and sound the alarm “The SWEEP is happening (or happened)” and the other person usually says “Oh (expletive deleted.)” So this term “sweep” might have just been created by one of the people within my group of friends that have websites. And within that group, the term “sweep” means that the spiders did their crawl thing and Google has done its re-indexing and page ranks have been adjusted. From what we have come to learn, Google does a “Major sweep” about four times a year. Sometimes more often, sometimes less often. You see, Google likes to keep all of the webmasters guessing as to when these activities will occur. And Google changes their algorithm of what items they are looking for, and how much value each item will have during the ranking process.

Opinion disclaimer
Opinion disclaimer