Web site optimization encompasses a number of processes and techniques to compose, lay out and improve the contents of a Web site. Preferably, a Web site should be optimized before it is submitted to a search engine. Web site optimization can be a rather lengthy and tedious process, and can sometimes turn into excessive nit-picking. Nevertheless, certain aspects of the optimization procedure should never be neglected.

Site Optimization Elements
The Web site optimization procedure includes the following main elements:

Search engine selection
Zooming in on a few key search engines and optimizing the Web site for their particular search criteria is generally the most effective way to improve the site's Web visibility.

Although several influential search engines and directories, including AltaVista, Google, the Open Directory, and Yahoo! allow free Web site submission to their indices, the industry trend appears to lean toward paid inclusion as more and more search engines either add paid, enhanced services to their offerings, or transform themselves entirely into paid-inclusion engines.

Implications of Paid Inclusion
Generally, paid inclusion will secure a shorter turnaround time and in some cases might boost the submitted site's ranking. Turnaround time is the time that elapses between the site being submitted and the search engine's editors getting around to reviewing, ranking and ultimately displaying the site in its listings. Turnaround times can vary significantly. A site that has been submitted at no cost might wait quite a long time before it is eventually reviewed.

Depending on the individual search engines' submission procedures a paid site submission might guarantee a close-to-instant turnaround. Generally, each engine will indicate the estimated turnaround time for submitted sites.

While free submission offers no guarantees regarding a submitted site's ranking, paid inclusion sometimes does. However, the only safe method of securing an opportune ranking for a submitted site is to use pay-per-click advertising and outbid the competition for a given search phrase. Still, even pay-per-click advertising might only secure a ranking for a limited amount of time, as other advertisers could decide to raise their bids and thus snatch the top ranking.

The flip side to paid inclusion is, of course, the one-time or recurring cost of having a site listed. As well, a one-time setup fee is often required. Although bulk discounts sometimes are available, submitting multiple URLs can be costly. In the case of pay-per-click advertising, the charge, of course, depends on the amount bid and the number of visitors to the site.

The most advanced — and most blatant — type of cloaking is IP cloaking. In order to use IP cloaking successfully, the Web site owner must know the Internet addresses major search engines are using when they access the Web. When a request comes from one of the known addresses, the Web site delivers the custom content. Requests from any other IP address will trigger the actual Web site content.

Besides increasing search engine rankings, cloaking is sometimes used to protect a site's <meta> tags. This is usually done by Web sites that operate in a highly competitive environment. Under such circumstances, cloaking can prevent competing Web sites from reading and copying a highly ranked Web site's code.

Why Not to Use Cloaking
To many search engines cloaking equals spamming. And the ramifications of submitting cloaked pages can be rather severe. Google, Lycos and HotBot are among the search engines that have implemented zero-tolerance policies on cloaked Web sites. If cloaking is detected, Web sites will generally be temporarily or permanently removed. With or without prior warning. At the very least, the ranking that was gained through cloaking will disappear. Therefore, Web developers are generally advised not to use cloaking as a means to boost Web site rankings. — Or to conceal code from the competition, for that matter.

Generally, search engines catch cloaked pages by revisiting indexed sites on a regular basis. Because there is no exact way to predict a search engine's revisiting intervals, Web site owners who use cloaking to secure their search engine rankings most likely will get caught sooner or later.
 

Regardless of the reason, the standard method of doing so is to create a "robots.txt" file, which can prevent search engines from visiting and indexing particular pages within a Web site. A robots.txt file can also be used to entirely block search spiders from crawling a site. Robots.txt files must be placed in the root server's HTML directory and can therefore only be used with Web sites hosted on servers that allow the file to be placed in their root directories.

The robots.txt file can either block spiders entirely, or it can prevent the search engines from accessing and listing specific directories, files or entire Web pages. To create the robots.txt file, open Windows' Notepad or any other plain-text editor. When the file is completed, upload it to the Web site's root directory. The robots.txt file must include the following code:

user-agent: Googlebot disallow: /

The first line —“user-agent:” — specifies which agents, spiders or browsers should read and obey the commands in the file. The second line — "disallow:" — defines which files and directories should be blocked from the search engines. An asterisk ("*") denotes "everything." If you wish to target Google's spider only then use the following syntax:

user-agent: disallow: /

Preventing Google From Caching Pages
Google routinely stores many of the indexed pages in its cache. This allows users to view copies of those sites, even if the servers they reside on are temporarily unavailable. If so desired, you can prevent Google and other spiders from saving cached versions of your page content. To do so, insert the following command in the <head> section of the Web pages you don't want archived:

<META NAME="ROBOTS" CONTENT="NOARCHIVE">

The above command tells robots not to archive the page. Google will continue to index and follow links from the page, but will not present cached material to users. If you want to allow other robots to cache your content, but prevent Google's robots from doing so, use the following tag:

<META NAME="GOOGLEBOT" CONTENT="NOARCHIVE">

The change will take effect the next time Google crawls the page containing the NOARCHIVE directive in a <meta> tag.

About Google Spamming

Due to its dominant position on the search engine market, Google is a particularly tempting target for search engine spammers. Google in return has implemented strict anti-spam policies accompanied by a number of more or less severe countermeasures against perceived spamming.

Basically, Google bans most of the same spamming methods that other leading search engines do. And depending on its nature and severity Google spamming may lead to anything from having no impact on a site’s ranking, to a severe drop in its ranking, to permanent or temporary removal of the site from Google’s index.

In its Webmaster guidelines Google lists the following directions:

These quality guidelines cover the most common forms of deceptive or manipulative behavior, but Google may respond negatively to other misleading practices not listed here, (e.g., tricking users by registering misspellings of well-known Web sites). It's not safe to assume that just because a specific deceptive technique isn't included on this page, Google approves of it. Webmasters who spend their energies upholding the spirit of the basic principles listed above will provide a much better user experience and subsequently enjoy better ranking than those who spend their time looking for loopholes they can exploit.

It is highly recommended that you comply with Google’s anti-spamming guidelines.

Conclusion

Only snippets of Google’s ranking algorithm are disclosed to the public. Still, it is clear that among the more than 100 factors Google are believed to incorporate into its calculations, link analysis is the single most important element. Web site owners that aim for opportune Google rankings should focus on link building first and foremost. A reasonable number of quality links to your Web site will help boost the site’s ranking. Other important factors include frequent, but not excessive keyword usage, unique and well-compose page content, and proper HTML code.

Google makes no guarantees as to the inclusion or eventual ranking of indexed material. Therefore, no one can guarantee a high Google ranking for any Web site.

Obtaining an opportune Google ranking is a demanding, time-consuming and ongoing task. But for those who are willing to apply the necessary time and efforts to pursue the desired Google results, the hard work should eventually pay off. And once it does, that Google ranking is worth its wait in gold as you will gain unrivaled visibility on the Internet.