By Lee Cummings, June 2001
The World Wide Web has evolved from a
library of static web pages to an increasingly database-driven
and dynamic place. Ecommerce sites are springing up everywhere, and
even the most primitive of these is powered by some sort of database
mechanism. Unfortunately, in the haste to make a working shopping system,
the consideration of the search-engines is often set aside, and
the Web is awash in millions of pages which are completely invisible to
One reason for this is that search-engines
tend not to follow links with question-marks (?) in them, as that
clearly designates the page to be a script, as opposed
to a static HTML page. But why should the server be so worried about
indexing a script? There are several good reasons, and one might conclude
that the search engines are justified in their policies:
- A script implies that the output is in a state of change; if it was always the same,
why use a script? Since pages remain in the index for months if not years before
being re-indexed, it may not make sense to index very timely and changing
information at all.
- Some scripts map a space which is infinite, or extremely large. For example, consider a web script which plays a game of chess. Efforts to map a space of this size would consume massive resources, and ultimately fail.
- Disreputable practices that some web programmers use against the search
engines, such as cloaking, are more easily facilitated when scripts are indexed.
The consequences of this policy are simple to
understand: database driven pages are almost entirely absent in the search
engines, and that includes the product pages of over 90% of all ecommerce sites.
If you are involved in an ecommerce venture,
this news may be particularly disheartening. Go to your site and observe the
URLs of the product pages. Product pages are the most important pages on your
site for useful search engine traffic. If your URLs contain a "?", or the phrase
"/cgi-bin/," or simply the telltale ".asp" extension, you know that these pages
are not fully accessable to the search-engines.
What is to be done? One popular remedy
is to use cloaking, the disreputable technique mentioned above. In short,
the idea is to create a "dummy site" that contains normal HTML pages that
mirror the real site. When the search engines come, it sees only the
static HTML pages, which it indexes perfectly. But when real visitors come,
they get quickly whisked away to the corresponding page on the real server.
There are three problems
with cloaking for ecommerce sites. First, it requires a large amount of
maintenance, which would be unneccesary if
the site was constructed in a search-engine friendly manner to begin with.
Second, the search engines are strongly opposed to this practice, as it
interferes with their methods of rating web sites. In extreme cases,
this can trigger a retaliation, such as banning your site from their service.
Thirdly, search engines like Google are becoming very sophisticated
in their ability to rate a site based on incoming links (sites that link to you.)
The "dummy site" that is required for cloaking will suffer when evaluated
by such criteria; therefore, you may find that it's hard to drive significant
traffic to the site using such methods.
The best method is to simply
take search-engines into account from the very beginning. This relates
largely to the subject of compatibility. Just as we consider
WebTV users are real, legitimate users of our services, and make
accommodations for them, we must view
the search engines as a kind of user with a somewhat limited browser. The
search engines really want to index our pages, it's the designers
of ecommerce systems who have created the obstacles. This doesn't
mean abandoning scripts and databases, just to make sure that the
crucial product pages appear to the search-engine to be a normal
HTML page in every respect.
One online company who got their
technology correct from the beginning is Amazon. Their system is clearly database driven,
and yet the pages appear to be normal HTML pages, and do not
contain the telltale question-mark. In combination with their excellent
affiliate system, affiliates were able to insert their Amazon pages into
the search engines, providing the site with a constant stream of
highly targeted traffic.
When 20/20 Technologies set out
to design an ecommerce system, the search-engine compatibility issue was
paramount. By studying Amazon and other successful ecommerce systems,
we perfected a method of presenting normal HTML pages to the engines
which are, behind the scenes, dynamic database driven pages. The final
product, dubbed Cart Nouveau, implements the best ideas we learned
from Amazon and others. Skeptical
at first, our doubts vanished when we saw one of our clients secure the
#1 search result in Google for two highly competitive terms.
If you are involved in an ecommerce
venture, don't make the same mistakes that thousands have made before you.
Recognize that the ecommerce platform and search-engine promotions are inseparable. If you are interested in Cart Nouveau,
or if you have any questions,
please contact us today for a complete evaluation of your company's online promotion needs.
Copyright © 1994-2005 20/20 Technologies
718 NE 11th Avenue · Pompano Beach, FL 33060
Phone: (954) 934-7184 · Email: email@example.com