Webmaster Papers








Site Maps: Let Search Engines Find Your Pages


With 40 million websites in existence, and more than 3 billion web pages indexed by Google at the time of this writing (July 2003), it's no wonder that more and more people are relying on search engines to find their way through the unruly world that the web has become.

Nowadays, it is crucial to get your pages indexed by the most important search engines. To maximize traffic to your site, you must make sure that all your internal pages are indexed, not just your main page (homepage).

Fortunately, you don't need to submit each of your pages manually. The most efficient way is to create a Site Map (a list of links to all the pages in your site) and link to it directly from your homepage.

How Will A Site Map Help Me?

Search engines find pages by "crawling" the web. They go through the code of all the pages in their database (also called index), following links to other pages and adding them to the database (in fact, more pages are added this way than by manual submission).

However, search engines have trouble following links from pages buried too deep within the directory structure of a site. A Site Map solves this problem by giving the engines access to the links to all your pages once they follow the Site Map link in your homepage. For more effectiveness, place your Site Map in your root directory (where your index page is).

Site Maps: Not Just for Search Engines

While some web users will find their way through your site by following navigation links or by using the search box, others will turn to your Site Map. If you design your Site Map carefully, it will not only be useful to the search engines, but to your human visitors as well.

Here are some pointers:

  • The Site Map should act pretty much like the table of contents of a book.

  • The Site Map must clearly show all the sections of your site, and the information contained in each of those sections.

  • Every item in your Site Map must be hyperlinked to its URL.

  • If it's not too long or cumbersome, use each page's TITLE as the link text, since this tends to increase the relevance of your site. Otherwise, use the word or the short phrase that best describes the content of the page.

  • Make sure that you place the link to your Site Map at a visible location in your homepage (users shouldn't need a map to find your Site Map!).

  • Don't get creative: simply call the link "Site Map".

  • Make your Site Map a simple text link. If you use javascript the search engines will ignore it.

How can I check if my pages have been indexed?

Once you have created and uploaded your Site Map and placed a link to it in your homepage, submit both your homepage and your Site Map page to the search engines. You will then have to wait until the search engines do a web crawl. In the case of Google, the largest search engine, this happens approximately once a month.

To check if a page on your site has been picked-up and indexed by Google's, go to www.google.com and use the "allinurl" command in the search box:

allinurl:yourdomain.com/yourpage.html

Where "yourdomain.com/yourpage.html" is the URL of the page you want to check.

To get a list of all the pages in your domain that have been indexed by Google, you'll have to use the "site" command, followed by your domain name plus a word (or group of words) that you know appear in all your pages (for example, a copyright statement or some footer text):

site:yourdomain.com commonword

If after typing this command you get a list of all your pages (or at least a significant number of pages that weren't in the index before), this will be a strong indication that your Site Map has been successful.

-----

You can freely reprint this article. Just include the following resource box at the end:

About The Author

Mario Sanchez publishes The Internet Digest ( http://www.theinternetdigest.net ), an online collection of web design and Internet marketing articles and resources. You can freely reprint his weekly articles in your website, ezine, newsletter or ebook.

RELATED ARTICLES

Local Customers Know Where to Find Local Businesses... the Internet
Through search engines and directories, the Internet provides a quick and easy place to choose a local merchant or business.
Keywords, Ranking, & Search Engine Optimization Fun
I am a Search Engine Optimization newbie. I have read a little on various forums, browsed a few articles, and read through The Affiliate Masters Course (Ken Evoy) a couple of times.
?Web Content Management System fr Window?: Search Engine Typos
Oops! I meant "web content management system for windows." Do search engines understand consumer search engine typos? Typing something so close to what you are looking for, like typing "web content management system fr window" instead of "web content management system for windows" may not seem like a big deal, but search engine bloopers alter consumer searches more than we know.
Link Swapping - How to Win the Website Marketing Game in 3 Easy Steps
If you are a webmaster, chances are one of your main concerns is the ranking of your website with the major search engines. Since you are concerned with this, you are probably involved in acquiring links to your websites on the Internet to further increase the link popularity or your site. The reason gaining links to your website is important is because many of the large and popular search engines focus on the amount of links there are to your site when they begin ranking your site. The more links you have, the higher your website will rank. So, if you are not familiar with how to increase your marketing by acquiring links then you need to learn. Fortunately, it is not difficult and will not take a lot of effort, but you will need to be open minded, patient, and always play fair. Consider the following suggestions when you are beginning your link swapping or reciprocal linking campaign.
Search Engine Optimization - A Beginners Guide
Getting your site listed in the top search engines, such as Google, Yahoo, or MSN is no small job. There is lots of work that needs to be done to guarantee the highest placement possible, and even more work is needed to keep your ranking for any period of time. Here are some simple tips and strategies to keep your site listed, and listed well, without spending any extra cash on pay per clicks.
Its Not Just All About Google Anymore
Those webmasters that stick to the old ways and focus entirely on Google are missing out on a lot of search traffic these days if they are not also well ranked by Yahoo and MSN.
Google Patent Application - User Data As Part of Ranking Process
In this third article, we continue to dig into the patent application of Google regarding the method used to rank sites in search results. Interestingly, Google asserts in the application that it "might" consider user data as a factor in organizing search results.
Does Google Hire Mad Scientists?
Online search giant, Google, often tests the waters for new services by rolling out a "beta" (first) version of an idea to gauge public reaction.
5 Simple Steps to Great Search Engine Rankings
Having your website rank well in the major search engines is crucial to a successful Internet business. However, the information on doing so, which abounds on the Internet, can be perceived as complex and contradictive. Many website owners end up overwhelmed by all the data and simply quit and move on to other areas of promotion. By following the handful of steps presented in this article you will be well on your way to solid rankings.
Google Local Search And The Impact On Natural Optimization
With the advent of Google Local, a service that helps Web users find local businesses by typing in a search term and a city name, many questions arise concerning its impact on Natural Optimization.
PageRank and How It Gets Assigned
We know that each and every website page is assigned a Google Page rank, based upon a mathematical algorithm. Pages rank on a scale of 0 (zero) the lowest, and 10 (ten) the highest. Linking between websites both internally and externally pass a value or Page Rank.
The Business Case for SEO
It's interesting how potential clients have preconceived notions about which aspects of search engine marketing have the most value. In fact, they tend to fall into two camps that are 180° apart. The first camp believes completely in the value of pay-per-click marketing (PPC). It's easy to understand why. PPC provides immediate and measurable benefits. The ROI of PPC marketing is obvious. This group doesn't understand why it's necessary "to bother" doing SEO. The second camp believes the only way to go is SEO. Clicks are free and the branding benefits of high rankings have been well documented.
How to Verify and Monitor Your Search Engine Listing on Google?
Being listed in search engines and ranked high on searches is the overall goal a webmaster is trying to achieve when dealing with search engines. Search engine optimization is probably one of the most commonly used words among webmasters. In the Internet market becoming much more critical to business success this is almost natural.
The Secret Benefit Of Search Engine Optimisation: Increased Usability
A higher search ranking is what many website owners dream of. What they don't realise is that by optimising their site for the search engines, if done correctly, they can also optimise it for their site visitors.
Dont Make the Top 30 SEO Mistake
SEO consultants will tell you that you need to be in the Top 30 or you can pretty much give up any hope of getting a visitor to your web site.
Non-Reciprocal Link Building For Higher Search Engine Positioning
Non-Reciprocal Link Building For Higher Search Engine Positioning By Dave Davies, Beanstalk Search Engine Positioning, Inc.
Are You Losing The Battle For Search Engine Traffic?
Search engine traffic should be a priority for any online business and some level of optimization is apart of every effective marketing strategy.
Search Engine Optimization Strategies Guaranteed to Skyrocket your Rankings
The point of optimizing your website is so that you will get ranked higher in the search engines and receive more visitors to your site. As a result, you will increase sales and revenue. Consider the following tips so that you will be able to optimize your site and be returned as a higher result in the search engines.
How to Protect Your Search Engine Placement by Keeping Up-to-date on Industry Changes
There's no denying it: Search engines are a dominating force on the Internet, with millions of people going online to search on their topics of interest every single day.
How To Get Number 1 Spot In Google, Every Time, Guaranteed
Woaah! Wait a minute. Before I exclusively reveal these insider secrets can I ask you question? Well, actually, I want to ask you a series of questions.