|
|
Abandoned by Google! Googlebot, Wherefore Art Thou Googlebot?
As a search engine optimization specialist I often optimize existing web pages for small business clients, upload them to the site and see pages re-indexed by Google within a week. This only happens with existing business sites that have been online for a few years. Google seems to be updating their index as often as every other week at this point and older established sites that are already indexed seem to be re- crawled on that twice a month schedule on a fairly routine basis.
Two clients that hired me for recent work saw their rankings shoot to the top for a newly targeted search phrase in a weekend when I did optimization on a Thursday and they were ranked instantly by Saturday. Now keep in mind that this doesn't happen for everyone, only those that have been online for some period and already have significant content that simply needs tweaking and proper title and metatag information added. They usually have relatively good existing PageRank and do well for other RELEVANT search phrases already. I offer that warning only to avoid instilling false hopes in anyone hoping to achieve the same instant ranking boost overnight.
Those clients that do succeed in this way are often thrilled with the results accomplished in such short order. I'd love to be able to offer that type of ranking boosts to everyone, but some are more equal than others when it comes to easy, inexpensive SEO tune-ups that rev up your rankings overnight. Your mileage may vary.
WHY DO NEW SITES SUFFER?
What is going on with newer sites that don't get crawled for months? I've got a client, a newer attorney directory that offers tons of great information in the form of articles on specific areas of law, links to incredibly valuable and relevant legal sites and over 600,000 attorneys listed by practice area and state. Yet the site has not been re-crawled by Google for over 3 months! Now this would not be such a big issue for many sites, but this site is relatively new and we've optimized all the titles, tags & page text, created a complete site map and placed links to all these resources on the front page.
I know that the site is not being crawled because Google's cached copy of the front page shows it before we did the work four months ago, without the new links and without title tags. We've submitted the site by hand, (manually) once a month for three months via the Google Add URL page. http://www.google.com/addurl.html When the hand submission failed to get it re-indexed for four months, we submitted the sitemap page, which has not been crawled at all. Google shows only ONE page on this site, when in fact it has thousands of pages, a sitemap and dozens static pages!
Part of the problem is that this site must be dynamic, since a database of over 632,000 attorneys must be accessed, retrieved and served for any of those law firms searched for to be returned to the site visitor. Google warns owners of dynamic sites that Googlebot may not crawl dynamically generated pages with "?"" question marks in the URL. This is to avoid crashing the server with too many concurrent page requests from Google's spider. http://www.google.com/webmasters/2.html#A1
The solution to this dynamic URL problem has been discussed widely in search engine forums and solutions have been bandied about including software provided by SEO's, URL re-write techniques for dynamic pages on APACHE servers http://www.alistapart.com/articles/urls/ and PHP pages http://www.stargeek.com/php-seo.php to generate search engine friendly URL's. Others recommend simply adding static HTML sitemap pages as alternatives for the search engine spiders. In this instance the client's developer simply said "I can't do that (PHP solution) on this server". So we resorted to putting up the static HTML sitemap pages with hard-coded
URLS to the main 54 pages of the site at http://lawfirm411.com/Law-Firm-411-sitemap.html This should get at least those fifty pages crawled by Googlebot, but Googles' spider appears not to be crawling this site at all. How do we know this? See for yourself by using the following query in the search box at Google: allinurl:www.lawfirm411.com where the result page shows ONE page in the results. If you try that query on your own site (replace your own domain name for lawfirm411.com), you'll see the results lists ALL your pages.
The site home page was crawled by Google four months ago, when they took their "Cached Snapshot" of the page. You can see this by visiting the Google cached page here: http://66.102.7.104/search?sourceid=navclient&;ie=UTF-8&q=cache:www.lawfirm411.com where the date of this snapshot is "Apr 20, 2004 07:42:19 GMT" and they haven't been back since. The page in that snapshot has none of the newly added links, an outdated title tag, and old content.
This problem is not unique to this site. One client we worked with two years ago had a dynamically generated, framed site! Those two site structures have always given search engines trouble. Their site was not crawled at all and only the front page showed up. Our solution was to create a second domain (owned by the client), which had static HTML pages that precisely mirrored the content of the client's framed, dynamically generated site. Guess what happened after Googlebot crawled the static site? Google indexed the framed site in full and then banned the static site from the index! Not an approach we advocate, but the one that worked for this client.
We're still searching for ways to get Googlebot back to LawFirm411.com before creating that new static site, but decided to share this odd experience with the SEO community before going to any extremes. Google provides over 70% of most search engine referred traffic to ALL of our clients and we realized we can't site idly by and see a major client languish because Googlebot didn't like what it found at the client site on the first visit four months ago. This issue dogs newer sites in other places as well. The Open Directory Project has also become notoriously slow in adding new sites to the directory and in this case, has not picked up this site even after 6 regular monthly submissions. The web playing field may have begun tilting toward older, established sites and away from new ones.
------------------------------------------------------------
Mike Banks Valentine is the SEO for http://www.lawfirm411.com
Contact him at http://www.seoptimism.com/SEO_Contact.htm
Improve Your Small Business Online at our Ecommerce Tutorial
http://website101.com/Free-Tutorials/index.html
------------------------------------------------------------
Click Click Boom: a Linking Strategy that will Blow Away Your Competition
Web marketers, do you hear what I hear?
Duplicate the Exact Steps Used to Get a Number 1 Yahoo Ranking in Less than 30 Days
If you have ever been into a McDonalds you will understood the value of the Cookie Cutter Model. Every McDonalds you go in are the Same. The Kitchen's are laid out identically, the procedures from everything to checking the toilet paper to ordering Hamburgers are all Laid out in detail. Have you ever seen a McDonalds go out of business. McDonalds is like a legal cash printing press. If you have a Good Formula you just repeat it over and over again.
The Myth of Rankings - Beyond Search Engine Optimization
What follows is a condensed version of a conversation that happens all too frequently when I am approached by a prospect interested in search engine optimization (SEO):
Search Engine Optimization for Everyone
There are many facets to SEO and the search engines continually change their methods of indexing sites to help prevent the same sites from staying up in the ranks all the time.
Make The Search Engines Love Your Site
Most webmasters have no idea on how to make a search engine friendly web site. If you are one of them this will all change by following these steps below.
SEO Help: Dont Try to Fool the Search Engines
Writing articles is all the rage these days on the web. Or should I say getting other people to write articles for you, joining membership sites that provide "private label" rights -- that is all the rage.
How To Make Your Website More Successful? (Part II)
In part I of our series of how to make your website more successful we already showed you some important tricks to build a more successful website. This time we are going to expand the scope a little to further improve your website and to make it work harder for you on the Internet.
Site Maps: Let Search Engines Find Your Pages
With 40 million websites in existence, and more than 3 billion web pages indexed by Google at the time of this writing (July 2003), it's no wonder that more and more people are relying on search engines to find their way through the unruly world that the web has become.
Were You Bamboozled by Google?
It's still happening. "Florida," "Boston," "Ginger," and "Brandy" are just a few of the many Google updates that have hit within the last few months. And Google is still issuing periodic updates as we speak. No one can say for sure whats going on or what to expect. One thing is for sure... unless you want to get bamboozled by Google (again!), youd be wise to take action to prevent it now.
10 Easy Steps to Boost Your Search Engine Rankings!
In order that someone finds your website and buys your product on the Internet it is essential to make it known to the world. So the first thing you do is to submit to search engines.
How to Boost Your Traffic and Profits with Content!
Are you aware of how vitally important and valuable CONTENT is to your online business? In fact, content can do more to build your business and profits than just about any other resource or service available.
Use Search Engines For A Guaranteed Web Site Promotion
For your web site to succeed, you must use is search engines optimization. Web sites definitely need top rankings in major search engines such as Google, Yahoo!, AOL, and MSN. The higher the ranking, the more likely viewers will come and visit your web site. Your site should stay in one of the top positions in the search engines to draw the largest amount of customers.
Search Engine Success & The Google-Vision Secret
Want to know the secret to great search engine listings? Ignore Google.
Ten Steps To A Well Optimized Website - Step Seven ? Website Submissions
Welcome to part seven in this ten-part search engine positioning series. Last week we discussed the importance of human testing. In part seven we will cover the best practices of website submissions, where to submit your website to, and how to do so.
Search Engine Optimization: Creative Ways To Acquire Natural Back Links
Search engines use algorithms calculate the order in which the search results are displayed. Although no one outside the search engine companies know the actual algorithms, search engine optimization (SEO) experts agree that back links are heavily weighed. A "back link" is text on another website that links to your site. Search engines count these links as "votes" for your site. The more votes, the better.
10 Quick Ways To Kick-Start Your Profit Pulling Keywords
First, you must realize that targeting the right keywords or phrases is the 'key' to making any kind of profit from your site. Choosing the 'right' keywords (the exact keyword or phrase surfers type into the search engines to find your site or product) can make or break your online venture.
What Makes The Perfect SEO Firm?
SEO companies come in all shapes and sizes. You've got your solo SEOs that either a) do everything themselves and/or b) sub-contract out many aspects of each campaign while maintaining a tight control on the quality and results of the project. Then you have your big SEO firms that employ 20+ employees that handle various aspects of your account. These firms can often turn into SEO factories and can lack the ability to treat each client individually, because everything is done in bulk.
Whats with the Competition? Ever Heard of Cooperation?
I attended the SES Expo in San Jose, CA the other day. It was an hour from my home in Larkspur, CA just north of the Golden Gate Bridge. Along the way I was thinking this would be really neat to see, a lot of companies doing the same things really, and to see how they play together.
Affordable Small Business SEO: 5 Common SEO Errors and How to Fix Them
Don't put the cart before the horse.
Ranked #1 at Google for Invisible Entrepreneurs But No Traffic?
I am ranked #1 for that silly phrase at Google. So What?