Webmaster Papers








The Evolution Of Search


There have been many significant changes to the face of search over the last several years with engines becoming more intelligent than ever before. Today's users expect mainly fast, easy, relevant and satisfactory search results. In response to this search engines have responded by giving users more control over search results than ever through the emergence of alternative search engines.

One instance of these so-called alternative search engines goes by the name of Nutch (http://www.nutch.org/). Nutch is a two-year-old open source project, which has been hosted previously at Soundforge and backed by a non-profit organization. Since then it has been determined that the Apache license is the most appropriate, with Nutch no longer requiring the overhead of an independent non-profit organization. The board of directors and the developers both were in favor of the move to the Apache Foundation.

Nutch builds on Lucene technology, which was developed under the watchful eye of Doug Cutting, the primary developer for both of these open source projects. Doug has been working in the field for almost two decades and has spent three years at Apple, four years at Excite as well as 5 years at Xerox PARC, so it is safe to say that Doug definitely knows his stuff. Lucene is suitable for nearly any application that requires full-text search, especially cross-platform. It is a full-featured, high-performance, text search engine library, coded entirely in Java to implement web search. Nutch is an application; you can download it and run it. It adds a crawler and other web-specific stuff to Lucene as well as it's very own search algorithm and a link analysis module. Nutch aims to search the entire web like Google or Yahoo! but has a few tricks up its sleeve thanks to the beauty of open source licensing.

I recently had the privilege to interview Mel Strocen, the CEO of Jayde Online, Inc. (http://www.exactseek.com/), one of the Web's major online publication and search companies. Mel had some very exciting news to report on how Jayde is planning to utilize the Nutch application.

Jayde has been developing a customized version of Nutch for the last eight months and is planning to launch a search engine based on the Nutch technology within the next few weeks. The initial beta version will consist of a network of dedicated servers with an index of between 20 and 30 million website listings.

The real potential of this new search engine, and others using the Nutch technology, lies in the fact that it is open source and uses a "Plug-In Architecture". What this means is that the engine will be perpetually evolving and constantly improving to better facilitate the needs of searchers. One terrific example that shows us just how beneficial this type of open source plug-in technology can be is the FireFox web browser (http://www.firefox.com/).

FireFox, in its short existence has eaten up a significant portion of the once all mighty Internet Explorer's market share. The popularity of this browser is due to the fact that it is constantly making itself smarter. You can now find a plug-in for virtually anything that you require , ranging from web developer, downloading, and search tools to privacy, security, website integration and humorous plug-ins. You name it, there is an extension for it. The extension library consists of nearly six hundred different plug-ins and is growing daily thanks to the help of contributors everywhere.

Now just imagine implementing this type of plug-in technology to a search engine, with one type of plug-in for say searching MP3s and another plug-in for downloading PDFs. The possibilities of this new open source search technology are infinite. Now the term "open-source search engine" may make a lot of people's minds wander towards the idea of Black Hat search engine optimization. The primary developer of Nutch, Doug Cutting, feels that the closed-source advantage is not nearly as much of a factor as one might imagine it to be. The fact that the search engine is open-source allows sp@mmers to be detected far faster than that of closed-source search engines latest sp@m detecting algorithms. Either way, you know that the sp@mmers will eventually figure out how it works, the only difference is how quickly. So the top anti-sp@m techniques, closed or open source, are those that continue to function even when their mechanism is known.

Another type of alternative search engine technology has just recently been released to beta version is "Relevancy Rank" from the Claria Corporation (http://www.claria.com/relevancyrank/about/), the minds behind Gator. I had the pleasure to conduct an interview with the Vice President and Executive Chief of Marketing, Scott Eagle. He had some very interesting things to say about the launch of this new product and what exactly the benefit of Relevancy Rank has to the user. This unique search technology takes the results from the top search engines and applies its very own algorithms to output to the user the most relevant results.

Relevancy Rank is a combination of personalization, localization, time spent at any one site, click through rates as well as conversions. These are all taken into account to provide the most relevant results. "For an example, if you happened to be a zoologist who loved to search for different animals and information relating to animals and you entered the word "Jaguar" you would be returned far different results from say a car enthusiast who searched frequently for different types of vehicles and also typed in the word "Jaguar"", noted Scott. Relevancy Rank helps to provide you with the most relevant results based on your previous search behavior.

With the end users expectations continuing to grow, these twists on the way that results are gathered and displayed are an enormous help in satisfying the user's hunger to get to the results that they are looking for. I am quite anxious to see how these new forms of search technology fair out over the next several months. One thing is for sure, these new technologies are sure to revolutionize the way that web search is conducted and pave a new path for the evolution of search.

Tyler Huston is the SEO Manager for Beanstalk Search Engine Positioning Inc. Beanstalk is proud to offer their guaranteed SEO services to clients from around the world. To keep updated on the latest going's on in the search engine world watch for Tyler's posts on the Beanstalk SEO blog.

RELATED ARTICLES


The Most Important Aspect of Writing Web Copy
There is an ongoing debate about web copy. Some say that it should be similar to direct mail copy. Others state that is should be written in a more editorial, news offering style. However, both styles work. Both styles generate thousands of dollars of money for the website owners. Why is this?
Enhance Your Website With A Yahoo-Style Directory
Does your website have a links/resources page?
Writing Effective ALT Text For Images
Anyone who knows anything about web accessibility knows that images need alternative, or ALT, text assigned to them. This is because screen readers can't understand images, but rather read aloud the alternative text assigned to them. In Internet Explorer we can see this ALT text, simply by mousing over the image and looking at the yellow tooltip that appears. Other browsers (correctly) don't do this. The HTML for inserting ALT text is:
Traffic for Webmasters
"If you build it, they will come"; is an age old phenomenon for webmasters that they develop the website and visitors themselves would visit that. This may be true for only a handful of websites but the most important and crucial topic for any webmaster today is to how to get targeted traffic to his website.
The Power Is In The Pipes: How To Get Maximum Leverage From Your Website
What is the most important part of your online business?Many people would say: "my website". And that'sunderstandable ? it's the most visible part of an internetbusiness.
What is The Google Toolbar?
No matter what browser you may want to use, you should consider using Google's toolbar. Google.com, the innovative, stripped-down, add-free search engine that has taken the web by storm has provided an innovative interface through most web browsers; mainly Internet Explorer. This toolbar has many great features for searching around the Internet as well as blocking those annoying Pop-up ads that scream "BUY ME!" every twenty seconds.
Web Accessibility Myths
With more and more countries around the world passing laws about blind and disabled access to the Internet (including the Disability Discrimination Act in the UK), web accessibility has been thrown into the spotlight of the online community. This article attempt to put a stop to the misinformation that has been thrown around and tell you the truth behind web accessibility.
Top 7 Tips for Building an Antique Car Website
Like wine cars get more attractive to collectors as years pass by. The fact is there are only a finite number of cars made in the world in any model and make. As years pass by only a few of these manage to stay out of the graveyard. These are usually maintained by antique car enthusiasts. Then there is the collector who collects them for their value and sometimes as an investment. After the advent of Internet a lot of self made millionaires and billionaires are out there. These folk consider owning the antiques as prestigious. The current day business folks clearly understand the opportunity that is lying before them. This article provides 7 tips for launching a great antique car website.
Eight Deadly Web Site Mistakes and How to Avoid Them
Creating and maintaining an effective presence on the Web has become increasingly complex and challenging as the power of the Internet as a marketing tool becomes more and more necessary to entrepreneurs and emerging businesses.
Six Basic Reasons Why Visitors Stay On Your Web Site
1. The first page appears quickly.
Internet Marketing Website Promotion -The 7 Biggest Mistakes I See People Make With Websites!
1. Many people are not getting good or complete advice. Often for example people don't understand all the concepts of Internet Marketing and having a website so they simply pay to have a website developed. Often this website may look good but it falls far short in the area of being "search engine friendly". This is typically because many businesses that deal with the web are very one track focused.
Three Way Linking - Webmaster Strategy
Three way linking and concerns.
You Don?t Have to be Amazon.com to Achieve 12% Conversion Rates!
That's right. According to a recent study by Nielsen/NetRatings, Amazon.com converts 12.8% of its visitors into sales.
The Top Ten Benefits of Having a Web Site
Do you need a web site? Are you considering getting one but are unsure? Here we take a look at the main reasons why a web site could be beneficial to your organisation.
Art, Artists and the Web: Part 4--What to Do After a Website is Designed
What to do if you are an artist after you finish your website.
Search Engines and Customers Want Focused Web Site Content
How do you decide on the content, products and or services you will promote on your Web site.
Stop Losing Precious Web Site Traffic to the Dreaded World Wide Web Black Hole
You work hard to build traffic to your web page. If you are not doing 1 simple step you are loosing a portion of all your web site traffic to the dreaded World Wide Web Black Hole.
How One Word Or Even One Letter Can Boost Conversion Rates By Over 400%!
Recently I was reviewing the keyword specific conversion rate data of a consulting client of mine. I have been working with this client for a few months now, helping her improve the sales conversion rate of her website and we have had very good results, taking average conversion rates at her site from below 1% to just over 4.3%.
How To Convert More Sales On Your Website
One of the biggest mistakes that most online retailers make is they do not take into account typical buyer behaviour. The conversion from real world to online provides many benefits to the retailer, but present some real challenges for the customer because their buying decision is made more difficult in an online environment.
Build or Buy a CMS?
However, careful analyses often reveals dangerous pitfalls and serious short comings with many custom built content management systems.