There is a lot of talk about the importance if high search engine ranking. Almost everyone agrees that few people go beyond the first two pages of search engine results pages (SERP), so it’s critical that your targeted keywords place you in the top twenty sites listed. Any further down and your chances of getting search engine hits on those keywords are slim.

Getting Indexed

To raise your ranking you need to do a few things. First and foremost is to get your site indexed by the search engines. You can have the best looking, flashiest site around but if the search engines don’t find you or cannot crawl your site you will never see results. So a good first stop is to make sure that you submit your site to the search engines. To do this, you can submit your site name by hand to the major three search engines. Easy enough, right? But how? Well, just check out the following links:
  Google: http://www.google.com/addurl
  Yahoo: http://siteexplorer.search.yahoo.com/submit
  MSN/Bing: http://www.bing.com/docs/submit.aspx

Webmaster Tools

It may take a few days or even weeks before the search engines will get around to crawling your site. Hey, the Internet is big and these crawlers are looking all over the net. But in the mean time, you should determine how “readable” your site is to the search engines. The easiest way I’ve found to do this is using Google’s Webmaster Tools. Go to http://google.com/webmasters and create an account if you don’t already have one. Once you do that, use the “add a site” button to add your domain to the list of managed sites for your Google Webmaster Login. You will need to verify that you are the site owner before going much further. Once verified, there is a suite of tools that you have available. Among these, on the rights site of the page is a heading called “Crawl errors”. Here, you will find all the URLs that Google has found for your site. Which URLs may have problems, cannot be found, or are restricted. This is the area you should focus on. Any problems listed here means that Google is unable to find your content. And if Google can’t find your content, it may not be getting indexed. Just as important, if Google can’t find it, the other search engines are probably having similar problems.

Sitemap.xml and Robots.txt

While you’re here, you can check and make sure that your sitemap.xml file is correct, as well as your robots.txt file. There are links on the site that tell you what these are and how to set them but, but in essence the sitemap.xml file is used by the search engines as a starting point. It is a list of the URLs on your site and their relative ranking in importance. Other pages on your site that are linked to by the URLs listed in your site map will also be found so it’s not really critical to have every URL listed, but the more that are listed, the more likely it is that Google will index them all.

The robots.txt file is used to tell the web crawlers how to access your site and to exclude certain URLs from the search engines. Why would you want to exclude a URL? If your site has a login page does that really need to be indexed? Is the content of that page contributing to your overall PageRank? Probably not. Nor is the page that is displayed when you enter a bad password. These merely distract the search engines from focusing on your site’s real content: your products. You can also exclude a terms of service page if you think that some keywords on that page may interfere with your ability to rank well with those keywords on your products pages. These pages can also clutter up the search engine results with URLs that won’t help much in converting visitors to customers. Proper use of the sitemap.xml and robots.txt files can help you guide the search engines to where you want them to be spending their time on your site, rather than getting distracted on less important pages and content.

Keyword Ranking

The Google Webmaster Tools also gives you some very valuable information on how you rank with some keywords. It can automatically determine some of the keywords for your site (as described in your keywords metatag) and where you rank with that keyword. Any keywords listed here as ranking more than twenty is where you need to focus. You can decide to not attempt to rank higher with that keyword, or make changes to rank better with a given keyword. But knowing where you currently stand is invaluable information in this process.

Search Engine Optimization takes time and effort. You need to constantly check how you’re doing, make adjustments and track how your changes effect your ranking. When something works, keep with it. If it doesn’t work, try something else. Done properly, you can slowly crawl up the ladder and rank well with your chosen keywords. Using these tools helps a lot in the process. And since they’re free and available, you can be certain that your competitors are using them.