Friday, July 27, 2007

Underscores in URLs are now Treated as Word Separators by Google

Till now, Google used to treat dashes and underscores in the URL differently. A dash (hyphen) between two words in the URL (i.e. keyword-phrase) would tell Google that those are two different words (i.e. keyword +phrase). In the past, an underscore between two words in a URL (i.e. keyword_phrase) would tell Google that those words are a single word (i.e. keywordphrase). Now both underscores and hyphens in the URL are treated as word separators.

One key development that Matt shared with the audience was that underscores in URLs are now (or at least very soon to be) treated as word separators by Google. That's great news, because it historically hasn't been that way. Back in 2005, Matt stated that Google did not view underscores in URLs as word separators. That meant that in a URL like Googlebot couldn't "see" the words iphone or review. Instead it read iphone_review as one word. I wouldn't recommend targeting "iphone_review" as a keyword, as I doubt anyone will be including an underscore in their Google query.

So it used to be--until now--that any benefit that you would have gotten by having a keyword-rich URL was negated by the use of underscores separating those words. TypePad and Movable Type blogs were particularly affected by this, as by default, underscores were used instead of hyphens. This new change in the Google algorithm should make bloggers using the TypePad service or the Movable Type blog software (and anyone else using underscores in their URLs) very happy, as I anticipate their Google traffic will be going up.

A few other highlights from Matt's talk that you may find interesting:
  • Matt claims that Google treats URLs with a query string the same as static URLs. Caveat: as long as there are no more than two or three parameters in the URL, that is! Put another way, you won't take a hit in your Google rankings if you have a question mark in your URL; just don't have more than two or three equals signs in the URL.
  • Matt stated that the number of slashes in your URL (i.e. the number of directories deep your page is) isn't a factor in your Google rankings. He went on to say that although it doesn't matter for Google, it is rumored to matter for Yahoo and MSN (Live Search). Matt addressed this because I specifically asked the question from the audience.
  • According to Matt, the file extension in your URL won't affect your rankings. So it's inconsequential whether you use .php, .html, .htm, .asp, .aspx, .jsp etc. The one extension you should avoid for your Web documents? .exe.
  • Matt stated it was myth that Google uses its status as a domain registrar to access domain registration data to use it as a ranking signal. According to Matt, being a registrar doesn't grant one special access to other registrars' customer data. Note that Matt didn't state whether Google is or isn't using WHOIS data as a signal. I believe they are.
  • When asked about how to get one's blog into Google News, Matt shared one of Google's requirements for inclusion: the blog must have multiple authors. So those of you wanting your blog showing up in Google News results, I hope it's a group blog!

Friday, July 20, 2007

Yahoo! Search update - July 2007

Priyank Garg of Yahoo Search has mentioned in Yahoo Search Blog that an update is going on.

Priyank Garg's "weather report" said:

We've been rolling out some changes to our fresh web data and crawling, indexing and ranking algorithms over the last few days. We expect the update will be completed by the weekend. So, as you know, throughout this process you may see some changes in ranking as well as some shuffling of the pages in the index.

We expect to see some changes in the search results in the next couple of days

Thursday, July 19, 2007

Microsoft to Add to its Network that Display Paid Search Listings

Microsoft said it will add to its network of Websites that display paid search listings. Under the program, small businesses that purchase paid search ads through Microsoft's Office Live service will see their search listings appear on Web sites operated by and its partners, in addition to Microsoft's Windows Live and sites.'s partners include the second tier search sites Dogpile, MySearch and Excite.

Microsoft's Office Live customers have access to adManager -- a Microsoft tool that allows users to select and bid on keywords to be used in paid search campaigns. Under the alliance with, adManager will now automatically extend the campaigns beyond Microsoft sites to sites through integration with's Ask Sponsored Listings service, Microsoft said.

The deal could make Microsoft a more attractive service provider for businesses contemplating a paid search campaign. The company's Web sites captured about 13% of all search traffic in June, according to market watcher Comscore. The addition of's sites to its paid search service will expose Microsoft's customer listings to an additional 5% of search traffic.

That takes Microsoft's network a step closer to Yahoo, which in June captured 25.1% of all search traffic, and Google, which grabbed 49.5% of traffic for the month, according to Comscore.

With its software business under pressure from providers of open source and Web-based applications, Microsoft is looking for ways to boost its revenue from online advertising. In its most recent third quarter, the company said sales of online services—which includes paid search listings—increased 11% to $623 million.

Given its importance to the company's future, analysts will be watching that number closely when Microsoft reports fourth quarter earnings on Thursday.

Friday, July 13, 2007

Implementing 301 Redirect Properly

What is 301 redirect?

A 301 redirect is a command used to tell the search engines that a page has permanently moved, and that you want them to index the new page and drop the old one from their index. It’s basically a change of address card for the Web. As long as everything is done correctly, a 301 redirect will ensure that you keep the rankings earned by the old page and it will prevent the duplicate content nightmare that should arise if the engines were to index box version of your site.

How to Implement 301 Redirects Using .htaccess for Apache?

Before you get yourself all excited about the fun you’re about to have, make sure you have access to your server, your Apache Configuration file, and that you can use your .htaccess files. The ability to use .htaccess files will reside in a command called "Allow Override" in the Apache Configuration file. If you do not have this access, you’ll have to first call your hosting company and get access.

Once you know that you have access to this file, your next step is to locate it. The .htaccess file is a control file that allows server configuration changes on a per-directory basis. It controls that directory and all of the sub-directories contained within. In most cases, this file will be placed in the root Web folder for your Web site. If there’s no .htaccess file present, create one.
To begin using the .htaccess file to redirect page(s) on your Web site, open up your FTP and login to your Web site. Work your way into in the root Web folder in order to access your .htaccess file. This little gem is the file you’ll need to edit. (A little piece of advice: The dot in .htaccess makes it a hidden file, so make sure your FTP browser is enabled to view hidden files. Otherwise you’ll never find it and your IT guy will laugh at you when you ask him for help.)

When you start editing the file, use a UNIX style text editor, not Notepad. Typically, an HTML editor or code editor such as TextPad works just fine. To 301 redirect pages using the .htaccess file, you will add a line to the file that will tell the server what to do. You can do this a couple of ways:

RedirectPermanent /old-file.html
Redirect 301 /old-file.html

A redirection from one domain to another would be:

RedirectPermanent /

Our IT department reminds me there are some key things worth mentioning here:
  • The first section of the command tells the server what to do: RedirectPermanent or Redirect 301. Again, these commands are equivalent.
  • The second is the relative path to the file that you want to redirect. This must be the relative path to the location of the .htaccess file (i.e. /old-file.html).
  • The last section is the full path to the new file. This is a fully-qualified URL, meaning you need the http:// (

Once you have inserted the commands to 301 redirect your pages, you need to make sure that there is a blank line at the end of the file. Your server will read the .htaccess file line by line, which means at some point you’ll need to throw them an “endline” character to let them know you’re finished. An easy way to do this is to put a blank line at the bottom of the file.
You can test that your redirect was done properly by heading over to your search bar, typing in your old domain name, and hitting search. If you’ve correctly implemented your 301 you’ll be immediately taken to the new location. If you haven’t, go play in traffic. You’re useless and should be ashamed of yourself. Zing!

How to do a 301 redirect using a Windows server?

Okay, this is a bit trickier. First, navigate your way to the Internet Services Manager (Programs -- Administrative Tools -- Internet Services Manager). Once you’re there, select the site you’re redirecting from, and right click to open up the properties menu. This will open up the Context menu. (Sadly, there is no ‘viola’ once this happens.)

Inside the Context menu you’ll see the “Home Directory” tab, click on it and select "A redirection to a URL" from the radio buttons at the top. The default choice will be to select “A directory located on this computer” but that is NOT the option you want. Once you’ve made your selection, type in the new URL you want your old domain to point to. Make sure that the “A permanent redirection for this resource" is selected in the checkboxes underneath.

The other checkboxes listed handle some different options, depending on what you want to do:

  • "The exact URL entered above": This will redirect the search spiders and users to the exact URL you entered in the "Redirect To:" box. For instance, if you somebody went to, they would go to
  • "A directory below this one": Freaky Friday images aside, this redirects a parent directory to a child directory. For instance, if someone typed in to access the site and you wanted the request to be redirected to a child directory named "tools", then they would be sent to

If you need more options for what you’re redirecting to, you can add some control variables to the “Redirect to” box where you just entered in the new URL. The most useful variable for domain redirects is $V. This removes the server name from the original request.

For example, http://www.youroldsite/directory1/page1.html would redirect to Leaving this control variable out will only redirect the home page. If other pages are ranked in your site, then it would be extremely helpful to redirect those pages using these variables also.

If you don’t have access to your .htaccess file or your Windows Server Administration Panel, you can still implement 301 redirects with code on your old pages, but you have to do with one hand tied behind your back and wearing an eye patch. Fine, no eye patch. If your pages are in PHP, ASP, Java, or any other language that allows you to modify Response Headers, simply place code at the top of each page to do the Permanent Redirect.

Does 301 Redirect HURT Rankings ?

Aaron Wall posted a case study on a recent site move. He documented the traffic to the site, before, during and after the big move. He also documented how long Google, Yahoo, and Microsoft picked up the site on the new domain.

Google to Introduce New Meta Tags

The ‘unavailable_after’ meta tag will soon be recognized by Google according to Dan Crow, Director of Crawl Systems at Google and Jill Whalen of High Rankings. Google is coming out with a new tag called “unavailable_after” which will allow people to tell Google when a particular page will no longer be available for crawling. So, it gives you the option to tell Google when Googlebot should no longer crawl that page.

This tag comes in handy when you have a limited time offer promotional page, and on this page, the promotion will expire on a specific date. For instance, if you have a special offer on your site that expires on a particular date, you might want to use the unavailable_after tag to let Google know when to stop indexing it. Or perhaps you write articles that are free for a particular amount of time, but then get moved to a paid-subscription area of your site. By using the "unavailable_after" tag, you can tell Google that they should not crawl this page, after the promotion expires. There are several practical scenarios for this tag.

For example, if you have a page that you would like to be removed from the search results at 6pm EST on July 23, 2007, you would add the following Meta tag:

meta content="unavailable_after: 23-Jul-2007 18:00:00 EST" name="GOOGLEBOT"

Once Google crawls the page and sees this Meta tag, it will take about a day for the page to be removed from the search results. Also this tag is only supported with web search. To remove the page completely from Google, including the cache copies, use the Google removal tool.
Google has also added support to control access to other types of documents, such as PDF files, audio, xls documents and so on via new X-Robots-Tag directive in the HTTP Header used to serve the file.

Here are examples of how to use the X-Robots-Tag:

Don't index this document:

X-Robots-Tag: noindex

Don't show cache or snippet in search results for this file:

X-Robots-Tag: noarchive, nosnippet

Don't index after a specific date (i.e. the "unavailable_after" tag):

X-Robots-Tag: unavailable_after: 7 Jul 2007 16:30:00 GMT

You can also combine these tags as well.

Wednesday, July 11, 2007

Tools you can Consult to do Keyword Research

Tools you might consider to do Search Term Research

Free Tools

  1. Google AdWords Keyword Tool: Enter a search term or terms, and you'll be shown other keywords related to that term. You can then sort keywords in terms of search volume popularity.

  2. Google Trends: Got some terms in mind and wondering how they've been trending, such as becoming more or less popular? Google Trends provides a way to go back in time and chart keywords, based on the accumulated search data that Google sees. Also see Google Meme: Hot Trends Added To Google Trends, which explains more about the day-to-day reporting you can get from Google Trends.

  3. adCenter Labs Keyword Forecast: Similar to Google Trends, this shows you the popularity over time of terms you enter. In addition, you can see how popular these terms are by age and gender of those searching. adCenter Labs also offers a variety of other keyword-related tools.

  4. Wordtracker Free Keyword Suggestion Tool: Enter a term and get a list of 100 terms related to (and including it), ranked by daily popularity. Data is based on Wordtracker's search term data collected from the Dogpile and Metacrawler meta search engines. Wordtracker also offers a much more robust fee-based keyword research tool.

  5. Trellian Free Search Term Suggestion Tool: As with Worktracker above, enter a term and get back a list of 100 terms related to and including it. They'll be ranked by how often they're searched for each year, rather than each day. Data comes from a variety of search engines. Trellian Keyword Discovery is fee-based tool hitting the same database and providing more features.

  6. Yahoo Keyword Selector Tool: Long-standing tool offered by Yahoo (formerly Overture) that provide searches done on the Yahoo network in a given month. It no longer works as well as in the past, however -- it often fails to provide information or when it does, the data is old. A new tool is offered to advertisers who are logged in, but it fails to provide as specific of figures as the old tool or sort terms by popularity.

  7. SEO Book Keyword Suggestion Tool: Taps into the Yahoo tool above and provides links and information to many of the other tools also listed above, so that you can get more information directly from them. A variety of other keyword tools are also offered.
Other Tools

  1. dWoz Keywords & Search Phrases: A collection of places that provide keyword suggestion data, tools and search "voyuer" services.

  2. Hitwise Search Intelligence: Provides the ability to tap into Hitwise's database of how people search, based on ISP data it gathers. Offered on a custom pricing basis. Hitwise Keyword Intelligence offers fewer features but is also available for a lower, fixed cost.

  3. comScore qSearch: Allows you to tap into comScore's database of search activity, gathered from a large panel of web users that it monitors.

  4. eBay Marketplace Research: Mine data on how people are searching at eBay, for a fee.

  5. Good Keywords: Free software for Windows designed to help with keyword lists.

  6. Google Suggest: Start entering a word, and you'll be shown popular searches that are related (based on spelling) to that word.

  7. Google Webmaster Central: Provides a way to see the top search terms that are sending you traffic, according to Google.

Wednesday, July 4, 2007

Google Websearch / Search URL Parameters Explained

If you are really interested in knowing the different parameters appended at the end of the Google search URL, then go through the list given below. The list is originally created by Joost de Valk containing the different parameters you can add to the Google search URL.

Parameter Description (Value)
Search modifiers
q / as_qthe search query
as_epqmatches exact phrase, same as searchphrase surrounded by quotes in searchbox
as_oqwith at least one of the search terms, same as searchterms combined with OR in searchbox
as_eqwithout these searchterms, same as searchterm prefixed with - in the searchbox
qdr / as_qdr Date restriction, show results up to either 3 months old, 6 months old, or 1 year old (3m 6m y)
as_occt query term occurs in (any title body url links)
as_filetype specify filetype (pdf ps dwf kml kmz xls ppt doc rtf swf)
as_ft exclude or include solely files with filetype given in as_filetype (e i)
sitesearch the same as site: (URL with or without http://)
as_sitesearch same as sitesearch, but shows site:URL in the search box, can be used together with as_dt (URL with or without http://)
as_dt together with as_sitesearch shows you results from URL, or from all except that URL (i (only this URL) e (except this URL))
as_rq find sites related to URL, same as related: in the search box (URL)
as_lq find sites linking to URL, same as link: in the search box (URL)
hl specifies the interface language
lr language restrict ()
ie input encoding
oe output encoding ()
safe safe search filtering (active (default) off)
filter 0

Result modifiers

num number of results (<0..100>)
newwindow open results in new window (1)
pws disable personalized web search (0)
adtest allows testing of ads without registering an impression, url's are not clickable (on)

btnG normal search from webinterface
btnI I'm feeling lucky search (I%27m+Feeling+Lucky)

Localization (taken from:

gl country, for instance "US" ()
gll latitude&longitude, (,)
gr region, for instance GB-ENG ()
gcs city, to use this gr must be set to the same value ()
gpc postal code, only works if gl is set to "US" ()
gm US designated metropolitan areas ()

The Blogosphere