• Recently Written

  • SEO for an existing site – It’s like baseball
  • Mine your Keywords Report - Take Immediate Action
  • The Two Key Factors for Ranking Well in Search
  • Cut-off Your Head; Grow a Longer Tail
  • Introduction to SOLR “Enterprise Search”
  • High Performance Lucene Indexing
  • Multi-Field Lucene Example
  • Simple Lucene Example
  • On-Page SEO is not about Optimization
  • MSN Comes Clean on Fake Search Traffic

• Categories

  • Lucene
  • SEO
    • Google
    • Microsoft
    • PageRank Sculpting
  • Web Analytics

• Archives

  • July 2008
  • May 2008
  • February 2008
  • December 2007
  • October 2007

• Blogroll

  • Bruce Clay
  • Marketing Pilgrim
  • Matt Cutts
  • Search Engine Journal
  • Search Engine Land
  • SEO Book
  • SEOmoz
  • Webmaster World

• Admin

  • Login

• Search

On one of the large sites I SEO and monitor traffic for, we’ve seen a large bump in search.live.com traffic over the past few months. The keywords used don’t always make sense for the page being landed on, and the keyword phrase is always a single word.

I’ve assumed (shame on me?) that it was a bug somewhere related to search.live.com that is only allowing the first keyword through to our WebTrends Analytics tool. Andrew Urquhart has commented on his similar theory for the problem.

For yesterday’s stats, MSN had a significantly large volume of traffic that just didn’t seem to make any sense. If it’s wrong, it’s no longer statistically background noise, but rather seriously impactful data that puts the legitimacy of the log data into question.

Looking for answers, I directed my browser over to WebMasterWorld to check out the MSN forum. The Strange Referrer Activity post was right near the top. Among other things, this post on Sept. 5th 2007 was interesting, and a major cause for concern:

Thanks for all the feedback on this thread.

First, we appreciate the concerns and issues that have been raised and apologize for any incovenience this might have caused.

Second, we want to explain what this is all about. The traffic you are seeing is part of a quality check we run on selected pages. While we work on
addressing your conerns, we would request that you do not actively block the IP addreses used by this quality check; blocking these IP addresses could prevent your site from being included in the Live Search index.

Please keep the feedback and thoughts coming as we will use this to help improve this process and make sure that it impacts your sites as little as possible.

thanks
- msndude (msd)

That seems to confirm that at least some of this may be bogus traffic. How much is bogus? We have no easy way to know…

That should be very concerning to any webmaster who values the credibility of their log data. If nothing else, that puts thousand of historical search.live.com referrals for the past few months into question.

It concerns me that:

1. Microsoft seems to not seem to care about messing with the validity of log files for websites on a global scale.
2. Microsoft is making it look like websites are getting human traffic from search.live.com which in fact may be a Microsoft bot. I heard the phrase “fake it until you make it”, but that should not be a valid search engine market share tactic!
3. There is no obvious and/or official way to correctly cleans the corrupted log files of this bogus traffic. (Not to mention the time I need to spend to do it.)
4. Microsoft has mostly been silent on this issue.
5. The bot is ignoring the robot.txt rules, and does not identify itself as a bot.

I don’t follow MSN very much since they don’t send very much traffic, so I don’t really know where to look for these things, but I’ve found no official mention of this other than the WebmasterWorld post.

Frankly, this seems to be to be very unethical of Microsoft on a number of levels. For one thing, many website opperators probably think they actually ARE getting an increase in traffic from search.live.com.

If you run a Website, this should concern you greatly. We need answers and solutions. Please comment if you have either.

Matt Cutts opened up a whole new world of possibilities when he told Rand from SEOmoz that it’s acceptable for a site owner to use nofollow on internal links to control the flow of PageRank within a site.

There’s a whole can of worms that has been opened up by internal use of nofollow being considered white hat. Matt muddied the waters a bit in this comment, but still indicates internal use of nofollow is a white hat tool webmasters can use by saying “It’s available if you want to get into that much fine-grained control.”

So far the best information I’ve seen about this is the Third Level Push (modified Siloing) For Deeper Index Penetration. That’s an excellent post, tool, and process, but it’s not the whole story. It’s also a little hard to get your head around.

On most sites, especially smaller ones, I don’t think PageRank sculpting is generally required. It may actually cause more harm that good.  On larger sites, especially where SEO is not a primary concern, PageRank Sculpting could make a huge difference in how linkjuice flows through your site.

Here’s one example.

PR Sculpting using internal nofollow tags may be helpful for a site with extensive multiple layers of site navigation as part of the template for a site. (Think your entire sitemap on every page.)

The one example I’m thinking of right now has over 80 links in css based drops downs in the top navigation bar. Granted, there are over a thousand pages on the site, so it still only links to a small fraction of the overall pages.

If your site only has a handful of site wide navigation links, link sculpting isn’t for you. However, if you have multiple levels of CSS based dropdowns leading deep into your site, including to areas not very valuable in terms of search traffic, you may be a good candidate for internal use of nofollow.

Having a large quantity of deep links from every page on your site can only help, right? Well, maybe not. By linking to a large number of pages from your site template, you end up spreading your link juice very thin over your entire site, like peanut butter on bread. While that may be what’s best, sometimes that does more harm than good.

Instead of spreading your link juice thinly over a large portion of your site, you may be better off strategically directing a larger portion of your PageRank to certain pages by properly implementing nofollow on some internal links. We’ll get into that in future posts.

Is your site navigation like peanut butter on bread? Is it helping or hurting the PageRank flow within your sites?  This is like a sharp knife…sometimes it’s the best tool for the job, but if you’re not careful…it could hurt or kill your site.

← Previous Page