Every now and then an article comes along that you could have written yourself.  Align Your SEO Strategy With Site Structure summarizes my core SEO strategy quite well.  I can’t put into words how much value I know is hidden on the words of that page.

The article has one flaw.  It only looks at two aspects of a website, the site structure and SEO Strategy.  It misses the whole idea that those two elements are a subset of a number of things which all must be working together to have a site that not only not only ranks well in search engines, but actually deserves the ranking achieved.

The Whole Picture:

• Site objectives
• Content
• Site structure

All of those elements, together, must be aligned and tightly knit together for a strong website.  If either one of those four items is lacking, you’ll have a site worth less than the sum of the parts.

Are your site objectives not aligned with everything else? You could have first page rankings for high volume keywords, have excellent easy to find content that people love to read and even tell their friends about, but not accomplish what you set out to achieve.  It’s a failure.

Bad content? Traffic comes in like a tidal wave from terms perfectly on target with your site objectives.  One problem.  Lots of them to be exact.  One hit wonders.  People are not satisfied and hit the back button a few seconds in.

Bad site structure? You have it all.  Traffic by the freighter full, content that people love, and visitors who fit your target site objectives perfectly.  Each visitor lands on a page and instantly thinks “this is the best content I’ve ever seen!”  They read the page three times from top to bottom.  They are intensly interested in the content you have to offer.  Unfortunately, they don’t see that your site has more to offer.  The site navigation doesn’t match their needs.  They hit the back button;  you failed.

Bad SEO Strategy? Great site, no traffic.

The Ideal SEO Strategy?

Align site objectives, content, and site structure while applying solid SEO tactics to all three.

The most common reason to use a friendly URL for search engine optimization has to do with keywords the URL giving the page a boost for searches with those terms.  While that seems to be true,  there is an additional reason that I don’t see discussed often.  The Human Algorithm.

First, what’s a friendly URL?  Let’s look at an example:

http://www.bitworm.com/search/2008/long-tail-keywords/

vs.

http://blog.searchenginewatch.com/blog/050314-164653

If you search for something related to the “long tail”, saw those two in the search results, which would you click?  Why?  Your “human algorithm” probably decided the first one is more relevant than the second, even though both pages in reality have to do with the long tail of search.

I know, there is more to the search results than just the url.  The are:

1. The Title
2. The Description
3. The URL

The battle for search traffic is not about ranking well.  Really, it’s not!  It’s about ranking well AND getting people to click your listing.

Your best chance of getting a click, especially if you don’t rank #1 for a phrase, is to have all three items (title, description, url) targeted to trigger the human brain to see relevance to the search intent.

It’s the human algorithm.  

Just as a search engine makes split second judgements to decide which 10 pages out of a billion are most relevent to a query, the human algorithm takes those ten results and determines which page to click.  

For me, and I’m sure many other searchers as well, this process happens in a second or two.  In that short amount of time, I know I’m not reading every single word being returned, but my brain quickly scans for signals of relevance and quality to chooses a page to click.  Keywords in the URL is one of those signs of relevance.  

Your goal is to not only be at the top of the search engine algorithm, but the human algorithm as well.  Friendly URLs help with both.

I just read an interesting post over at SEO Bandy about valid code giving a significant boost in the search engine rankings.  In a way it makes perfect sense, and in a way it’s hard to believe.

What makes this more interesting is that there are roots in a Matt Cutts quote.  Matt is in an interesting position.  He knows a lot of ranking information he can’t directly talk about.  That’s why his words are examined by SEO’s is if this were Wall Street disecting the words of the Fed Chairman.  Could be nothing, or he could know a lot more than he’s letting on.

I’d agrue returning “valid” pages is in Google’s best interest in terms of generic page quality.  A web page that passes HTML validation should in theory render well in any web browser a searcher happens to be using.  If a page isn’t valid, it’s a gamble whether the page will render for the searcher, although in practice the vast majority will render well enough.

From that standpoint, I can see why Google may give a boost to valid pages.

On the flip side, experience tells me that the vast majority of pages on the Internet won’t pass a validation test.  Sure, most are very close to valid, but few actually pass.

I’ve thought for a long time that there is a certain quality bar that needs to be crossed.  For example, I wrote a crawler recently which collected some key stats for every page on a site.  I was suprised to find a handful of pages where the resulting info just didn’t make any sense.  On investigation, it seems extra body tags were the culprate.

My crawler established it’s own quality bar.  Each of those pages with extra body tags would have been improperly indexed by my crawler, and possibly Google’s as well.

So, is having code that passed validation really a factor?  Is this why I never run into “best view on Internet Explorer” pages any more?  I don’t think the experience of SEO Bandy is enough to base a decision and reaction on…but I smell a SEO test in my future.

Google’s chrome isn’t a direct attack on Microsoft, unlike what the general consensus of many news agencies and blogs seem to be reporting. 

At least, it isn’t any more of an attack than Ford’s Model T was an direct attack against buggy whip manufacturers.  Ford didn’t set out to hurt the buggy whip companies, but rather to make a car everyone could afford.

Might Microsoft be hurt?  Probably.  However, any pain would be collateral damage due to lack of their own proactive action.  Words mean things.  “Attack” is a word that implies intent.  I don’t think that’s present here, or at least not as a primary driving factor.

Let’s look at the current day reality of the browser market and Google apps.

Google makes advanced web applications that push the limits of the current browser technology, just like how Microsoft’s Windows OS pushed the limits of Intel’s hardware with each release.  Moore’s law is a great thing for Microsoft.  The hardware keeps get better by orders of magnitude.

Here’s the rub – Google has pushed the limits of the current generation of web browsers, while the browser technology has stood still.  Yes, FireFox 3 is an improvement, as is the IE8 beta, but they’re both merely a step in the right direction, not the leap that is needed…and Google’s apps still don’t have any head-room.

Google needs drastic improvements in certain features to push their overall apps and technologies forward.  They’ve been waiting a while, and the market hasn’t responded, so they’ve decided to take matters into their own hands. 

The browser being Open Source is evidence of this.  They don’t really want to “own” this thing, but they know they need it badly enough to make it themselves.

To Microsoft, this may look like an attack.  From my view of the current browser market, it’s a company that is reliant on a commodity product deciding the commodity quality is not acceptable, and taking action in the most direct way possible.

Performing SEO for an already live site is much different from integrating SEO during a ground up site build.

Analogies are a simple tool that we can use to explain technical SEO concepts to our clients.  Here are a few that I’ve found myself using recently with a baseball analogy that have been well received and gotten the point across.

1. You may strike out…and that’s OK

Generally speaking, professional baseball players strike out two out of every three times they step up to the plate.  On average, if this was the only indicator of success, they are failures.

Despite striking out two out of every three times, runs are still scored and the game always ends with at least one team with one or more runs.  Each team gets a minimum of 27 times at bat.  Some strike outs are OK, and long as there are some hits in the mix.

Not every site tweak or change will get you ahead.  Due to the massive complexities and mystery surrounding the algorithms and data, some changes may even set you back.  You should never hinge an SEO project on a single change.  A multifaceted approach is key.  On the whole, you will still end up ahead.

2. Don’t always swing for the fences

Not every hit is a home run.  If fact, most hits only get you to first base.  It is entirely possible to win a baseball game one base hits alone.

In SEO, not all site changes will equate to homerun traffic increases.  Many small traffic improvements can add up.  In fact, I’ve more than tripled traffic for an established site without achieving a single high volume keyword ranking.

If you try for a base hit each time up at bat, you are more likely to succeed than if your strategy is to swing for a homerun every time.

If you target popular high volume keywords phrases to the exclusion of everything else, you may bite off more than you can chew and, in the end, have no traffic increase at all!

3. Base hits are investments in future potential runs

Runs are all that count in the game of baseball.  Nobody ever wins for getting players on base.

Search optimization is about getting runners on base and waiting for them to come home.  This is not a game of short feedback loops.  Although the timing of things is getting shorter, you may not see the results of specific optimization changes for days, weeks, or even months after a specific change is made.  If you sit around on your keister waiting for results before you make the next change, you’ll only ever take baby steps.

You must continue making changes based on the belief (based on experience) that the optimizations you are performing will pay traffic dividends down the road.  If you perform a single optimization then wait to see results before doing more, you will lose the game.  The feedback loop takes too long.

Web Analytics tools are nice to show you how many people are visiting your site, but their value can go well beyond that. Here is one simple way you can use your web analytics package to gain information you can immediately use to improve your site and generate more traffic.

There are two reports in most analytics packages that show how people found a specific page on search engines.

Keyword Phrase Report - This report show the entire phrases that searchers typed into the search box to find your site. “Blue Widgets”, “Red Widgets” would be an example.

Keywords Report - This report list the individual words searchers used to find your page. “Widgets”, “Blue”, “Red” would be an example.

I will explain on way to get information you can immediately act upon out of your Keywords Report.

On the surface, especially at the site level, this report may not tell you anything you don’t already know. If your site is about widgets, “Widgets” will of course be at the top of the list day after day, likely followed by the other primary keywords your site generally targets.

Filter the Keywords Report down to the single page level.

Pick a specific content page to target and improve, and filter the Keywords Report to only show keywords for that one page. Ideally, pick a page that gets decent long-tail traffic. If you can, open up the report in a window right beside the webpage itself so you can easily compare the two side by side.

Look at the words people are using the find the page. You will probably see a long tail trail off of a handful or two of words a lot of people use, trickling down to lots of words only a few people use.

Start by focusing on your top 5 words. Look at how you use them on the page. Check the title tag, headings, and links. Usually this won’t be too surprising. The top 5 words are likely prominent in these places.

Keep going down the list until you start seeing keywords that you don’t prominently use. What are they, and how do they relate to the page? This is where the opportunity is hiding.

Are groups of people finding the page using a word you didn’t expect or target?

Take Action!

1. Re-write your title, headings, links, and content to include some of these newly found keywords. However, don’t remove or de-emphasize the other more important words. If that doesn’t make sense for this page and these words, then:
2. Consider creating new pages to target these specific keywords.

Not every page will have these opportunities staring you in the face. Pick 10 pages to go through, and a bet you’ll find a couple good opportinities for improvement.

Don’t force these changes into any page where they don’t make sense. In my experience, the best page modifications not only include these keywords for search engine sake, but also make the page better for human visitors as well, after all, you’re now speaking their language.

It’s been said that Google takes over 100 different considerations into account when figuring out where to return your page in a search result. Even though that sounds like a lot, all those 100+ factors can be boiled down into two core concepts. To rank well, you need to maximize both.

Ranking = Strength + Relevance

Page Strength

I know a lot of SEO’s love to say PageRank is dead. I disagree, and see evidence of it every day. Sure, PageRank is certainly not as important as it used to be, but it’s certainly not dead.

PageRank is a measure of how important a page is on the Internet as a whole. Every page that links to your is essentially counted as a vote of confidence for the page. Links from important pages like the New York Times count more than links from lesser pages like your brother’s personal blog about his dog. To over-simplify things greatly, Google’s PageRank score is a sum of the value of all those votes in comparison to all the other pages on the web.

There are other factors adding to a page’s strength as well. Such as:

• Is the page new?
• Has the page and/or domain been around for many years?
• Is the domain registered for a long term between renewals, like 5 years?
• Is the domain or page penalized for not following the webmaster guidelines?
• Is the domain selling links?
• Is the page located in Wikipedia? (OK - maybe not that one)

I know I’m missing some factors, but you get the general idea about what might play into strength.

What’s clear though, is that your page is never going to rank well on strength alone.

Ranking well needs one additional key factor…relevance.

Query Relevance

If strength was all that was required, a site like www.google.com would rank #1 for all queries…it’s a PageRank ten out of ten. That obviously would not make sense.

Relevance is a measure of how related a page is to what the user has search for. Your brother Joe’s blog about his dog, even though not a strong site, may very well be the most relevant destination for people searching for “joe dog blog”.

Relevance factors include:

On-Page:

• Query Terms in the title?
• Query Terms in header tags?
• Query Terms in body text?
• Query Terms in outgoing link text?

Off-Page:

• Query Terms in incoming link text?
• Query Terms in “on page factors” of the pages linking in?

It’s possible for a page to rank well based on a strong relevance while having a low pagerank, which is why some people say pagerank is dead. (That doesn’t mean PageRank is dead, it just means there are other factors than can overcome a low pagerank.)

Burn the Candle on Both Ends

Strength and Relevancy are simple high level concepts that can be easily explained and understood by both technical and non-technical users alike.

Some tasks, like link building, will have an impact on both.

For a strong ranking, your strategy should be to maximize both strength and relevancy. Don’t give one the blind eye.

A couple months back, I decided to take a calculated risk with my most popular page; I cut off my head. It’s worked out for the better, and is an interesting case study. Let me explain…

A page on one of my sites was ranking a very solid #1 for a single head keyword. That’s great, right? Maybe.

Let’s say my site is about Widgets. This one head keyword, let’s call it WidgMaster was bringing in a decent amount of traffic.

While the page was certainly related to the WidgMaster variant of Widgets, it was also very relevant to learning about widgets in general. As evidence, the page currently ranks on page 3 of Google for the much more broad “widgets” term.

In terms of traffic, the page was successful, but unnecessarily pigeon-holed into a very small niche.

It’s worth noting that this page is linked to more heavily than any of the other internal pages on this site, having been on the front page of Slashdot once, and digg twice. It has more “link juice” than any other page on the site, and pulls in three times as much traffic as the homepage.

I wanted to see if I could make the page pull in a more general audience. I was looking for more “Widget” focused long tail traffic, as described very well recently in “Deep Links, Longtail Keywords, and Why you Should Love Them Both.

I performed a simple and minor, but crucial strategic change. I figured the chance of success was 50/50, and there was certainly potential for traffic loss.

I reordered the words in the page title.

Learn about WidgMaster Installation and Configuration - Widgets

Became…

Learn about Widget Installation and Configuration - WidgMaster

Almost immediately, I lost the #1 spot that I had held for WidgMaster, which had been solid #1 for over a year. The page now ranks around #6 on Google, high enough to still get a trickle of traffic, but less than 10% of what was previously coming in for that keyword.

Interesting, and supporting my initial hypothesis, the traffic levels stayed about the same.

I was hoping for an increase, but the resulting wash isn’t all bad.

The page now ranks much better for more general long term “Widget” phrases. The resulting visitors are less WidgMaster focus “one hit wonders”, and much more interested in sticking around my site after their initial landing.

While this strategy is not right for all situations and pages. Much more usable for “short head” terms than “tall head” terms, it’s certainly a strategy and method that you should consider experimenting with on a small scale, and having in your SEO Bag o’ Tricks.

SOLR bills itself as an open source enterprise search engine.  I would not go as far as to call it “enterprise,” but certainly believe SOLR is a nice value added wrapper around the already powerful Lucene package.  It may become “enterprise class,” but it has a ways to go.  That being said, it’s certainly good to aim high!

Raw Lucene search engines have a few issues that SOLR very nicely addresses.  SOLR addresses Lucene issues including:

• Platform Lock-in (Java)
• Indexing requires custom Java coding
  
• Existing document update issues
• Index Replication
• IndexSearcher warming (fake automated queries to prepopulate the cache)
  

SOLR runs as a java web application.  The nightly version os SOLR that I downloaded came bundled with Jetty in a very easy to  run, test, and even deploy into a light duty production role.

Instead of being a Java library that you use directly from your Java code, SOLR works as a web application that you POST documents via HTTP to index documents, and query using HTTP GET requests.  This HTTP interaction means your application does not need to be written in Java.  Your application can be in any language that can post data and request data via http.

When it comes to setting up your index, instead of having to code in your field information into Java code. It is setup in an XML file.  This file, among other things, includes the list of document fields and document primary keys needed to maintain your index.

If you are familiar with Lucene, my mention of the primary key may have peaked your interest.  To those not familir with Lucene, it does not have any notion of primary keys or document updates.  To update a document in Lucene, you first needed to locate and delete the previous version of the document through a rather indirect process.  When provided with a primary key, SOLR will handle that process automatically for you.

The SOLR example that is included with the nightly build includes a failr simple script and sample documents to show how indexing works.  Run the examples to index the samples, then you can run searches through the admin interface.  To use the search results in your own application, you query the same url as shows up in the admin interface, and parse the xml response.

As an initial experiment, I have used SOLR to index product information on my sheetmusic site.  It’s used for the sheetmusic searchengine as well as related products query.  My searchengine implementation is only really “quick hack” quality (I have not implemented next/previous page links yet) , but the related products part usage is more polished.

In some instances, indexing into RAM rather than direct to disk can create a large indexing performance increase. Here’s one way to do it. You may need to increase the Java JVM memory parameters with the arguments -Xms128M -Xmx256M, of course modifing the sizes to fit your needs. Tweeking the foldCount size with affect how much memory is required by setting how large the RAMDirectory is allowed to grow in terms of the number of Lucene Documents it can hold. Each time the foldCount is reached, and/or when indexing is complete, the index will be flushed to disk.

Lucene Example Code: RAM to Disk

int foldCount = 500000;
int indexSize = 0;
int count = 0;
try {
   RAMDirectory ramDir    = new RAMDirectory();
   IndexWriter  ramWriter = new IndexWriter(ramDir, analyzer, true);

   IndexWriter writer = new IndexWriter(indexDir,analyzer,true );
   writer.mergeFactor = 100000;

   while(rs.next()){
      Document doc = new Document();
      ramWriter.addDocument(doc);
      count++;
      indexSize++;
      if(indexSize == foldCount){
         foldToDisk(ramDir, ramWriter, writer);
         ramWriter = new IndexWriter(ramDir, analyzer, true);
         indexSize = 0;
      }
   }

   foldToDisk(ramDir, ramWriter, writer);
   writer.optimize();
   writer.close();
} catch (IOException e) {
   e.printStackTrace();
}


public static void foldToDisk(RAMDirectory ramDir,
			IndexWriter ramWriter,
			IndexWriter writer) throws IOException {
		ramWriter.close();
		Directory dirA[] = new Directory[1];
		dirA[0] = ramDir;
		System.out.print(".");
		mergeDirs(writer, dirA);
		System.out.println(".");
}