Predicting Your Search Engine Rankings

I've been writing on keyword efficiency, and in a coming week I will write about one factor that an ideal keyword efficiency metric would take into account: keyword difficulty. Obviously the holy grail of keyword research is the identification of the keyword targets that, if optimized for, would provide the highest return on investment. To find it, all you need to do is:

1. Find the traffic for a keyword (or average traffic for keywords in a given market).
2. Find the keyword difficulty.
3. Using (2), predict the rank in Google your site can achieve.
4. Using (1), (3), and data on search engine click-through rates, calculate how many hits your site will receive.
5. Using (4) and data on conversion rates, calculate how many sales your site will make.
6. Using (5) and your profit margin for the product that best matches the keyword, calculate your total return.
7. Using (2), calculate how much you need to invest.
8. Using (6) and (7) calculate your return on investment.
9. Repeat for other keywords, and using (8) for each, find the keywords with the highest ROI

This week I want to introduce an idea for a formula for step 3. When meeting with a prospect the other day in a coffee shop in Coeur d'Alene, he began telling me about his practice and a new product that will soon be all the rage. I whipped out the SEOmoz Keyword Difficulty tool to get a rough and ready picture of how competitive the search market was and though I was glad to be able to go get a feel for the market beyond how just numerous the competition was, I began wishing that I could go even further than merely the difficulty of ranking for a keyword. I wished I could get an estimate of what I could actually rank for my keyword in Google.

So I started thinking of a way of trying to project what one could rank for a given keyword, given its difficulty. Here is the current state of my idea, which is admittedly in need of some development: Make a table with all the URLs you've ever optimized in column A, the keywords for which each URL ranks in column B (requiring there to be duplicates in column A), the Keyword Difficulty of each in column C, and the position of each URL for each keyword in column D. Your constants in a SERP Position Prediction Formula ("SERPPPF" for short) are: your average rank (taken from averaging column D) and your average difficulty (taken from averaging column C). The variable in your SERPPPF is the Keyword Difficulty for the keyword you are thinking of targeting.

The rank prediction formula goes like this: (aR x KD) ÷ aKD = R

In the formula above, aR (the average rank you are able to achieve for URLs you optimize) and aKD (the average keyword difficulty for terms you target) are the two constants you derived from your historical ranking data. KD (the keyword difficulty score assigned by SEOmoz's tool) is the variable that you will look up and punch into your formula whenever you want to see how well you could do for a given keyword you are considering.

Voila! Now you have a rough picture of what kind of position you could achieve for new keywords you consider, given what you've been able to accomplish historically.


WEAKNESSES OF THE RANK PREDICTION FORMULA

As it stands, this formula doesn't take SEO timelines into account. Your historical data are going to represent work in various stages of the SEO process, and this new keyword you might target comes with its own timelines as well. It also fails to take into account the resources behind the campaign associated each keyword. I've worked with some clients unwilling to invest in their own marketing and wanting me to do it all for them (which sucks), and others chomping at the bit to write high quality articles, leverage existing client and vendor relationships for link building, and coming up with neat link-baiting ideas while they lay awake at night. Ok, I lied about that second kind. But you get the point. Different SEO campaigns have different factors driving them.

One way to mitigate this weakness is to only include URLs in your dataset that represent work in similar stages of development and with similar backing. Calculating those values will have to be a post for another day! And obviously, the bigger the dataset, the better.

I think that insofar as one is trying to estimate ROI, my formula for predicting search engine rankings is certainly better than a shot in the dark.


THINKING AHEAD

You know what could be cool? If there were a way to aggregate data like this for professional SEOs, such as those in the SEOmoz community. My SEOmoz Pro account already knows my keyword targets and their difficulty ratings, as well as my rankings for them. This whole process could easily be automated.

Furthermore, why not allow mozzers to opt-in to contributing toward community benchmarks? Wouldn't it be great to see how you stack up to other SEOs? What if the Keyword Difficulty tool displayed something like "The average mozzer is able to rank 1st in Google for keywords with this difficulty rating." in its results page?

"Locksmith Supplies": A Case Study in Quickly Identifying Opportunity

I have been blogging about keyword research, and keyword "efficiency" in particular. This week I want to take a break from that series to write about a rough and ready method for quickly identifying profitable opportunities in an existing website's keyword strategy by looking at a case study.

A couple buddies of mine own and operate one of the biggest and most respected manufacturer of locksmith supplies in the US, and while I was digging around in their traffic and ranking data I happened upon a simple opportunity—one keyword target that could represent thousands of sales for them. Here's how it all went down.

By far, their highest performing keyword is "locksmith supplies". It may not be the most efficient all things being equal, but it is a high performer for their site in particular. Namely, of the set of all keywords whose monthly visits are over 10 and whose average pages per visit are over 12, traffic from this KW comprises the 3rd highest percentage of new visitors behind “locksmith supplies online” and “key blanks wholesale”. Of the same set, "locksmith supplies" generates the most amount of traffic, at more than double the second place keyword

So, out of curiosity I took this set of wholesale locksmith-related keywords, and whipped up an impromptu efficiency metric for them. I assigned a value to each word in the list based on a function of visits, pages per visit, average time on site, percentage of new visits, and bounce rate.

Performing keyword research on existing sites can fine-tune your strategy and raise your profits, because you have access to more metrics.

In this case, I doubled the traffic since it’s the most important of the values, took the inverse of the bounce rate since a lower rate is better, and 10% of the average time on site, pages per visit, and percentage of new visits, and multiplied all of the values together. The formula is pretty rough and very subjective because I weighted each value according to how important I guessed them to be, just off the cuff, and different ways of calculating efficiency would yield different results altogether.

That said, it was a very quick way of ordering a list of keywords to determine which ones to pay more attention to. “Locksmith supplies” was almost triple in its efficiency over “locksmith supplies online”. That was almost exclusively due to its much larger traffic volume, however I was able to confirm that it didn’t just bring in more traffic, it wasn't worthless by other methods of evaluation.

Sometimes we find keywords that look good from one angle but are worthless because of something else we failed to take into account.

There was also synergy among some of the top keywords. The second and third place phrases contained this one in them, the fifth place phrase was a conjugation of it, and the rest, save one, contained the CLK brand name.

So this warranted further investigation. I took the average number of hits per day that CLK garnered from this keyword, found its ranking, and used clickthrough rate (CTR) figures to triangulate how many hits it would receive if it were to rank first in Google. As far as I know there are only two places to get CTR data, and so I ran the numbers using both of them in turn. I will talk more about CTR in future posts.

From there all I had to do was chop the additional traffic volume down by the percentage of hits that would likely convert to sales, and voila! I had a projection of how many additional sales CLK Supplies could make by further optimizing their website for locksmith supplies.

But there was another approach to calculating that same figure that I wanted to try. Instead of taking their existing traffic data and using it to determine the traffic they could get for position 1 in Google, I looked up traffic volume data for the keyword (first using the WordTracker number, and then again using the Google Traffic Estimator figure) and use it to estimate their potential traffic volume instead.

It turned out that, any way you slice it, bumping this keyword up in the search engine results pages (SERPS) just a two positions could represent thousands more sales per year for my client. In a matter of minutes we were able to identify an accessible keyword target that could bring in a substantial return on investment.

The moral of the story is that keyword research needs to be done more than once. Ongoing research functions much like a sluice, filtering gold from the gravel so that you know how to really cash in on your SEO efforts.

---

For those interested, the star product that CLK Supplies sells is something they actually invented, called "Aerokey Software". It solves a problem often faced by locksmiths doing complex lock pinning. When a locksmith needs to key an apartment, school, church, office, or other group of related rooms and buildings, for which the client needs or would like a master key as well as manager and worker keys, it involves a significant amount of math (as you can imagine!). With CLK's locksmithing software, all of the work is done for the end user, even showing them a pinning chart, specifying the sizes needed!

The New Keyword Efficiency Index ("KEI3")

In my last post I introduced something called KEI, which is a measure of keyword efficiency. This week I'd like to write about another efficiency metric that was developed in response to KEI's weaknesses called "KEI3".

While some SEO tools are still only relying on KEI, an SEO tool called WordTracker began incorporating KEI3 at least as far back as 2009 (which was many, many Google algorithm updates ago ^[1]). KEI3 is calculated by taking the number of searches (and not squaring like KEI does) and dividing it by the number of websites competing for your keyword.

KEI3 = Queries ÷ Competition

Queries = The number of times per day that your keyword is thought to be used as a search.

The number of searches is derived using the same method as KEI.

Competition = The number of sites with your keyword in their incoming hyperlinks and your keyword in their titles.

Instead of using InAnchor to determine the competition for your keyword like KEI does, KEI3 uses the InAnchorAndTitle (IAAT) operator. IAAT works just like InAnchor, but adds InTitle as well. What this means is that it finds pages that each have at least one backlink with your keyword in its anchor text, and which has your keyword in its meta title tag too ^[2][3].

The main benefit of KEI3 over KEI is that it only counts webpages as competitors if they are at least somewhat optimized for your keyword (indicated by the use of it in their meta title tag), rather than counting pages that just so happen to have the keyword in the anchor text of one or more of their incoming links.

KEI3 shares many of the same weaknesses of KEI, including some not mentioned in the previous post: popularity isn't derived from actual Google data (but from the WordTrcker database, which comes with its own weaknesses), and the traffic volume sample set is pretty small (1% of US search).

A unique disadvantage to KEI3 however, is that keywords with low traffic might look really good, because for keywords with the same traffic-to-competition ratio, KEI3 isn't biased in favor of the keywords with more traffic overall. For example, let's say you are considering keyword A and keyword B. Keyword A has 100 queries and 100 competitors, and keyword B has 1 query and 1 competitor. Both keywords will have the same KEI3. However if you are going to target one or the other, wouldn't you want to target keyword A, since it has more traffic overall? The only reason you might not want to is because the competition is overwhelmingly high. But the point is that you can't tell one way or the other when all you're looking at is KEI3. Your quantification of the efficiency of your candidate keywords neither favors keywords with higher traffic when all else is equal, nor informs you of whether a keyword's traffic is too high simpliciter.

In summary, KEI3 is a measure of a keyword's efficiency with its own set of strengths and weaknesses. Using KEI3 to to determine your keyword targets is better than a shot in the dark, but it is still too simplistic like KEI.

---

[1] http://www.seomoz.org/google-algorithm-change
[2] For example, if you were to search "InAnchor:Example+InTitle:Example", Google would return pages to which one or more sites link with the text "example", which also have "example" in their titles. However, it is interesting to note that in WordTracker's case, they actually pull data from the index of a company called Majestic SEO, which correlates highly with Google.
[3] This page explains how WordTracker itself views KEI and KEI3: Finding Profitable Keywords.

The Keyword Efficiency Index ("KEI")

Do you want to find keywords for which your site has a good chance of ranking, but that will actually bring you substantial traffic? If so, you may be interested in learning about what professional SEOs call "KEI". Many SEO products will calculate the KEI of keywords that you are trying to get your site to rank in Google for. It is a single number assigned to a keyword, and the higher the number, the better. It goes up when a keyword has lots of traffic, and down when a keyword has lots of competition. For two keywords whose ratio of traffic and competition is the same, the keyword with more traffic will have a higher KEI. All things being equal, you'd prefer more traffic, right?

As far as I can tell, KEI was first proposed by Sumantra Roy, and is sometimes said to stand for "Keyword Efficiency Index", and other times for"Keyword Effectiveness Index"^[1]. It is calculated by taking the number of times for which a keyword is searched per day, squaring it, and dividing the product by the number of websites competing for that keyword.

KEI = Queries² x Competition

Queries = The number of times per day that your keyword is thought to be used as a search.

The number of times for which a keyword is searched per day is determined using WordTracker's "database of searches over the past 365 days[, which] constitutes just under 1% of all US search, and the data is gathered from metacrawler.com and dogpile.com. The database is updated every day, and new data is between 15 and 30 hours old when it hits the live servers."^[2]. The upside to this is that it is derived using real search data that is somewhat resilient to incidental and seasonal influences. The downsides are that it is not Google data and it is a small sample size. However the figure doesn't give you any sense of seasonality. So for example, if it is a tax-related keyword it might be heavily queried in the Spring, but you can only get a sense for how many times over the last 365 days it has been queried.

Competition = The number of sites with your keyword in their incoming hyperlinks.

The number of competing websites for a given keyword is determined using something called an advanced search operator—specifically the one called InAnchor ^[ibid.]. You can use the InAnchor operator in Google to search for pages that are linked to with certain words. For example, if you were to search "InAnchor:TheEspresseo", Google would return pages to which one or more sites link with the text "TheEspresseo". The words that a site uses to link to another site are referred to as that link's "anchor text". So if I were to link to my own SEO website with the words "Coeur d'Alene SEO", then "Coeur d'Alene SEO" would be considered the anchor text. Having keywords in the anchor text of a hyperlink to your site helps your site rank well in Google and other search engines for those keywords. That's why the traditional keyword efficiency metric, KEI, uses the number of sites with a given keyword in any of their inbound links as the number of sites competing for that keyword.


WEAKNESSES OF KEI

KEI doesn't distinguish between links that contain an exact match to your keyword, or those that contain only a partial match. Both kinds of links may be important, and we have actually have reason to believe that in fact partial match backlinks are more highly statistically correlated with higher search engine rankings than exact match backlinks. But KEI doesn't respect the nuance.

In fact, KEI doesn't attempt to evaluate the links to competitor sites at all. How many backlinks do the pages that these links come from have? Are these relevant links or links from totally unrelated pages? What about the links to the sites that link to the competitor sites? These factors matter to Google, but not to KEI.

Not only is KEI simplistic in its evaluation of the link profiles of the competition, it's simplistic in its evaluation of the on-page optimization of sites it considers competitors as well. It is precisely because KEI merely considers any site with an exact or partial match backlink to be a competitor, that it may count sites that aren't really competitors at all. This lends itself to exploitation: since KEI doesn't take into account the fact that anyone can link to anyone else's site from one or more of their own sites, using any anchor text they want, a blackhat SEO could "poison the well" by linking to tons of random, non-optimized sites using a keyword they suspect you might be considering, making it look like your competition is more numerous than it actually is. This could scare away rival SEO's from targeting a lucrative keyword that they could then have all to themselves.

Blackhat SEO scenarios aside, even if there are thousands of sites that are in fact trying to target your keyword and have each managed to get themselves at least one exact match inbound link, you are still only getting an idea of how 'numerous' your competitors are relative to the traffic, not how 'well optimized' each of them is.

It could be that there are thousands of poorly optimized sites out there "competing" for your keyword such that any SEO worth his salt could get your website to the top of Google.

In summary, KEI is a measure of a keyword's efficiency that goes up with increased popularity and down with increased competition, and favors popularity when all things are equal. The weakness of KEI is that it is too simplistic, failing to account for nuances in both off-page and on-page ranking factors.

---

[1] http://www.link-assistant.com/blog/studying-wikipedia’s-collective-mind
[2] https://keywords.wordtracker.com/help/metrics_explained

Welcome to My North Idaho Web Design & SEO Blog

I run a one-man search engine optimization (SEO) business out of scenic Coeur d'Alene, Idaho called The Espresseo. I started up this simple SEO blog to share some basic SEO knowledge with fellow webmasters in the area and hopefully gain some exposure to potential clients as well. I'd like to kick it off with a series on my specialty: keyword research.

This first post is simply an introduction to the idea of "keyword efficiency". Enjoy.

Knowing where to start is more than half the battle in SEO. Don't believe me? Think about it. If you rank first in Google for "blue midgets" but sell blue widgets, you've gained nothing. If you rank first for "blue widgets" and sell blue widgets, but there isn't enough traffic to capitalize on, you've gained nothing. It gets worse: If you rank first for "blue widgets", sell blue widgets, and there's plenty of traffic, but it doesn't convert to sales, you still have yet to win. Furthermore if you rank first, get plenty of traffic, and get a substantial increase in sales, but it cost so much to get there that your profit margin became too narrow, then all your effort is all for naught.

In SEO, the bottom line is the bottom line.

And the way to raise your bottom line is to find that cluster of keywords, or "search market" represeting the highest return on your investment (ROI). The way I approach that is by identifying the market with the most efficient mix of:

1. High traffic
2. Low competition
3. High commercial intent

Going through this process helps us identify markets to consider targeting. By this point we will have found groupings of keywords that are related to your product, searched for frequently (the more popular the better), plausible to rank for (the lower the competition the better), and reflect a commercial intent (rather than an information intent). Each keyword is assigned a single number as a measure of a its efficiency, and then keywords are grouped together according to different categories, and their values are averaged to give us a grade for each market. The higher the grade, the better.

Once we know the efficiency of all the potential markets, we can start to calculate your ROI. In order to calculate ROI we need to know two values: what your SEO campaign will cost, and what it will bring in.

To calculate what it will bring in we need to take the amount of traffic a given keyword grouping has, calculate how much of that we can drive to your site, project how many of those visitors will buy your product, and finally factor in how much profit (not counting SEO expenses) that would represent for you.

Your local SEO consultant or marketing consultant should be up-front about their costs (especially after computationally evaluating the market and getting to know your company and your goals), so once we know your profit we will be able to compare that to your marketing costs, and then calculate ROI. This should be done for each market (different markets may represent different products, margins, costs, etc.) and then you can sit down and make a very well-informed decision about which keywords you should target.

Figuring out where to start, answering the question "what keywords should I target with my SEO efforts?", is one of the most important steps in the SEO process. Fortunately, it's also one of the most computational. In the following series of posts I will elaborate more on keyword efficiency, specifically how it has been calculated in the past, what the weaknesses of the current methods are, where we can get our data, and how I am able calculate it now. See you next week!