Marketing

How to measure SEO success or failure?

Measuring the impact of your SEO work is a necessity regardless of the nature of the project you are working on. I share my thoughts on how to create predictions for SEO, what KPIs are worth tracking and how.

Norbert Hires

30 May 2022 — 14 min read

When I started in SEO I felt setting up (and commenting on) monitoring reports a waste of time. I could write articles, fix technical shortcomings or build links for clients in that time instead of talking about what's going as planned or where we are falling short.

But I was wrong.

Sometimes it matters more how you present the results than the results themselves. If you are working with clients or internal stakeholders and not on your pet project then this is even more true. The correct way of measuring the impact of your SEO initiatives can make or break a project and can make you win or lose clients.

Sure. Bad SEO monitoring is not as tragic as a migration without redirects, but still: not being able to create a strong case for SEO based on your past success can result in the same outcome.

In this article, I share my biggest learnings when it comes to measuring the impact of SEO. In the past few years, I worked with clients in an agency setting, and with agile teams as an in-house SEO, while running and measuring my own silly SEO projects.

What will we cover?

This is not an in-depth article about the perfect SEO KPI or SEO monitoring in general. However, we touch on several important elements of how to measure SEO success:

KPIs for SEO (not necessarily SEO KPIs)
How to monitor these metrics?
What tools to use for SEO monitoring?
How to make predictions?
What is messing up our predictions?

Who is this article for?

I am writing partially for fellow SEOs, but mainly for Product Managers, Product Owners, and leaders who are making a case for SEO in their respective companies.

When I get asked the same question all over again I usually try to get a better understanding of the problem. Thinking through a recurring challenge in writing is a good way to clarify my thinking while hopefully solving a problem for others as well.

This piece of writing can be useful to you, if:

you have SEO projects within your company, and you are reviewing KPIs set by others while don't have a clue about SEO at all.
you are an SEO / PM and struggling to select the best KPIs, and monitoring tools for your project.
you are expected to do traffic projections but don't know-how.

Good enough KPIs for SEO

When I say there isn't a perfect KPI for SEO, I mean it. I have reported based on most of the industry-wide accepted metrics but after at least a quarter with each of them, I had to realize all of them have different shortcomings.

Sampled data, database increase, messed up measurement, unexpected trends, and several others can mislead your calculations and make you (and everyone around you) think about whether it makes sense to follow your chosen number at all.

When choosing a KPI to measure the progress of your SEO project you have to accept that you can not make perfect predictions whichever you choose.

The goal here is to pick a metric that is reliable enough to influence your behavior when something goes wrong and is trustworthy enough to back up when you reach your goals.

From money metrics to pure SEO metrics

Revenue, transactions

If you are making a case for SEO you will probably ask for money, or time to devote to SEO (which are cost at the end of the day as well).

Questions are naturally arising in most of these conversations like:

How will the project impact the business?
How much money will the project generate?
What's the cost of inaction?

You are often expected to tie SEO impact to business-relevant metrics. I also think metrics that are closely tied to how the given business makes money should be used as a KPI. This helps you work on opportunities that truly matter and avoid chasing vanity metrics.

You can measure

revenue,
form submissions,
subscriptions,
leads,
transactions

which are accredited to SEO / Organic Search based on your analytics system.

In an ideal world, it would be this simple. In reality, I have never seen any example when measuring a revenue-related metric that could give a perfect overview of how an SEO project is going.

There are too many interconnected pieces that impact how revenue is generated within the business. You can increase relevant organic traffic while revenue is decreasing because of forces outside of your control:

The business can run out of stock for important products.
They can change pricing.
New features and new designs can mess up the conversion rate and require more traffic to reach the same results.

And we didn't even touch cases when conversion tracking is not working as expected and you successfully increase leads from Direct traffic or from Paid Search.

However, if you have a good measurement setup and you have visibility for most of the aspects impacting your chosen metrics outside of SEO, then revenue-related metrics are the best to follow.

Users / Sessions / Page views

The most well-understood and accepted KPIs for SEO are Users / Sessions coming from Organic Search.

Your data source is most of the time Google Analytics or some form of GA alternatives like Adobe Analytics. To work well with these metrics you need to have at least a limited understanding of the platform your company is using for analytics. This can be considered a disadvantage compared to Google Search Console-based metrics which most SEOs are already familiar with.

While we have 16 months of data for Search Console (if we are not storing SC data ourselves), until then we have a much larger timeframe in most of the Analytics tools working with Session-related metrics. This can come in handy when doing year-on-year comparisons or projections.

Users and Sessions could be a good KPI for SEO, but they don't show you the complete picture. You have page-level performance but an accurate way of measuring keyword-level data is missing.

If you are working on a site with a matured brand then you can not measure effectively branded and non-branded traffic using only Users / Sessions.

Clicks, Impressions, Average ranking, CTR

Search Console provides the best data on what happens in Google Search. You can filter to pages, keywords, and devices. You can differentiate traffic coming from Google Discover, Image Search, and a lot more.

You can measure the impact of SEO initiatives very well by using data provided by Google. However, there are several shortcomings worth considering if you pick one of these metrics as your KPI.

First of all outside SEOs, almost nobody knows these metrics. You have to explain what they are and convince others to care about them. Often you have to show the connection between clicks and sessions, which makes the question obvious: why don't you use sessions as a KPI then?

Another shortcoming is that you have sampled data, which makes your life very hard when you are working on a bigger site. Simply you won't have any data for certain keywords or page filters even if you know you are ranking for those terms and have traffic there. You can decrease sampling by adding extra Search Console properties for parts of your site, but even this can't solve the problem perfectly.

If you have the idea to pick a bucket of keywords and check their average ranking as your KPI using Search Console data then you will be surprised that often you don't have accurate data. You can not measure precisely gain or loss in organic traffic this way because missing data is impacting averages.

Using these KPIs as site-wide health metrics also can be misleading.

You should increase overall clicks in the long run. That is a valid expectation. But changes in Impressions, Average ranking, and CTR can lead you astray without context. Having a goal like decreasing site-wide average rankings is for example counterproductive.

Lower average positions do not necessarily mean better results. If you are creating new landing pages and ranking for a high number of new keywords then your average position will go up but the number of clicks as well. Also if you are losing visibility for non-branded keywords then your average positions will decrease but clicks as well.

Better average positions do not mean more traffic.

Site Visibility, Number of keywords

SEMRush, Ahrefs, and Sistrix have a solid database of keywords. When something worthwhile happens (like a Google Update) checking metrics like "Number of keywords in Top 3 positions" can provide valuable insights especially given that you don't have to wait three days like in the case of Search Console.

However, using these signals as KPIs to report is risky. These providers are always tweaking their algorithm, increasing their coverage of keywords, or sometimes even dropping keywords.

If you are planning to use them, then it is a better idea to use their Rank Tracking functionality and monitor the performance of your top keywords.

You can use average ranking and visibility there.

Which metrics should you use as an SEO KPI?

SEO should contribute to the main KPI which drives the business, but SEO is not the only area that impacts that. This makes it hard to use only one broad metric to measure the impact of SEO initiatives. If you can't impact the whole user journey then it makes sense to go one level deeper.

But keep in mind that every metric which doesn't have a link to revenue can be misleading. Increasing the number of sessions from SEO does not matter if you can not monetize the traffic.

Flexing on skyrocketing ahrefs traffic charts doesn't mean anything without context.

Proxy KPIs

SEO is more than traffic and can be a slow and unpredictable process.

How do we make sure that we are going in the right direction?

While monitoring traffic-related metrics often it makes sense to track deliverables. It is good for SEO's mental health, especially until we don't see any visible results on traffic from our efforts.

At the early stages of an SEO project, we can track the input but not yet the outcome.

Improving metrics that we know can have a positive impact on traffic over time as a supporting KPI can come in handy.

Some of these can make sense to follow:

Core Web Vitals
The ratio of Crawled / Indexed URLs
Indexed URLs without Clicks
Errors / Warnings / Excluded URLs in SC
Published articles / Generated landing pages

How to monitor KPIs?

You got your number to track.

What now?

Tools

You need to pick your tools. Sometimes your metrics determine the tool, but there are exceptions.

My weapon of choice is most of the time Data Studio because I can use metrics from different tools with it and create custom dashboards to have a more comprehensive overview.

But if you use the native UI or take advantage of the automated reports of any SEO tools I can not blame you.

Tools worth considering to track the impact of your efforts:

How should you measure the impact?

It is not the case most of the time that you will have an enormous site-wide impact and incredible year-on-year growth. If you are working with sites that were optimized before often it happens that your predecessors already took advantage of the best SEO opportunities.

Site-wide impact measuring makes more sense to be able to track the impact of Google Updates, check when your competitors are actively doing SEO and getting ahead of you, and monitor SEO fuckups released without your supervision. And over time with accumulated small wins your work may have a noticeable impact as well.

What are the alternatives for measuring site-wide impact?

First, you can narrow down your focus to the given part of the website you are working on. If you are releasing landing pages under a subdomain, or you are working on a given category, then URL filters are your friends and Data Studio is your biggest ally.

Following a similar logic but a different route is selecting a cluster of keywords and tracking your performance against them. We always find opportunities first and optimize pages later. Why shouldn't we track our positions for the keywords we are actively chasing?

Whether you plan to check site-wide impact, measure the performance of a subsection of your site or track your positions for a selected group of keywords you need to set up your measurement tool of choice first.

If you plan to use Google Search Console or Google Analytics then you need to make sure then they are correctly set up, but apart from that, you don't need extra work.

For 3rd party data, you need to set up your project before any action happens. This is especially true for tracking a cluster of keywords.

How to make predictions?

My take on any prediction for SEO is summarized best by the Danish proverb that states:

"It is difficult to make predictions, especially about the future,”

But anyway most of the time you are expected to come up with some sort of number. If you start with a good KPI even then you can mess up a project with too ambitious predictions or simply by not taking into consideration certain events (we touch upon this later).

David Epstein wrote about predictions made by experts in his book, Range. Several pieces of research show that the more experience and credentials you acquire, the less chance you have to make good predictions in your field of expertise.

So we are not in a good position when creating predictions for SEO, but there is still some useful information worth incorporating to be as precise as possible.

Top-down or bottom-up forecasting

We have two scenarios when creating predictions.

With top-down forecasting we take a general metric or trend we would like to influence and use that as the starting point of our forecast. Then we find the opportunities to make that growth possible.

For example, a top-down forecast is a +20% growth in page views from SEO or a +4% increase in revenue from SEO.

Bottom-up forecasting is taking the opposite approach. First, we collect the opportunities, then we summarize their potential impact and calculate how they can affect our KPI.

For example, by launching a new category we can increase page views from SEO by 4%, by optimizing images we calculate with a +2% increase so we can have an approximately +6% increase in page views from SEO with our initiatives.

From an SEO point of view, it is more accurate to go with bottom-up forecasting. Choose this approach when you can. But in reality, we often have to declare the target, and just after that can we start the discovery of the initiatives we will actually work on.

Make predictions with caution

In reality, our predictions can be far away from reality with both approaches. We don't know the exact impact our initiatives will have. Even with the number of keywords we can target, search volumes and average CTR benchmarks often we can go astray.

I suggest setting reasonable confidence intervals on projections (don't promise incredible growth in the first month) and highlighting the limitations of any SEO predictions.

Events outside of our control often mess up our predictions.

I collected some of them which you should watch closely.

Elements making predictions harder

Continuously changing SERP landscape

Google is always experimenting with new features on the Search Engine Result Pages. Often these changes lead to a decrease in Click-Through-Rate and a decrease in attainable clicks at the end of the day.

When Google introduced Google Flights, the SEO predictions (or even the business model) of the flight comparison websites proved entirely wrong.

With Google Jobs, they stepped on the toe of job marketplaces decreasing their organic visibility in several countries. Google Shopping further decreased the achievable clicks for another huge set of keywords.

The Knowledge Panel made it harder to drive traffic from keywords with informational intent. Do not even mention paid search placements and their searches over time.

Next time maybe there is a change that will hugely impact your site. Are you in the car buying/selling business? It is worth checking Google Vehicle then.

Google Updates

Google is changing its algorithm on the go but still, there are bigger updates that shake up the status quo. The bad thing about Google Updates is that most often you don't know why you won or lose traffic.

The big algorithm updates don't target specific issues or niches. They don't even have a name anymore, they are just "Core Updates".

It can happen that you lose a significant amount of keywords and organic traffic while there is nothing wrong with your site. Even Google's official response is that often there is no problem with your pages and you can do anything about the impact of an update:

"There's nothing wrong with pages that may perform less well in a core update. They haven't violated our webmaster guidelines nor been subjected to a manual or algorithmic action, as can happen to pages that do violate those guidelines. In fact, there's nothing in a core update that targets specific pages or sites. Instead, the changes are about improving how our systems assess content overall. These changes may cause some pages that were previously under-rewarded to do better." - Source

Will a Google Update happen in Q2?

I have no idea.

Can a Google Update make your predictions nonsense?

Absolutely.

You should count on the possibility of Google Updates, but you can not really prepare for one.

Non-predictable trends in search demand

Predict your SEO Traffic based on last year, do nothing, and fail to hit the numbers. If all the above mentioned frequent changes wouldn't be enough there are global trends that impact how people search:

COVID-19: The start of lockdowns was the first time I saw extreme changes in YoY clicks. Some enormous websites simply lost 50% of their organic traffic YoY simply because people didn't care (and search) for the type of keywords they were still ranking in the number one position.
Global conflicts: After the Russian invasion of Ukraine on the 24th of February similar changes were visible in several countries. There were certainly more searches for keywords connected to the conflict that "benefited" news websites but the event was hard on other industries. You don't plan your vacation or daydream about buying a new car when much more important things are at stake.

While the above-mentioned unpredictable events have a real impact on how people search (and how many visitors land at your site), they also have a cascading effect.

Just think of the normalization of remote work, the big resignation, the supply chain issues, the global car chip shortage, and the decreasing stock prices.

Do they impact how people search in your industry?

Or do they affect whether you have products on stock or not?

Probably.

Hard-to predict true impact

We have some benchmark metrics but they are far from being precise. Even calculating with average CTRs can mislead our predictions since they vary in countries, in SERPS with different intent or with the number of paid placements present for the keyword.

There are too many unanswered questions we would need to make precise predictions. You see a keyword with search volumes, you have content for it and can create a page targeting the keyword.

But after that:

If you create a new page will it rank?
In which position will it rank?
How many users will click through?
How much will convert?

Sometimes we just don't know the impact, but we know it is the right thing to do.

How do you quantify this?

To sum it up

Try to find a KPI which shows the impact of SEO initiatives metrics relevant to the business.
Use supporting metrics to measure your progress when the impact of SEO is not visible yet.
Choose your tools for measuring impact wisely and set up them correctly before the start of the project.
Use bottom-up or top-down forecasts, but always point out the limitation of SEO forecasts.