Why It's Getting Harder to Rank for Some Commercial Keywords (+ What You Can Do About It)

Our inbound lead generation is highly dependent upon SEO — it’s how we get more than 70% of our leads. So, naturally, we got a tad nervous when we saw some of our top commercial landing pages dipping out of top-five positions for their targeted search terms.

When we took to the SERPs to investigate who was unseating us, we discovered something equal parts unsettling and interesting: Our high-value, commercial-intent landing pages weren’t being displaced by competitors. They were being ousted by business directories and “listicles” that rank or compare multiple companies.

In fact, an evaluation of seven of our most important commercial keywords demonstrates a 50% increase in the number of lists and directories in top-five organic SERP real estate year-over-year (excluding SERP features such as People Also Ask).

Here’s an example for the keyword “SEO content writing services.” The image on the left is the SERP in 2022, and the one on the right is the SERP in 2021:

Notice the difference? The Featured Snippet, which we owned in 2021, is gone, we’ve dropped to position 5, and there are now three lists in the top-5 organic results where there used to be none.

We saw this again and again, and it left us with three questions:

1. Is there, in fact, an intent shift happening here, where Google is favoring lists, directories, and company rankings (consciously or unconsciously) over traditional commercial landing pages?

2. If so, what can be done about it?

3. What does this mean for the future of SEO?

Let’s tackle them one at a time.

Part 1: Are there more lists in 2022 than in years prior?

The answer: Based on a random sampling of commercial-intent keywords, yes. To what extent and who is most affected are the key questions, though.

Let’s look at how we figured this out, and evaluate the evidence

First things first: What do we mean by “lists” in SERPs?

For our purposes, we’ll define a “list” as any Google search result that compares, ranks, or evaluates multiple businesses. An example is something like, “8 best coffee grinders of 2022.” A Clutch directory listing for “Top Content Marketing Agencies” would also apply.

These could be off-site content from a third party writing about other companies, or an on-site list in which the publisher shrewdly ranks multiple companies, including their own. Here is NP Digital doing that very thing:

These different formats of “lists” are unified by a comparison-type commercial intent.

Second: How do we see a SERP change over time?

If we want to see how SERPs have changed from one year to the next, we need to be able to see what today’s SERP for a certain keyword looked like a year ago.

Here’s what the SERP for “mesh wifi router” looked like in 2021, based on Ahrefs data:

Third: How exactly should you compare one year’s SERP to another?

For it to be a fair assessment, I’d have to select keywords completely at random and then see how their SERPS differ from 2021 to 2022. The problem with Ahrefs’ historical SERP data, however, is that it plays poorly with local search intent.

For instance, I noticed that the search for “Landscaping services” was showing me results from Los Angeles.

I conducted most of this study from the literal opposite side of the country (Portland, ME), and I most certainly never see Los Angeles-specific results when I search for services here.

I noticed this with several other local-intent terms, like roofing for instance. Why does it happen? I suspect it has something to do with how they scrape or archive data to give their best estimate of what a SERP looked like at a point in time. Regardless, this means that my selection of commercial keywords had to be limited to terms that don’t have a strong local leaning.

Here is what that random selection of 25 keywords ended up looking like:

1. Content marketing agency
2. Moving services
3. HR Services
4. Website hosting services
5. Graphic design services
6. Email marketing services
7. Shipping service
8. Cloud migration service
9. Data center services
10. Staffing agency
11. Supply chain software
12. Keyword research tool
13. EHR system
14. Architecture firm
15. Cloud hosting service
16. IT consulting company
17. Accounts payable software
18. Marketing consultancy
19. Small business loan
20. Affordable laptops
21. Mesh wifi router
22. Water pick
23. Small coffee grinder
24. Budget printer
25. B2B lead generation service

So what did I do with these keywords?

1. Determined whether more lists are showing up in 2022, 2021, or 2020 SERPs. I simply counted the number of “lists” in the SERP in 2022 and compared it to the years prior.

2. Determined whether more lists are ranking in the top five organic listings in 2022, 2021, or 2020. I did this, again, by counting the number of lists for each year that appear in the top five. The point here is to see not just if there are more lists in SERPs, but whether more of them are ranking higher in 2022.

Note that I did not count SERP features, as I was primarily concerned with the actual listings.

Disclaimer: This is by no means an exhaustive or completely conclusive study. It’s really more of a random sampling, or a “head count.”

Finally: What did the results say?

Let’s look at the numbers. Below is the total lists from the 25 SERPs I looked at:

Overall, there was an 8.5% increase in the number of lists between 2021 and 2022 (and the difference between 2020 and 2021 is nominal).

That might not seem groundbreaking, but the sample size is not insignificant: I looked at approximately 200 total listings (using 25 keywords, most of the SERPs had nine or 10 listings).

So that 8.5% increase carries some weight.

Things get more compelling when we look at the top five results:

The number of lists in the top five positions increased by 35.5% from 2021 to 2022. In this case, our sample size is a hard 125 listings, since we counted the first five organic results for every single one of our 25 keywords.

Again, not a nail in the coffin, but it is meaningful.

In summary

Here’s what we know:

• 2022 had more lists than 2021 and 2020.

• 2022 had double-digit percentage gains in terms of lists in the top-five results over both 2021 and 2022.

• But, 2021 actually has slightly fewer lists than 2020.

• And 2021 saw a decline in lists in top-five results of nearly 12% from 2020.

Here’s what it likely means:

• Overall, the number of total lists and top-five lists has trended upward since 2020, with the latter trending upward much more significantly.

• Pulling from this data pool, there is sufficient evidence showing that lists are more dominant in the top five positions on SERPs for commercial keywords in 2022 than either of the preceding years.

Here’s what it does not mean:

• That this trend is universal. After all, we left out local intent, which is a big deal for some companies. The data seems solid for non-location searches, but we can’t evaluate what we can’t reliably see (local-intent commercial searches).

This is what it all looks like visually:

So why the slight dip from 2020 to 2021? It reads like a blip in what is otherwise an upward trend, but we can only speculate. Three years of data, while not bad, isn’t enough to conclusively signify a long-term trend.

However, it’s more than enough of a change to affect any company that relied heavily on those top-five positions in 2021. If you’ve seen this change (like we have), you know how much it can hurt.

Which brings us to:

Part 2: What can be done about it?

The answer: Get listed in relevant directories, make your own lists, and infiltrate others’ lists.

Again, we’ll go one by one.

1. Get listed in relevant directories

If a directory is outranking your landing page for an important commercial term, you want to be on that directory, and as high up in it as possible. Invest time and energy into sourcing the reviews needed to improve your rank.

For several years, we more or less owned the number one spot for “content marketing agency.” Today, we’re ranking third, and we are only one of two commercial LPs on the entire SERP.

The number one spot is now held by Clutch. When we saw Clutch coming for us in 2020, we ramped up our efforts to acquire more reviews on the platform. This got us ranking highly in their directory.

The result: It’s one of our top sources of referral traffic and the top source of conversions from referral traffic.

Before pursuing paid options with business directories, I’d recommend doing everything possible to bring your organic ranking up, as many directories use organic rank as a tie-breaker in their “bidding.” I’ll add one extremely important note here: You never know which directory will be the next big winner in the SERPs. Sure, target most of your reviews for the ones that sustain a strong presence, but also scour the web for directories that list your competitors, but not you.

Link intersect tools from the likes of Moz can help you do this, and it’s so worth the time. Another way is to just explore SERPs of some of the relevant keywords in your industry.

In mid-2021, our link building specialist spent literally five minutes getting us listed in a directory called Growth Marketing Pro. As a result of those five minutes, we’ve since earned over 1,500 users, and 46 conversions.

Just look at all the keywords that directory is on page one for — not to mention some of that keyword volume:

It’s just such an easy win.

2. Make your own lists

We’ve only recently begun exploring this option, and as you can see, position 14 is nothing to really brag about.

But we’re playing a long game. Our post about content marketing platforms went live a few months ago. It hasn’t landed on page one yet, but our hope is that we’ve created something that will build backlinks over time, and that we can update and re-promote year after year. And each time we update it, we’ll optimize the on-page copy to make sure it’s strongly positioned to succeed in search.

As with most of the strategies referenced in this section, it’s still too soon to attribute success to this, but it’s something to think about. If other sites can create comprehensive lists that feature their own product, you can, too. It’s just a matter of creating said list strategically, and then promoting it and monitoring performance over time.

3. Infiltrate others’ lists

If you can’t beat ‘em, join ‘em. Or at least try by brokering your media strengths in exchange for a spot on their page one listing.

We’ve had mixed success here. In some cases, the promise of a link exchange, a guest blog post, or a guest appearance on our podcast were enough to get us included on a strategic list.

Appearing on those lists is great, as it gives us more visibility on SERPs for high-value keywords. But much to our existential chagrin, publishers have (on more than one occasion) flat-out told us we need to pay to be listed on their site. I’ll save the far-reaching, philosophical implications of this for Part 3 of this post.

For now, my advice is to add what we’ll call “referral SEO” to your existing backlink efforts. Think of it as targeted brand mentions. Your targets are based on what list is ranking for a keyword that is highly important to you. Once you’ve earned your placement, keep an eye on your referral traffic from that source.

Part 3: What does this mean for the future of SEO?

Answer: It’s complicated — and a potentially explosive subject.

Off the cuff, there’s a few big questions to unpack:

1. As directories and lists capture more page one SERP territory, they’re looking to cash in. One publisher literally told us they’d consider putting us on one of their lists if we paid them $3,500. Does Google care about that? Worse, at what point does commercial SEO become a pay-to-play situation, whereby if you want your company on a SERP, you have to pay your way onto someone else’s page?

2. When should you stop investing in certain commercial landing pages? Like I said, commercial pages are going extinct on the SERP for one of our most important keywords, “content marketing agency.” For now, we’re in position three. But we’ve seen all but one of our competitors get knocked onto page two. What happens then?

I can’t answer these questions confidently, but here goes:

I like to think the answer to the first question is that Google will catch on to what’s happening here, and start penalizing lists and directories that are presenting results based on who’s paying them, as opposed to what might actually be the best, most meaningful result for that query. But I have no idea if or how Google will do that.

For the second question, we’d have to shift our keyword target. At some point, we may need to bow out and understand where we can’t compete with these directories head-on, and assess whether there’s another keyword of equal value that we should target instead.

My first answer is hopeful and tenuous, and my second opens the floodgates to a whole lot of future work. But unfortunately, that’s the best we can do right now.

The final takeaways

Yes, there is some evidence that lists are now more prominent on page one for commercial keywords.

Yes, there is something you can and should do about it — and as always with organic SEO, you have to keep playing the game and run alongside the SERPs. Improve your rankings in directories. Get listed in new directories. Make some lists. Trade some backlinks. Get wily and start infiltrating some SERPs.

And no, we don’t quite know what this means for the future of SEO.

All you can really do is stay informed, stay data-driven, and keep updating your strategies to have a fighting chance of maintaining some sort of presence on page one for your most important commercial keywords.