The Ultimate 2018 Keyword Research Manifesto

Been thinking about writing this one for several weeks and finally decided to write it. There are many misconceptions when it comes to keywords and the keyword research process.  People are treating SEO like it’s some sort of luck game when it really isn’t. Know what you’re doing and you’ll see patterns and consistent results most of the time. Easier said than done nevertheless, but keyword research is definitely something you should be paying more attention to. Build a site around the wrong keywords and no link package will save you – been there, done that.

In this article I’m going to talk about some practical ways of finding good keywords and keeping the good ones, without getting too side-tracked. I’m going to focus on a few key tools – these are in no way my entire arsenal, but they can definitely get much of the job done. This should be enough to have you on the right track of doing your keyword research properly. Anyway, let’s get started.

Expectations

First things first, before you start your keyword research, you need to get real. There is no magical 10,000 US monthly searches buyer intent keyword for a $500 product, or anything similar. I’m yet to see one of those in 2+ years of ACTIVE keyword research, yet newbies come and ask me to find them these “gems”. There isn’t some secret club where someone all holds all keywords like these (regardless of what some sellers would say) – and if there is, I’m certainly not a member.

Also, don’t expect to find keywords that none of your competitors would have ever heard about. If it has the monthly searches, it means that enough people are putting that into the little Google search box. And if enough people are doing that, why would you think that none of your competitors would have thought of that particular keyword as well?

Found some huge MS keyword that all the tools say is “low competition”? I’ll take a wild guess and say that’s it’s an ecommerce keyword that HUGE, million (or even billion) dollar business are ranking for with their category pages (don’t confuse category pages with product pages – those are way easier to outrank). Some overpriced tool like Long Tail Pro might say that it’s “low competition” because it has 0 links or whatever it is they account for today with their ridiculous formula, but it won’t be even close. If something looks too good to be true, it is.

What you CAN hope for is finding keywords that not too many of your competitors know about or are targeting, or keywords that are mostly targeted by pages with either bad onpage, a lack of links, or both.

Short/Long Term Keywords

I mean short/long not so much in terms of time, but more as a way to separate the keywords that you will be able to rank faster (short term) vs the keywords that you can rank for eventually (long term).

I don’t see this discussed often at all, yet I think it’s quite crucial when building a site. This part goes hand in hand with silo structure, so if you don’t know much about that you better read some here.

I’m going to shamelessly steal the image from the post above:

dogfood silo structure
An example from an online forum explaining the silo structure of a hypothetical dog food website.

Without going into too much detail about silo, here are a few points regarding short/long term keywords:

  1. The review keywords would be considered short term, since you’d be far more likely to rank for those faster, compared to the ones listed below. Easier to rank keywords are important in my opinion, as you can start seeing some page 1 ranks way before you start ranking for your real money keywords.
  2. Beef dog food, best chicken dog food, best dog food, dog food… – these would be more long term keywords, with some being more long term than others. The further “up” the silo you go, the tougher the keywords – or at least it should be that way. Competition for these (non-review) keywords will be usually higher.
  3. Before picking the review keywords you’re going to target, I would look at the product names themselves. These would be more long term and slightly harder competitiveness than the review keywords themselves, but still definitely worth targeting as they will have X times the monthly searches of the review keywords and you can rank for these eventually as well.

Keyword ideas

Before you start targeting a niche, you should definitely get familiar with it first. Do the products go by any other names (for example, British vs American English, slang words, etc)?  Is there a super related product that you could be targeting?

Try to get as many keyword ideas as you can. This is where I’m going to mention the AdWords Keyword Planner – even though I currently don’t see another use of it and I’d say you’re better off without it. Something I find more useful is simply LSIGraph.com – finding words semantically related can help a ton with your research. LSIGraph can also come in handy for on-page optimization, but that’s a whole other topic.

Sign up for my newsletter to receive SEO tips, discounts, and freebies.

   

Another [VERY] simple trick you can use to come up with more keyword ideas is simply scrolling to the bottom of the Google search results and copying their “related” searches terms.

Google Related Keywords
Google will show you the related searches for each query on the bottom of the page

How NOT to look for keywords to target

Every keyword research tool that I’ve tried can be good for two things only – getting the monthly searches (now that the almighty Google doesn’t reveal them to the average mortals) and filtering out high competition keywords. Other than that, tools like KWFinder  fail horribly at coming up with new keywords to target in your niche. I am going to talk more about KWFinder and what it can be useful for later on, but this is definitely not its best usage.  In fact, all the KWFinder “Suggestions” option does is shamelessly take the same results the Google AdWords Keyword Planner provides. Don’t believe me? Here’s what both of them have to output when you input “keywords”

Adwords Keyword Ideas
The Keyword Ideas option in the AdWords Keyword Planner tool isn’t the best one to use for SEO, even though it’s considered an industry standard.

 

kwfinder suggestions
KWFInder’s keyword suggestions are no different than AdWords’

Oh, and they both conveniently provide just 554 results for the term “keywords”. Shocking, I know.

KWFinder and blindly trusting competitiveness “scores”

Let’s talk about competitiveness formulas. I’m guilty of using Long Tail Pro in the past, until I realized it’s good for nothing. My tool of choice right now is KWFinder, although I already mentioned that it isn’t the best for coming up with keyword ideas. Apart from that, their “queries per 24 hours” thing is total rip off, just as their daily limit itself. Let’s say you haven’t used the tool in a couple of days and decide to use it for an hour Sunday night. No matter in which time zone you are, you will have to wait until Monday night until your daily limit resets. This would be fine only if there was a specific server or local reset time. This way, you’re using your Sunday queries (after not using the tool for a couple of days, remember), BUT you have to wait 24 hours until it resets. I guess I went on a bit of a rent about this, it’s just really annoying for a tool that isn’t very cheap in the first place.

You will also need to get the Agency plan if you want to do any real research, otherwise you just won’t have enough queries. Aaand, they often have “connection problems” that will make you pull your hair out while trying it for the Nth time. Oh, and did I mention that their Black Friday deal was so unjust to current customers that I had to downgrade and then upgrade again to basically pay them $60 just so they would treat me with the respect and pricing a new customer would get (but that’s a whole different story).

Not everything is horrible though, and what KWFinder is actually good for is getting rid of high competition keywords and providing monthly search volumes. Their filtering options come in handy as well, even though there isn’t a button to mass check competitiveness for keywords since they’re cheap little pricks and they don’t want you to use up your queries. Still, I’m yet to find a better alternative to get a certain part of my filtering process done.

Step 1 – Finding keywords and building an initial list

You should probably use Keyword Planner (or KWFinder’s suggestions, same thing) for the initial ideas and keep whatever it has to provide, just in case you don’t miss out on the few good keywords that it might suggest. However, keep in mind that Keyword Planner is a tool meant to get people to spend money with AdWords. It is NOT an SEO tool, and it is not in their interest to get you the good SEO keywords.

So, how do you actually find good keywords?  Remember LSIGraph, the keyword ideas tool I mentioned earlier? Go there and take every word that’s actually related to what you’re looking to target.

There are a few things you can do with these:

  • Use keyword generators based on Google autocomplete, such as UberSuggest.io, KeywordTool.io or KeywordShitter.com

This is one of the most underestimated ways of finding keywords. And, it’s free! All of these tools do more or less the same – they use the Google Autocomplete option to find you what people are searching for. It finds all kinds of stuff related to a certain term, and can definitely be a good way to gather hundreds of keywords. However, this also has its limitations, since it you will literally also get all kinds of unusable crap that you will later have to filter out.

Another little trick you can do is use the Keywords Everywhere chrome extension to get the monthly searches on websites like UberSuggest and KeywordShitter

Uber Suggest
UberSuggest, in combination with the Keywords Everywhere Chrome extension will both find keywords using Google autocomplete and display their search volume

With KeywordShitter, make sure you don’t input more than one keyword (or at least a few super related ones), or your output is going to be a gigantic mess and you won’t be able to separate the different niches and product types.

  • Use Scrapebox’s Keyword Scraper

This can take the above step to the next level since you can use proxies and multithreads to scrape keywords from sources other than just Google’s autocomplete – you can get the keywords users search for from Amazon, YouTube, Ebay, Wikipedia, Shopping.com, and even Alibaba. This can come in quite handy for product research as well. You can also use the keywords you get as seed keywords later (Transfer Results to Left Side) and run it again.

Scrapebox Keyword Harvester
You can use Scrapebox’s keyword Scraper tool to find keyword suggestion without only relying on Google autocomplete
  • Use SEMRush’s related keywords option

SEMRush is one of the better digital marketing tools out there, but I’m not gonna go in depth about what it does since you can easily Google that. When it comes to keyword research, we need it for one simple reason – because of their “Related Keywords” option. Since we’re building a huge list of keywords that we can later filter out, anything we can find to it will help.

Just input a keyword into SEMRush, and Export its Related Keywords. You can do this for more than one term for optimal results.

Semrush Related Keywords
Semrush’s related keywords option lets you easily export a ton of keywords related to your term.
  • “Steal” your competitors’ keywords using Ahrefs

And I don’t mean ONE of your competitors. No single website is going to target all the good keywords in a single niche. So, take your main (long term) keywords and the good ones that came out of LSIGraph and start Googling them hard.  Your objective is to find as many websites in your niche as you can which you think could be targeting some low competition keywords. Make no mistake – these could be category pages on large websites, microniche sites and even horrible little affiliate sites. You never know where the best keywords will be.

Sign up for my newsletter to receive SEO tips, discounts, and freebies.

   

Once you have a list of URLs, you will need to use the Site Explorer option in Ahrefs. Even though some prefer to use SEMRush for this, I’ve found that Ahrefs provides much more data, whether it’s related to backlink analysis or organic keywords. It’s expensive but worth it – nobody said this was free. Anyway, plug your URLs into Ahrefs one by one, and Export the Organic Keywords these sites are ranking for.

Ahrefs Organic Keywords
Ahrefs let’s you export the keywords your competitors’ are ranking for – well, at least the ones their crawler found and depending on your subscription plan.

Go through all the URLs one by one, and you should have a bunch of csv files you can use. The problem with these is that a lot of keywords will be repeated multiple times, so what you can do is combine all the CSV files you get, and then remove duplicates.

On Windows, you can do this by opening the Command Prompt (Run > cmd), then typing in pushd “directory” – where “directory” is the location of the folder you have all your CSVs in (make sure you don’t have any other ones in there).  After this, just type in copy *.csv combine.csv – where combine.csv is the name of the CSV file in which you want all this data combined.

Command Prompt
Combining CSV files using the Windows Command Prompt is fast and easy.

After this, all you need to do is copy all the data from the Keyword column, and then remove the duplicates (and possibly empty lines). A simple tool that does exactly this is this one.

If you did everything right, you should now have a list of hundreds, if not thousands, of your competitors’ keywords, as well as Google autocomplete keywords ready to be filtered out.

Step 2 – Filtering the trash and high competition keywords out

Now, this is where KWFinder comes in handy. You should already have a large list of keywords you want to filter out thanks to the tools mentioned above. We want the filtering part of the process because of two reasons – we want to get rid of the keywords with low monthly searches, as well as the obviously high competition ones. I say “obviously” because keyword competitiveness formulas aren’t the greatest thing ever, as we already saw.

First of all, you want to start importing your keywords in KWFinder. You can only import 700 at a time, and you will probably have thousands of them to report. To make this easier, I use a little free tool called TextWedge.

If you go to Dialogs > Configure Standard Splitting, you can split one text file in multiple parts, each containing no more than 700 lines.

Text Wedge Configuration Settings
Choose how you want to split your text file – the directory, file name (followed by the number) and on which lines you want the split

Then, just navigate to Action > Spilt, and Save Chunks and your 700-line text files will be in your chosen folder (I used the Text Wedge folder where the .exe to run the program is located).

Now, you will start importing these into KWFInder. After each import, add a filter of your choice. I recommend you filter through your results twice – the first time for lower MS, short-term keywords, and the second for your higher MS, long-term keywords. You can save these in Lists that you can organize/filter further later.

For short term keywords, I would do minimum 100 monthly searches (unless you have more expensive products for which lower MS would be worth pursuing) and 35 maximum Difficulty. For my Long-term keywords, I would want 1000+ MS keywords (again, depending on the niche – this is just the low limit) and Difficulty not higher than 45. This is not a written rule, and your filters should be flexible in order to fit your goals. Keep in mind that if you’re applying more than one filter for a certain group of keywords, your filters shouldn’t overlap since you don’t want to be manually looking through keywords twice, as you will have enough work as is. KWFinder (nor any other tool, for that matter) can’t distinguish between keywords Google wouldn’t consider different, such as “keyword research tool”, “tool keyword research” and “tool for keyword research” and you will manually have to sort through these.

KWFinder Imported Keywords Filter
Choose your preferred average monthly searches, number of words, and SEO difficulty scores for the imported keywords using KWFinder’s filter options

Now, note that after you apply the filter, you still have to check competitiveness for all of the keywords that are not calculated already. KWFInder only saves a KW in their database for 30 days (or so they say) after it’s difficulty is first calculated, since SERPs often change. You will notice this yourself when you manually go through the keywords – right after you click, it will often show numbers quite different than what was calculated before.

Step 3 – Manually picking through whatever’s left

Once you’ve filtered all of the keywords in your list (which should have been split in 700-line chunks), you want to go through them. There is no easy way of doing this, but the first I recommend doing is removing what is obviously unusable from your lists.

Sign up for my newsletter to receive SEO tips, discounts, and freebies.

   

After that, use your common sense to find the best candidates to target as your long term keywords. Note that there WILL BE other affiliate sites on the SERPs for these. Any long term keyword worth pursuing is already known to affiliates. These will include a lot of “best” and “reviews” keywords – doesn’t mean they’ll be bad keywords to target long-term, it just means that you won’t magically appear in SERPs for them right away, which you should expect.

As for your short-term, product-review-type keywords, I tried but couldn’t really come up with a super specific set of rules to follow. However, here are a few things you should and shouldn’t do:

  1. Look beyond page 1 of the SERPs to get rid of “False Positives”

Like I already said, Moz (whose API is used by KWFinder to provide the metrics for each results) will often not see the links some of the best ranking affiliate sites have. However, after page 1 you have pages 2, 3, 4, 5… These WILL have sites that are not blocking crawlers whether they’re PBNing or not. With KWFinder, you can do this by simply clicking “Load More” on the right side of the screen, right under the SERP results.  The beauty of this, is that you can see KWFinder’s “Rank” for each site – if you see a lot of green on page 1, but orange or red on the pages that follow – this would likely be a “false positive” low competition keyword that you should keep away from.

KWFinder Load More
KWFinder let’s you look at other pages of the SERPs, not just page one. You can also do this with SERPChecker, which comes along with KWFinder
  1. Look for low Page Authorities

This is the one metrics KWFinder gives the most value to – 40 percent, in their formula. It’s also one I look a lot at myself. You might be able to even tell how hard a keyword will be to rank for just by looking at the DA/PA of the top results – it’s where their “Rank” colors come from anyway.

  1. Check some of the SERP results manually

See how much content, optimization, views, likes, etc some of the sites in the SERPs have. A YouTube video with 2000 views and a Facebook page with 200 likes on page 2? Good signs. It should give you an idea if that’s an easy keyword or not. I had a customer send me a sample a keyword seller gave him – it was full of SEO-optimized results for that keyword up until PAGE 9 of the SERPs – this isn’t even what I’d consider a long-term keyword, let alone call it “low competition”, even though this seller was trying to justify it.

  1. Don’t spend too much time on Links, focus on content more

Check out the numbers available, but keep in mind that if you go looking at the Backlink Analysis for each SERP results using Ahrefs or whatever, your research will never end. It’s smart to do this for your Long-term keywords just so you know what to expect, but over-analyzing each and every results for each and every keyword is a waste of time. You will get a similar idea just by quickly looking at the content and optimization of a certain page for a particular keyword, along with it’s PA. If a site isn’t well-optimized, it’s likely they’re not spending too much on Links anyway.

  1. Look at the Trends of a keyword

This part can be a little tricky, since we already filtered for average Monthly Searches (which are yearly). However, that’s why I recommended these MS filters – in order NOT to miss out on any potentially no competition keywords rising in popularity. You can quickly look through the trends of the keywords in your lists by checking out the following graphs:

KWFinder keyword trend column
You can quickly look through the trends of the keywords in your lists by checking out the Trend column. We can see here what these little graphs mean.

 

A few other things worth mentioning

Further Investigation and Keyword Trends

After you’re done manually looking through the SERPs, you should do your own detailed analysis, especially for your long-term keywords. Here are a couple of keywords I found recently – one was for a client, while the other one was for one of my affiliate sites:

Keyword Trends 1
The first keyword with good trends, found when doing keyword research for a client.
Keyword Trends 2
The second keyword’s trends, which I found while doing research for one of my affiliate sites. I first though I struck gold, and even mentioned this keyword in a previous post.

Now, for both of these keywords it’s important to notice that the searches in the last couple of months are significantly higher than the average monthly searches – had I picked different filters, I never would have found these.

However, can you tell which of these keywords would be better one to target? No way you could without further research. So, after looking into these, I found that the first keyword was a $700 product that recently came out. Commission for this would be quite decent, and competition was quite low since the competition didn’t seem to be catching up just yet – surely something my client would love to get.

The second keywords, however – even though it had higher monthly searches and even lower competitiveness, was one that people were only searching for because they were seeing a very particular Facebook ad for it. I checked Google Trends, and as I suspected, the person had already stopped running ads for this product and monthly searches were now close to none. However, KWFinder hadn’t updated the monthly searches after November 2017.

Sign up for my newsletter to receive SEO tips, discounts, and freebies.

   

Another thing you should do when further investigating a keyword is finding long tail and LSI keywords (can use LSIGraph.com again here) to target in your article. You will often  find hundreds of monthly searches of keywords than can bring extra traffic if you at least mention them in your content – you don’t even necessarily have to be targeting all of them properly. Not only can these bring more visitors, they can also help rank better for the initial keyword , since Google lately is really placing an accent on semantic search.

Alternative keyword research techniques/tools you might want to look into

Here are some other ways you can make your life easier when searching for keywords:

  1. Filtering for CPC

For most niches, it’s true that a higher Cost Per Click will correspond with a higher SEO competitiveness. You can use KWFInder to filter for CPC and get rid of keywords before you have to check their SEO difficulty scores. Careful when you’re doing this though, as you could potentially miss out on good keywords.

  1. Using Ahrefs Keyword Difficulty

This is another way to filter for high difficulty – I like to run my keywords through Ahrefs one more time and get rid of anything over 5 Difficulty for short term, and 20 for long term keywords.

  1. Moz’s Keyword Explorer

I like this Moz tool for a few reasons:

  • the graphs they show for each keyword make it visually easy for me to make a quick decision on whether that’s a good long-term keyword or not
  • Their Organic CTR and Priority scores make it easy to find potentially profitable keywords to targets
  • They have a “Keyword Suggestions” option, which, much like SEMRush, helps you find related keywords you can be targeting
  • Moz Lists make it easy to keep the keywords you’re targeting organized
  1. Using KeySearch

This is probably the best KWFinder alternative out there. Other than the fact that they’re cheaper, they do have a Bulk check option and let you look at some on-page factors of the top results, such as the title and meta. They don’t let you import too many keywords at once, though and that’s something I find necessary. They obviously have a different competitiveness formula, but that’s not a very big deal as you can compare the two tools and pretty much figure out where they’re different.

KeySearch SERP results
KeySearch can be a useful KWFInder alternative, especially if you’re dealing with a lower amount of keywords
  1. Consider using Custom Tools

If you do keyword research as much and as often as I do, getting your own tools built to match your process and make it easier become a necessity. Think out of the box, hire a programmer to automate parts of the process, or outsource someone to do that. You will still need to manually pick through the keywords, but a custom tool or two can help dramatically. For example, you can take advantage of SERPChecker (comes along with KWFinder) to extract data to a sheet that will show you whatever it is you find important to look for in the SERPs. You can also get a tool that does all the data processing for you, instead of having to use some of the little tricks I mentioned. Another idea would be an improved version of the tools that use Google autocomplete. But, I’ll let you figure that out yourself.

The newbie FAQ

This isn’t something I would normally do, but it’s funny to see newbies post the same questions over and over again. So, to answer a couple of the questions being frequently asked:

“Is xxxxxx a good keyword?”

-No one can answer this for you without looking at the SERPs themselves. BUT, chances are, if you only have ONE keyword that you’re looking to target, you will fail miserably. Good keyword research [most of the time] doesn’t mean that you should have one magical keyword that you will rank for and solve all your life problems, but instead a group of high potential keywords that you can build a site around and target.

“How many links will I need to outrank everyone or to get to page 1”

-This is a rather annoying one, because it implies that the number of links is in any way relevant to whether you can outrank a site or not. It’s not the quantity, but the quality of links that counts. If you’re asking this, you should probably learn more about SEO before ever trying to outrank someone.

“X tool shows Y competitiveness score, should I target this keyword?”

Again, keyword competitiveness scores are just that. They’re in no way a super accurate representation of the SERPs, and definitely not representative of anything beyond page 1. Further research will be required – if you’re asking this, you should at least provide details of what the SERPs look like further than page 1, and probably use more than one tool to get the difficulty score.

“Can I outrank Amazon/Ebay/Walmart/Etsy/Target/Another-Large-Ecom-Site”

Yes, and no. Are these product pages or category pages? What sort of keyword are you are you targeting? For best, review and few other types of keywords it will be mostly affiliate sites on page 1 anyway. However, the trouble can come when you’re targeting product names – I would target these with their appropriate review keywords, along with LSI and long tails only, and use them as long term goals rather than something you can target easily – although a lot of product name keywords tend to be quite low competition and you can appear on the first couple pages of the SERPs with nothing but good content and on-page. With that being said, I would stay away from keywords where page 1 is filled with category pages of the large ecommerce sites – these are almost never even medium competitiveness, let along easy.

Anyway, it’s day two of writing this article so it’s time to stop here. I tried to put this together so it makes as much sense as possible and if anything is unclear, please ask. There’s definitely more to say about keyword research, but let’s save a little more something for later.

Sign up for my newsletter to receive SEO tips, discounts, and freebies.

   

6 thoughts on “The Ultimate 2018 Keyword Research Manifesto

  1. Wow, great guide, I am already your customer from (BHW). But this is the exact piece of info i was looking for . Many people write about “how to find easy keyword” but no one provides what do serp look like for easy kw and you pretty much nailed it. Keep up the good work..

  2. Thanks for writing this. It is just so much info and so many factors play an important part. It makes the whole process very difficult.

    • Thanks. And yes, keyword research isn’t the easiest thing ever if it’s not something you’ve done successfully over and over.

Leave a Comment

Need a hand? Get in touch and let me know what your SEO troubles are.
Holler Box