How to block bots and crawlers

Share on Facebook0Share on LinkedIn0Share on Google+0Tweet about this on Twitter

When you’re building a site in a competitive niche, you know that everybody is doing A LOT of research on their competition.

cut-throatAnd when the competition is cut-throat, then that research can and will be used against you.

One of the best ways to get a good look at your competitors’ backlinking is by using Majestic, Ahrefs or Moz.

When they find your backlinks they can:

  • impersonate an agency doing SEO for you and request removal of “bad links”,
  • duplicate your backlinks if they’re open to all,
  • report your PBNs to Google’s web spam team.

 

So the best way to hide yourself from the competition is by blocking all of the mentioned bots.

There are multiple paid and free options out there, from Spyder Spanker, Link Privacy to Spider Blocker. I prefer and use Spider Blocker on all my websites because it’s published on WordPress.org.

Download it for free on WordPress.org >>

Share on Facebook0Share on LinkedIn0Share on Google+0Tweet about this on Twitter
The following two tabs change content below.
Ben Anderson is an SEO and affiliate marketer living in San Diego, California.   He began his career in Internet Marketing in 2005 and went full time in 2010 when he left his full-time job as a developer. Now he builds affiliate sites and helps local business rank on top of search engines.   Ben likes surfing, coffee and everything computer.

Latest posts by Ben Anderson (see all)

Leave a Reply

Your email address will not be published. Required fields are marked *