Login or register
Back to list

Crawler overview

Crawler

Like
30
Tweet
17
+1
4
Prevent your site from future visits of search bots.
If you noticed a burst of activity in the referral traffic reports, your website is likely to be visited by robots of the Semalt web analytics service. Keep calm, because they are aimed at statistical data harvesting. It takes only a few minutes to prevent future visits of these search bots to your website.
 
Use Semalt Crawler and your website will be completely removed from the Semalt database.
On the Semalt Crawler page you can find the description and detailed instructions how to work with this tool. It allows to prevent visits of the Semalt search bots without using filters and editing the .htaccess file.
 
How does Semalt Crawler work?
Semalt Crawler is an online form where you can add your website URL and all its subdomains. Visit the Crawler page — How to remove website for more information.
Semalt Crawler is a simple method of removing your website from the list of web resources visited by the Semalt robots. It doesn’t require registration.
               
  • Why are the bots visiting my website?

If you don’t use the Semalt service for web statistics monitoring, you website could be marked as a competitor for one of our projects. In this case, our robots can visit your resource every day in order to harvest statistical data.
 
  • Can the robots harm my website?

These bots are designed for data and statistics harvesting. They cannot cause any harm to your resource.
 
  • Do the bots affect my statistics?

No, the bots do not affect either your search engine positions or the bounce rate. They cannot change the actual conversion value on your website, since they are not unique users.
 
  • I have added my website to Semalt Crawler but the bots are still visiting my website.

There can be several reasons:
  • you haven’t added all the subdomains of your website
  • you have entered the web address without http:// attribute
  • not enough time has passed since you added your website to Crawler (it takes 30 minutes to apply changes)
  • you have another additional domain name/website

If you followed all the instructions properly, but you still didn’t manage remove a site from our seed list, please contact our user support service.
 
 
avatar

Contacts

Ivana Fedorova 32 b1 st,
Kiev, Ukraine
© 2017, Semalt.com. All rights reserved
Have a question?
We′ll call you back.