Login or register
Back to list

How to remove website

Crawler

Like
30
Tweet
17
+1
4
Learn how to set limits for Semalt bots that are visiting your website.
Go to the page Semalt Crawler to find the detailed information on the search bots technology we apply for statistics monitoring.
You can restrict the access of Semalt bots to your website data. You don’t need to register. All you need is to perform these simple tasks:
  • Go to the page Semalt Crawler and read the information about crawlers.
  • Remove the site from our seed list: enter your website URL in the field at the lower part of the page.
  • You can enter several URLs – each in a new line – to speed up the process.
  • If your website contains subdomains, you must add them to the list as well.
 
Be attentive! URL http:/www.domain.com/ and http:/domain.com/ are two different web addresses!
  • You must enter all URLs in the format http://domain.com/
  • Click the button Remove under the URL field.
The changes will be applied within 30 minutes and Semalt search bots will never visit your websites again.
 
Check your website statistics the next day after you used Semalt Crawler to make sure you performed all the tasks properly.
 
  • I have added all my websites to Semalt Crawler but the robots are still visiting my pages.

You probably failed to enter all the subdomains or the changes have not been applied yet.
 
  • When I add my website to Semalt Crawler, will it be removed from the Semalt database once and for all?

Yes, the website will be completely eliminated from the list of web resources we visit.

Please feel free to contact our support service if you have any questions about Semalt Crawler.
avatar

Contacts

Ivana Fedorova 32 b1 st,
Kiev, Ukraine
© 2017, Semalt.com. All rights reserved