Warm up now. Book January. New year head start: 30 days free on any paid plan. Claim 30 days freeConnect your own mailbox and we add a free one on our domain, for warmup and replies. See how it works

Skip to content
Crawler

WarmerlyBot

WarmerlyBot is the web crawler run by Warmerly. If you found it in your server logs, this page explains what it is and how to control it.

Lines in robots.txt to block it
Month, at most, to act on a removal

Last updated 2 October 2026

1. How it identifies itself

One user agent, always the same.

If a request says this, it is ours. If it does not, it is not WarmerlyBot.

Every request carries this user agent: WarmerlyBot/1.0 (+https://warmerly.com/bot).

WarmerlyBot/1.0 (+https://warmerly.com/bot)
  • WarmerlyBotThe name to look for in your logs, and the name to write a robots.txt rule for.
  • 1.0The version number, as it appears in the string.
  • +https://warmerly.com/botThe address of this page. The crawler's user agent points here, so you can always check what it is.

To find it, search your access logs for the word WarmerlyBot. The Prospect Privacy Notice adds that the crawler identifies itself in its user agent and obeys robots.txt.

2. What it does

Reads what a business already publishes.

Public pages, public details, and nothing behind a login.

It reads the public pages of business websites (typically the home page and a few contact or about pages) to find the contact details and basic facts a business publishes about itself, which power the Lead Database in Warmerly. It does not log in to anything, submit forms or read pages behind a login.

The facts it looks for
  • Company name and legal name, website, description, industry or business type, approximate size, country and the year it was founded.
  • Registration details a business publishes, such as a company number or VAT number, and whether it links to a careers page.
  • Structured data the site publishes for search engines, as well as the visible page.
The contact details it looks for
  • Email addresses, a phone number, a business address, a contact-form link and opening hours.
  • Links to the business's LinkedIn and other social profiles.
  • Most published addresses are role-based, such as info@, hello@ or contact@. Where a business publishes an individual's work address, or runs from a personal or free-mail address, that is what is held.
What it never does
  • Log in to anything.
  • Submit a form.
  • Read a page behind a login.
From address to search result

The route one website takes

  1. It finds a siteCompany web addresses come from publicly available datasets of website addresses, such as the Chrome UX Report and Common Crawl.
  2. It reads robots.txtBefore fetching a site, WarmerlyBot reads its robots.txt and skips any path you disallow. If a site disallows it, the site is not crawled.
  3. It reads public pagesThe home page and a few contact or about pages. It does not log in, submit forms or read pages behind a login.
  4. It picks out business detailsAutomated processing extracts the details a business publishes, and classifies businesses by industry and size.
  5. They become search resultsThe details power the Lead Database in Warmerly. Companies that have opted out never appear in anyone's results.

The crawl is Warmerly's own: company records are crawled first-hand, not licensed or resold. Counts in the company directory are totals only, never individual companies, and an opted-out company is never counted. More on the lead database.

3. It follows robots.txt

Your robots.txt decides.

A file you already control sets what WarmerlyBot may fetch.

Before fetching a site, WarmerlyBot reads its robots.txt and skips any path you disallow. To keep it out of your whole site, add:

User-agent: WarmerlyBot
Disallow: /

A rule that names WarmerlyBot takes priority over a User-agent: * block. If your site has no robots.txt, public pages are treated as allowed.

One rule is enough. The line that matters is the Disallow: a path you list there is skipped, and if a site disallows WarmerlyBot it is not crawled.

Examples

Three ways to write the rule

Keep it out of one folder
User-agent: WarmerlyBot
Disallow: /private/
Paths you disallow are skipped. Everything else public is treated as allowed.
Keep it out of everything
User-agent: WarmerlyBot
Disallow: /
The two lines from the section above.
Different rules for different crawlers
User-agent: *
Disallow: /staff/

User-agent: WarmerlyBot
Disallow: /
A rule that names WarmerlyBot takes priority over the User-agent: * block, so this keeps WarmerlyBot out of everything while other crawlers are only kept out of /staff/.

robots.txt controls what the crawler fetches. To have details Warmerly already holds removed, ask for removal in section 4.

4. Contact and removal

A problem, or a removal? Email us.

No account, no justification, and an answer within a month.

If the bot is causing a problem, or you would like business details about you removed from our database, email privacy@warmerly.com. See also the Prospect Privacy Notice.

  1. Tell us the domain or email addressEmail privacy@warmerly.com or use the privacy request form, with the domain or email address concerned. No account needed, no justification needed.
  2. We may ask you to confirmYou may be asked to confirm that you control the address or domain, so that nobody can remove, or see, someone else's details.
  3. We act within one monthUsually much sooner.

What removal does: the email address goes on the platform-wide suppression list, the company record is marked as removed, and only the suppression record is kept to make it last.

  • The address is suppressedYour email address is added to Warmerly's platform-wide suppression list, which stops every Warmerly customer sending campaign email to it through Warmerly, including customers who add it later.
  • The record is marked removedThe company record or address concerned no longer appears in search results or exports, and the mark stays in place if the website is crawled again.
  • One record is keptOnly the suppression record needed to make this last. Suppression records are kept indefinitely, because deleting them would allow you to be added back.

What removal cannot reach. Copies a customer exported before your request, and mail a customer sends from their own systems outside Warmerly. For those, contact the sender directly or use the unsubscribe link in their message. Where Warmerly can identify a customer who obtained your details from it, it may pass your objection on.

What is held

A database about businesses, not people.

From the Prospect Privacy Notice: what the lead database holds, what it deliberately does not, and how long records are kept.

Organised around businesses

For a given company domain it may hold the business's name, description, size, country and the contact details it publishes. If you run a business on your own, some of these details, such as a business address or phone number, may also be personal data about you.

What it does not collect

No special category data, such as health, ethnicity or religion, and no criminal offence data, intentionally. No home addresses or personal phone numbers, except where a business publishes them as its own contact details.

Not the crawler

The email finder is a separate lookup that a customer starts for a named person at a company. Its individual-level results are kept for 24 months and then deleted. That is not something WarmerlyBot does.

Kind of recordHow long it is kept
Company recordsWhile the business appears to be trading and the record remains useful for business prospecting. Records for domains that no longer resolve or no longer accept mail are retired.
Deliverability check resultsFor as long as they help keep Warmerly's deliverability checks accurate.
Suppression recordsIndefinitely, because deleting them would allow you to be added back.
Who uses it

What the details are used for.

Warmerly is a UK-based sole trader and the controller of this database. The notice sets out the purposes and the basis.

Used for
  • Operating the lead finder and email finder, which customers use to identify businesses, and people in their business roles, who might want to hear from them.
  • Checking whether addresses are likely to be deliverable, so customers do not send mail that bounces.
  • Keeping suppression records so that people who asked not to be contacted are not contacted through Warmerly.
  • Maintaining and improving the database, including aggregated statistics that do not identify anyone.
The ground rules
  • Customers can search the database and export records for their own outreach, under the Lead Database Use Terms.
  • Warmerly does not sell or license the database as a dataset to data brokers or list sellers.
  • Where a customer exports your details, that customer becomes the controller of that copy.
  • The lawful basis is legitimate interests. You can object, and there is no test you have to pass first.
Questions

Questions about WarmerlyBot.

What is WarmerlyBot?

The web crawler run by Warmerly. It reads the public pages of business websites to find the contact details and basic facts a business publishes about itself, which power the Lead Database in Warmerly.

How do I recognise it in my server logs?

Every request carries this user agent: WarmerlyBot/1.0 (+https://warmerly.com/bot). The address in the brackets is this page.

Does it obey robots.txt?

Yes. Before fetching a site, WarmerlyBot reads its robots.txt and skips any path you disallow. If a site disallows it, the site is not crawled.

How do I keep it out of my whole site?

Add two lines to your robots.txt: User-agent: WarmerlyBot, then Disallow: /. A rule that names WarmerlyBot takes priority over a User-agent: * block.

What happens if my site has no robots.txt?

Public pages are treated as allowed.

Does it log in, fill in forms or read private pages?

No. It does not log in to anything, submit forms or read pages behind a login.

What does it read?

Typically the home page and a few contact or about pages, including the structured data a site publishes for search engines as well as the visible page.

What does it collect?

Business details: the company name, description, industry, approximate size, country and year founded, and the contact details the business publishes, such as email addresses, a phone number, a business address, a contact-form link, opening hours and social profile links. Also registration details a business publishes, such as a company number or VAT number.

Where do the details go?

Into the lead database that powers Warmerly's lead finder. Customers search it and export records for their own outreach under the Lead Database Use Terms. Warmerly does not sell or license the database as a dataset to data brokers or list sellers.

How do I get my business details removed?

Email privacy@warmerly.com or use the privacy request form, with the domain or email address concerned. You do not need an account or a reason.

How long does a removal take?

Warmerly acts within one month, and usually much sooner. It may ask you to confirm that you control the address or domain first, so that nobody can remove, or see, someone else's details.

Will the details come back if the site is crawled again?

No. The company record or address is marked as removed, so it no longer appears in search results or exports, and the mark stays in place if the website is crawled again. Warmerly keeps only the suppression record needed to make this last.

Can removal reach copies a customer has already exported?

No. Removal cannot reach copies a customer exported before your request, or mail a customer sends from their own systems. For those, contact the sender or use the unsubscribe link in their message.

Which law does Warmerly rely on?

Legitimate interests, Article 6(1)(f) of the UK and EU GDPR. You have the right to object to processing on that basis, and there is no test you have to pass first.

A problem with the bot, or a removal request?

Email privacy@warmerly.com, or use the privacy request form. No account needed and no reason required.