Active crawler · operated by webatla.com

WebatlaBot

Automated web technology & domain intelligence crawler.

If you reached this page from reverse DNS or your server logs, WebatlaBot is the crawler behind the request. It collects publicly available metadata about domains — DNS records, response headers, detected technologies — to power the global domain database at webatla.com. Below is everything you need to identify it, block it, or opt your domain out.

Block this crawler Contact operator

Identity

Quick facts

The exact identifiers WebatlaBot uses in every request. Use these to match it against your access logs, firewall rules, or whois lookups.

Bot name
WebatlaBot
User-Agent
WebatlaBot/1.0 (+https://bot.webatla.com; abuse@webatla.com)
IPv4
178.104.181.169
IPv6
2a01:4f8:c17:e9cd::/64
Reverse DNS
bot.webatla.com
Operator
webatla.com
Abuse contact
abuse@webatla.com

What it does

Purpose of the crawler

WebatlaBot only visits publicly accessible endpoints and never attempts authentication, form submission, or anything destructive. Its entire job is to build a structured picture of the public web.

Domain discovery

Processes public domain zone files, RDAP and WHOIS responses to maintain the current state of active domains worldwide.

Technology detection

Inspects response headers and publicly served HTML/JS to identify CMS, frameworks, analytics and hosting used by each site.

Ranking & metadata

Aggregates public metadata — canonical URLs, language, OpenGraph, DNS topology — into the domains database at webatla.com.

Our policy

How WebatlaBot behaves

We run the crawler like a polite citizen of the web. If you ever see behaviour that violates one of these rules, please email us — we treat it as a bug.

  • Respects robots.txt for WebatlaBot and * user-agents.
  • Respects Crawl-delay directives when specified.
  • Rate-limited to roughly one request per second, per host.
  • Exponential back-off on HTTP 429 and 5xx responses.
  • Sends an identifying User-Agent with contact URL and email.
  • Never scans RFC1918, loopback, multicast or known sinkhole networks.
  • Does not follow login forms, submit forms, or probe authenticated endpoints.
  • Caches responses and uses If-Modified-Since / ETag to minimise load.

How to block

Block WebatlaBot

The preferred method is robots.txt — WebatlaBot re-reads it on every visit and will stop crawling your site within hours. If you need an immediate hard block, use firewall rules.

robots.txt Recommended
User-agent: WebatlaBot
Disallow: /
Firewall Hard block
iptables -A INPUT -s 178.104.181.169 -j DROP
ip6tables -A INPUT -s 2a01:4f8:c17:e9cd::/64 -j DROP