A user clicking a phishing link, malware trying to connect to its command server, a script pulling a file from a newly registered domain. All three start in the same place: a DNS query. DNS filtering evaluates that query before answering it and doesn’t resolve the malicious domain. It’s cheap, it has broad coverage, and if it isn’t set up correctly it’s easily bypassed. Below we explain hands-on the architecture options, the RPZ approach, closing the bypass routes and using DNS logs in incident investigation.
Which threats DNS filtering stops
Because DNS filtering kicks in before a connection is made, it’s protocol-independent: HTTP, HTTPS, SMTP or a custom command-and-control protocol make no difference. If the domain isn’t resolved, the connection isn’t made. Areas where it’s effective:
- Known phishing and malware distribution domains
- Command-and-control (C2) infrastructure and names generated by domain generation algorithms (DGA)
- Newly registered or recently first-seen domains (the provider needs to support this category)
- Categories against company policy: gambling, anonymization services, personal file sharing and so on
Its limits are clear too. It can’t stop connections made directly to an IP address, malicious content on a compromised legitimate site, or communication that doesn’t use DNS at all. Techniques such as DNS tunneling are noticed only if query patterns are monitored. That’s why DNS filtering doesn’t replace web filtering, email security and EDR; it’s a layer that works in front of them. For a layered approach against phishing, the reducing phishing risk in enterprise email article is a complement.
Architecture options: resolver, forwarder, endpoint agent
In most organizations, clients use Active Directory DNS servers, and these servers forward external queries somewhere. Where you put the filter determines both coverage and the visibility you’ll have during an incident.
| Option | How it works | Advantage | Limitation |
|---|---|---|---|
| Filtering forwarder (cloud or NGFW) | AD DNS forwards external queries to the filtering service | Easy setup, no client changes | Only the AD DNS servers appear as the source IP on the filter side |
| RPZ on your own resolver | A policy zone in BIND, Unbound or a commercial DNS product | Full control, local logs | The threat feed and maintenance responsibility are yours |
| Endpoint agent / client | Intercepts and filters DNS on the device | Devices outside the office are protected too, with per-user logs | Agent deployment, and no coverage on unmanaged devices |
| DNS inspection on the NGFW | The firewall inspects DNS traffic passing through it | Existing device, central policy | Can’t see encrypted DNS; only traffic passing through the firewall |
The visibility problem is an often-overlooked point. If AD DNS servers forward to a filtering forwarder, you can’t see in the filter logs which client a blocked query came from; there’s only the DNS server’s IP there. If there’s no answer to “which machine?” during an incident, the log is useless. The solution is to enable analytic/debug logging on the AD DNS servers or to choose an architecture that carries the client IP. In practice, the common model is: a filtering forwarder or RPZ inside the office, and an endpoint agent for laptops outside the office. The engineering side of DNS infrastructure, i.e. split-horizon and resolver design, is described in the enterprise DNS architecture guide on the network site.
The Response Policy Zone (RPZ) approach
RPZ lets a DNS resolver change its responses according to a specially prepared zone file. BIND, Unbound, PowerDNS Recursor and many commercial DNS products support RPZ. Windows Server DNS, however, doesn’t support RPZ directly; in Windows environments RPZ is usually applied on an intermediate resolver that AD DNS forwards queries to. DNS policies in Windows Server 2016 and later (Add-DnsServerQueryResolutionPolicy) can be used to block individual domains, but they don’t consume threat feeds in RPZ format, and management quickly gets difficult with large lists. Threat intelligence providers can distribute RPZ feeds via zone transfer.
A local RPZ example on BIND:
// named.conf
options {
response-policy { zone "rpz.corp.local"; };
};
zone "rpz.corp.local" {
type master;
file "/etc/bind/db.rpz.corp.local";
allow-query { none; };
};
logging {
channel rpz_log {
file "/var/log/named/rpz.log" versions 5 size 50m;
print-time yes;
};
category rpz { rpz_log; };
};
; /etc/bind/db.rpz.corp.local
$TTL 300
@ IN SOA localhost. hostmaster.localhost. (2026111001 3600 600 86400 300)
IN NS localhost.
; Return NXDOMAIN
malicious-example.example CNAME .
*.malicious-example.example CNAME .
; Redirect to the block page
gambling-example.example CNAME block.corp.local.
; Exception: pass through without evaluation
safe-example.example CNAME rpz-passthru.
CNAME . produces an NXDOMAIN response and CNAME *. a NODATA response; rpz-passthru. exempts the domain from subsequent policies. After a change, validate the zone file with named-checkzone rpz.corp.local /etc/bind/db.rpz.corp.local and load it with rndc reload rpz.corp.local. Forgetting to increment the SOA serial number is a classic mistake that results in secondary servers never receiving the change. Redirecting to a block page gives the user an explanation; but on HTTPS sites the browser shows a certificate error. That’s why most organizations prefer NXDOMAIN and handle user communication through logs and the help desk process.
Preventing bypass via DoH, DoT and direct DNS
DNS filtering relies on the assumption that clients use the corporate resolver. There are three ways that break this assumption; each needs its own countermeasure:
- Direct DNS (UDP/TCP 53): A client or malware sends queries directly to a public resolver. Countermeasure: on the firewall, egress to port 53 is allowed only for the corporate resolvers and their forwarder targets; for everyone else it’s blocked and logged.
- DNS over TLS (TCP 853, RFC 7858): Because it uses a separate port, it’s easy to block; egress on 853 is closed except for the corporate resolvers.
- DNS over HTTPS (TCP 443, RFC 8484): Because it’s carried inside normal HTTPS traffic, it can’t be blocked by port. The countermeasures are layered: disabling DoH or pointing it to the corporate resolver through browser and operating system policies; blocking the category or application signature of known DoH providers on the NGFW or web filter; returning NXDOMAIN for the
use-application-dns.netcanary domain for Firefox.
The canary domain only disables the DoH that Firefox turns on by default; it doesn’t help if the user has enabled DoH manually. Distribute browser policies centrally on managed devices (DnsOverHttpsMode for Chrome and Edge, enterprise policy for Firefox). Example firewall egress logic:
Rule 10: Source=DNS_SERVERS Destination=FORWARDER_TARGETS Service=DNS(53/udp,53/tcp) Action=allow, log
Rule 11: Source=any_internal Destination=any Service=DNS(53), DoT(853/tcp) Action=block, log
Rule 12: Source=any_internal Destination=DoH_PROVIDER_CATEGORY Service=HTTPS Action=block, log
The place of these rules in the policy and their maintenance should be part of the periodic review; the ownership and review approach in the firewall rule cleanup guide applies here too.
Designing the category and exception policy
Policy design determines whether users will accept DNS filtering. A badly designed policy means hundreds of exception requests and “turn off the filter” pressure in the first week. The approach we recommend:
- Security categories mandatory for everyone: Malware, phishing, C2 and crypto mining categories are blocked without exception.
- Newly registered domains: Start in monitoring mode first; move to blocking after seeing it doesn’t break workflows.
- Policy categories by group: Categories such as social media or video are decided together with HR and management; it isn’t something the security team should decide alone.
- Exception process: A process where users can apply about a misclassified domain, the security team evaluates it quickly and every exception is recorded with its justification. Exceptions are made per domain, not per category.
- Server networks: Servers’ internet-facing DNS resolution can be limited to only the domains they need. This is the DNS counterpart of the default-deny approach.
Using DNS logs for detection and incident investigation
DNS logs are one of the cheapest data sources for answering “which device asked for which domain, and when” in an incident investigation. For them to be useful, there are three conditions: the log should include the client IP (preferably the user or device name), it should be possible to correlate it with DHCP logs, and it should be retained for as long as incident investigation will need. A log that isn’t retained doesn’t exist.
Detections that can be built with DNS logs in a SIEM:
- Queries blocked by the RPZ or filter: repeated blocks from the same client may be a sign of malware running on the device.
- Query sequences containing high-entropy or very long subdomains: an indicator of DNS tunneling and DGA.
- A client receiving a large number of NXDOMAIN responses in a short time.
- Retroactively searching for a domain newly published in threat intelligence: thanks to retained logs, the question “who asked for this domain before?” can be answered.
Detection scenarios that evaluate firewall and DNS logs together are described in detail in the threat detection from firewall logs article. Designing DNS filtering together with segmentation and egress policy can be handled within network security and segmentation, and implementing the egress rules within firewall rule management.
Frequently asked questions
Does DNS filtering replace web filtering?
No. DNS filtering works at the domain level; it can’t see the URL path, page content or direct IP connections. It’s a broad, low-cost first layer; it should be used together with web filtering and EDR.
Can RPZ be used on Windows Server DNS?
Windows Server DNS doesn’t support RPZ directly. The common approach is for AD DNS servers to forward external queries to a resolver that supports RPZ or to a filtering DNS service. In that case, logging on the AD DNS side should be planned separately for client visibility.
Is it possible to block DoH completely?
It can’t be blocked by port, because DoH is carried inside normal HTTPS traffic. When browser and operating system policies, blocking known DoH providers on the firewall or web filter, and the Firefox canary domain are used together, the risk of bypass drops significantly.
How are laptops outside the office protected?
An in-office forwarder or RPZ stops applying once the device leaves the corporate network. These devices need endpoint agent-based DNS protection or a remote access architecture that routes all traffic through a corporate control point.