I fed Cloudflare, Quad9, NextDNS, and AdGuard the identical malicious domains to see who really stops them

Security-focused DNS companies all make roughly the identical promise. Point your gadgets at their resolver, and identified malicious domains can get stopped earlier than your browser ever connects to them. Cloudflare, Quad9, NextDNS, and AdGuard all provide some model of that safety, however I needed to see how typically they might really cease the identical known-bad hostnames.

So I gave all 4 the identical 100 malware and phishing domains and recorded what got here again. I by no means opened any of the websites, downloaded something, or let a browser close to them. This was strictly a DNS-resolution take a look at, and the 4 companies agreed far much less typically than I anticipated.

I began with greater than 41,000 domains and lower the record to 100

Dead domains make very beneficiant safety exams

Finding 1000’s of malicious URLs was simple. Turning them right into a DNS take a look at I may really belief took much more pruning.

For the malware facet, I used URLhaus’s domain-only host file, which accommodates hostnames tied to energetic malware URLs or URLs added throughout the earlier 48 hours. URLhaus additionally excludes hostnames from domains within the Tranco Top 1M to cut back false positives. For phishing, I used PhishTank’s verified-online dataset.

After pulling out the hostnames, eradicating duplicates, and discarding uncooked IP addresses and different unusable entries, I used to be left with 392 distinctive URLhaus hosts and 41,522 distinctive PhishTank hosts. I randomly sampled 75 from every supply, giving me 150 candidates to start out with.

Before testing any security-focused resolver, I despatched all 150 domains by way of Cloudflare’s common unfiltered DNS-over-HTTPS endpoint at cloudflare-dns.com/dns-query. I needed to know which domains nonetheless resolved usually earlier than giving any filtering service an opportunity to dam them.

That management run left me with:

• 131 domains that also resolved
• 19 that didn’t
• 0 question errors
• 74 surviving malware hosts
• 57 surviving phishing hosts

From there, I locked the ultimate dataset at 100 domains, cut up evenly between 50 malware and 50 phishing hosts. Every one in all them nonetheless returned an IPv4 handle through the closing management test.

That cleanup was vital as a result of lifeless transport systems could make each resolver look smarter than it’s. If a malicious hostname has already vanished from DNS, a filtered resolver might seem to have blocked it although its threat-detection system by no means made that decision.

I ran the ultimate take a look at, sending the identical 100 A-record queries over DNS-over-HTTPS with dnspython and HTTP/2 to Cloudflare’s malware-filtering resolver, Quad9, a recent NextDNS profile utilizing its default safety settings, and AdGuard’s default public resolver. Immediately earlier than scoring every hostname, I checked it towards the unfiltered management once more. All 100 have been nonetheless resolving when their 4 filtered queries ran.

I additionally needed to normalize what counted as a block as a result of the companies don’t all report one the identical method. Cloudflare returned 0.0.0.0 for domains it categorised as malicious, and NextDNS used that very same unspecified handle for blocked A-record queries throughout my run. Quad9 signaled a block with NXDOMAIN and AUTHORITY: 0, whereas AdGuard ceaselessly returned its 94.140.14.33 block handle. If I had handled each failed-looking response the identical method, the comparability would’ve been badly skewed.

Malware was simple in contrast with phishing

The total scores cover the mess beneath

I queried the identical 100 domains towards every of the 4 filtered resolvers, giving me 400 scored DNS queries to match.

AdGuard and Cloudflare completed just one area aside total, however that tiny hole turned a lot much less fascinating as soon as I separated malware from phishing.

Malware produced a reasonably constant image. AdGuard and Quad9 blocked all 50 malware hosts, Cloudflare caught 48, and NextDNS stopped 47. More tellingly, 45 of the 50 malware domains have been blocked by all 4 companies, whereas the remaining 5 have been nonetheless caught by three.

Phishing was nowhere close to that tidy. Cloudflare blocked 37 of the 50 phishing hosts, AdGuard caught 36, NextDNS stopped 23, and Quad9 blocked 18. Only eight phishing hostnames have been blocked by all 4 resolvers.

The relaxation fractured shortly. Eleven have been blocked by three companies, 20 by two, 9 by only one, and two slipped previous all 4. That means 40 of the 50 phishing hostnames received a unique block-or-allow end result relying on which resolver answered the question.

If phishing safety is one purpose you’re switching DNS suppliers, that’s the half I’d take note of. The malware hosts in my pattern have been a lot simpler for these companies to agree on, whereas phishing produced way more disagreement from one resolver to a different.

I wouldn’t flip 18 out of fifty right into a blanket declare that Quad9 is unhealthy at phishing. These have been 50 hostnames from one feed at one cut-off date, and threat-intelligence databases are always altering as malicious pages seem, disappear, and get reclassified. If I run the identical take a look at every week later, a few of these numbers may simply change.

The phishing pattern additionally shared plenty of transport systems. Thirty-six of the 50 hostnames matched recognizable internet hosting or proxy platforms akin to Weebly, Firebase Hosting, Cloudflare Pages, Wix Studio, Blogger, and a handful of different companies. That helps clarify why URL-level phishing reviews don’t all the time translate neatly into hostname-level DNS blocks.

The overlap between suppliers made that even clearer. Out of 100 domains, 53 have been blocked by each service, and solely two slipped previous all 4. The remaining 45 produced a minimum of one disagreement, which implies virtually half of this intentionally small take a look at set received a unique reply relying on which resolver dealt with the request.

Nearly an identical totals may cover very completely different decisions too. AdGuard completed at 86 and Cloudflare at 85, however that doesn’t imply they blocked the identical 85 or 86 hostnames. Different menace feeds, classification guidelines, replace schedules, and filtering insurance policies can produce virtually an identical totals by way of utterly completely different paths.

The scores aren’t the one purpose to choose one

What I’d really select every resolver for

The experiment left me way more considering how these companies differ than in pretending an 86 versus 85 end result settles something. DNS choice is about more than speed, and the pattern is just too small and too depending on timing for that form of verdict. The variations between the companies themselves are a lot simpler to make use of when selecting one.

There are additionally free DNS servers built for different jobs, so the precise selection relies upon closely on what you really need your resolver to do.

If I have been selecting between them, I’d break it down like this:

• Cloudflare if you’d like simple malware and phishing filtering with little or no setup.
• Quad9 if you’d like a security-focused resolver with out advert or broader content material filtering.
• NextDNS if you wish to tune safety protections, blocklists, logging, and different DNS conduct your self.
• AdGuard if you’d like malicious-domain safety bundled with network-level advert and tracker blocking.

That context additionally helps put my NextDNS lead to perspective. I examined a recent profile with its default safety configuration reasonably than switching on each aggressive safety accessible, and NextDNS offers you significantly extra management over filtering conduct than a set public resolver akin to Cloudflare’s malware-filtering endpoint.

Quad9 is equally simple to misinterpret if you happen to look solely at its total 68/100 end result. It caught each malware hostname in my pattern, so most of its deficit got here from the phishing half reasonably than a normal lack of ability to determine malicious transport systems. That distinction in priorities can be why a switch from Cloudflare to Quad9 can don’t have anything to do with uncooked velocity.

AdGuard’s end result comes with its personal wrinkle as a result of the default public resolver additionally filters advertisements and trackers. That could be a plus if you’d like one DNS service doing a number of jobs, but it surely additionally makes a direct “who blocks probably the most” comparability much less helpful than understanding what you really need the resolver to filter.

A clear DNS response does not imply a clear website

The greatest takeaway for me wasn’t that one resolver blocked a couple of extra domains than one other. It was {that a} hostname resolving usually doesn’t show it’s protected. Three suppliers may block the identical area whereas the fourth lets it by way of as a result of its menace intelligence or filtering coverage reached a unique conclusion.

Filtered DNS can cease a foul connection earlier than the browser ever reaches the positioning, which is precisely why I exploit it. I simply deal with a profitable lookup because the absence of a warning, not as proof that the vacation spot is reliable.