Asynchronous Reverse DNS

Introduction

Current official versions of nmap use the system's DNS resolver to resolve IP addresses to their registered DNS names. This works well and is quite portable (As portable as getnameinfo()). Unfortunatley, it can suffer from performance problems because the result from each query needs to be received before the next one can be sent.

I'm working on an asynchronous DNS patch for nmap that makes use of the nsock library (shipped with nmap) to send and receive many DNS requests in parallel to increase performance in large scans with many hosts.

This patch performs best with multiple fast DNS servers and large scans

Current downloads

Asynchronous Reverse DNS Patch for Nmap 3.96BETA1 - Development release

Old downloads

Asynchronous Reverse DNS Patch for Nmap 3.95 - Development release

Asynchronous Reverse DNS Patch for Nmap 3.95

OLD Asynchronous Reverse DNS Patch for Nmap 3.93

Usage

Applying the patch:

doug@darkmatter:~/nmap/nmap-3.95$ patch -p1 < /path/to/nmap-3.95.rdns-devel.patch

And then ./configure and make like usual.

To use the patch, simply use nmap as normal. If you use the -v flag you can get a short description of how much time nmap took to perform the RDNS lookup. If you use a debugging level of 1 -d you can get brief summaries of the progress of the DNS resolution. If you use a debugging level of 4 -d4 you can get detailed packet transmission information.

To fall back on the system resolver (getnameinfo()), use the --system_dns switch.

To specify the DNS servers to use on the command line, you can use the --dns_servers ip1,ip2,ip3 option.

Windows support

Windows support seems to be working fine! Either try compiling the code yourself or use the windows binaries I've supplied.

To compile on windows, you need to add nmap_dns.cc and nmap_dns.h to the build and link with the advapi32.lib library (for registry access functions).

Features

Tunable by changing parameters at the top of nmap_dns.c
Falls back to system resolver for IPv6 addresses
Parallel resolving
/etc/hosts support

Todo

Performance improvements
More reliability testing

Resolve IPs that resolve to CNAMES: (very rare - only affects some misconfigured servers)

$ host 217.31.174.62
62.174.31.217.in-addr.arpa is an alias for 62.0-25.174.31.217.in-addr.arpa.
62.0-25.174.31.217.in-addr.arpa domain name pointer bitnetcomp62.bitnet.nu.

Test on big endian and aligned architectures (should work OK)

Algorithm

As described in the file nmap_dns.cc:

// Algorithm:
//
// A batch of num_targets hosts is passed to nmap_mass_rdns():
//   void nmap_mass_rdns(Target **targets, int num_targets)
//
// mass_dns sends out CAPACITY_MIN of these hosts to the DNS
// servers detected, alternating in sequence.

// When a request is fulfilled (either a resolved domain, NXDomain,
// or confirmed ServFail) CAPACITY_UP_STEP is added to the current
// capacity of the server the request was found by.

// When a request times out and retries on the same server,
// the server's capacity is scaled by CAPACITY_MINOR_DOWN_STEP.

// When a request times out and moves to the next server in
// sequence, the server's capacity is scaled by CAPACITY_MAJOR_DOWN_STEP.

// mass_dns tries to maintain the current number of "outstanding
// queries" on each server to that of its current capacity. The
// packet is dropped if it cycles through all specified DNS
// servers.

More information along with exact details can be found in the file nmap_dns.cc.

Performance/reliability tests

Experiment #1

I created list of 1000 random IPs. I'm using 2 fairly fast, close DNS servers. I then used the asynchronous resolver against the same list of IPs 4 times:

DNS resolution of 1000 IPs took 63.32s. Mode: Async [#: 2, OK: 168, NX: 761, SF: 2, RE: 525, DR: 69]
DNS resolution of 1000 IPs took 56.54s. Mode: Async [#: 2, OK: 168, NX: 760, SF: 6, RE: 519, DR: 66]
DNS resolution of 1000 IPs took 58.51s. Mode: Async [#: 2, OK: 168, NX: 761, SF: 1, RE: 509, DR: 70]
DNS resolution of 1000 IPs took 58.79s. Mode: Async [#: 2, OK: 167, NX: 761, SF: 4, RE: 514, DR: 68]

Then, I used the system resolver against the same list of IPs 2 times:

DNS resolution of 1000 IPs took 959.90s. Mode: System [OK: 168, ??: 832]
DNS resolution of 1000 IPs took 967.78s. Mode: System [OK: 168, ??: 832]

As you can see, the asynchronous resolver is about 15 times faster with very little loss in accuracy. By performing the async requests first, we rule out the possibilty of any of the IPs being cached to begin with for the async scan. Not that this, in effect, gives the system resolver an advantage. The rest of the experiments use the same technique.

Experiment #2

This experiment is the same as the previous one, with similar results.

DNS resolution of 1000 IPs took 73.24s. Mode: Async [#: 2, OK: 178, NX: 724, SF: 4, RE: 652, DR: 94]
DNS resolution of 1000 IPs took 66.82s. Mode: Async [#: 2, OK: 179, NX: 724, SF: 6, RE: 598, DR: 91]
DNS resolution of 1000 IPs took 70.31s. Mode: Async [#: 2, OK: 179, NX: 724, SF: 8, RE: 628, DR: 89]
DNS resolution of 1000 IPs took 67.02s. Mode: Async [#: 2, OK: 179, NX: 724, SF: 3, RE: 580, DR: 94]

DNS resolution of 1000 IPs took 1328.22s. Mode: System [OK: 179, ??: 821]
DNS resolution of 1000 IPs took 1371.33s. Mode: System [OK: 179, ??: 821]

As can be seen, the async DNS resolver improves the total DNS time by factors between 15 and 20 with little to no loss in accuracy.

Experiment #3

I wanted to see how much of a speed boost the async resolver would give compared to the system resolver. This graph show the time it took each resolver to scan a collection of hosts of different sizes. I used the same technique as above - performing the async requests before the system requests to avoid giving the async resolver any speed/reliability advantage.

This graph nicely illustrates the power of parallel requests. There isn't a large speed advantage for numbers of hosts below about 10, but after that we start to see amazing performance improvements.

I originally plotted the number of "errors" (DNS names the system resolver found but the async resolver didn't) on the plot as well, but it ran right along the bottom - no errors in this scan.

Experiment #4

Experiment 4 is the same as the previous except with larger numbers of hosts. The improvement is even more dramatic. There were only 3 "errors" (see previous experiment) in the entire scan.

Experiment #5

I ran an experiment against 4096 random hosts using the same technique as above - performing the async requests before the system requests to avoid giving the async resolver any speed/reliability advantage. There were 12 "errors", giving an error rate of about 0.3%.

Experiment #6

This graph shows even larger host groups but the system resolver wasn't used at all because it would've taken too long. This means we can't determine the number of "errors" (see experiment #3). As expected, we see a fairly linear time increase.

Author

The excellent open-source security scanner Nmap is created by Fyodor and other contributors. This asynchronous reverse DNS patch is created by Doug Hoyte.