HCSW Technical Blog

Wow! Between jan-02-2006 and apr-20-2006 Nmap users submitted 948 fingerprints covering a huge range of diverse protocols. That's nearly 9 submissions per day! No other product on the planet has a database like we are building! Thanks go to everyone who submitted. Keep those submissions rolling!

I hope you enjoy a few miscellaneous notes I made regarding services I haven't covered in previous blog entries:

• Every once in a while I get a great submission that includes not only fingerprints but also match lines that I can adapt for the nmap-service-probes file. This time around I received one of the best I've ever seen. This submission deals with customised BIND versions and I incorporated it mostly unchanged into the nmap-service-probes file. I'll let the submitter's words speak for themselves:

  This is an example of a customized ISC Bind version - this is commonly done to obscure the exact version. The following match rule can be used as a catch-all for many common choices: # This fallback is because many people customize their BIND version to avoid # revealing specific version information. This rule should always be below the # detailed rules above. match domain m|\x07version\x04bind.*[\0x4-\x1f][\x03-\x1e]([-\w._ ,;?()[\]+:/@\n]{3,30})|s p/ISC Bind/ v/($1)/ Incidentally, the second ISC Bind version match rule has a glitch in that the byte-length character range doesn't match the corresponding string regexp; these should be:

  # Allow 3-20 character version numbers
  match domain m|\x07version\x04bind.*[\x03-\x14]([-\w._ ]{3,20})$|s p/ISC Bind/ v/$1/
  match domain m|\x07version\x04bind.*[\x08-\x19]BIND ([-\w._]{3,20})$|s p/ISC Bind/ v/$1/
  

  There's an interesting audit of BIND version information at http://www.phaze.org/dnsaudit/bindaudit-20010203.txt - I used this to generate the set of random punctuation for the catch-all pattern; although five years old, it gives a general idea of what one might expect to find.

  To the submitter: Thank you very much! I'm more than happy to give credit where credit is due: just email me from the same email address you put in the web form!

• An unusual DNS submission that I couldn't really determine the source of is the following:

  "\0\x89\0\x06\x85\0\0\x01\0\x01\0\0\0\0\x07version\x04bind\0\0\x10\0\x03\xc0
   \x0c\0\x10\0\x03\0\0\0\0\0_\^If you have a legitimate reason for requesting 
    this info, please contact hostmaster@Level3\.net"
  

  The submitter described it as Trick Or Treat Deamon: a DNS-proxy for IPv4/IPv6 translation. I'm not saying I don't believe this, I just need to be sure this is consistent for this application - I need more submissions! Also notice the interesting warning from the level3 hostmaster!

• Another interesting submission that speaks for itself:
  
  The mysterious Lexmark 9200 port. Googling around reveals something about perhaps being used to print directly from as/400!? Witchcraft, I say.

  "\x1e\0\0\0\0Dell Laser Printer W5300\0"
  

  More information on this? Email me!

• Unfortunatley, with some protocols we don't get very much version information in the normal responses. Take VNC for example:

  match vnc m|^RFB 003\.00(\d)\n$| p/VNC/ i/protocol 3.$1/
  

  However, sometimes we get lucky! :)

  "RFB 003\.003\n\0\0\0\0\0\0\0jServer license key is missing, invalid or has expired\.\n
   Visit http://www\.realvnc\.com to purchase a licence\."
  

• Finally, let's not forget you have to be careful when running Nmap against mission critical services. But also remember attackers probably won't be so careful!

  Description: Intel RAID Configuration Service 5.3.0.13
  
  Currently a service scan causes this service to spike the CPU to 100% until the service is restarted. I plan on doing a little more work to identify what causes this.

FTP can be a problematic service to match reliably. This is mostly due to how configurable a typical ftpd is! FTP servers often let the user customise the banners, the authentication prompts, etc, etc. Nmap's multiple probes can help with this though - FTPds are commonly fingerprinted by the NULL probe, the GenericLines probe, and the TCP Help probe.

• Some easy submissions are for products we already have match lines for and are merely slight variations caused by changing versions, overly specific regular expressions, unusual network conditions (banned IP, etc), and so on. These usually require a small tweak to the existing match lines or a new match line addition. For example, there was a match line for an AXIS video server that worked great until AXIS released the AXIS 2401+. Can you tell why from the following patch?

  -match ftp m|^220 AXIS (\d+) Video Server (\d\S+) (.*?) ready\.| p/AXIS $1 Video Server/ v/$2/ i/$3/
  +match ftp m|^220 AXIS ([+\d]+) Video Server ?(\d\S+) (.*?) ready\.| p/AXIS $1 Video Server/ v/$2/ i/$3/
  

• Some servers, like ncftpd - although not required by any standard - check the connection to see if the client is using another protocol altogether. Here is ncftpd's response to the GetRequest probe:

  220 OK\r\n502 This is not an HTTP server\.  Goodbye\.\r\n
  

  This is a good idea in order to defend against this sort of attack.

• In some cases, multiple embedded devices use the same underlying embedded software making it difficult to distinguish between the actual devices themselves. In some cases, especially httpds, we can usually distinguish the device models by looking at the first few hundred characters of the HTML page. The contents of the <title> tag are particularly fruitful.
  
  In other cases, we simply have to match the underlying embedded ftp server. As an example, both the TOSHIBA e-STUDIO350 and the Sharp FO-DC500 use the NetSilicon DPO-7300 embedded ftp server and there is no easy way to distinguish the two, we have to match like so:

  match ftp m|^220 DPO-7300 FTP Server ([\d.]+) ready\.\n| p/NetSilicon DPO-7300 ftpd/ v/$1/
  

  Although it isn't FTP, here's another obvious example of 2 different manufacturers using identical firmware:

  match telnet m|^\xff\xfb\x01\xff\xfb\x03\nNRG Maintenance Shell\.   \n\rUser access verification\.\n\rPassword:|
     p/NRG maintenance telnetd/ d/printer/
  match telnet m|^\xff\xfb\x01\xff\xfb\x03\nRICOH Maintenance Shell\.   \n\rUser access verification\.\n\rPassword:|
     p/RICOH maintenance telnetd/ d/printer/
  

  An interesting research project might be discovering if it's possible to build some sort of automated procedure to discover the lineage of different servers. What were they forked off of? How many generations of change have they gone through?

• And, finally, don't let anybody say OS/2 is dead - at least not to Nmap users!

  match ftp m|^220  IBM TCP/IP for OS/2 - FTP Server ver ([\d:.]+) on .* ready\.\r\n| p|IBM OS/2 ftpd/ v/$1/ o|OS/2|
  

One unexpected annoyance of submission processing is service submission spam! Here's an example:

Service: http://www.CUT.info
Platform: I am new in the forum but already excited of it.It contains a lot of things, really usefull for all of us. But a lot of my time I spend on <a href="http://www.CUT.info">My Lame Poker Site</a>and dont know how to stop it...My wife very angry.. :(
Description: Many usefull advices how to win and how to stop playing poker if something's wrong. It contains a lot of things for newbees in gambling and some interesting sights of poker players.
Notes: Havajan

Especially since the text included an HTML link, it's obvious that this was entered by some kind of crawl bot in the hopes that it would be posted on some publicly accessible site - increasing the linked site's PageRank, etc. I wonder what the ramifications of this are. Also, what is the best way to protect against this sort of garbage? Multiple steps to posting anything? Register/login everywhere? CAPTCHAs?

Always the largest, often the majority: HTTP and related protocols create probably the most time consuming job of service submission integration. The overwhelming prevalance and sheer diversity of the HTTP protocol is very impressive and, as I recall Fyodor commenting once, causes many internet users to think that it and port 80 are the internet.

• We always have to be careful not to include erroneous match lines based on strange data. Changing the identifying banners of servers is remarkably popular despite the fact that the security benefit is, in all likelihood, very small. This is obvious in this case:

  HTTP/1\.1 200 OK\r\nDate: Tue, 14 Feb 2006 10:52:25 GMT\r\nServer: Microsoft-IIS/4\.0 \(Unix\)/2\.0\.48 \(Unix\)\r\n
  

  but probably not as much so here:

  HTTP/1\.1 200 OK\r\nDate: Thu, 02 Feb 2006 17:23:34 GMT\r\nServer: CERN httpd 3\.0B \(VAX VMS\)\r\n
  

• Sometimes we can't do much with a submission! A windows Nmap user submitted the following fingerprint with no comments save that it runs on the platform Windows XP:

  HTTP/1\.1 200 OK\r\n\r\n\r\n404 Not found
  

• Somebody submitted a fingerprint of a lighttpd install explaining he'd customised the Server line banner. I'm sorry but I can't justify adding a match line to the official distribution if it's likely to only ever run on a single server, no matter how clever that banner might be:

  Server: lighttpd maybe its time for something less patchy\r\n
  

• Every probe in the service file gives us an opportunity to look for variations and clues a server might leak out. The more angles you look at something from the closer you are to identifying it. Sometimes software authors or administrators decide to hide or obscure their servers but don't realise that software usually has very distinctive fingerprints and making a program behave in an identical manner to some other piece of software generally requires signifigant rewriting - not just editing a string here and there. People who disguise Apache often give themselves away with Apache's directory listing banner:

  match http m|^.*<address>Apache/([\d.]+) Server at ([\w-_.]+) Port \d+</address>\n</body></html>\n|si p/Apache httpd/ v/$1/ h/$2/
  

  Or its characteristic and widely imitated

  HTTP ...\r\nDate ...\r\nServer: Blah/ver.si.on\r\n...
  

  Probes like HTTPOptions and RTSPRequest come in handy in situations like this one:

  ---------- GetRequest ----------
  HTTP/1\.0 403 Forbidden\r\nDate: Fri, 17 Feb 2006 15:05:01 GMT\r\nConnection: close\r\nContent-Type: text/html; charset=ISO-8859-1\r\n\r\n<HTML><HEAD><TITLE>403 Forbidden</TITLE></HEAD>\n<BODY><H1>403 Forbidden</H1>\nYour client does not have permission to get URL / from this server\.\n</BODY></HTML>\n
  
  
  ---------- HTTPOptions ----------
  HTTP/1\.0 501 Not Implemented\r\nDate: Fri, 17 Feb 2006 15:05:01 GMT\r\nConnection: close\r\nContent-Type: text/html; charset=ISO-8859-1\r\n\r\n<HTML><HEAD><TITLE>501 Not Implemented</TITLE></HEAD>\n<BODY><H1>501 Not Implemented</H1>\nPOST to non-script is not supported in Boa\.\n</BODY></HTML>\n
  

  Note how we can see this is the Boa Webserver because of the HTTPOptions request but cannot from a conventional GetRequest! In order to exploit this even further, I am considering a new probe that would attempt to always generate a 404 error from the server by requesting a URL that certainly shouldn't exist. Something like

  Probe TCP FourOhFourRequest q|GET /0wned/by/Nmap.txt HTTP/1.0\r\n\r\n|
  

  :)

  The trick will be in figuring out the ordering and probable ports that will cause the least (hopefully 0) impact on the existing match line database.

• Sometimes an obvious coding mistake gets duplicated between products - making the ancestry crystal clear! Compare this

  match http m|^HTTP/1\.0 401 Unauthorized\r\nDate: .*\r\nCache-Control: no-cache,no-store\r\nWWW-Authenticate: Basic realm=\"\.\"\r\nContent-Type: text/html; charset=%s\r\nConnection: close\r\n\r\n<html>\n<head><title>401 Unauthorized</title></head>\n<body>\n<h3>401 Unauthorized</h3>\nAuthorization required\.\n</body>\n</html>\n| p/m0n0wall FreeBSD firewall web interface/ o/FreeBSD/ d/firewall/
  

  with this

  match http m|^HTTP/1\.0 401 Unauthorized\r\nDate: .*\r\nCache-Control: no-cache,no-store\r\nWWW-Authenticate: Basic realm=\"\.\"\r\nContent-Type: text/html; charset=%s\r\nConnection: close\r\n\r\n<html>\n<head><title>401 Unauthorized</title></head>\n<body>\n<h3>401 Unauthorized</h3>\nAuthorization required\. HuaCheng Technologies\n</body>\n</html>\n| p/HuaCheng firewall http config/ d/firewall/
  

  and either give thanks or curses, as your preference may be, to BSD-style licensing!

• A reader makes an interesting suggestion that would involve a serious version detection overhaul:
  
  Apparently nmap doesn't follow a "Moved Temporarily" header properly. It is still a HTTP 302 Error like "Found" and does basically the same thing. If it follows the link, it should be able to grab the server headers and recognize them. This particular server doesn't send the server header until the link is followed. I used wget to print the headers and find out the version information above.

• I finally made the difficult decision to re-include apt-proxy into the probes file. This is, unfortunatley, too general of a match line and results in a few (I hope) extremely rare mis-identifications. The sheer number of independant submissions (20+) convinced me!

  # This one can cause false results!
  match http m|^HTTP/1\.0 404 Not Found\r\nConnection: close\r\n\r\n$| p/apt-proxy httpd/
  

• Here are some other mildly interesting replies to the GetRequest probe:

  "HTTP/1\.1 200 OK\r\nDate: Fri, 31 Mar 2006 17:14:54 GMT\r\nServer: All attacks reported to ISP and local Police\r\n"
  
  "HTTP/1\.0 200 Okay\r\nConnection: close\r\nServer: BaseSwitch 801FM\r\n"
  
  Kolban Webcam32 JavaCamPush Version 2.0
  "HTTP/1\.0 400 OK\nContent-Type: text/html\n\n"
  
  HP Systems Insight Manager 5.0
  "HTTP/1\.0 400 String index out of range: -1\r\nContent-Type: text/html\r\n\r\n"