The Nmap Scripting Engine

David Fifield

Schedule

NSE for end users (10 minutes)
NSE for developers (20 minutes)
Applications (10 minutes)
Questions (10 minutes)

Audience handout: http://www.bamsoftware.com/talks/fosdem-2010.pdf.

Quick review on Nmap

Download from http://nmap.org/download.html.

$ nmap scanme.nmap.org

Starting Nmap 5.21 ( http://nmap.org ) at 2010-01-31 19:40 MST
Nmap scan report for scanme.nmap.org (64.13.134.52)
Host is up (0.078s latency).
Not shown: 994 filtered ports
PORT      STATE  SERVICE
25/tcp    closed smtp
53/tcp    open   domain
70/tcp    closed gopher
80/tcp    open   http
113/tcp   closed auth
31337/tcp closed Elite

Nmap done: 1 IP address (1 host up) scanned in 9.19 seconds

Motivation for NSE

NSE is a combination of the Lua programming language, networking libraries, and Nmap. It can be thought of as a kind of extended version detection that can carry on protocol negotiations or do anything else that can be programmed. It is suitable for rapid exploit research and development (both detection and exploitation).

How to use NSE

$ nmap -sC scanme.nmap.org

Starting Nmap 5.21 ( http://nmap.org ) at 2010-01-31 19:47 MST
Nmap scan report for scanme.nmap.org (64.13.134.52)
Host is up (0.070s latency).
Not shown: 994 filtered ports
PORT      STATE  SERVICE
25/tcp    closed smtp
53/tcp    open   domain
70/tcp    closed gopher
80/tcp    open   http
|_html-title: Go ahead and ScanMe!
113/tcp   closed auth
31337/tcp closed Elite

Nmap done: 1 IP address (1 host up) scanned in 26.98 seconds

$ nmap --script=safe scanme.nmap.org

Starting Nmap 5.21 ( http://nmap.org ) at 2010-01-31 19:43 MST
Nmap scan report for scanme.nmap.org (64.13.134.52)
Host is up (0.073s latency).
Not shown: 994 filtered ports
PORT      STATE  SERVICE
25/tcp    closed smtp
53/tcp    open   domain
70/tcp    closed gopher
80/tcp    open   http
|_html-title: Go ahead and ScanMe!
| http-headers:  
|   Date: Mon, 01 Feb 2010 02:44:20 GMT
|   Server: Apache/2.2.3 (CentOS)
|   Accept-Ranges: bytes
|   Content-Length: 739
|   Connection: close
|   Content-Type: text/html; charset=UTF-8
|   
|_  (Request type: HEAD)
|_http-date: Mon, 01 Feb 2010 02:44:26 GMT; +11s from local time.
113/tcp   closed auth
31337/tcp closed Elite

Host script results:
|_asn-query: No Answers
| whois: Record found at whois.arin.net
| netrange: 64.13.134.0 - 64.13.134.63
| netname: NET-64-13-143-0-26
| orgname: Titan Networks
| orgid: INSEC
|_country: US stateprov: CA 

Nmap done: 1 IP address (1 host up) scanned in 35.40 seconds

Script selection

Default scripts

nmap -sC
nmap --script=default
nmap -A

Specific scripts

nmap --script=ssh-hostkey
nmap --script='http-*,snmp-*'

By category

nmap --script=safe,default,malware

Boolean selection

nmap --script='(default or discovery) and not intrusive'
nmap --script='smb-* and not smb-brute'

Script arguments

nmap --script=ssh-hostkey --script-args ssh_hostkey=all
nmap --script=snmp-sysdescr --script-args snmpcommunity=test1
nmap --script=pjl-ready-message --script-args 'pjl_ready_message="TAKE ME TO YOUR LEADER"'
nmap --script=smb-server-stats --script-args smbuser=admin,smbpass=password1

Highlighted scripts

description = [[
Gets the date from HTTP-like services. Also prints how much the date
differs from local time. Local time is the time the HTTP request was
sent, so the difference includes at least the duration of one RTT.
]]

---
-- @output
-- 80/tcp open  http
-- |_ http-date: Thu, 23 Jul 2009 23:15:57 GMT; -6s from local time.
-- 80/tcp open  http
-- |_ http-date: Wed, 17 Jan 2007 09:29:10 GMT; -2y187d13h46m53s from local time.

author = "David Fifield"

license = "Same as Nmap--See http://nmap.org/book/man-legal.html"

categories = {"discovery", "safe"}

require("http")
require("shortport")
require("stdnse")

portrule = shortport.port_or_service({80, 443, 631, 8080},
	{"http", "https", "ipp", "http-alt"})

action = function(host, port)
	-- Get the local date in UTC.
	local request_date = os.date("!*t")
	local response = http.get(host, port, "/")
	if not response.status or not response.header["date"] then
		return
	end

	local response_date = http.parse_date(response.header["date"])
	if not response_date then
		return
	end

	-- Should account for estimated RTT too.
	local diff = stdnse.format_difftime(response_date, request_date)

	return string.format("%s; %s from local time.",
		response.header["date"], diff)
end

Required fields

description
categories
portrule/hostrule
action

The difference between `portrule` and `hostrule` scripts

`banner.nse`

---
-- Script is executed for any TCP port.
portrule = function( host, port )
  return port.protocol == "tcp"
end

action = function( host, port )

`asn-query.nse`

---
-- This script will run for any non-private IP address.
hostrule = function( host )
  return not ipOps.isPrivate( host.ip )
end

action = function( host )

Parallelism

	local socket = nmap.new_socket()
	-- These look like blocking calls to the script,
	-- but other scripts get to run while they execute.
        socket:connect(host.ip, port.number, port.protocol)
        socket:send("USER anonymous\r\n")
        socket:send("PASS IEUser@\r\n")

Files to know about

nse_main.lua — main engine, itself written in Lua.
nselib/*.lua — NSE libraries. Examples are dns.lua, http.lua, snmp.lua.
nse_*.cc — Lua-to-C++ interfaces. Some libraries, like openssl and bit, are written in C++ with a Lua wrapper.
scripts/script.db — the script database that maps script names to categories.

Tips for script development

nmap --datadir .

uses files in the local directory instead of those in /usr/share/nmap.

Don't forget to run

nmap --script-updatedb

or the script won't be selectable except by file name.

Use -d for backtraces:

./scripts/ssl-cert.nse:155: attempt to perform arithmetic on a nil value
stack traceback:
        ./scripts/ssl-cert.nse:155: in function 'stringify_name'
        ./scripts/ssl-cert.nse:110: in function <./scripts/ssl-cert.nse:84>
        (tail call): ?

How to submit your new script

Write to nmap-dev@insecure.org. For example:

http://seclists.org/nmap-dev/2009/q4/508

Patrik Karlsson:
I finished two more Nmap scripts that hopefully will be of more use than
the first one. nfs-showmount.nse replicates the functionality of
showmount -e in order to list remote NFS shares.
citrix-published-applications.nse queries Citrix (1604/udp) for the list
of published applications.

http://seclists.org/nmap-dev/2009/q4/529

David Fifield:
I have only one concern from looking at the scripts, and that is the
undocumented packet contents. You need to add comments explaining what
all those hex digits mean, and links to references if available.

http://seclists.org/nmap-dev/2009/q4/537

Patrik Karlsson:
Ok, so I've made an attempt to clean up and document the script(s).
There is little documentation on this protocol or at least that I can
find. I have documented the bits and bytes I am using when processing
the response and the bytes I was able to figure out in the packets that
are sent.

http://seclists.org/nmap-dev/2009/q4/535

David Fifield:
I added your script in r16210. Well done!

Resources

Nmap Network Scanning chapter 9. http://nmap.org/book/nse.html
Programming in Lua. http://www.lua.org/manual/
Online NSEDoc. http://nmap.org/nsedoc/
nmap-dev@insecure.org mailing list. http://seclists.org/nmap-dev/

NSE in action: HTTP favicon survey

From http://nmap.org/nsedoc/scripts/http-favicon.html:

Gets the favicon ("favorites icon") from a web page and matches it against a database of the icons of known web applications. If there is a match, the name of the application is printed; otherwise the MD5 hash of the icon data is printed.

A chance to scan! From favicon-survey.nse (not included with Nmap):

Gets favicons and makes a database with their frequencies.

The script downloads each favicon it finds and stores it in a file in ICON_DIR
named after the MD5 hash of the favicon.

$ cd ~/favicon
$ ls icon
17F03417CBF92B80992B7CA7A566FB0C.ico  C89ECD7675567625E5755A7A9C31632D.ico
379A65BEB4D412765FCF9FBBDEECD416.ico  C8BFCB5728998AC6C3DA90EA5CD2340A.ico
7131EF7073ED685BF2987B9061C65D36.ico  CB5AA723DDDB0734CEC459F2B9C3B1C4.ico
88733EE53676A47FC354A61C32516E82.ico  D16A0DA12074DAE41980A6918D33F031.ico
A3C7BE1BCF382EA413C30453A4ACF638.ico  D41D8CD98F00B204E9800998ECF8427E.ico
A8FE5B8AE2C445A33AC41B33CCC9A120.ico  D4DA62A788942AAB81D033C9E49D57CB.ico
B6141EFEE8D8E64DBC23539F99F7238E.ico  ECF508711C226CCDA02D58853B31D7A7.ico
C3FB27F0BF8AC3171C8105726D61380A.ico

It also stores the host name and port of each host that had a favicon in a file
in HASH_DIR, also named with the MD5 sum.

The result:

# These are the most common icons from a scan of all dmoz sites in January
# 2010. Icons are listed in decreasing order by frequency.

# Commented-out lines are those that are too broad to be useful in
# identification.

# D41D8CD98F00B204E9800998ECF8427E: empty file
9CEAE7A3C88FC451D59E24D8D5F6F166: Parallels Plesk Panel
63B982EDDD64D44233BAA25066DB6BC1: Joomla! CMS
DCEA02A5797CE9E36F19B7590752563E: Parallels Plesk Panel
64CA706A50715E421B6C2FA0B32ED7EC: Parallels Plesk Panel
6C4EC806C82AB04D6B7D857E6FB68F95: Doteasy web hosting
1CE0C63F8BD1E5D3376EC0AE95A41C08: Parallels Plesk Panel
# ECAA... is a UTF-8 byte order mark, so may be more general.
ECAA88F7FA0BF610A5A26CF545DCD3AA: Demand Media domain parking
5B0E3B33AA166C88CEE57F83DE1D4E55: DotNetNuke CMS

What do you do with a million tiny icons?

489 megapixels of frequency data

More information at http://nmap.org/favicon/.

Questions

David Fifield
david@bamsoftware.com

nmap.org
nmap-dev@insecure.org