ScanEngine

Almost all of Nmap's scans happen in the function ultra_scan in the file scan_engine.cc. Here is its declaration:

void ultra_scan(vector<Target *> &Targets, struct scan_lists *ports,
        stype scantype);

About the parameters

The class Target stores information about a scan of a single host. As you can see, an entire vector of targets is scanned at once.

struct scan_lists (defined in global_structures.h) is just a list of TCP or UDP ports and IP protocols:

struct scan_lists {
        unsigned short *tcp_ports;
        int tcp_count;
        unsigned short *udp_ports;
        int udp_count;
        unsigned short *prots;
        int prot_count;
}; 

stype is an enum in global_structures.h:

typedef enum { STYPE_UNKNOWN, HOST_DISCOVERY, ACK_SCAN, SYN_SCAN, FIN_SCAN,
XMAS_SCAN, UDP_SCAN, CONNECT_SCAN, NULL_SCAN, WINDOW_SCAN, RPC_SCAN,
MAIMON_SCAN, IPPROT_SCAN, PING_SCAN, PING_SCAN_ARP, IDLE_SCAN, BOUNCE_SCAN,
SERVICE_SCAN, OS_SCAN, SCRIPT_SCAN, TRACEROUTE, REF_TRACEROUTE}stype;

Data structures

UltraScanInfo

Each scan creates an UltraScanInfo instance. An UltraScanInfo is created using the same parameters as ultra_scan: a list of targets, a list of ports, and a scan type. It includes a bunch of bool variables like tcp_scan, udp_scan, icmp_scan, and others, which I believe are a convenient broken-down form of the scan type. There are a libdnet Ethernet handle, a pcap handle, and a raw socket descriptor.

UltraScanInfo contains a GroupScanStats, which contains statistics for the entire scan. It also contains a list of incomplete hosts (hosts which haven't finished being scanned). Each incomplete host is a HostScanStats which has the state for one host (things like outstanding probes and retransmission counts).

UltraProbe

An UltraProbe represents one of the probes that can be sent in ultra_scan. The list of outstanding probes in a HostScanStats is a list of UltraProbes. You can set it to be an IP, a connect, or an ARP probe using the methods setIP, setConnect, and setARP. When you set it to IP, you give it a struct probespec as a prototype.

struct probespec

typedef struct probespec {
  u8 type; /* PS_NONE, PS_TCP, PS_UDP, PS_PROTO, PS_ICMP, PS_ARP. */
  u8 proto; /* If not PS_ARP -- Protocol number ... eg IPPROTO_TCP, etc. */
  union {
    struct probespec_tcpdata tcp; /* if type is PS_TCP */
    struct probespec_udpdata udp; /* PS_UDP */
  } pd;
} probespec;
struct probespec_tcpdata {
  u16 dport;
  u8 flags;
};
struct probespec_udpdata {
  u16 dport;
};

A probespec is a probe type, a protocol (?), a destination ports, and flags. It can be thought of as a condensed version of UltraProbe.

ultra_scan walkthrough

ultra_scan goes like this:

• If Targets is empty, return (whew!).
• Start the timeout clock on tagets that haven't timed out. Each target has its own timeout clock.
• Create an UltraScanInfo object (see below) called USI.
• begin_sniffer(USI, Targets). This calls my_pcap_open_live in tcpip.cc, which in turn calls pcap_open_live to start sniffing. begin_sniffer builds up a Berkeley Packet Filter filter string based on Targets and the scan type in USI. This limits the packets received to only potentially interesting ones. The filter string is printed out with -d2.
• While there are incomplete hosts in USI:
  • doAnyPings. Ping any incomplete hosts if we've sent at least 10 probes to that host since the last ping, it's been at least a pingtime (struct ultra_scan_performance_vars) since the last response and last sent ping, and the congestion-control/rate-limiting mechanisms say it's okay. This doesn't happen (I think) the first time this function gets called, just because a pingtime hasn't elapsed yet.
    The pings are sent using either sendConnectScanProbe or sendIPScanProbe, the same functions that do normal port scan probes. They're called in a special way that causes them to send ping probes, encoding the retransmission number in the source port field.
  • doAnyOutstandingRetransmits. This runs through the list of outstanding probes for each host and retransmits them as appropriate, subject to rate limiting.
  • doAnyRetryStackRetransmits. Outstanding probes are moved to the "bench" when they expire or reach the maximum number of retransmissions. If the maximum number of retransmissions is increased, they are moved from the bench to the retry stack (see HostScanStats::retransmitBench). retransmitBench doesn't actually retransmit the probes; that's done by doAnyRetryStackRetransmits and sendNextRetryStackProbe.
  • doAnyNewProbes. This calls get_next_target_probe to get the next TCP port, or UDP port, or protocol, or whatever (depending on scan type), encapsulated in an struct probespec. The probe is then sent using sendArpScanProbe, sendConnectScanProbe, or sendIPScanProbe (recall that two of these functions are also used to send pings). The probes are sent with an encoded retransmission number of 0, because they're new.
  • waitForResponses. This gets a packet or ARP reply from the pcap sniffer started in begin_sniffer (or just does a select if doing a connect scan), and interprets it. There's a bunch of nested logic to determine if the packet is relevant and what to do with it if it is. This includes marking hosts as being up or down, or marking ports open, closed, or something else. This step is where host and port states actually change (well, actually in the call to ultrascan_port_probe_update.
  • processData. Remove completed hosts from the incomplete lists. Expire timed-out probes. When a probe gets no reply, set the port or host state accordingly (see scantype_no_response_means).