PortSetGraphics

Here are graphics showing the set of ports scanned in several scenarios. Each one shows 65,536 ports in a 256 × 256 bitmap. Each black dot is a scanned port. Read left to right, top to bottom. Port 0 is in the top left and port 255 is in the top right.

These are as of August 20, 2008.

 Old-style nmap-servicesNew-style with frequency data (nmap-bhdc08 branch)
Default TCP scan
Fast TCP scan
Default UDP scan
Fast UDP scan

Analysis of magic table

My conjecture is that the magic table reverse-engineered from the Conficker.C source code at http://mtc.sri.com/Conficker/contrib/#example-code is a blacklist of port numbers. Each bit represents 32 ports (probably to save space), so the 2048 bits in the table cover all 65,536 port numbers. The first two 0xffffffff entries blacklist the ports 0–2047.

The magic_shift macro could be called port_is_blacklisted. To understand how it works, imagine writing it this way:

/* Look up a bit in a bit vector composed of uint32_t. The low-order
   5 bits of b are a bit index into an element of array; the high-order
   bits index the array directly. */
#define BIT_IS_SET(array, b) ((1 << ((b) & 0x1F)) & array[(b) >> 5])

int port_is_blacklisted(uint16_t port)
{
    uint16_t reduced;

    /* Discard the low-order 5 bits. This is where the blocks of 32 come from. */
    reduced = port >> 5;
    return BIT_IS_SET(magic, reduced);
}

Mentally making this change in the code, we can see that the generator loops while (port_is_blacklisted(...)) (and while port numbers are duplicated), expressing the presumed intention of the code in a clear way.

Probably the blacklist is to avoid commonly used ports. I checked to see how many of Nmap's default scanning ports are covered by the blacklist, thinking that the amount of overlap would indicate whether the blacklisted blocks really do cover commonly used port numbers. I modified the port set generation program to mark the blacklisted blocks (shown in dark gray). The ports in Nmap's database are marked in black. These are Nmap's ports on a default TCP scan using both the old-style (no frequency data) nmap-services and a new-style file.

Old-styleNew-style
Blacklisted: 1628 (94.8%) Not blacklisted: 89 (5.2%)Blacklisted: 754 (75.4%) Not blacklisted: 246 (24.6%)

There is a good deal of overlap in both cases (95% and 75% respectively), so I think it's reasonable that the purpose of the magic table is to blacklist certain ports.

Here are the data files to reproduce the above graphics: Attach:port-graphic.py Attach:ports-tcp.txt Attach:ports-tcp-freq.txt

I wrote a program that iterated the non-week-dependent random port number generation program, starting with the address 0.0.0.0 and using v.u32[0] as the address for the next round. I pulled off only one of the two port numbers generated each time. Here are pictures of what port numbers are generated for different numbers of iterations. The generator doesn't hit every single non-blacklisted port, but it may eventually hit them all in different weeks.

IterationsPort numbers generated
100
(100 ports generated)
1000
(997 ports generated)
10000
(9185 ports generated)
100000
(44242 ports generated)
1000000
(44242 ports generated)
Page last modified on April 06, 2009, at 11:07 PM