LogArchive2007

April 2007

• 12: Got the official acceptance notice.
• 13: Found out about the Subversion repository.
• 15: Learned about the various databases Nmap ships.
• 19: main.cc investigation.
• 28: libdnet.
• 29: Wrote a libdnet ping of death program.
• 30: Found an assertion failure in traceroute and started investigating it. Refined the ping of death.

May 2007

• 6: Read the reference guide and the version detection paper. Checked out targets and some of the scanning mechanism.
• 7: More research into the traceroute assertion failure.
• 8: Submitted a bug report about the traceroute issue. More traceroute research.
• 10: Read the idle scan paper. I get it now! That's cool.
• 12: Browsed scan_engine.cc, trying to get the idea.
• 13: Spent some serious quality time with scan_engine.cc. Scan engine.
• 15: Read parts of the Nmap and Subversion books. Checked out and built using my infrastructure credentials. Wrote up my notes on the ultra_scan research.
• 16: Looked at Eddie's reason patch. Sent some observations and suggestions. Read more from the Nmap book. Wrote a version detection line for monchat.
• 17: Skimmed over RFC 2553, on IPv6 and things like sockaddr_storage. Read the OS detection chapter of the Nmap book. Walked through a Wireshark trace of OS detection.
• 18: Merged from trunk and wrote a little how-to. Submitted a patch to force overwriting script files. Read the Nmap book chapter on ping scanning.
• 19: Committed a change to use svn export when installing script files.
• 21: Committed a modified change after discussion on nmap-dev.
• 22: Merged from trunk. I'm the "SVN merge expert" now! Had my very first change committed. Fixed a bug with list and ping scan not working with the enhanced port selection patches.
• 23: Read the NSE chapter of the Nmap book and parts of the Lua tutorial.
• 24: Read more of the Nmap book.
• 25: Chatted with Fyodor on instant messaging. Agreed on tasks to get started with. Looked at some more NSE implementation. Spent some time experimenting with weeding out header files. Graphed build dependencies. Wrote a program to automatically compare the number of dependencies in makefile.dep.
• 26: Got my book from Google in the mail. Figured I should start reading the mailing lists. Looks like I have to send in some forms.
• 27: Researched ways to keep the version string in just one place.
• 28: Sent status report #1. Submitted a version string patch.
• 29: Worked on a way to show the locations of used data files. Committed a patch to do the --services option.
• 31: Created an nmap-dependency branch. Did a lot of dependency reduction work.

June 2007

• 1: Installed a Windows compiler. Committed the NMAP_VERSION patch. Reported a Subversion keyword problem with nmap-service-probes. Committed the dependency-reduction patch. Renamed the --services option to --servicedb. Added a --versiondb option.
• 2: Committed a change to keep from copying .svn directories into the Windows zip file.
• 3: Got my first payment. Investigated a weird Windows bug that appears when running a Debug build but not a Release build.
• 4: Sent status report #2. Worked on ideas for presenting data file paths and sent it to nmap-dev. Deleted the dependency branch.
• 5: Worked on an implementation for verbose data file paths.
• 6: Worked (for a long time) on the verbose data file paths. Came up with a pretty good solution. Committed a patch. Debugged the issue that appeared only on Windows, turns out it's not just a Windows issue.
• 7: Worked on rewriting the basename and dirname replacements so they can go in nbase. Talked with Fyodor on IM. Committed a correction to my reason patch.
• 8–12: Away on vacation. (Sent status report #3.)
• 13: Moved dirname and basename replacements into nbase. Reported a bug having to do with --script-updatedb (fixed by Stoiko). Removed INTERACTIVE_NAMES.
• 14: Committed a change to make init_updatedb use the new path_get_basename. Did OS database integration training with Fyodor. Wrote up some OS detection integration notes.
• 15: Looked at merging identical lines in OS fingerprint submissions. Wrote a merging test program.
• 16: Made the nmap-dev utilities warn when they see an illegal empty value, and keep fingerfix from printing spurious empty values.
• 17: Did about 100 OS submission integrations.
• 18: Sent status report #4.
• 20: Committed a change that handles the case where the initial TTL is calculated to be greater than 255. Committed another to handle negative distances. Talked to Fyodor after I discovered that some reference fingerprints have a TTL greater than 255. We decided to allow it and make the fingerprint utilities warn if it's seen.
• 21: Refactored the fingerprint-checking code in the OS database utilities. Added a test for TTLs greater than 255. Worked through about 120 submissions.
• 22: Worked through about 250 submissions.
• 23: The disk on the Subversion server filled up. Fixed a bug with fingerfix having to do with the removal of empty tests. Worked through about 200 OS submissions.
• 25: Sent status report #5. Worked through about 410 OS submissions.
• 26: Worked through about 220 OS submissions.
• 28: Worked through about 40 OS submissions (mostly finished).
• 29: Made mergeFPs merge fingerprints more intelligently.

July 2007

• 2: Sent status report #6. Sent OS detection leftovers to Fyodor. Reviewed the top ports patch and found a few small bugs.
• 3: Had a long meeting with Fyodor to work through the last of the OS detection submissions. We decided I should start working on the massping migration.
• 4: Studied host discovery.
• 5: Created a massping migration branch. Found and fixed a bug in my previous mergeFPs change that kept first-generation fingerprints from being merged. Started hacking in the branch, mainly in the code that surrounds massping (nmap_main and nexthost).
• 6: Sent a request for advice and a brief proposal to nmap-dev describing my ideas for the massping migration. Researched the issue some more.
• 8: 4.22SOC1 released. Poked at massping migration some more.
• 9: Sent status report #7. Worked on hacking up a prototype of an ultra_scan-based massping. Did a little bit of canonicalization in the OS databases, which edits Fyodor beat me to.
• 11: Very carefully worked on implementing TCP SYN ping scan in ultra_scan. Got it working, though without short-circuiting and with a problem with dismissBench. Also it only handles TCP replies, not ICMP.
• 12: Worked some on the massping migration. Got it mostly working for TCP SYN ping scan.
• 13: Sent status report #8. Finished my ultra_scan-based massping prototype. Notified nmap-dev. Solicited code review from the soc list.
• 13–21: Away at a leadership camp.
• 22: Caught up on email.
• 23: Send status report #9. Made ping scan bail out after discovering a host's state. Make ping scan support TCP ACK.
• 24: Dealt with an issue having to do with interrupted system calls in pcap_select.
• 25: Got mail from a Google recruiter.
• 26: Added support for UDP ping scanning. Gave ultra_scan the ability to send ICMP packets.
• 27: Had a phone call with the Google recruiter. Added support for ICMP ping scan.
• 28: Started implementing TCP connect ping scan.
• 29: Finished implementing TCP connect ping scan.
• 30: Sent status report #10. Did a 50,000-host massping test.
• 31: Wrote a script to compare host lists. Sent a message to nmap-dev to ask for help in debugging the new host discovery code.

August 2007

• 1: Had a telephone interview with a Google engineer. Talked with Fyodor about how to correct the ultra_scan host discovery deficiencies.
• 2: Committed a change to reduce the number of calls to UltraScanInfo::numIncompleteHosts. Did some more host discovery testing.
• 5: Did some port scans of the hotel using the old and new host discovery code. Saved the logs for later review.
• 6: Sent status report #11.
• 10: Did more host discovery research.
• 11: Did host discovery benchmarking and experimented with changes in my working directory to improve it.
• 12: Found that hosts being removed from incompleteHosts is the primary reason for failing to mark hosts as up. Committed a change to keep a list of completed hosts around.
• 13: Did a bunch of host discovery benchmarking and found a few cases where the new ping scan code is superior. IMed with Fyodor about host discovery. Sent status report #12. Had another Google phone interview.
• 14: Did more extensive benchmarking. Committed a change to use timing pings in host discovery, which at first glance looks to speed it up a lot.
• 15: Made a few small changes to host discovery: send ICMP message with a non-zero identifier, and mark a host up when a destination unreachable comes directly from it. ultra_scan-based host discovery is starting to be competitive with massping. Found out that TCP connect scans sometimes hang.
• 16: Ran a big 50,000-host benchmark, and got more hosts in less time with ultra_scan than with massping.
• 17: Did more benchmarking. Fixed a bug related to the shortening of TCP connect ping scan timeouts.
• 19: Experimented with retaining the group timeouts between invocations of ultra_scan.
• 20: Sent status report #13. Made a change to have massping save timeouts across invocations. Wrote a script to graph congestion control variables.

End of Summer of Code 2007

August 2007

• 22: 3 h. Made saved timeout values specific to an interface. Fixed a bug that kept SSL in nsock from working. Worked on graphing performance parameters.
• 23: 3 h. Fixed some bugs reported on nmap-dev. Spent a lot of time researching connect ping scan drops. Google invites me to an interview in California, but I object to their NDA requirement.
• 24: 6 h. Bug fixing. Host discovery performance investigation. Book proofreading.
• 25: 4.5 h. More work on drops and timing.
• 26: 2.25 h. Extensive testing of connect ping scan, performance improvements.
• 27: 4 h. Discussion with Fyodor about progress and merging. Did the big merge. Tested the merge.
• 28: 4.25 h. Split the logic between updating timeouts and updating congestion control variables. Found and fixed a bug having to do with retransmits exceeding the congestion window. Wrote a letter to nmap-dev explaining the migration.
• 29: 5 h. Fielded host discovery testing reports from nmap-dev. Made some recommended changes. Revised the Nmap book section on "Scan Code and Algorithms".
• 30: 2.25 h. Added an EACCES handler for connect scans. Worked on debugging ping scan issues.

September 2007

• 1: 2.5 h. Made graphs for the book.
• 2: 1 h. Integrated OS correction submissions.

Total August 20–September 2: 37.75 h.

• 3: 1.25 h. Integrated about 50 OS fingerprint submissions.
• 4: 3.5 h. Made a few congestion control tweaks. Innovated with congestion control algorithms: Response rate scaled congestion control.
• 5: 1.75 h. Sent a message to nmap-dev about the congestion control changes. Made some other small changes.
• 6: 3 h. Worked further with testing host discovery. Integrated about 150 OS submissions.
• 9: 2.5 h. Integrated about 100 OS fingerprints.
• 10: 4.75 h. Integrated about 100 OS fingerprints. Worked on graphs for the book.
• 11: 1.5 h. Made some graph tweaks. Worked on interpreting testing reports.
• 12: 2.75 h. Integrated about 200 OS submissions.
• 13: 4.25 h. Integrated about 100 OS submissions. Debugged a Windows crash.
• 14: 2.5 h. Fixed the Windows crash. Read up on Python distutils. Wrote a patch for an annoying MODULE_DIR Umit problem.
• 16: 1.75 h. Merged some revisions from umit trunk.
• 17: 2 h. Merged the response rate scaled congestion control.
• 18: 0.25 h. Merged from Umit trunk.
• 19: 1.5 h. Updated a graph for the book. Investigated scaling host congestion variables.
• 20: 1.75 h. Started documenting what I could about Umit. Committed some small patches.
• 21: 6.25 h. Got set up to build Umit on Windows. Committed some more patches. Made icon loading better. Merged three MANIFEST.ins into one.
• 23: 2.5 h. Created a SourceForge account. Merged a bunch of stuff from Umit trunk.
• 24: 5.5 h. Had a IM meeting with Fyodor. Merged some changes back into Umit trunk. Made Umit's pickling scripts use local files. Got another message from Google offering to let me see the NDA.
• 25: 3 h. Tried to benchmark host scaled congestion control but unreliable network conditions skewed the results. Updated Umit version numbering.
• 27: 3.25 h. Worked with Umit version number handling. Found a GTK bug.
• 28: 4.5 h. Set up another installation to build Nmap and Umit on Windows. Fixed the GTK bug. Fixed Windows build and install scripts. Worked on getting "make distro" to work for me. Worked on improving Umit version number updating, but didn't commit it yet.
• 30: 1.25 h. Committed a change to Umit versioning.

Total September 3–September 30: 61.25 h.

October 2007

• 1: 4 h. Did many more benchmarks with the host scaled congestion control. Merged the host scaled congestion control. Talked with Leslie Hawthorn at Google.
• 2: 3.5 h. Merged per-host scaled congestion control. Analyzed the effect of raising the increment cap: Performance graphs#timeouts. Fixed pcre_chartables.c auto-generation.
• 3: 5 h. Fixed line endings in a couple of NSE scripts. Merged from Umit trunk. Fixed a formatting bug in the script that generates os_classification.dmp. Emailed Adriano about merging back to Umit trunk.
• 4: 1 h. Fixed a crash in the command wizard caused by a misspelled identifier. Moved *.xml files to /usr/share.
• 9: 0 h. Had a meeting with Fyodor. Contacted Bret about vectorizing the eye logo.
• 11: 1.75 h. Made various Umit improvements.
• 12: 1.25 h. Removed dead code and the splash screen from Umit.
• 13: 4.75 h. Made the Windows icon integrate different sizes and bit depths. Merged back to Umit trunk. Solved the makefile dependency problem that was causing nmap to be always rebuilt. Unified setup.pys a little more.
• 14: 4.5 h. Unified setup.py. Deleted a ton of obsolete files.
• 15: 0.5 h. Had a chat with Fyodor. Looked for OpenBSD shell accounts.
• 16: 0.75 h. Fixed the OpenBSD but another was reported. Worked on debugging a segfault.
• 17: 2 h. Worked on setting up the shell account. Debugged compilation on OpenBSD. Researched about libtool to fix one remaining issue.
• 18: 7 h. Extensively analyzed the OpenBSD build issue and made a recommendation on how to fix it. Investigated the --script=ripeQuery.nse segfault and found that it was probably due to my machine having IPv6 functions but not supporting IPv6. It's difficult to make NSE report an error when this occurs, however.
• 22: 4.5 h. Committed a fix for a GTK+ configure test. Committed the OpenBSD fix. Started working on Zenmap renaming.
• 23: 2 h. Did most of the Zenmap renaming.
• 24: 3.75 h. Did the rest of the preliminary Zenmap renaming (phase I). Started work on the About screen.
• 26: 3.75 h. Finished an About dialog prototype.
• 27: 1.25 h. Tested Kris's IP protocol ping scan. Changed Umit to Zenmap in the documentation.
• 28: 1.25 h. Started working on better port selection for ping scan.

Total October 1–28: 52.5 h.

• 29: 5.25 h. Port selection. Got a prototype of a refactored getpts that passes the test suite.
• 30: 1 h. Ping scan port selection design.

November 2007

• 1: 6.75 h. Removed NmapFE. Read about creating RPMs. Committed ping scan port selection. Worked on improving the internals of port selection. Emailed nmap-dev about NSE exceptions.
• 2: 5 h. Removed first-generation OS detection. Fixed port selection wildcards in Zenmap. Worked a bit on makefile improvement.
• 3: 4.25 h. Worked on a Zenmap RPM. Deleted some more Zenmap files. Worked on the build system a bit.
• 4: 3 h. Made Zenmap's installation process more RPMable. Finished the Zenmap RPM.
• 5: 2 h. Worked on making the Zenmap RPM work with Python 2.4 and lib64.
• 6: 1.75 h. Made the Zenmap RPM work with Python 2.4 and lib64. Tried splitting it out into its own spec file.
• 7: 1 h. Split the Zenmap RPM into its own spec file and tested.
• 8: 1 h. Added checks for nse->iod->sd == -1 to nsp_add_event. Made nmapfe and xnmap symlinks. Changed the bug report window.
• 9: 0.5 h. Added a header to the bug report window. Made configure check for Python.
• 10: 0.25 h. Responded to mailing list questions, looked for a Umit branch with a reported bug fix.
• 11: 4.25 h. Looked at bugs in the Umit tracker. Fixed a bug that kept nbase from building on Solaris 9.
• 12: 4 h. Worked on the Zenmap man page. Fixed compilation and alignment (bus error) on Solaris. Discovered the cause of lazy binding errors on Mac OS X.
• 14: 0.5 h. Started working on the WinPcap installer.
• 15: 3.25 h. Upgraded WinPcap to version 4.02. Changed Zenmap file names.
• 16: 4 h. Did extensive research and found a solution for building version-independent Zenmap RPMs.
• 18: 2.5 h. Updated configure scripts to Autoconf 2.61. Looked at how libdnet handles Windows interfaces. Made Zenmap emit a more meaningful error message when a dump file can't be loaded.
• 19: 5 h. Made better error messages on import errors in Zenmap. Made Zenmap look relative to its script directory for certain data files. Cleared out a bunch of Zenmap bugs. Added a check for strip in configure. Found a possible solution to multiple IP addresses per interface on Windows. Later revised my patch.
• 20: 4.75 h. Went to school and played with py2app. Wrote a patch to allow testing whether SSL is supported from NSE scripts. Worked on cleaning up the build system.
• 21: 1.5 h. Committed the patch that allows testing for SSL from within NSE. Committed a build enhancement that makes nmap be rebuilt when something in a subdirectory changes. Worked on skipping Zenmap at configure time when Python is not detected.
• 23: 0.5 h. Did research on wireless problems on Windows Vista. Posted to the mailing list about it.
• 24: 0.5 h. Made Zenmap show an error message when it's run under Python 2.3.
• 25: 2.25 h. Removed unused command line options from Zenmap. Committed a first version of the Zenmap man page.

Total October 29–November 25: 64.75 h.

• 26: 3.75 h. Fixed interface aliases on Windows. Added a debugging message for unknown interface types in libdnet. Worked on removing superfluous config.guess, config.sub, and install-sh files (and some others). Put zenmap.1 into the install and package building process.
• 27: 0.5 h. Mailing list issue investigation. Burned through OS correction spam.
• 28: 0.5 h. Mailing list issue debugging.
• 29: 5.75 h. Fixed the bug that the nselib-bin subdirectory wasn't being configured. Removed a bunch of autogenerated files from nselib-bin. Did 100 OS submissions. Debugged Windows Vista interface names. Sent a summary and a proposed fix.
• 30: 1.25 h. Renamed Zenmap packages to avoid conflicts with those of Umit. Documented some nmaplib functions in the NSE section of the book. Did 100 OS submissions.

December 2007

• 2: 5.5 h. Did 450 OS submissions.
• 3: 5.5 h. Did 450 OS submissions.
• 4: 1.5 h. Debugged Zenmap issues. Did 50 OS submissions.
• 5: 6.5 h. Did 500 OS submissions.
• 6: 1.25 h. Did all 67 non-spam OS corrections. Initial debugging of mailing list reports.
• 7: 0.25 h. Debugging of Mac OS X bug reports.
• 8: 2.5 h. Fixed and tested absolute script paths. Fixed some script problems.
• 9: 0.75 h. Debugged a Zenmap crash. Sent mail to submitters of unusual OS fingerprints.
• 10: 1 h. Forgot about my weekly meeting with Fyodor. Wrote an outline of the Zenmap docs. Got the HTML files built.
• 11: 1.25 h. Wrote sections on saving and loading results and the Zenmap database. Tested 4.50 Windows packages.
• 12: 5 h. Fixed instructions in zenmap/install_scripts/README. Eliminated double-spacing in Zenmap when loading results files. Escaped OS fingerprints in XML output. Made Zenmap use -PN instead of -P0. Made leading-backslash paths absolute on Windows. Committed Rob Nicolls's fix for --resume on Windows. Fixed skipped host message ("Are what?"). Regenerated .dmp files. Updated the Zenmap web site in the About dialog.
• 13: 1.5 h. Debugged mailing list issues. Wrote some more Zenmap documentation.
• 14: 0.25 h. Zenmap documentation.
• 15: 3.75 h. Zenmap documentation. Committed a first incomplete version of the Zenmap Guide.
• 16: 0.75 h. Wrote another section of Zenmap documentation.
• 18: 2.25 h. Wrote a section on Zenmap's control files and part of a section on the profile editor.
• 19: 3.5 h. Wrote sections on searching and comparing.
• 20: 4 h. Did some more work on the Zenmap Guide, also added sections on command line options and history. Fixed a Windows bug (unprivileged scan against localhost).

Total November 26–December 23: 58.75 h.