Measuring the accuracy of a high speed port scanner to perform web census
LE3 .A278 2015
Master of Science
The high-speed port scanner, ZMap, was used to conduct an Internet-wide scan of Port 80 in an attempt to determine the size of the Web. The size of the Web was estimated from the number of Web servers found during the high-speed scans of IPv4 addresses. The focus of the research was to see if an accurate census could be made in a reasonable amount of time using ZMap. The metrics for determining the success of this research were a decrease in time needed to perform an Internet-wide scan, compared to historical attempts, and a comparison between scan data collected in this research with data collected by Rapid7 Labs to test for accuracy of the ZMap scans. ZMap was used in combination with a program named Banner. Banner sent HTTP GET requests to confirm and count the number of Web servers. An optimal scan speed of 1000 pkts/s by ZMap and 3000 concurrent connections made by Banner improved the performance and accuracy of an Internet-wide scan such that the scan time decreased by a factor of 5 compared to historical attempts. Comparing data from several small-scale scans of a local Class B network and data from Rapid7 Labs, Project Sonar, revealed no significant differences between the two data sets. Any noted differences were attributed to the dynamic nature of the Web, and the local Class B network scans were deemed to have a high level of accuracy. This thesis research showed that the scan speed was substantially faster and more accurate than previous Web census techniques. Therefore, ZMap is recommended as a high-speed port scanner for future Web census research.
The author grants permission to the University Librarian at Acadia University to reproduce, loan or distribute copies of my thesis in microform, paper or electronic formats on a non-profit basis. The author retains the copyright of the thesis.