This experiment evaluate the accuracy of different OSD tools to identify the actual operating system of a computer. The experiment relies on 95 targets with different OSes as part of the vlab testbed at CRC Canada. We rely on a set of 6,656 traffic traces to evaluate passive tools, while active tools had access to the actual targets. We fed each trace to a passive tool and record its output as the set of possible OSes. For active tools, we run the tool once against each target and use the output as the set of possible OSes for all the traces related to that target.
Experiment Results
Recall
The recall measure is computed by the number of traces for which the tool provided the good answer (i.e., the actual OS was among the set of possible OSes provided by the tool) divided by the number of traces analyzed.
Hence, the higher the reaall is, the better.
Precision
The precision measure is computed as the average size of the possible OSes set provided by the tool for the traces on which the tool provide the correct answer.
Hence, the lower the precision, the better.
Target descriptions
Click here to view the description of the targets used in the dataset.
Exploit descriptions
Click here for the description of the exploits used in the dataset.
Tool Outputs
| posd | (Engine version: 0.2) | (Rule Version: 0.2.6) |
| aosd | (Engine version: 0.2) | (Rule Version: 0.2.6) |
| hosd | (Engine version: 0.2) | (Rule Version: 0.2.6) |
| p0f(RstAck) | (Engine version: 2.0.8) | (Rule Version: Release) |
| p0f(SynAck) | (Engine version: 2.0.8) | (Rule Version: Release) |
| p0f(SynAck) | (Engine version: 2.0.8) | (Rule Version: Release) |
| p0f(Syn) | (Engine version: 2.0.8) | (Rule Version: Release) |
| SinFP | (Engine version: 2.00-8) | (Rule Version: Summer 2006) |
| Siphon | (Engine version: 0.666beta) | (Rule Version: Summer 2006) |
| Ettercap | (Engine version: NG.0.7.3) | (Rule Version: 22-03-2007) |
| Nmap | (Engine version: 4.20) | (Rule Version: April 2007) |
| Nmap | (Engine version: 2.0.3) | (Rule Version: April 2007) |