{"id":105,"date":"2015-06-19T13:46:28","date_gmt":"2015-06-19T13:46:28","guid":{"rendered":"http:\/\/carleton.ca\/nmai\/?page_id=105"},"modified":"2026-02-10T14:58:51","modified_gmt":"2026-02-10T19:58:51","slug":"experiment-2009-04","status":"publish","type":"page","link":"https:\/\/carleton.ca\/nmai\/research-projects\/hosd\/experiment-2009-04\/","title":{"rendered":"Experiment 2009-04"},"content":{"rendered":"\n<section class=\"w-screen px-6 cu-section cu-section--white ml-offset-center md:px-8 lg:px-14\">\n    <div class=\"space-y-6 cu-max-w-child-5xl  md:space-y-10 cu-prose-first-last\">\n\n            <div class=\"cu-textmedia flex flex-col lg:flex-row mx-auto gap-6 md:gap-10 my-6 md:my-12 first:mt-0 max-w-5xl\">\n        <div class=\"justify-start cu-textmedia-content cu-prose-first-last\" style=\"flex: 0 0 100%;\">\n            <header class=\"font-light prose-xl cu-pageheader md:prose-2xl cu-component-updated cu-prose-first-last\">\n                                    <h1 class=\"cu-prose-first-last font-semibold !mt-2 mb-4 md:mb-6 relative after:absolute after:h-px after:bottom-0 after:bg-cu-red after:left-px text-3xl md:text-4xl lg:text-5xl lg:leading-[3.5rem] pb-5 after:w-10 text-cu-black-700 not-prose\">\n                        Experiment 2009-04\n                    <\/h1>\n                \n                                \n                            <\/header>\n\n                    <\/div>\n\n            <\/div>\n\n    <\/div>\n<\/section>\n\n\n\n<p>This experiment evaluate the accuracy of different OSD tools to identify the actual operating system of a computer.&nbsp;The experiment relies on 95 targets with different OSes as part of the vlab testbed at CRC Canada.&nbsp;We rely on a set of 6,656 traffic traces to evaluate passive tools, while active tools had access to the actual targets.&nbsp;We fed each trace to a passive tool and record its output as the set of possible OSes.&nbsp;For active tools, we run the tool once against each target and use the output as the set of possible OSes for all the traces related to that target.<\/p>\n\n\n\n<h2 id=\"experiment-results\" class=\"wp-block-heading\">Experiment Results<\/h2>\n\n\n\n<h3 id=\"recall\" class=\"wp-block-heading\">Recall<\/h3>\n\n\n\n<p>The recall measure is computed by the number of traces for which the tool provided the good answer (i.e., the actual OS was among the set of possible OSes provided by the tool) divided by the number of traces analyzed.<\/p>\n\n\n\n<p>Hence, the higher the reaall is, the better.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/HOSD-experiment-recall.xlsx\">Per trace<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/HOSD-experiment-recall2.xlsx\">Per target<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/HOSD-experiment-recall3.xlsx\">Summary<\/a><\/li>\n<\/ul>\n\n\n\n<h3 id=\"precision\" class=\"wp-block-heading\">Precision<\/h3>\n\n\n\n<p>The precision measure is computed as the average size of the possible OSes set provided by the tool for the traces on which the tool provide the correct answer.<br>\nHence, the lower the precision, the better.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/HOSD-experiment-precision-per-trace.xlsx\">Per trace<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/HOSD-experiment-precision-per-target.xlsx\">Per target<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/HOSD-experiment-precision-summary.xlsx\">Summary<\/a><\/li>\n<\/ul>\n\n\n\n<h2 id=\"target-descriptions\" class=\"wp-block-heading\">Target descriptions<\/h2>\n\n\n\n<p><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/HOSD-Targets.xlsx\">Click here to view the description of the targets used in the dataset<\/a>.<\/p>\n\n\n\n<h2 id=\"exploit-descriptions\" class=\"wp-block-heading\">Exploit descriptions<\/h2>\n\n\n\n<p><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/HOSD-exploits.xlsx\">Click here for the description of the exploits used in the dataset.<\/a><\/p>\n\n\n\n<p>Tool Outputs<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th><\/th><th><\/th><th><\/th><\/tr><\/thead><tbody><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/posd2009-04.zip\">posd<\/a><\/td><td>(Engine version: 0.2)<\/td><td>(Rule Version: 0.2.6)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/aosd2009-04.zip\">aosd<\/a><\/td><td>(Engine version: 0.2)<\/td><td>(Rule Version: 0.2.6)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/hosd2009-04.zip\">hosd<\/a><\/td><td>(Engine version: 0.2)<\/td><td>(Rule Version: 0.2.6)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/p0fRstAck2009-04.zip\">p0f(RstAck)<\/a><\/td><td>(Engine version: 2.0.8)<\/td><td>(Rule Version: Release)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/p0fSynAck2009-04.zip\">p0f(SynAck)<\/a><\/td><td>(Engine version: 2.0.8)<\/td><td>(Rule Version: Release)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/p0fStrayAck2009-04.zip\">p0f(SynAck)<\/a><\/td><td>(Engine version: 2.0.8)<\/td><td>(Rule Version: Release)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/p0fSyn2009-04.zip\">p0f(Syn)<\/a><\/td><td>(Engine version: 2.0.8)<\/td><td>(Rule Version: Release)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/SinFP2009-04.zip\">SinFP<\/a><\/td><td>(Engine version: 2.00-8)<\/td><td>(Rule Version: Summer 2006)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/Siphon2009-04.zip\">Siphon<\/a><\/td><td>(Engine version: 0.666beta)<\/td><td>(Rule Version: Summer 2006)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/Ettercap2009-04.zip\">Ettercap<\/a><\/td><td>(Engine version: NG.0.7.3)<\/td><td>(Rule Version: 22-03-2007)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/Nmap2009-04.zip\">Nmap<\/a><\/td><td>(Engine version: 4.20)<\/td><td>(Rule Version: April 2007)<\/td><\/tr><tr><td><a href=\"http:\/\/carleton.ca\/nmai\/wp-content\/uploads\/sites\/199\/Xprobe2009-04.zip\">Nmap<\/a><\/td><td>(Engine version: 2.0.3)<\/td><td>(Rule Version: April 2007)<\/td><\/tr><\/tbody><\/table><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>This experiment evaluate the accuracy of different OSD tools to identify the actual operating system of a computer.&nbsp;The experiment relies on 95 targets with different OSes as part of the vlab testbed at CRC Canada.&nbsp;We rely on a set of 6,656 traffic traces to evaluate passive tools, while active tools had access to the actual [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":100,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","meta":{"_acf_changed":false,"_cu_dining_location_slug":"","footnotes":"","_links_to":"","_links_to_target":""},"cu_page_type":[],"class_list":["post-105","page","type-page","status-publish","hentry"],"acf":{"cu_post_thumbnail":""},"_links":{"self":[{"href":"https:\/\/carleton.ca\/nmai\/wp-json\/wp\/v2\/pages\/105","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/carleton.ca\/nmai\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/carleton.ca\/nmai\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/carleton.ca\/nmai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/carleton.ca\/nmai\/wp-json\/wp\/v2\/comments?post=105"}],"version-history":[{"count":3,"href":"https:\/\/carleton.ca\/nmai\/wp-json\/wp\/v2\/pages\/105\/revisions"}],"predecessor-version":[{"id":1158,"href":"https:\/\/carleton.ca\/nmai\/wp-json\/wp\/v2\/pages\/105\/revisions\/1158"}],"up":[{"embeddable":true,"href":"https:\/\/carleton.ca\/nmai\/wp-json\/wp\/v2\/pages\/100"}],"wp:attachment":[{"href":"https:\/\/carleton.ca\/nmai\/wp-json\/wp\/v2\/media?parent=105"}],"wp:term":[{"taxonomy":"cu_page_type","embeddable":true,"href":"https:\/\/carleton.ca\/nmai\/wp-json\/wp\/v2\/cu_page_type?post=105"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}