TR-05-06: Addressing Malicious SMTP-based Mass-Mailing Activity Within an Enterprise Network

Abstract

Malicious mass-mailing activity on the Internet is a serious and continuing threat that includes massmailing worms, spam, and phishing. A mechanism commonly used to deliver such malicious mass mail is an SMTP-engine, which turns an infected system into a malicious mail server. We present a technique that enables, in certain network environments, detection and containment of (even zero-day) SMTP-engine based mass-mailing activity within a single mailing attempt. Contrary to other massmailing detection techniques our approach is content independent and requires no attachment processing, statistical measures, or system behavioral analysis. It relies strictly on the observation of DNS MX queries within the enterprise network. Our approach can be used as an alternative to port 25 blocking and in conjunction with current proposals to address mass-mailing abuses (e.g. SPF, DomainKeys). Our analysis on network traces from a medium sized university network indicates that MX query activity from client systems is a viable SMTP-engine detection method with a very low false positive rate. Our detection and containment approach has been successfully tested with a prototype using a live massmailing worm in an isolated test network.