TR-07-09: What Happened to Anomaly Detection?

Abstract

Mimicry attacks have motivated much of the recent work in program-level intrusion detection. Whether based on runtime monitoring or static analysis, work by Kruegel (2003), Giffin (2005) and others have focused on the challenge posed by attacks that are designed to emulate normal program behavior. Although mimicry attacks can be a significant threat in certain circumstances, we argue that they are not more significant than issues such as false positives and overall attack coverage. Furthermore, given the inevitable compromises involved in creating intrusion detection systems, a focus on mimicry attacks almost necessitates a compromise on these other qualities. In this paper we present a simple model of program-level intrusion detection systems that captures these trade-offs, and we apply this model to the work of Forrest (1996), Giffin, Kruegel, and Sekar (2001). This analysis shows that advances in mimicry resistance have been accompanied by an overall reduction in performance. We then propose an alternative interpretation of mimicry attacks that suggests improved designs for next generation systems.

TR-07-09.pdf