Load Testing with Artillery

In the ITS department the servers that run our mission critical software need routine maintenance and replacement. When we replace computer hardware we work with the vendor to know what performance level to expect, but these numbers are calculated in laboratory conditions. Real world implementation is very different from an isolated testing environment, so in the course of the replacement project it was important to collect and measure the difference in performance between the systems.

To answer the question of the performance delta, we turn to load testing. Essentially, you use a tool to send a large amount of network traffic to the system you want to test. Generally the tools used to craft these tests allow you to specify certain behaviour patterns to follow.

I worked closely with student support staff to determine what the most common performance issue are experienced, and we found one of the largest pain points to be the timetable calendar, and individual events. We determined it to be prudent to test a sustained load, representing average use, as well as a spike of traffic that mimics registration.

After assessing a variety of options available on the market I landed on an javascript package called Artillery. Artillery is written in modern javascript and runs on a platform called nodejs. It can be obtained from the npm registry, and documentation can be found on the Artillery website. As a command line tool, Artillery is run from a terminal window and uses test cases you have written in the YAML scripting language. In the case of our script we log in with a test user account, navigate to the user’s schedule, view an event, and then log out. Each of these actions returns a status code, which I monitor in real time as the tests run. When I no longer receive 200 OK responses from the server I know the server’s CPU has been overwhelmed. We record and graph the server’s load average during the tests with a piece of software called ORCA. When our tests are complete we are able to compare the results with our original hardware and can validate that the performance deltas are within expected levels.

Load testing is an instrumental validation step in the IT hardware life cycle. Our systems are tied to mission critical processes, and we must ensure they are able to meet the performance needs of our users.