By Tyrone Burke
Photos by Fangliang Xu
The Morris Island Conservation Area is only about an hour’s drive west of Ottawa, but on a brilliant autumn morning, the city feels a world away.
The 47-hectare mix of forested woodlands and wetlands is bursting with crimson and gold, with reflections of towering maples and a cloudless sky shimmering in the glassy back bays of the Ottawa River. And even though it doesn’t look the part, Morris Island is a stand-in today for distant James Bay, which stretches south from Hudson Bay about 1,000 kilometres north of the National Capital Region.
Jeremy Laliberté (left) and Gerhard Bruins.
Backed by low-lying marshland, James Bay is sparsely populated and lined with broad beaches and nutrient-rich mudflats. It’s an ideal place for more than two dozen species of shorebirds to rest and feed before they continue their epic migrations.
Every summer, a team of volunteers travels to this roadless region; they count shorebirds by walking back and forth on transects that run perpendicular to the shore — about 10 kilometres each day — noting which species they spot and how many there are. But this type of survey can be invasive, disturbing birds such as the endangered red knot — a hefty, long-legged sandpiper that flies to Tierra del Fuego at the tip of South America — and adding risk to southbound journeys on which they must navigate coastal habitats damaged by ports, resorts and urban development.
Which is why one of those volunteers, Gerhard Bruins, asked an engineer at Carleton how the bird counts could be made more comprehensive, accurate and efficient. And why a small group of professors and students have gathered at Morris Island to test fly a new approach.
Using Drones to Enhance Wildlife Conservation
At the conservation area, master’s student Brendan Ooi is tiptoeing along the water’s edge, planting hyper-realistic 3D-printed models of shorebirds including red knots and greater yellowlegs among the rocks. Overhead, a small drone is hovering. The low buzz of its rotors is interrupted only by the clicks of its integrated camera system and the occasional splash of a Canada goose coming in for a landing.
The drone is piloted by Mechanical and Aerospace Engineering professor Jeremy Laliberté, Ooi’s academic advisor and one of the country’s leading autonomous aerial systems researchers.
Brendan Ooi
Laliberté and Ooi are working out how best to use aerial photography to count birds on the ground. They’re taking photographs and video footage of these model birds from different elevations to determine the optimal balance between covering terrain and accurately identifying birds.
“We’ve been trying to find the ideal flight path,” says Ooi.
“We want to know whether it’s better to fly in a grid pattern or in a line. We want to automate the image taking and video recording process, so a drone can be set up in advance and is easy enough to use that the volunteers can go north and run the bird count program even with little or no experience flying drones.”
But automating drone operations is one thing; processing the imagery is another. “Sometimes you can collect too much data,” says Ooi, “and you don’t know what to do with it.”
A Creative Collaboration Between Engineers and Biologists
That’s where Cheryl Schramm comes into the picture. The Systems and Computer Engineering instructor was the first person approached by Bruins, a former Canadian Wildlife Service scientist and director of the ecological consulting firm Species Inc., who has travelled to James Bay three times to count shorebirds the old-fashioned way.
Schramm recognized that the bird counting process could be automated by training a neural network to recognize aerial images of shorebirds — and that the artificial intelligence application could process this visual data much more efficiently than humans. But getting that information into a format the neural network can handle is an additional challenge, and two of Schramm’s fourth-year computer science students are working to solve it.
Simon Dewilde and Rachel Myrah are developing a photo stitching system that will merge the imagery captured by the drones into larger files and automatically eliminate the overlap between images. They’ve also started training a neural network to recognize shorebirds, so they can automate the process of counting birds in those stitched-together panoramas.
“We want to automate as many of the steps as possible,” says Dewilde, “to reduce the amount of human effort required. People have to go out there for weeks at a time and can be in dangerous situations when they do.”
Dewilde and Myrah are striving to make a system that can count birds at least as accurately as people can, while allowing volunteers to stay close to the hunting camps they rent from local First Nations during these annual counts (and far from any polar bears that happen by). But training a neural network to recognize and classify features in images can take thousands of photos; each classification can require a thousand images. And when you’re dealing with rare or endangered birds, there aren’t thousands of photos to choose from, especially from an aerial perspective.
“It requires a ton of data,” says Myrah. “Right now, we’re getting sample images, and even though they are not actual birds, they’ll let us train the neural network, which will hopefully enable us to count the real birds.”
Realistic 3D-Printed Birds Help Train Neural Network
For this approach to work, the model birds needed to be realistic. And for that, the researchers turned to biology professor Jeff Dawson.
“We collected photographs of the different shorebird species,” says Dawson, who worked with undergraduate computer science student Skyler Bruggink to create the digital models.
“To make sure we had modelled the birds accurately, we compared the 3D models to actual taxonomic specimens and made sure they had accurate dimensions and attributes.”
Skyler’s father Ed Bruggink, a wildlife artist who has managed the biology department’s greenhouses for nearly 40 years, then painted the models to give them a lifelike look and feel.
For ease of use, Laliberté chose to work with an off-the-shelf platform. The DJI Mavic 2 is a small quadcopter with a high-quality integrated camera. It’s a widely used aircraft for drone photography. By using a ready-made setup, Laliberté and his team could focus on how best to collect the data, rather than having to build a drone or design a photography system for one.
As Ooi places the model birds among the rocks on the sandy shores at Morris Island (whose maple trees belie the simulation of sub-arctic tundra), he notes that even if all the technical hurdles can be overcome, there will be one more challenge: not disturbing the living, breathing birds that are sharing Morris Island’s airspace with the drone.
“The biggest issue is how they react,” he says.
“If the birds don’t like you, you can’t really do this. If they stopped flying, we wouldn’t fly either.”
