New Cornell Lab tool identifies birds in photos

HarlequinDuck_MerlinPhotoID_Christopher_Wood-440
As shown in this photo of a Harlequin Duck, users of the Merlin Bird Photo ID identifier draw a box around a bird and click on the bill, eye, and tail. Merlin does the rest, using computer vision to identify the bird. Photo by Christopher L. Wood

The folks at the Cornell Lab who brought us the excellent Merlin bird-ID app for smartphones have just released a new tool for identifying birds, and wow, is it impressive!

It’s a free web-based photo identifier that is capable of recognizing 400 of the most commonly encountered birds in the United States and Canada. (See the list.)

Called Merlin Bird Photo ID, the identifier “gets the bird right in the top three results about 90 percent of the time,” said Jessie Barry, Merlin’s project leader. “It’s designed to keep improving the more people use it. That’s truly amazing, considering that the computer-vision community started working on the challenge of bird identification only a few years ago.”

To see if Merlin can identify the bird in a photo, you start by uploading an image and telling the application where and when you took it. Then it prompts you to draw a box around the bird and click on its bill, eye, and tail.

Within seconds, Merlin looks at the pixels and combines powerful artificial intelligence techniques with millions of data points from humans and presents the most likely species, including photos and sounds.

I tried it with seven photos I took this spring: a straightforward shot of a Solitary Sandpiper, an upper-body image of a Sandhill Crane, small-in-the-frame pictures of Wilson’s and Chestnut-sided Warblers, a Baltimore Oriole partially obscured by vegetation, a calling female Red-winged Blackbird, and a somewhat-out-of-focus Hermit Thrush that I photographed through a dirty (but bird-friendly) windowpane.

Merlin’s first choice was correct on six of the images, and on the thrush shot, it listed Swainson’s Thrush first and Hermit second. That’s a solid A in my book. Here is what I saw after the application analyzed my so-so Wilson’s Warbler photo:

Merlin-screen-shot-Wilsons
This screen shot shows my Wilson’s Warbler photo on the left and Merlin’s best matches on the right.

The photo tool has been in development since 2010. Its success relies on collaboration between computers and humans. The tool learns to recognize each species from tens of thousands of images identified and labeled by birdwatchers. It also taps in to more than 70 million sightings recorded in the eBird database, narrowing its search to the species found at the location and time of year when the photo was taken.

Because the photo identifier uses machine-learning techniques, it has the potential to improve the more people use it. For the time being, it works on computers running only a recent version of Chrome or Safari. After it can reliably identify photos taken with smartphones, the team will add it to the Merlin bird-ID app.

And the developers plan to add more North American species, followed by species from other parts of the world.

We can’t wait! — Matt Mendenhall, Managing Editor

Read more articles about bird identification.

Read “On the Move,” our regular column about migration from the leaders of eBird.

Crowdsourced eBird data shows that songbirds utilize weather patterns during migrations.

 

Read our newsletter!

Sign up for our free e-newsletter to receive news, photos of birds, attracting and ID tips, and more delivered to your inbox.

Sign Up for Free