Air Date: Week of July 8, 2011

LeafSnap uses the intricate details of the leaf to come up with a tree species identification. (Wikipedia Creative Commons)

Curious about that tree in your neighborhood? Take a picture of its leaves with LeafSnap on your iPhone. It’s an app that sends information about the leaves of a tree to a database and retrieves an identification. Host Bruce Gellerman talks with LeafSnap co-creator John Kress about what the app can do for nature awareness. Photo: LeafSnap uses the intricate details of the leaf to come up with a tree species identification. (Wikipedia Creative Commons)

Transcript

GELLERMAN: It's Living on Earth, I'm Bruce Gellerman. I think that I shall never see, an app that can I.D. a tree…. but now I can! Poems are made by fools like me but it took John Kress to develop a smart phone app to find the right tree. Kress is Chief Botanist at the Smithsonian Institution. He and some computer experts at Columbia University have created LeafSnap. Dr. Kress, welcome to Living on Earth.

KRESS: Oh it's great to be here and talking to you.

GELLERMAN: So, LeafSnap. You take a picture with your iPhone, then what happens?

KRESS: “Then what happens” took us about eight years to develop! It then generates an image of it. It segments the leaf into kind of its basic shape. Then it sends that segmentation over the net up to the server at Columbia where we’ve stored a giant library of other images of leaves. So it compares your leaf to all those images of leaves, gives you the best match and then sends back the top choice.

GELLERMAN: How big is your database?

KRESS: Right now we have about 200 species of trees, primarily from Central Park in New York City and Rock Creek Park here in Washington. That accounts for almost 10,000 different images that we have up on the server. We built a library for each species of any types of leaves that you’d find the forest because we all know nature varies and no two leaves are alike, so, we really wanted to represent each species by more than one leaf. And so, for Magnolia grandiflora, we may have 50 different leaf images in the library that your image than gets compared to.

GELLERMAN: So, our producer Stephanie went out and got some leaves and she picked them and they’re right here, and I’m going to take a picture with a borrowed iPhone. Saving the image…it’s uploading the image over our wireless network…so I’ve got results here - I’ve got a bunch of results here, actually. It’s got the picture that I took and it’s got almost like an x-ray, like a silhouette.

KRESS: That’s the segmentation of the leaf. It’s separated out its background and it’s separated out its shape there, so what you’re seeing is just the shape of that leaf - there’s no veins, there’s no color, there’s no nothing.

GELLERMAN: Well, there’s the results that says: London Plane Tree or Red Maple or Sugar Maple. Now there’s a lot of different trees there!

KRESS: Yeah, but look at the leaves, they’re all somewhat similar – they all have that serrated jagged edge with maybe five or six main lobes on it. And that’s the fascinating thing about nature. Nature has come up with the same sorts of shapes in many different ways in many different species, and our challenge was to separate those out there. And we focused on using the leaves because one, they’re two dimensional, as soon as we move into the three-dimensional realm of a flower or a fruit, the problem gets a lot harder, and two, there’s a lot of information in those leaves. If you look at, it's not only the basic shape, but it’s the lobes on the leaves, it’s the little serrations or the smooth edges - all of that helps us identify what species it.

GELLERMAN: I think I found it! It’s a Striped Maple. So I’ve got Acer Pennsylvanicum here.

KRESS: Yep. If you were standing outside next to that tree, and you came up with the identification, then you could compare the photo of the bark with the photo of the tree. And if there were any fruits hanging down you could compare the photo of the fruits, and then you can really confirm your identification that way.

GELLERMAN: So now you’ve got Central Park and Rock Creek Park in D.C. mapped out here - what’s next?

KRESS: Right as we speak we have colleagues collecting images and data on all the trees of the northeastern United States. Because we are going to expand this across the United States starting with the Northeast and then probably all the trees east of the Mississippi River and then head west. And we hope within a few years if the funding is available, we’re going to be able to have a system that will work anywhere in North America.

GELLERMAN: Now, I can see a potential snafu to LeafSnap, though, you know? You’re out there in the great outdoors and you’re hiking in the middle of nowhere and you don’t have 3G or wireless access.

KRESS: Yeah, that is a problem. But I think that’s probably going to change as time goes on. I just came back from eastern Africa, and even there, out near the Serengeti, I had access to some sort of communication.

GELLERMAN: So I don’t have an iPhone, I’ve got an Android phone. You got an app for that?

KRESS: We’re working on an Android version of LeafSnap right now. We hope, probably by the end of the summer, early fall, you’ll also be able to put this on your Android phones.

GELLERMAN: And it’s a freebee app, right?

KRESS: It’s free. We had long discussions about that too. This project was supported, in part, by the National Science Foundation who gave us the original grant to put it together. But we felt that this application that we really hope brings people closer to nature and closer to understanding nature, would be much better to be a free app than any sort of cost to it.

GELLERMAN: You could also broaden horizons, you could turn kind of a casual person into a citizen scientist, actually.

KRESS: That’s just what we hope, Bruce, that’s just what we’re aiming for. In fact, we’ve already, since the release of LeafSnap, we’ve already had a lot of inquiries from school teachers, from high school teachers, from even college professors that really see how LeafSnap can be adapted and become part of their curriculum, whether it’s five year olds or whether it’s twenty five year olds, and that’s what we’re shooting for.

GELLERMAN: Well, Dr. Kress, that was a lot of fun. Thank you, I really enjoyed it!

KRESS: My pleasure and I hope everybody else enjoys using Leafsnap.

GELLERMAN: John Kress is chief botanist at the Smithsonian Institution and helped develop the new LeapSnap app – for more details check out our website – loe.org.

 

Links

Learn more at LeafSnap