The original mission of the Mars Helicopter (named Ingenuity) was to successfully complete a single 30-second long flight on Mars. That happened back in April. After several more successful flights, Ingenuity’s 30-day mission came to an end, but the helicopter was doing so well that NASA decided to keep it flying. Several months later, JPL promised that Ingenuity would “complete flight operations no later than the end of August,” but as of late November, the little helicopter has completed 17 flights with no sign of slowing down.
NASA has kept the helicopter operational, in part, because it’s transitioned from a pure technology demonstration to an operations demonstration. In fact, Ingenuity has turned out to be quite useful to both the science team as well as the roboticists who operate the Perseverance rover. While NASA never planned to have Ingenuity make occasional scouting flights, its having that capability seems to have paid off. To understand just how much of a difference the helicopter is making to Perseverance’s mission, we talked to one of the Mars rover drivers at JPL, Olivier Toupet.
Toupet has been at JPL for nine years, and he’s the supervisor of JPL’s Robotic Aerial Mobility group (which includes key members of the Mars Helicopter team). He’s also the deputy lead of the rover planner team for Perseverance, meaning that he’s one of the folks who tells the rover where to go and how to get there. In his role as a Perseverance rover driver, Toupet specializes in strategic route planning, which means listening to where the scientists want the rover to go and thinking about how to best reach all of those targets while considering things like safety and longer term goals. “We design routes to visit the targets that scientists are interested in, or we tell them that it’s too dangerous,” Toupet tells us.
“Initially there was a lot of pushback, even from the science team, because they thought it was going to be a distraction. But in the end, we’re all very happy with the helicopter, including the science team.”
—Olivier Toupet, NASA JPL
Toupet was also one of the rover drivers on the Mars Exploration Rovers (MER) and Mars Science Laboratory (MSL) programs, and over the years, he and his team have developed a solid intuition about how to drive Mars rovers over different types of terrain—how to do it efficiently, but also minimizing the chances that the rover could get damaged or stuck. Obviously, the stakes are very high, so the rover team takes no chances, and sometimes having even a single picture from Ingenuity of a potential route can completely change things, says Toupet.
IEEE Spectrum: How much of a difference has it made for you to have Ingenuity scouting for Perseverance on Mars?
Olivier Toupet: My team designs the routes for the rover to drive, and typically we have orbital imagery [from Hi-RISE], which is as you can imagine very low resolution, and then we have imagery from the rover on the surface, but it can only see a few hundred meters. With the orbital imagery, we can’t see rocks that are smaller than typically about a meter. But a rock that is taller than 35 centimeters is an obstacle for the rover—it can’t put its wheel over a rock that size. So it’s been really helpful to have that helicopter imagery to refine our strategic route and plan to avoid challenging terrain well before the rover can see it."
Animated gif cycling between blurry orbital imagery, less blurry rover imagery, and high resolution helicopter images This animation shows the different kinds of imagery that the rover planners are able to use for route planning, including imagery from the rover’s own cameras, images taken from orbit, and helicopter images.NASA/JPL
What about planning for day-to-day rover operations?
We do look at the helicopter images when planning our daily drives, but we can’t fully trust the 3D mesh obtained from pairs of overlapping images because we don’t know the exact distance the helicopter flew in between each one. We use the images in a qualitative way, but we can’t tell where obstacles are with the precision that we’d need for drive planning—we can’t entrust the life of the rover to those images.
You and your team must be highly skilled at understanding Martian terrain from the relatively low-resolution orbital images, since JPL has been planning for rovers on Mars based entirely on orbital images for decades now. With that in mind, how actually useful is high-resolution imagery like the helicopter provides?
I was actually a rover planner on Opportunity, Curiosity, and now Perseverance, so I’ve been doing this for a long time! But it’s a fair question. You are correct that we’re very experienced with interpreting orbital imagery, but there are still some cases where higher resolution imagery can be very important. With Curiosity, there’s a place called Logan Pass, where of course we had relied on orbital imagery for our strategic route planning.
Panoramic image of the Mars surface showing a sandy depression with hills and mountains in the background View southeastward towards Logan Pass from Curiosity’s Mast Camera, taken in May of 2015.NASA/JPL-Caltech/MSSS
We thought there was a shortcut to get there that we could squeeze through. We drove all the way there to the start of a slope that we were going to have to drive on with a large field of sand dunes beneath it. We’d thought that the slope was likely to be compacted sand, which would have been fine, but what we couldn’t see on the orbital imagery was that the slope was actually a thin layer of sand on top of pebbles, and when the rover tried driving on it, it began to slip substantially down towards the sand trap. We tried to get across the slope a couple of times, but we ended up deciding that it wasn’t safe at all, so we had to take a pretty substantial detour because that strategic route wasn’t feasible.
Curiosity’s path shown as a white line which goes into a dead end and out again Orbital imagery of Curiosity’s route showing attempt to traverse Logan Pass, followed by detour through Marias Pass.NASA
So overall, it’s true that typically orbital imagery is good enough, especially on terrain that’s pretty benign. But there are times where having higher resolution imagery ahead of time is very valuable for route planning.
What about for Perseverance? Are there any examples of specific ways in which detailed imagery from Ingenuity caused you to change your mind about a route?
We landed right next to an area called Séítah, which is actually very hard to drive through because it’s full of large sand dunes. And getting stuck in sand is the nightmare of every rover planner, because it could be mission-ending. Right after landing, the scientists were saying, “let’s cross over Séítah and get to the delta!” I said, that’s not going to happen, we have to drive around it.
Orbital image showing Perseverance’s route as a white line traveling around an area of hills and sand dunes View of Perseverance’s route around Séítah and current position of the rover and helicopter on the south side of Séítah.NASA
While we were driving around, the helicopter just flew right over to the west side of Séítah on Flight 9. That was really interesting, because it gave us excellent images and we realized that while there were some places we wouldn’t want to drive in, there were other places that actually looked traversable.
Image of the sandy, rocky surface of Mars, with the shadow of the Mars helicopter in flight at the bottom Image taken by Ingenuity showing bedrock poking through sand, suggesting that some areas might be traversable by the Perseverance rover.NASA/JPL-Caltech
And so it was really helpful to have that helicopter imagery over Séítah to refine our strategic route. Thanks to the helicopter, we ended up modifying our route—we were initially going to drive over a kind of hill, but the helicopter flew right above that hill, and I was able to see that the hill looked much more challenging than I thought from the orbital imagery. In the end, we decided to drive around it.
Aerial image of a hill on Mars showing a red line labeled "Initial Route" going over the hill and a green line labeled "Refined Route" going around the hill Image taken by Ingenuity of the hill Perseverance had planned to climb, which helped the rover planning team decide to drive around the hill instead.NASA/JPL-Caltech
If we hadn’t had the helicopter imagery, I think we would still have made it work and found the same route. But having the helicopter, we were able to plan the route ahead of time, and make a much better estimate of how long it would take, which helps the whole Perseverance rover team to plan more efficiently. That’s pretty valuable.
What has the reaction been to having the Mars Helicopter stick around as a scout?
The whole team, we all love it! We didn’t know we were going to love it—it’s really interesting, I think initially there was a lot of pushback, even from the science team, because they thought it was going to be a distraction. But in the end, we’re all very happy with the helicopter, including the science team. The more information we have the better—for the science team, for example, the helicopter can save us a lot of time by quickly investigating potentially interesting areas.
“We’ve found a way to do both rover and helicopter activities in parallel, in a way that’s very low impact and very high value.”
—Olivier Toupet, NASA JPL
For example, when we flew the helicopter over Séítah, over the area where the scientists wanted the rover to go, the pictures that the helicopter took enabled the scientists to decide whether it was even worth trying to drive the rover that far—it would have taken us two or three weeks to even get there. But the images from the helicopter led the scientists to say, “hey, yeah, this area is actually really interesting, and we see valuable rocks that we’d like to go and sample. And so it enabled us to make that decision early on rather than potentially wasting two to three weeks driving over there for nothing.
At one point, JPL said that even if everything with the helicopter was working great, flight operations would cease “no later than the end of August.” Obviously, the helicopter is still flying—how much of a surprise has that been?
Frankly, it’s been a big surprise, but we should have known better! Opportunity was supposed to be a 90-day mission, and it was still going 14 years later. Some of us suspected that the helicopter mission would continue to be extended, but the helicopter team played their cards pretty close to their chest. Obviously, they were very focused on accomplishing the tech demo, and that was always a top priority. So whenever we’d ask, “what happens next,” they’d tell us not to get distracted because a successful tech demo was why the helicopter was funded to go to Mars.
But I remember being in a meeting with someone from NASA HQ, who said something that is very true, which was that the tech demo is great, but the long-term goal is to show the potential of flying on Mars. I really hope that Ingenuity being such a success means that in the future there will be another helicopter mission to Mars. You can imagine a helicopter flying into Vallis Marineris, the largest canyon in the solar system. It would be amazing.
The official story was always that there were going to be five flights and that was probably going to be the end, and so I’m glad that we’re now at flight 17, and the helicopter has been extremely successful. I can’t wait to see all the things we’ll be able to do in the months to come, including when we reach the delta—there are many steep slopes, and lots of dunes, so having the helicopter there is going to be especially valuable.
Black and white image taken from orbit giving a 3D effect of an ancient river delta Oblique view of the Jezero crater delta looking west.NASA/MSSS/USGS
Do you think that part of the reason that the mission keeps getting extended is because NASA is realizing just how valuable having a helicopter scout can be for a rover like Perseverance?
Yes, I think maybe we didn’t initially realize just how useful the helicopter would be in supporting our scientific mission. I would also say that another reason the helicopter mission keeps getting extended is because it’s turned out to have a fairly minimal impact on the rover team, in the sense that the helicopter team has been pretty independent and they are only flying once every two to three weeks. We’ve found a way to do both rover and helicopter activities in parallel, in a way that’s very low impact and very high value.
It sounds like having a helicopter scout would make an especially big difference once Perseverance reaches the delta. Are you hoping Ingenuity will survive that long, and that it’ll be able to scout for the rover indefinitely?
I definitely hope so! Initially, there were some concerns about whether the helicopter’s electronics will be able to survive the winter [through March of next year]. There are still some questions about this, but things are looking promising. There are also communications challenges—so far, the helicopter has been staying pretty close to the rover, within about 300 meters. But once we’re done with this area to the southwest of Séítah, the rover will be driving very quickly back around Séítah to the foot of the delta. Specifically, we’ll be using multi-sol autonav, which is where we tell the rover to keep on driving itself autonomously as quickly as it can to its destination over multiple Martian days. Put the pedal to the metal! And so there is a little bit of concern whether the helicopter can keep up with us. It’s funny, I love the helicopter, but I also work on the autonav software, so I hope the rover goes fast.
Orbital image of Jezero crater with a dotted yellow line taking a winding kilometers-long route around craters to a location called Three Forks. Perseverance’s planned route from Séítah to Jezero’s river deltaNASA
But I think it’s going to be fine. The helicopter team is working on improved capabilities, including the capability to pop up in the sky and talk to the rover, and that could substantially improve the communications range, perhaps even to kilometers. So while they’re going to do their best to try and keep up with the rover, in parallel they’re working on improving the capability of the helicopter to stay in communications even from farther away. So I’m very hopeful that Ingenuity will be around for a long time!
As someone who’s been working on several generations of Mars rovers, what would you like to see from the next-generation Mars helicopter?
The big advantage of a helicopter is of course that it can fly, and the Mars Science Helicopter will be able to fly tens of kilometers in a single day. To give you a sense of perspective, we’re hoping that Perseverance will be able to drive a few hundred meters in a day. So the helicopter would have several orders of magnitude more range, which is amazing—you could imagine going not just to one site on Mars, but to multiple sites.
A rendering showing on right the Ingenuity Mars Helicopter, 0.5m across, next to the Mars Science Helicopter concept, which has six rotors and is six times the size. Mars Science Helicopter concept compared to Ingenuity.
But the big disadvantage of the helicopter, unfortunately, is the payload. A rover can carry a lot of science instruments, while the helicopter, because the air density is so low on Mars, has a much lower maximum payload, which restricts how much science you can do. That being said, you could imagine being able to swap instruments—what if you could carry just the instrument that was necessary for the specific site you’re visiting that day? Of course there are technical challenges with that, but yeah, personally I do think that the next mission should be a helicopter just by itself. It would be great to see that in the future.
And when we send another rover to Mars, should it have its own helicopter scout?
That’s a great question, and a controversial one, because the next mission to Mars is about sample return, and the European Space Agency is making the rover, not NASA. And so, I don’t know who gets to make such decisions, but I personally do think that a helicopter would be extremely valuable—not just as a scout, but potentially also as a backup, that could retrieve the samples if the rover had some issues. That would be great to have for sure.