A.J.S. Rayl • Apr 30, 2009
Mars Exploration Rovers Update: Spirit Endures Reboots and Amnesia, Opportunity Races Forward
The Mars Exploration Rovers challenged their ground crews with an April full of high drama, a little suspense, and a lot of energy. While Spirit lived through a kind of robot soap opera, complete with bewildering reboots and bouts of amnesia, Opportunity roved forward and back into the fast lane on a restored front wheel, slowing down for a brief visit to a series of small, intriguing craters and an unplanned close encounter with a pesky little purgatoid.
At Gusev Crater, Spirit began the month finishing up a close-up examination of another light-toned patch of soil that turned out to be rich with opaline silica and iron sulfate. It then carried on with its trek along the western side of Home Plate, heading for von Braun and Goddard, its next big attractions to the south. Despite slipping repeatedly in the fluffy, powdery Martian soil, Spirit was persevering as usual. Then mysteriously, on the morning of April 9, it overslept, missed three wake-up calls and three planned events, and lost a whole sol in the process. Its alarm clock finally woke it up 27 hours later and for unknown reasons the rover decided to reset itself.
Spirit, in effect, suffered a “fault,” according to Bill Nelson, engineering-team chief for the MER project, perhaps in the high gain antenna subsystem (HGA) toward the end of the communications session where ground control uplinks commands to the rover. It might have been the software that keeps track of the antenna direction may have somehow lost track and marked its positions 'unknown.’ In a situation like that, the rover is instructed to not move its antenna and declare a fault, he explained. But that’s just a guess. They don’t really know what happened.
The very next sol, Spirit unexpectedly suffered a “fatal” event on its system – much like a hard disc crash on a personal computer, Nelson explained. That caused the rover to reboot. What made these episodes bizarre was that the rover also suffered a couple of bouts of robot amnesia around the same time. In rover terms, that means Spirit failed to record any data in its flash memory, which is where the MERs store and preserve engineering and science data for later downlinking to Earth; therefore, after these events, its memory was blank. The absence of any data left the engineers at the Jet Propulsion Laboratory (JPL) – where the rovers were designed, built, and are being managed – mystified.
Since Spirit put itself into auto mode and later communicated with crewmembers, informing them that it was stable, the events were not disasters. In fact, it appears that the rover woke up after its “fatal” crash event in the “crippled mode,” Nelson said. In that mode, the rover automatically assumes its flash memory isn’t working, so it stores its data onto its random access memory (RAM). The only problem was that Spirit went back to sleep and the data was lost, because RAM is a volatile form of memory, which means it isn’t saved – as it would be on flash – when the rovers shut down.
During the week that followed, Spirit seemed fine and the team engineers allowed it to continue in auto mode, essentially waking up, communicating, and going right back to sleep, while they tried to figure out what happened.
Then, on April 17, Spirit suffered another bout of amnesia and the sol after that, it rebooted again, reliving a similar scenario to that of the previous weekend. Three bouts of unexplained amnesia, along with the reset and reboots – it was downright odd.
The MER team convened its Council of Experts, as Nelson put it, some of the original crew members who knew these rovers when they were just bits and pieces and parts. But they too were “baffled,” said Steve Squyres, the principal investigator for rover science, of Cornell University.
New R9.3 software had been uploaded onto Spirit in March, but that hardly seemed suspect since it was really just a patch to prevent a possible Y2K type of problem with dates. Moreover, Opportunity has been functioning with the same software even longer without any kind of problem. “The interesting thing is that it's an intermittent problem, the hardest kind of problem to troubleshoot, because they don't happen all the time,” Squyres pointed out.
Worse, the engineers had absolutely no data – zilch – nothing to look at, making the task of troubleshooting almost impossible. They don’t even know if the reboots had anything to do with the bouts of amnesia. “It's like the 18-minute gap in the [Nixon] White House tapes,” said Squyres. [Note to whippersnappers: See Frost-Nixon for more information.]
With Jake Matijevic, chief rover engineer and a member of the MER Council of Experts, at the helm, the JPL engineers dug in and began doing everything they could to figure it out. Plenty of speculations have been laid on the table – everything from a problem with the flash memory hardware to a corrupted file to issues with defragged files to some kind of problem stemming from the intimate relationship between hardware and software. “We just need grist for the mill,” said Nelson.
“People are concerned, but during all this time Spirit kept positive power, was never really out of control in the sense that it always got itself into a well-defined state, was willing to communicate with us, and it kept itself healthy," reminded Bruce Banerdt, MER project scientist, of JPL. "That’s a good sign.”
The council advised the team to change the rover’s nap time, so that if it displays this behavior again and saves its data to the random access memory (RAM) system, then at least it can send data to Odyssey for downlinking to Earth before it goes to sleep. That would give them something to go on. They also decided Spirit should keep on keepin' on, something the MERs have always done well.
One week ago, on April 23, Spirit powered up and roved onward, for the first time since April 8. It didn't get terribly far, because of the slippery terrain, but it did make some progress and it did everything it was commanded to do.
Spirit drove again this past Monday and Wednesday, making only a few centimeters worth of progress each sol, but at least it was moving and following commands. “We're keeping our fingers crossed because the next two weeks will be critical in terms of getting through the southern part of West Valley and into South Valley,” said Ray Arvidson, deputy principal investigator for rover science, of Washington University St. Louis.
The reboot-amnesia drama remains a mystery for now. Some mission followers wonder if Spirit isn’t just displaying a little jealousy, like it did right before Opportunity was to land way back in January 2004. And, to be fair, once again, this rover is left to deal with the harder road – and a completely broken front driving wheel – while its twin, Opportunity, is roving right along once again, without a seeming care in the solar system. “Maybe she has been throwing a tantrum,” considered Squyres.
Despite the frustrations, “the team is doing great actually,” said Squyres. “This has been an interesting puzzle. Certainly we don't like problems, but at the same time when problems and anomalies occur, it makes us use parts of our brain that we don't use otherwise and this is a group of people that kind of enjoys that in a way.”
“Our people here are always of the attitude 'we'll find a way,'"Nelson agreed. "If we can’t go over it, we'll go around it. We'll do anything except give up.” Not surprising, really. It’s the team, after all, that has imbued the rovers with their can-do, never-say-never principles, something that is sometimes forgotten given the rovers' vibrant personalities.
It’s now been 12 going on 13 sols since Spirit has exhibited any strange behavior. The team, however, does expect the drama will continue. Actually, Nelson said they’re hoping it will. “We’re very anxious to try and catch this amnesia event and reboot episode in our event records and log file,” he explained. “We're hopeful for a clue as to how and why it’s doing this.”
Over on the other side of the planet, at Meridiani Planum, Opportunity after several weeks of a stationary in situ science campaign on targets in an area of exposed bedrock called Cook Islands, was finally given the greenlight to rove on – and the rambunctious rover did just that. The trouble it had with its right front wheel overheating, an issue that began in February, had forced it to slow down. Its handlers had ordered it to take it easy, drive little and almost entirely backwards and take rests. That prescription served the purpose of allowing the lubricant inside the wheel assembly to redistribute.
Cook Islands had been a perfect place to take a good long rest, because it also allowed Opportunity the chance to study some bedrock and soil for the systematic science campaign it’s conducting en route to Endeavour Crater. Once it finished its close-up examination of Penrhyn, a target on Cook Islands, the ground crew let the rover go.
Opportunity took off like Danica Patrick, zipping along to make up time, as well as some serious tracks on its long journey to Endeavour still a long, long ways away. “We wrapped up the Penrhyn investigation and are proceeding at a high rate of speed once again to the south,” said Squyres.
Actually, April was all about driving with a little bit of crater hopping. Based on orbital images, the team has chosen a route that takes Opportunity by a series of small, shallow craters and around a group of gnarly puragatoids, large dangerous sand ripples in which the rover could get stuck. These large ripples were named for Purgatory Dune, which snared the rover for some 5 weeks back in 2005.
Opportunity pulled up to the series of small, young craters, which the team named after Captain James Cook's ships, other than the Endeavour and cruised along, taking breaks for drive-by shootings – of pictures that is, with its panoramic camera (Pancam) of Discovery, Resolution, Adventure, Rayleigh, and Marquis. “We chose the path that would take us by as many of them as we could, because we wanted to get a good sampling of this crater field,” said Banerdt.
“This is one of the first times we've wandered into a cluster of craters that look like they might be genetically related to one another,” said Squyres, and that made them particularly inviting. Opportunity photographed about a half-dozen craters, though there were many more in the immediate area.
Some of the craters were right on top of the ripples. "In this area, it's pretty hard not to hit a ripple," noted Rob Sullivan, of Cornell University, MER science team member.
Interestingly, they are probably the youngest craters observed by either rover to date and “likely formed in the past 100,000 years,” according to Matt Golombek, a senior research scientist at JPL, rover science team member, and Mars Exploration Program landing scientist. Even more interesting, they are peppered with very small pebbles that are either meteoritic or, if the craters are secondary craters, formed by ejecta from the impact of another larger crater, pieces of Mars from elsewhere.
So, in a “new tweak” to the science agenda, as Banerdt put it, the rover will now be on the lookout for some of these pebbles that are large enough to examine with the instruments on its instrument deployment device (IDD) or robotic arm.
Once it finished photographing the craters, Opportunity hit the road again. It was obvious this rover was back in the fast lane and happy about it, a little too happy maybe, because it soon roved right into a closer than wanted encounter with a purgatoid. But this one was small, said Banerdt. “It was more like a – purgatina.” In any case, it wasn’t about to stop this rover, no way, no how. “We know how to get out of these things now,” said Squyres, “so we just popped right out,”
The MER mission crossed an anticipatory sort of milestone in April -- the "minus 1st anniversary” of Spirit breaking the duration record for longest-operational spacecraft on Mars, as Jim Bell, principal investigator for the Pancam and Planetary Society President, put it. Less than one year from now, on April 26, 2010, Spirit will pass Viking Lander 1's duration record of 2244 sols on Mars. Opportunity will break that record three weeks later. “Can't wait,” said Bell.
Meanwhile, the team has kept its collective eye on the weather. On April 21, Mars moved into the closest point it will come to the Sun in its 23-month, elliptical orbit and in about three weeks the planet’s equinox will mark the start of summer in Mars’ southern hemisphere. This atmospheric-warming combination makes this the most likely time of the Martian year for dust storms severe enough to minimize or thwart the activities of the rovers. “Since the rovers are solar powered, the dust in the atmosphere is obviously extremely important to us,” said Nelson.
While Spirit and Opportunity continue to point their cameras toward the Sun to check the clarity of the atmosphere almost every day, cameras onboard the Mars Reconnaissance Orbiter (MRO) and Mars Odyssey also make regular observations of the Martian atmosphere that have dramatically supplemented the rover’s own skywatch. The Mars Climate Sounder (MCS), the Mars Color Imager (MARCI) camera on MRO, and the Thermal Emission Imaging System (THEMIS) on Odyssey monitor changes in airborne dust or dust-related temperatures in Mars’ upper atmosphere.
MARCI actually observes the entire planet every day at resolutions comparable to weather satellites around Earth. “We can identify where dust is rising into the atmosphere and where it is moving from day to day,” said Michael Malin, of Malin Space Science Systems, principal investigator for camera. “Our historical baseline of observing Martian weather, including data from the Mars Global Surveyor mission, helps us know what to expect. Weather on Mars is more repetitive from year to year than weather on Earth. Global dust events do not occur every Mars year, but if they do occur, they are at this time of year.”
While widespread haze had reduced Spirit’s daily energy supply by about 20 percent and Opportunity’s by about 30 percent in March, the storm that had been brewing to the far south did not whip up into a major event. Weather reports from Bruce Cantor of Malin’s MARCI team confirmed that the March increase in haziness was not the front edge of a bad storm. “Bruce’s weather reports have let us be more aggressive about using the rovers,” noted Mark Lemmon, a rover-team atmospheric scientist at Texas A&M University. “There have been fewer false alarms. Earlier in the mission, we backed off a lot on operations whenever we saw a small increase in dust. Now, we have enough information to know whether there’s really a significant dust storm headed our way.”
Having these global perspectives on Mars' weather and advance notice of dust events when they do whip up enables the rover team to take steps, if necessary, to conserve energy on their robot field geologists. They can reduce the length of time their charges will be active, for example, or can shorten or delete some communication events as they did back during the global storm of 2007 that threatened Opportunity and the November 2008 regional storm that dumped on Spirit.
This time, the dust event “kind of flared up, then went away,” said Banerdt. The skies over both Gusev Crater and Meridiani Planum cleared pretty quickly and the taus – or dust levels – at both sites came back down to more or less previous levels. “Right now, we're hanging in at a tau just below 1, which is pretty normal,” he added. But the season isn’t over, so the team is keeping its collective eye on the sky and relying on a little help from their friends on MRO and Odyssey. “We're watching all the time,” Banerdt confirmed. “Right now we're in good shape, but there’s still a strong possibility for storms."
Winds that can lift dust into the air also blow dust off the rovers’ solar panels as this mission has witnessed repeatedly. In fact back in the beginning of the mission no one expected the rovers to last much past five months, because the assumption had been the solar arrays would become covered in dust and thus put the rovers' lights out. The beneficial winds changed everything and are a prime reason Spirit and Opportunity have been able to rove long and prosper. This month both rovers got lucky.
Amidst the darkness, there was light, sunlight for Spirit, and a lot of it – and that was the best news from Gusev Crater this month. The dust-laden rover was finally gifted with two solidwinds gusts that blew enough dust off its arrays to boost its power level from 240 watt-hours to a whopping 370 watt-hours, energy this rover hasn’t had in more than an Earth year. An estimated 42% of the sunlight hitting the arrays is now getting through and being converted to power. “Every little bit helps for Spirit, so that was very good news,” said Squyres. Indeed, the news made some MER followers positively giddy.
Opportunity, meanwhile, experienced a cleaning event in early April, jumping to a high of 515 watt-hours of power. It now ends the month at around 500 watt-hours, while still converting an estimated 62+% of the sunlight on its solar panels to power.
Both Spirit and Opportunity passed their monthly physicals with flying colors. “Spirit kind of got weird on us for a while there and I think it would be optimistic to think it's not going to happen again, so we'll just prepare for it,” summed up Squyres. “Opportunity's in excellent health and doing well.”
So, as the month comes to a close, the rovers are wrapping this April – their sixth on Mars -- with happy Hollywood-style endings, raring to rove on into a Martian summer sequel. The agenda now, Squyres said, is simple: “Press hard to the south with both vehicles.”
Spirit From Gusev Crater
The objective for Spirit in April remained the same as it was in March – make tracks around the western edge and to the south of Home Plate, where its next big attractions await. Von Braun, an intriguing mound, and Goddard, a pit in the ground that team members suspect is a volcanic vent – named for the two leading rocket pioneers – have been on Spirit's science wish list now for more than two Earth years and at long last the rover is closing in.
Spirit began the month of April in the midst of a science campaign on Kit Carson, light-toned material unearthed during the final week of March. It turns out that Kit is rich with iron sulfate and opaline silica, according to Arvidson.
Given than a Mars sol is 39 minutes longer than an Earth day and the fact that the MER team uses a “hybrid solar time” for most of the planning, as Nelson explained, there was no sol that corresponded to April 2, 2009 on Earth. That Earth day was essentially lost to the time drift between the local times on the two planets, because they rotate at different rates. So, on Sol 1866 (April 3, 2009), the rover resumed driving and spent a good part of the first week of the month on the road.
While Spirit only progressed about 3 meters (10 feet) on that sol because of excessive slip, it managed to avoid troublesome terrain during the next drive sol, Sol 1868 (April 5, 2009), and put another 17.5 meters (57 feet) behind it. The rover wrapped up the first week and moved into the second week of April with a pair of drives on Sols 1870 and 1871 (April 7 and 8, 2009) during which it got another 20 meters (66 feet) closer to von Braun and Goddard.
During the second week of April, Spirit, beginning Sol 1872 (April 9, 2009), experienced a setback with a series of anomalous events. It all started, or so it seems, when the rover seriously "overslept," even though its alarm clock was sounding. “The alarm clock is a hardware countdown timer, loaded with the number of ‘ticks’ to wait until the next event – a ‘tick’ is 2 seconds,” explained Nelson. “When it counts down to zero, it triggers an interrupt to the battery control board to wake up the computer. The computer then writes new wakeup times. If, as we believed happened during the Sol 1872 anomaly, the hardware doesn't respond, the count-down timer rolls over from zero to its largest value (equivalent to 27 hours) and proceeds to count down again.” Finally, 27 hours later, the expiring alarm finally roused the little rover. But Spirit had missed three planned events and it promptly went back to sleep.
On Sol 1874 (April 11, 2009), Spirit woke up only to experience a fault of some kind, reset its computer without any command to do so, then suffered a bout of robot amnesia. That means, it did not record any data in flash memory, a type of computer memory where information is preserved even when power is off, such as when Spirit naps to conserve power. Therefore, the team really had no data, no real clues as to what exactly happened, other than what they know about the rover systems.
“A fault usually affects some subsystem,” Nelson said. “For example, we believe the fault on Sol 1874 was in the high gain antenna (HGA) subsystem, something that would mark that subsystem as effectively out of service." The HGA is what the team uses to uplink commands to the rover. "We don't necessarily think there is anything wrong with the HGA," he said. "The most typical reason we get an HGA fault is because the actuator positions are 'unknown.' The software that keeps track of where the antenna is pointed somehow loses track, then marks the actuator positions 'unknown.’ In that situation, the rover is instructed to not move the antenna at all and declare a fault.”
The fault doesn't appear to have affected the HGA pass. "We were uplinking,” Nelson continued. “But one of the effects of this fault is that it kills all the sequences. That prevented us from stowing the actuator and doing a regular shutdown. Because of that, we did not ‘right out’ the Pancam mast assembly (PMA) actuator positions, so those were not saved and the result was they too became 'unknown.' If we attempt to use the PMA after that point, we'll get a PMA fault.”
Even before the ground crew could get a hypothetical handle on that reset event, Spirit on the very next sol suffered a “fatal” event, something like the hard disc on your computer crashing, though the rovers don’t actually have hard disks. “On Sol 1875 (April 12, 2009), we had a 'fatal' during the HGA pass," said Nelson. "The symptoms of that indicated a corrupted file, which we have not been able to verify. We've looked at the file and it looks fine. So we're not sure what's going on there either. But the affect of the 'fatal' was that it took everything down, rebooted the computer, and because it happened during the HGA pass, the HGA actuator positions were not saved either." That left them with both PMA and HGA actuators unknown. So, between the Sol 1875 pass and the 1876 pass, there was no uplink and nothing the team could do about.
On Sol 1876 (April 13, 2009), Spirit apparently woke up to get its dsily commands and promptly declared an HGA fault because the actuators were 'unknown,' then it had another bout of robot amnesia. "It woke up in this strange mode, which we think was crippled mode, so not only do we get the HGA fault on Sol 1876, the rover didn't save any data products to flash memory," said Nelson.
That's because in crippled mode, the rover is programmed to presume that its flash memory is not working; consequently, it puts its data into the random access memory (RAM). Spirit has 128 megabytes of available RAM that can store data, as long as the rover is awake. "We believe the 1876 session took place just as it should and all the data products were created and they were saved in RAM and everything went pretty much as you would expect," Nelson said. "The only problem was it was that the rover then went to sleep, so everything disappeared, because the RAM is volatile memory."
Since Spirit has been so power challenged for the last several years, the engineers instituted a nap in its daily schedule to conserve energy, which is usually taken between its main activities of the sol and the main downlink transmission of data, usually through Odyssey, to Earth. So it was on its regular schedule, but when it woke up for the Odyssey pass, it had no data to downlink. So the team had no idea what had happened. "None of our log files were saved and we've got no record," said Nelson.
The only good news really was that Spirit continued communicating with its crew throughout that weekend and into the new week, although some of the communication sessions were irregular. The rover team, fortunately, has the advantage of multiple communication options. So, even with the HGA out of commission, the rovers can communicate at a slower data rate through a low-gain antenna that does not move. Additionally, communications can be relayed by Mars orbiters via the UHF (ultra-high frequency) transceiver, a separate radio system on the rover. To avoid potential problems using the HGA, they decided to communicate by UHF relay via Odyssey or by using the low-gain antenna.
When Spirit woke up on Sol 1877 (April 14, 2009), it put itself into auto mode and was taking care of itself, just fine. “If it doesn't have any instructions from us, it's got its own little sequence of things it does in auto mode, which is, basically, wake up and execute communications, then go back to sleep," explained Nelson.
While they had no explanation for what had happened yet, they did know that Spirit was power positive, its batteries were fully charged, its solar arrays were producing energy and all temperatures were well within allowable ranges, according to John Callas, of JPL, MER project manager.
Spirit stayed in auto mode through Sol 1879 (April 16, 2009) as the team began systematically resetting subsystems to bring the rover back to normal operations, while continuing to investigate the strange, anomalous behavior.
Both rovers' onboard software has been updated several times to add new capabilities for the mission, and Spirit got the new R9.3 software last month. The team is looking into whether the unexpected behavior in recent days could be related to the new software, but it's not likely. "There's no evidence to suggest it has anything to do with the new flight software," said Squyres. "The same software is on both vehicles and we haven't seen any evidence of any of this on Opportunity. But one can imagine ways in which there are complex hardware-software interactions that are unique to the Spirit vehicle.” Or, it could just be "age-related effects," as Callas has suggested.
Just as Spirit seemed to be returning to normal, however, the rover woke up and suffered another bout of amnesia on Sol 1880 (April 17, 2009), then once again rebooted its computer on the following sol, behavior that was now getting pretty eerie. Once again, Spirit put itself back into auto mode, just chilling and keeping itself healthy. They could find no causal link between the amnesia events and the reboots. "This is a true anomaly," said Nelson.
The team revised plans in order to regain Earth control of the rover's operations and resumed what diagnostic and recovery activities they could. "We are proceeding cautiously, but we are encouraged by knowing that Spirit is stable in terms of power and thermal conditions and has been responding to all communication sessions for more than a week now," Sharon Laubach, chief of the rover sequencing team, which develops and checks each day's set of commands, announced early last week.
Past those undesirable incidents, Spirit, being the good-girl rover it's always been, was "well-behaved," according to the mission mangers. Even Mars took sympathy and offered up a small dust cleaning on Sol 1881 (April 18, 2009), something that boosted not only the rover's power by more than 10%, but every team member's spirits by an incalculable percentage. Still, three amnesia events in 10 days, plus one on January 25 of this year were not events that brought smiles to the faces of the MER team.
The JPL engineers considered all possibilities. During diagnostic activities, Spirit successfully moved its HGA and its Pancam mast, indicating there weren't any mechanical issues with those components that could be related to the problems. They further worked to determine if some capabilities were ultimately shown to be diminished what the best way to keep using the rover would be. "For example, if we do determine that we can no longer use the flash memory reliably, we could design operations around using the random-access memory," Laubach said.
“Like everything else with these vehicles, as they age, the best thing to do is to come up with the best way to work around it,” said Squyres.
But the incidents were so bizarre and so confounding, they decided to convene their "Council of Experts," as Nelson put it, "the people who were involved in actually building and programming the rovers." That council includes Matijevic, former chief of MER's spacecraft and rover engineering team, Jim Donaldson, the lead engineer for the Avionics Subsystems, whose team designed and built the electronic boxes that contained the rovers' computers, as well as interfaces for communicating with other systems; Glenn Reeves, chief flight software engineer and software architect; Tracy Neilson, fault-protection software writer; John Waters, electronics engineer; Todd Litwin, software developer; Dave Hecox, software developer, and Rob Manning, development manager for the rover's descent through Mars' atmosphere and landing on the surface.
"These are the big stars from the development days of the vehicles," said Squyres "and they were as baffled as we were. So we have an interesting puzzle on our hands. But given the nature of this team and the smart engineers we have, it's actually been something that's been quite a stimulating thing to try to deal with and we're working it."
The council made a number of suggestions, however, that are being instituted. “One is to have Spirit stay awake through the Odyssey pass, so if it's stores data in RAM, we keep the power on until after we've had the chance to at least downlink the highest priority stuff before we shut down," Nelson explained. And, because there is a need to get back on the road and keep trying to get to Goddard and von Braun, the council, after determining that it wouldn't hurt for the rover, also recommended that Spirit carry on with the understanding that these events may occur again.
"We decided not to wait until finishing the investigations before trying to drive again," Laubach said. "Given Spirit's limited power and the desire to make progress toward destinations to the south, there would be risks associated with not driving."
In addition, the team is considering reformatting the flash drive, which would be a real pain in that it means taking the memory off the rover or clearing the slate, reformatting, then putting everything back on, much like users of pcs have to do when their systems go wonky. And, they're thinking about putting in some software triggers that will alert them if the rover goes into certain modes and save that data in some non-volatile memory. “Since we can't guarantee it would save into flash, and of course we can't guarantee Sprit will be awake if it's saved into RAM, we're trying to work out a way where we could save it into EEPROM – electrically erasable programmable read only memory," said Nelson. "That's a little tricky, because that's an area that's normally an area only used for sequences, so it's not easy to have the sequence change the sequence memory, although it's possible. We're looking into it."
For now, the plan ahead is to treat Spirit as if it's healthy and to resume near-normal operations. "Other than these anomalies, the rover is healthy," underscored Nelson. “It's power-positive. It's thermally stable. It's commandable. So, even though we know the rover isn't fully 100% and we don't understand why it's doing these things, as long as it's functioning, we're just picking up and starting over again,” he said. And, they have changed the rover's siesta time to before the Odyssey pass instead of afterward, so now any data the rover saves to RAM will not be lost.
“Clearly we have some kind of poorly understood intermittent problem, but this team is a team of problem solvers and when something like this comes along, it keeps the job interesting,” said Squyres. "We're just going to work around it as best we can and keep pushing southward. Maybe we'll get lucky and get some information along the way that will help us better understand what is it on the vehicle that's making these things happen."
With the next Martian winter on the not-so-distant horizon, time is now really of the essence for Spirit to get to von Braun and Goddard and they scheduled a short drive for Sol 1886 (April 23, 2009). The rover responded to all commands last Thursday and, for the first time since Sol 1871 (April 8, 2009), Spirit roved on, albeit for a very short distance of about 1.7 meters (5.6 feet), because of the slippery, powdery soils.
Spirit drove again on Monday, its Sol 1890, and again only progressed a few centimeters. Actually, the drive was automatically stopped because of excessive slipping in the fluffy terrain. “It took one step forward and one back,” said Nelson. “Because we drag this broken right front wheel, the rover tends to yaw counter-clockwise, so we move forward and then we try to correct the yaw by doing a turn that brings us back to the direction we want to go, then move forward again, then turn to correct and so forth.” As a result, it’s been slow going in this area."
“It's been easy until now, but the drive over the weekend was slightly up hill and in this sandy material," said Arvidson. “We exposed some more light-toned material, but didn't have time to stop and smell the roses. We’re really putting all the watt-hours into driving and post driving imaging.".
“We're getting into some challenging terrain,” added Squyres. “We'll deal with it as we always do, one step at a time. No one expected this drive was going to be easy the whole way and there are some tricky patches along the way. We seem to have gotten into one and we'll see how it goes."
Meanswhile, Mars continued to be supportive and sent winds to clear more dust from Spirit's filthy solar arrays just a couple of sols ago, Banerdt informed, allowing about 42% of the sunlight hitting the solar panels to get through and be converted to energy. That boosted the rover's power levels to 371 watt-hours, giving Spirit energy it hasn't had in more than an Earth year.
During the last day or so, the team has decided to have Spirit to change tact in the West Valley. “We're looking at perhaps backing out of the loose soil that we seem to be kind of bogged down in and moving around a little bit to the right and downhill a bit,” said Banerdt. “There's sort of a high road and low road, as we're referring to it. The high road goes uphill a little bit and opens up into a wider sort of range in which we can drive south, but we've been unsuccessful of getting up that lip. Alternatively, we can go just a little to the west and down the low road, which is a fairly narrow pathway that is kind of rocky and has some dips and depressions filled with loose soil, little areas with which Spirit has had trouble before. There's not much room to maneuver in there, so if we run into problems going down this corridor, it could be dicey. Both ways are a little hairy. But we're going to take a shot at going on the low road now.”
This weekend, as May blooms at Gusev Crater, Spirit will continue to moving toward the low road to head south through the more narrow pathway. Von Braun and Goddard are about 150 meters (about 500 feet) away, but are now in clear site. The rover is about two-thirds of the way there and it's only a matter of negotiating the difficult terrain.
The agenda for Spirit in coming sols is a lot like that of its twin: “Drive, drive, drive,” as Arvidson put it. “Our intention is to continue to the south,” confirmed Squyres. “Right now I would say there are no indications that suggest we cannot continue to the south. We don't view the path to be blocked in any substantive way right now. We'll just press ahead. If we learn more and the situation changes, we'll adapt to it. We always do."
Meanwhile, the engineers are ready for more of same reboot and amnesia anomalies. "As strange as it may seem, I am actually hoping that it does, because I want us to have a chance to identify what's happening so we can fix it," said Nelson. "We need data and we're going to run this rover as if it's problem free until trouble shows up again.”
opportunity from meridiani planum
As April dawned at Meridiani Planum, Opportunity was in the midst of finishing up its close-up examination on the stretch of exposed rock outcrop called Cook Islands, homing in on Penrhyn, a target named for the most remote and largest atoll of the 15 Cook Islands on Earth.
It's part of a systematic study of bedrock and soils that the rover is conducting on its long journey to Endeavor, the 22-kilometer (14-mile) diameter that the team hopes to get to sometime in 2010. The rover continued that work through the first week of the month.
On Sol 1845 (April 2, 2009), Opportunity positioned its robotic arm or instrument deployment device (IDD) and gently placed its Mössbauer spectrometer on the outcrop target Penrhyn for a multi-sol integration, to determine its iron content.
The stop for science came an opportune time, since Opportunity was forced to take a break to rest its right front wheel. About two months ago, that wheel was getting a little too hot from all the driving the rover had been doing. The actuator or motor was drawing more than normal current, a symptom the team had seen before fortunately. Basically, it indicated that the lubrication inside the wheel assembly was unevenly distributed, something that happens after a period of intense driving and they knew just what to prescribe: driving backward, a mitigation technique to redistribute the lubricant, and periods of rest.
By the end of the first week of the month, Sol 1850 (April 7, 2009), the team provisionally lifted the temporary stand-down on driving. Opportunity retracted its IDD and moved into the driving-stow position, then roved backward about 62.5 meters (205 feet). The drive went well, but the good news was that the right-front actuator exhibited currents near normal levels. More good news came in the form of another cleaning event, this one boosting the rover’s power levels by 40%, to 515 watt-hours. The dust factor improved to 0.642, meaning that 64.2 percent of sunlight hitting the solar array was penetrating the layer of accumulated dust and being converted to energy.
“We never really managed to get Opportunity very dirty," said Squyres. "And so the cleaning event is certainly welcome, but not really a surprise given its history."
Back on the road to Endeavour, Opportunity followed the course charted by its planners, veering to the west, away from a sea of purgatoids and by a series of small, shallow craters that were too inviting to just fly by. These small craters -- just a few meters or yards in diameter – are, the scientists believe, from fairly recent impacts, occurring in the last 100,000 years, according to Matt Golombek, MER science team member and JPL senior research scientist.
The team named them Discovery, Resolution, Adventure, Rayleigh, and Marquis after Capt. James Cook’s ships (other than the Endeavour). “Because of their proximity to one another, there’s a good chance that these craters formed at the same time," noted Squyres. "That means they're all the same age and by looking at how craters of different sizes have aged, and if you know they've aged at the same time, you can actually learn something about the crater degradation process.”
They could be primary craters created directly by impacts or secondary craters, formed by fallout emanating from bigger impacts. Some of them are, intriguingly, right smack on top of the ripples.
Opportunity slowed down and from Sols 1854 to 1858 (April 11 to 15, 2009) conducted some drive-by shootings with its Pancam. “We took a lot of images of those craters as we passed through that field,” Squyres continued. “It will take some time to create the stereo models and do all the geometry to really work out the science associated with it, but we've collected what I think is going to be a really good data set on those.”
“This cluster is similar," said Golombek, "to many craters observed to form in the past 10 years. They are younger than the large ripples and scattered across the craters are small dark pebbles. Because the pebbles are superposed on the craters, they must be younger or have come with the crater and thus impact derived."
Opportunity has been seeing these pebbles since Eagle crater, Golombek pointed out. "The question is: are the pebbles meteoritic or if the craters are secondaries pieces of Mars from elsewhere?”
The MER scientists are now on the lookout for pebbles big enough pebble to analyze in order to determine which of these two possibilities is correct. “We're keeping our eyes open for either concentrations of these that we can look at – like the blueberries back at Eagle – or perhaps we can find larger samples,” said Banerdt.
Once it finished with its photography, Opportunity hit the road again, using a combination of forward and backward driving. The longer drives – such as those on Sols 1859 (April 16, 2009) for 62 meters (203 feet) and on Sol 1860 (April 17, 2009) for 88 meters (289 feet) – included slip checks. Happily, the wheel currents all showed normal current levels.
During the last couple of weeks, Opportunity's agenda has shifted almost completely to again driving. Last week, it drove for three consecutive sols, logging 96 meters (315 feet) on Sol 1863 (April 21, 2009), 137 meters (449 feet) on Sol 1864 (April 22, 2009), and 95 meters (312 feet) on Sol 1865 (April 23, 2009). It seemed like nothing could stop this rover, then on the last of those drive it experienced a close encounter small purgatoid or "purgatina," as Banerdt put it.
Although Opportunity wound up having to spend a little time extricating itself, it was "only a slight embedding event," said Squyres, and "not a big issue. The backup drive on Sol 1866 (April 24, 2009) had only limited success, with Opportunity moving only about 28 centimeters (11 inches) before limits stopped the drive, but on the next sol the rover completed the backup, roving away for about 3.7 meters (12 feet). The rover drivers now have experience getting out of these sneaky sand drifts, and they also have experience “not getting into them too,” Banerdt noted. “What stopped the rover this last time was the slip check. There's a certain amount of slip it will tolerate, then it stops the drive and waits for further instructions.”
Spirit and Opportunity have always had the capability of doing slip checks. In fact, when Opportunity got really stuck way back in 2005 in Purgatory Dune, it was because the team had disabled the slip check feature. “Everything had been working so well, it was like we don't have to watch where we're going, Banerdt said. Safety takes time. “The slip checks require the rover to stop and do some imaging and analysis. That tends to slow the rover down. The frequency with which you do slip checks and the limits you put on them have a big affect on the maximum drive you can accomplish,” he explained. “There's a trade-off between safety and boldness in terms of how much terrain you can cover.”
Actually, Squyres in his talks and presentations on the rovers reflects back sometimes on those 5 excruciatingly long weeks in Purgatory Dune. “What we were doing was equivalent to driving down the freeway at 100 miles per hour with our eyes closed," he's said. It turned out to be a lesson learned in prudent driving for the whole team. And Opportunity has never gotten stuck like that since.
Last week, Opportunity took off again, like a bat out of -- well you know, and made some more really serious tracks. “We managed to get back onto better terrain," said Nelson. "In the last week and a half or so, we've driven another half-kilometer of distance, so it's really been moving along,” Banerdt added.
This week, Opportunity logged another 64 meters (210 feet) during drives on Sols 1870 and 1871 (April 28 and 29, 2009), which took it around the centerline of one of "these little noses that stick out" from the sea of purgatoids, as Nelson described it. But, on Sol 1871,they noticed a return of the increase in the amount of current drawn by drive actuator in the right front wheel. Now the engineers are again considering mitigation practices of resting the actuator and/or driving backwards for a while.