Mars Pathfinder: What Actually Occurred?

You’ve most likely heard of Mars Pathfinder, the NASA lander and its Sojourner rover that took up residence on Mars in 1997. At first, all was effectively, however after a number of days, the lander suffered a collection of complete system resets. A watchdog timer detected an issue and compelled reboots. [Vivek Bhageria’s] current publish appears into what happened and the eventual solution.

The root trigger was precedence inversion. This occurs when a lower-priority activity holds a useful resource wanted by a higher-priority activity and prevents the higher-priority activity from executing. In this case, the working system was VxWorks, however this may occur on any form of precedence scheduling system.

The simplified model is that there have been two duties that every used the identical mutex to keep away from stepping on one another. The highest precedence was an information distribution activity. The lowest precedence activity handled meteorology information processing. There was additionally a medium-priority activity that dealt with bus upkeep.

The failure would happen when the low-priority activity had the mutex and misplaced its time slot earlier than releasing it. The high-priority activity would then discover it couldn’t take the mutex. However, the low-priority activity didn’t get management once more due to the medium-priority duties ready to run.

In impact, the high-priority activity was now ready on one thing that couldn’t change into accessible till the lowest-priority activity was in a position to run. That’s precedence inversion.

Sadly, engineers seen this taking place often throughout testing however, as we are sometimes vulnerable to do, they chalked it as much as glitches. As certainly one of our previous bosses used to say: “If you don’t know why it occurred, you then don’t know that it received’t occur once more.”

VxWorks, in fact, understands that this may occur and, fortunately, there was sufficient functionality on the spacecraft to show precedence inheritance on for the mutex in query. This permits the low-priority activity to get a lift as a result of a higher-priority activity is ready on it.

You can learn extra concerning the particulars within the publish. We additionally coated this a decade in the past with our own analysis.