Evaluate autonomy through its actual decision scope

Autonomy means a system performs some decisions or actions without immediate human direction. It can use explicit rules, planning, image processing or learned models. Calling a system autonomous does not mean it chooses every mission goal, and calling an algorithm AI does not identify its verified operating limits.

In July 2016, JPL described Curiosity using AEGIS to select some ChemCam targets from images according to criteria set by scientists. That was a specific onboard capability supporting scientific work. It did not mean the rover independently determined every scientific priority or replaced the ground team.

For a fictional classifier, imagine 100 labeled terrain patches: 90 traversable and 10 hazardous. A system that calls every patch traversable scores 90% overall accuracy but misses every hazard. The aggregate score hides the error type that matters most for the proposed task.

A better evaluation reports missed hazards, false alarms, the test conditions and how labels were established. A classifier also needs a defined role in the larger decision process. Its output might advise a planner or be constrained by independent checks; the allowed action must be clear.

Overall accuracy can hide the critical error

A model trained or tuned on one terrain or lighting range may behave differently elsewhere. Representative evaluation should examine the conditions the mission expects, uncertainty handling and what happens when the system cannot support a confident decision. Success on a selected demonstration is not evidence for unlimited operation.

For a fictional proposal, ask: which decisions are delegated, what information is used, what constraints remain, and what evidence supports performance? Preserve the boundary between tested capability and intended future work. This allows a useful autonomous function to be understood and trusted for the specific job it has actually demonstrated.