Verification and Validation

Since this library was developed at JPL, we take the idea of verification and validation very seriously. We believe that this suite presents a significant improvement in our ability to do last-mile development in a more agile way than ever previously, we still have to answer to NASA, which means we need to create V&V artifacts.

This suite is meant to save users having to do work that other projects have already completed by providing standardized, clean, and complete implementations of common planning and data analysis fundamentals. But one of our fundamental principles is that our customers should own their own risk decisisions and be able to posture themselves around any risk tolerance they see fit. For this reason, we respect that some projects will not want to just trust that these libraries work as expected.

Verification

Verification is the "does it do the thing right?" part of V&V. For this we rely primarily on unit tests, which you can read about elsewhere in these docs.

Validation

For the "does it do the right thing?" part of V&V, projects cannot rely on our unit tests alone. Because even with perfect coverage, unit tests only test the code that exists. It does not test that the code that exists does the thing that it is supposed to do. Historically the only way to do this was to manually run the code, inspect its outputs, and make sure they match your subjective understanding of what they should do.

This is typically not very shareable nor very repeatable, and historically has been done mostly implicitly.

For this reason, we are developing a strategy to document validation reports directly alongside the autogenerated documents. TO DO: Actually implement this feature and document it better here.

A Word on V&V

While we try hard here to provide our customers with V&V documentation as well as code with out deliveries, we are also very clear that we offer this with no explicit warranty. Every project must take ownership for its own risk management and therefor V&V program. All of our unit tests and validation reports are meant to bootstrap projects into (what we think is) a good V&V program, but it is not a substitude. Each customer project should review our work in these areas before accepting it as their own.