Module 5: Validity, Reliability, and Model Improvement¶
This module will discuss model validity and reliability, two critical aspects of meaningful model-building. Validity is the process of determining that your model has been built correctly and is giving values that make sense within the confines of your system. Reliability covers a number of topics, such as how consistent your model is over repeated runs and how well it runs across various data sets. This section will discuss methods for ensuring validity and reliability, as well as some suggestions for improving models in preparation for publication and sharing.
5.1: Validation vs Verification¶
Ensuring model validity is often a complicated process involving multiple rounds of testing and refining. Although not impossible, it is rare that a model will give you exactly the values you expected the first time through. Taking time to test your model and ensuring that it is behaving as expected is a normal part of the process that should jot be overlooked. Before we discuss methods and examples of how to do this, there is a key distinction that should be made up front: the difference between verification and validation.
Verification is the process of determining whether your model is functioning as expected. In other words, verification involves making sure that all of the calculations, algorithmic steps, and processes are being executed correctly.
Validation is the process of determining whether your model is producing values that make sense given the conceptual bounds of the real-world system that it represents.
You can think of verification as the technical aspect of validity and validation as the conceptual aspect. Both of these processes are critical to model validity, so it is important not to confuse the two when assessing model validity. A model that has only been verified may still not be producing useful output, regardless of how robust the code itself is.
5.2: Methods for Verifying a Model/Simulation¶
[In Progress: To be included in update on 09/19/25]
5.3: Methods for Validating a Model/Simulation¶
[In Progress: To be included in update on 09/19/25]
5.4: Pairing Models with Experiment¶
[In Progress: To be included in update on 09/19/25]