Skip to content

Data driven tests

BHoM has several ways to cover developed functionality with Tests. An automated strategy for covering possible regression (i.e. loss of functionality erroneously introduced by code changes) can be done with "Data driven tests".

Data driven tests are simply a way to take a "snapshot" of the input and output of a specific method. The input and output are stored in a dataset, together with the name of the method used to produce the output from the input. This data can be then used to automatically trigger the method at a later time, or periodically, to check that the method has not been broken e.g. with side-effects of other code changes elsewhere.

To record the test data, you simply need to run a target Engine method with some specific input data. The input data and the ouput of the method, together with the method name, will be recorded. When the data-driven test will be run, it will simply call again the method in question with the stored input data, and compare it with the output data. This way, it is possible to check that Engine methods keep behaving reliably.

This kind of "Data-driven Unit test" can be run automatically via CI/CD for an automated checking of the functionality.

Storing test data for Engine Methods

To store data for tests, you can use the Test_Toolkit and the Unit Test component.

image

Procedure

  1. Compile the Test_Toolkit - it contains some useful methods that are not shipped in the BHoM installer.
  2. Drop a Unit Test component in a script.
  3. Right click the component and use the search field to find the method you want to store test data for.
  4. Produce the test data for the method. The test data should be any input object that you may want to feed to the Engine method. The test data should be representative of the general usage of the method.
  5. Connect the test data to the Unit Test component. The compnent will execute the target method in the backend with the provided data, and it will return one or more Unit Test objects, which contain the input and outputs related to the method execution.
  6. The Unit Test objects should now be stored in the .ci folder of the repository that defines the method being tested. We store the objects as json. In order to do this reliably, you can use the Test_Toolkit's StoreUnitTests function, like so:

image

The StoreUnitTests function will save the test data in the .ci folder of the repository. Make sure to commit and push the data in your PR.

Last update: September 20, 2023