Principle of use for the base of tests

The purpose of the base is to compare two versions of the code for various configurations and options, in order to detect, for example, developments that can have an unexpected impact in other parts of the code. In general, the base aims at:

• precisely evaluating the differences introduced by a development version, compared to the official version it’s based on.
• consolidating this development version by testing it in a large range of SURFEX configurations.

We ask that each developer runs the base of tests on its own branch, during or at the end of its development time, as he prefers. Before he can propose its development to enter a new official version he has to provide a text document that:

• sums up the results of the run of the base of tests: list of the test cases that give results different in the two versions.
• explains and justifies the origin of these differences for each concerned test case.

An example and framework for such a document is proposed here:

Once the branch of the developer is validated through the base of tests, it can be included in the building process of the next official version, and a Pull Request can be done on the SURFEX GitHub repository.

At the end of the call for contributions, the SURFEX team will review the different Pull Requests, merge them and solve the eventual conflicts (depend on the content of the PRs), and will run the base of tests for a last time.
The SURFEX team will use the documents containing information about changes in results, provided by developers, to summarize the scientific input of the new version to be published.