3. Principle of operating, for the base of tests

The base relies on a sort of "over-namelist" containing about all the possible switches and options for SURFEX. This namelist is called OPTIONS.nam_000.

The procedure is divided into two parts:

• shell scripts are successively called to run SURFEX with different options. They define all test cases to be run. For that, they modify the namelist OPTIONS.nam_000 through some “sed” commands. The two versions to compare are run, and output NETCDF files for these two versions are stored in two different directories.
• Then, for each test case, a python program compares the two series of output files, and gives results in the form of text summary files. important note: scripts have recently been adapted to Python3. Please verify that by default, python3 is used when you run STRAO.

Note that :

• the base of tests only compares NETCDF output files. So, it can’t be used to validate the effective contents of output files in other formats (LFI, FA, ASCII, BINARY, TEXTE). If some changes are performed in the writing part of the code associated to these other formats, the developer will have to design his own strategy of testing it.
• new versions of the code often bring changes in namelist blocks. Therefore, testing a new branch will inevitably lead to adapt, would it be just a bit, the base namelist, OPTIONS.nam_000. If names of namelist entries are changed, possible associated “sed” commands in shell scripts will need to be adapted too.

• so, in the case he tests a new version of the code, the developer has to:
  • adapt OPTIONS.nam_000 following his new version
  • verify, for example through a “grep” command among the shell scripts of the base, that no shell script refers to the namelist entry he modified, and if it refers to, adapt the shell script functions of his changes.
  • think if his modifications could lead to propose a new test case to enter the base. In this case, he can propose a new test case to the SURFEX team and it will be included in the next version of the base of tests.
    
• Another point is that often, the “old” version tested has not exactly the same namelist blocks as the new one. For this purpose, a script named script_to_old.sh is delivered inside the base. This script needs to be adapted too: OPTIONS.nam_000 contains namelists blocks for the new version tested, and script_to_old.sh allows to go back to the old ones. script_to_old.sh is composed of “sed” commands.

Concretely, each tester will have its own version of the base of tests, corresponding to the options or modifications of existing blocks he introduced in the namelist with his new version.

At the end of the process of building a new official version of SURFEX, the SURFEX team will take charge of merging the versions of the base of tests coming from the different contributers.