Next Course of Action for Testing

Now that build problem and go.s are fairly robust, the following remains for wiki integration. Let me know if you believe there is a more efficient or effective way of organization.

I created a new SVN repository at /data/repositories/tests on clover. This repo is currently integrated under the Browse Source tab.

This repository will store files of interest from /data/tests on clover.

The files will be organized as follows:

testname/logs/[date]/(testlog, rel_errors.data, abs_errors.data)

testname/images/[date]/sim.png

testname/images/ref.png

after go.s runs, the entire /data/tests directory will be checked into the repository. All files in this directory will be ignored except for those mentioned above.

A wiki plugin will be created which iterates through the repository and embeds the images in a wiki page. For example: 

[[Tests]]

would output the references and simulation runs for all tests for all dates, as well as a success or failure flag. 

[[TestList]]

would output a list of all tests currently on file in the repository.

Since only one reference is listed in the subversion repository for each test, any changes to this reference should the original be determined inaccurate would automatically update on pages using this plugin. One concern is if Trac will allow me to modify the "success" flag to invalidate all test results preceding the date of the reference change.

The results could also be culled. For example: 

[[Tests(FieldLoopAdvection, 11-11-2010, 11-11-2011)]]

would output the references and simulation runs for the FieldLoopAdvection test over the course of a year. This will allow for more complicated WikiFormatting structures, such as comparisons between test runs of different dates in tables.

Specific data can also be represented: 

[[TestResults(FieldLoopAdvection, 11-11-2011)]]

would only output "success" or "failure" for the FieldLoopAdvection test run on 11-11-2011. 

[[TestSim(FieldLoopAdvection, 11-11-2011)]]

would only output the the simulation picture for the FieldLoopAdvection test run on 11-11-2011. 

[[TestReference(FieldLoopAdvection)]]

would only output the reference picture for the FieldLoopAdvection test.

Please let me know what features/data culling abilities/syntax you would like to see if it is not adequately represented in the format above.

Comments

No comments.