Debug Team Procedures

New Tickets

1. When a new ticket arrives, the debug team leader will assign a debug team member (owner) to the ticket.

2. The ticket owner should establish contact with the reporter and verify:
   • That their environment settings, compiler flags, and submission scripts are correct.
   • They are using the most current version of the code.

3. The ticket owner will report back to the debug team leader, who will decide whether or not to change the ticket's priority. Most tickets will probably fall within the 2-4 range; only the debug team leader may set tickets as priority 1.

4. The ticket owner should work with the reporter to reproduce the bug. This will involve access to some or all of the following:
   • The reporter's data files.
   • The reporter's AstroBEAR build.
   • The cluster on which the error occurred.
   • The PBS script used to submit the job.
   • The reporter's .bashrc and .bash_profile contents.

5. Run the problem with debug flags turned on. If any pointer or memory allocation issues are found, then fix them first.

6. If no pointer/memory errors are found and the problem persists, then try the following options:
   • Lower the domain resolution.
   • Turn off AMR (i.e., set MaxLevel in global.data to 0).
   • Turn off source terms.
   • Set lGudonov to true.
   • Set InterpOpts to 0 in physics.data.
   • Use a different problem module.

The goal here is to simplify the setup by turning off extra features until the error no longer manifests. Once the error goes away, then try switching back on things that had been turned off until you have the closest thing to the original setup that still works. The remaining deactivated features should provide further clues about the fault.

7. Prepare a report (i.e., summary comment on the ticket) for weekly debug meetings. This comment should describe the results of the ticket owner's tests. If possible, the ticket owner should also recommend further courses of action.