Running a 2³ simulation of the colliding flows module with all the physics I will be using turned on, to get an idea of the baseline memory usage for this module.

I extended the time out to t=1000 so get a longer time on the processors. On bamboo, I run the job on 8 processors and give the command top. The virt column shows the memory usage to be 186 mb per process.

To test the difference in memory usage with the different physics on/off, compiled astrobear with and without hypre. This is the only expected multiphysics to produce a change in memory usage, as it uses external libraries.

Self-gravity flag	Machine	Processors	Memory per process (mb)
1 (on)	Bamboo	8	186
0 (off)	Bamboo	8	NA (bug in compiling with flag=0)
1	Bluehive, debug queue	8	196

(Note: all data files were same)

Results:

1. Baseline memory can be different on different machines, probably not by much.
2. Astrobear on disk (ls -lh astrobear) is 7 mb on bluehive, 6 on bamboo.
3. This baseline memory > 20*astrobear on disk. Meaning, in addition to holding the executable in memory on each processor, there is an additional ~180mb being loaded on each processor. This could be due to shared libraries, mpi, etc.??
4. Then in addition to total memory being used for info allocations (in astrobear.log), one should also add this "baseline" memory when trying to estimate the total memory the simulation is using. For example, if my baseline is 0.2 GB/processor, than the baseline = 1.6 GB per node with 8 processors. Now we can use that when trying to understand the total memory being used, as reported in bluehive.log at the end of a simulation, as the following tables shows.

For total info allocations being reported as ~ 200 mb (~.2 GB), 1 node/8 processors :

Resource	Requested	Used
memory (GB)	16.0	2.0

For total info allocations being reported as ~ 200 mb (~.2 GB), 2 nodes/16 processors:

Resource	Requested	Used
memory (GB)	16.0	3.7

For total info allocations being reported as 50 GB, 8 nodes/64 processors:

Resource	Requested	Used
memory (GB)	40	80