Scaling on Bluestreak

Intro:

All tests done with Shear15, ~¼ way through simulation. I am assuming that the other runs will scale similarly. Astrobear was built with hypre library 2.9.0 without global partition.

Method:

In global.data, changed number of frames and restart frame by factor of 10. So final frame went from 200 to 2000, and restart frame was changed from chombo00050.hdf to chombo00500.hdf. Running this simulation then produces frames with dt = 1/10th of original frame, which I then multiply run time by 10 to get an estimate of the run time per frame.

Results:

Nodes	Start time	Frame times	Average run time (not including first frame)	Avg. time to write to file	Frames/hr
32	3:09	3:43, 4:15, 4:49	33 min/ .1 frame	1.9 min	.19
128	3:09	3:29, 3:43, 3:57, 4:12	14.3 min/ .1 frame	3.7 min	.55
256	3:14	3:29, 3:42, 3:55, 4:09	13.3 min/ .1 frame	6.8 min	.83
512	5:17	5:34, 5:47, 6:00, 6.13	13 min/ .1 frame	9.4 min	1.3

In the table, I do not include the first frame when averaging the run time as it seems unusually slow given the additional time to reload the frames upon restart.

Note!!! As I increase the number of nodes, the time to write to file increases as well. Therefore, to get a truer estimate of the run-time for one frame, we need to be careful to remove the write-time before multiplying by 10, and only add it after (see next section).

Frame / hour calculation:

The frame rate R in hour/frame is calculated as follows for the 32 node case. To get frames/hour, take the inverse R^-1. This assumes the write time will be the same at the end of the normal dt frame rate.

32 Nodes Example

R_{32} = (Total ~time - write ~time)*10 + write~time

To get in hours per frame, divide by 60:

R_{32} = [(33 - 1.9)*10 + 1.9 ]min\times \frac{1 ~hr}{60 ~min} = 5.2 ~\frac{hr}{frame}

R^{-1}_{32} = 0.19 ~\frac{frames}{hour}

Similarly,

R^{-1}_{128} = 0.55 ~\frac{frames}{hour}

R^{-1}_{256} = 0.83 ~\frac{frames}{hour}

R^{-1}_{512} = 1.3 ~\frac{frames}{hour}

Choosing the best set of runs: