Posts in category bluestreak

Meeting Update 09/08/2014

Over the past few weeks I've spent my time shuffling around data and organizing things to make visualization not… impossible? So now we have all of the data together in their respective directories, shared between bamboodata and grassdata. Although one can use Erica's directories in grassdata as "home base" as I made sym-links to the bamboodata directories. So everything seems like it is stored with its applicable files on grassdata.

Current Disk Usage:

/bamboodata/ 86%
/grassdata/ 89%

Now we're doing post processing on the data, along with resolution testing. We have updated our wikipage (see: ​CFRunStatsHighRes).

I am also studying for the physics GRE, and given my study/work schedule I might not show up for group meetings. I'll still post updates.

Running Jobs

  • Currently:
    1. Bluestreak (BS):
      • Post processing (PP) on our chombos (reservation, for B1S60 and B10S15 256 nodes). Once those two runs are done, we'll do the other two. Eventually we'll need to throw the NoShear case into the mix (see 2.).
      • Convergence testing at AMR level 3 (pending in Standard queue for both shear 60 cases). Once these jobs are done, we'll do the other two if necessary.
    2. Stampede:
      • Running the NoShear case on there. Have done approximately 3 restarts. Current restart began 187, so this should be the last restart there. After each restart completes and I restart the next one, I tranfer files back to our local directories. Once we have all the files, I'll transfer them to BS to do PP.
  • Future:
    1. PP on BS of NoShear data.
    2. PP for B10S30 and B10S60 on BS.
    3. Convergence Tests for the rest of the data sets.

Tranferring Files

  • Once all PP on BS is done, transfer everything back to local directories.
  • Tranferring NoShear to BS and local machines.

Visualization

  • Erica submitted the proposal to the Vista Collaboratory Contest this past Friday.
  • Visualization on BS using x2go.

Meeting Update 08-05-2014

File Transfer from BS & Run Updates

  • Went through all of the chombos on visit. All look great, no issues.
  • Deleted all of the chombos on BS as I was over soft limit and could no longer write files.
  • Been updating Erica's Production Runs page: ​https://astrobear.pas.rochester.edu/trac/wiki/u/erica/CFRunStatsHighRes
  • Reservation starting Thursday at 8AM for 5 days. Plan on running B1S60 at 128 nodes, B10S15 at 256 nodes, and B10S30 at 128 nodes. Currently have B10S60 running in th standard queue, already at chombo/sink 157.

Post-Processing & Local Machines

  • All of the files I have run on BS are transferred over either on grassdata or bamboodata (some b10s30 (143-153). Can check ProductionRuns directories under madams in grassdata/bamboodata.
  • We have the following chombos: 144 (B10S15), 153 (B10S30), 153 (B10S60), and 130 (B1S60)
  • We have about ¾ of all the runs done.
  • Post-Processing is up to date for B1S60 (completed last night), currently doing post processing on both bamboo and grass (bamboodata) for B10S60 and B10S30 respectively.

​GIF ​GIF

Going Forward

  • Reservations on BS. E-mailed Carl to see if he could take off the soft limit for Erica's account, including my own.
  • Getting onto Stampede so we can split the jobs between the two machines.
  • Need more space on our local machines. Many of our files are scattered between our directories. It would be smart to manage them in the same directories.

Meeting Update 07/28

Now in full control of Erica's CollidingFlows data on Bluestreak. Completed first reservation (5 days) today. Can see our progress ​here (CFRunStatsHighRes).

  • E-mailed Carl about the soonest I can get another reservation (4-7 days on BS). (Update 14:47: Got our reservation extended to next Monday at 8am).
  • Erica is working on what is necessary to get us up on Stampede.

Currently:

  • scp-ing files from BS to /grassdata/madams/.
  • Post-processing on Bamboo.
  • Running B10S30 on our extended reservation on BS at 512 nodes for 48 hours (07-28-2014 14:49).
  • Submitted B10S60 to the standard queue on BS at 128 nodes for 24 hours… pending, but we'll see what happens.

Some highlights from CFRunStatsHighRes Page:

Beta Shear Machine, Path Frames Completed / Left Filling Fracs Walltime Taken / Left (1-2) Info, (3) Message allocations Framerate (hrs/frame), No. cores Notes
1 60 BS, /scratch/madams15/CollidingFlows/Beta1/Shear60/Restart/ 122/200 0.106 0.559 0.631 0.321 0.467 16.8days/0days 282.5 gb, 197.6 mb, 64.0 mb ~5hr/frame, 2048 cores Reservation. Stayed with 128 nodes. Started with 48 hours left on reservation.
10 15 BS, /scratch/madams15/CollidingFlows/Beta10/Shear15/Restart 139/200 0.386 0.698 0.616 0.496 0.468 22.1days/0days 422.4 gb, 296.5 mb, 64.0 mb 7hr/frame, 2048 cores Reservation. Restarted with 128 nodes on 07/26.
10 30 BS, /scratch/madams15/CollidingFlows/Beta10/Shear30 128/200 0.136 0.587 0.601 0.465 0.418 1.2mo/0days 377.0 gb, 252.4 mb, 64.0 mb 9hr/frame, 2048 cores Reservation. Started Jul 24 08:46, cancelled to test other runs, started again at Jul 26 16:41.
10 60 BS, /scratch/madams15/CollidingFlows/Beta10/Shear60/Restart/ 151/200 0.180 0.547 0.544 0.318 0.269 6.7days/0days 305.3 gb, 211.8 mb, 64.0 mb ~4hr/frame, 2048 cores Reservation.

Going Forward:

No. Nodes Case frames/hr frames/day Chombos Left Days to Complete Weeks to Complete
128 B1S60 0.2 4.8 78 16 2.29
B10S15 0.143 3.5 61 17 2.43
B10S30 0.111 2.67 72 27 3.9
B10S60 0.25 6 49 8 1.14
No. Nodes Case frames/hr frames/day Chombos Left Days to Complete Weeks to Complete
256 B1S60 0.4 9.6 78 8.1 1.16
B10S15 0.29 6.96 61 8.7 1.24
B10S30 0.22 5.28 72 13 1.86
B10S60 0.5 12 49 4 0.57
No. Nodes Case frames/hr frames/day Chombos Left Days to Complete Weeks to Complete
512 B1S60 0.8 19.2 78 4 0.57
B10S15 0.57 13.68 61 4.5 0.643
B10S30 0.44 10.56 72 7 1
B10S60 1 24 49 2 0.29
  • Depending on how quickly we can get onto Stampede, I am estimating that we'll be done with these in 2 weeks if we do this most efficiently.