Posts in category bamboo

Meeting Update 09/08/2014

Over the past few weeks I've spent my time shuffling around data and organizing things to make visualization not… impossible? So now we have all of the data together in their respective directories, shared between bamboodata and grassdata. Although one can use Erica's directories in grassdata as "home base" as I made sym-links to the bamboodata directories. So everything seems like it is stored with its applicable files on grassdata.

Current Disk Usage:

/bamboodata/ 86%
/grassdata/ 89%

Now we're doing post processing on the data, along with resolution testing. We have updated our wikipage (see: CFRunStatsHighRes).

I am also studying for the physics GRE, and given my study/work schedule I might not show up for group meetings. I'll still post updates.

Running Jobs

  • Currently:
    1. Bluestreak (BS):
      • Post processing (PP) on our chombos (reservation, for B1S60 and B10S15 256 nodes). Once those two runs are done, we'll do the other two. Eventually we'll need to throw the NoShear case into the mix (see 2.).
      • Convergence testing at AMR level 3 (pending in Standard queue for both shear 60 cases). Once these jobs are done, we'll do the other two if necessary.
    2. Stampede:
      • Running the NoShear case on there. Have done approximately 3 restarts. Current restart began 187, so this should be the last restart there. After each restart completes and I restart the next one, I tranfer files back to our local directories. Once we have all the files, I'll transfer them to BS to do PP.
  • Future:
    1. PP on BS of NoShear data.
    2. PP for B10S30 and B10S60 on BS.
    3. Convergence Tests for the rest of the data sets.

Tranferring Files

  • Once all PP on BS is done, transfer everything back to local directories.
  • Tranferring NoShear to BS and local machines.

Visualization

  • Erica submitted the proposal to the Vista Collaboratory Contest this past Friday.
  • Visualization on BS using x2go.

Computing Resources & Local Machines

Noticed as I am scp-ing data on the CollidingFlows runs that some of the typical machines I used are starting to become full. I also do visualizations on Clover or my own machine, however they can be quite hard on my own graphics card. This has led to talk to Baowei and others about potentially looking into developing a new machine quote (see: https://astrobear.pas.rochester.edu/trac/blog/bliu02072012).

Given that the quote is 6 months old, that the price is not the same anymore. It has been suggested to me that someone in the group should e-mail Sean Kesavan <sean@asacomputers.com> and ask him to give up an updated quote or change it to meet our requirements (referencing the quote number on the quotation). I also think these are the specs for Bamboo (https://www.pas.rochester.edu/~bliu/ComputingResources/ASA_Computers.pdf).

In my opinion it might be a good idea to look into more disk space/storage and also get a new machine for visualization. In the mean time I took a look at our local machines and who is using them.

Commands I am using:

*du -sh * 2>&1 | grep -v "du: cannot" prints the disk spaced used for each user by printing standard error to standard out.

*df -h inside of the directory to see disk spaced used (along with %) and space that is available.

alfalfadata

The Disk Space Used

DS Used User / Directory
63G bliu
13G ckrau
4.6G ehansen
227G erica
422G johannjc
16K lost+found
1.1T martinhe
4.0K modulecalls
20M nordhaus
1.3T shuleli
300M test.dd
3.5G trac_backup

The Disk Space Available

Filesystem Size Used Avail Use% Mounted on
aufsroot 11T 4.7T 6.3T 43% /
udev 7.9G 4.0K 7.9G 1% /dev
tmpfs 1.6G 1.3M 1.6G 1% /run
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 7.9G 108K 7.9G 1% /run/shm
none 100M 160K 100M 1% /run/user
/dev/md127 5.5T 3.3T 1.9T 64% /media/grassdata
deptdisk.pas.rochester.edu:/home/ehansen 11T 4.7T 6.3T 43% /home/ehansen
deptdisk.pas.rochester.edu:/home/shuleli 11T 4.7T 6.3T 43% /home/shuleli
alfalfa.pas.rochester.edu:/alfalfadata 5.3T 4.3T 802G 85% /mnt/net/alfalfadata

bamboodata

The Disk Space Used

DS Used User / Directory
1.2T bliu
4.0K brookskj
6.0G chaig
1.6G Defense.zip
8.0K ehansen
580G erica
1.1T johannjc
16K lost+found
2.6T madams
1.2T martinhe
21M rsarkis
4.9G ScalingTest
3.6T shuleli

The Disk Space Available

Filesystem Size Used Avail Use% Mounted on
aufsroot 11T 4.7T 6.3T 43% /
udev 7.9G 4.0K 7.9G 1% /dev
tmpfs 1.6G 1.3M 1.6G 1% /run
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 7.9G 108K 7.9G 1% /run/shm
none 100M 160K 100M 1% /run/user
/dev/md127 5.5T 3.3T 1.9T 64% /media/grassdata
deptdisk.pas.rochester.edu:/home/ehansen 11T 4.7T 6.3T 43% /home/ehansen
deptdisk.pas.rochester.edu:/home/shuleli 11T 4.7T 6.3T 43% /home/shuleli
alfalfa.pas.rochester.edu:/alfalfadata 5.3T 4.3T 802G 85% /mnt/net/alfalfadata
bamboo.pas.rochester.edu:/bamboodata 13T 11T 1.2T 91% /mnt/net/bamboodata

botwindata

The Disk Space Used

DS Used User / Directory
4.0K bliu
673G johannjc
904K orda
3.4G repositories
4.0K repositories_backup.s
11G scrambler_tests
2.6G tests
65G TO_CLEAN
75G trac
4.0K trac_backup.s
7.2G trac_dev
4.0K trac.wsgi
1.8M www

The Disk Space Available

Filesystem Size Used Avail Use% Mounted on
aufsroot 11T 4.7T 6.3T 43% /
udev 7.9G 4.0K 7.9G 1% /dev
tmpfs 1.6G 1.3M 1.6G 1% /run
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 7.9G 108K 7.9G 1% /run/shm
none 100M 160K 100M 1% /run/user
/dev/md127 5.5T 3.3T 1.9T 64% /media/grassdata
deptdisk.pas.rochester.edu:/home/ehansen 11T 4.7T 6.3T 43% /home/ehansen
deptdisk.pas.rochester.edu:/home/shuleli 11T 4.7T 6.3T 43% /home/shuleli
alfalfa.pas.rochester.edu:/alfalfadata 5.3T 4.3T 802G 85% /mnt/net/alfalfadata
bamboo.pas.rochester.edu:/bamboodata 13T 11T 1.2T 91% /mnt/net/bamboodata
botwin.pas.rochester.edu:/botwindata 890G 838G 6.3G 100% /mnt/net/botwindata

cloverdata

The Disk Space Used

DS Used User / Directory
28M afrank
368K ameer
72K aquillen
4.0K array.f90
4.0K awhitbe2
20K BasicDisk
137G bcc
31M blin
9.7M bliu
1.7G bobbylc
3.3G chaig
2.6M clover
37M cruggier
0 data
851M DB
4.0K devsrc
107M edmonpp
72G ehansen
301G erica
1019M iminchev
448K jnp1
3.0T johannjc
14M langao
154M laskimr
2.9M lijoimc
3.9G local
16K lost+found
221M madams
13G martinhe
8.0K MegaSAS.log
124K mendygral
37M MHDWaves
13M mitran
648M moortgat
852M munson
3.0G noyesma
20K odat1
4.0K old_accounts
45G pvarni
292K raid
3.3G repositories
5.8G repositories_backup
140M rge21
2.9G rsarkis
192K ryan
126G scrambler_tests
2.1T shuleli
16M spearssj
0 test
54M test3
0 test.me
840M tests
355M tests130
231M tests_old
452K tkneen
14G trac
27G trac_backup
60K wma
1.8M www

The Disk Space Available

Filesystem Size Used Avail Use% Mounted on
aufsroot 11T 4.7T 6.3T 43% /
udev 7.9G 4.0K 7.9G 1% /dev
tmpfs 1.6G 1.3M 1.6G 1% /run
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 7.9G 108K 7.9G 1% /run/shm
none 100M 160K 100M 1% /run/user
/dev/md127 5.5T 3.3T 1.9T 64% /media/grassdata
deptdisk.pas.rochester.edu:/home/ehansen 11T 4.7T 6.3T 43% /home/ehansen
deptdisk.pas.rochester.edu:/home/shuleli 11T 4.7T 6.3T 43% /home/shuleli
alfalfa.pas.rochester.edu:/alfalfadata 5.3T 4.3T 802G 85% /mnt/net/alfalfadata
clover.pas.rochester.edu:/cloverdata 11T 5.8T 4.5T 57% /mnt/net/cloverdata

grassdata

The Disk Space Used

DS Used User / Directory
184M afrank
4.0K bshroyer
711M bshroyerdata
8.5M chaig
4.0K cruggier
27M data4out.out
127M data5out.out
4.0K ehansen
1.4T erica
4.0K eschroe3
2.1G ferreira
3.2G fnauman
33G grass_swapfile
29G jnp1
119G johannjc
4.0K johannjc:~
4.0K langao
4.0K laskimr
16K lost+found
310G madams
656G martinhe
12M munson
1.4G noyesma
112M root
762G shuleli
1.4G slucchin
4.0K tdennis
1001M test.dd
4.0K testlog
403M tkneen
46M xguan

The Disk Space Available

Filesystem Size Used Avail Use% Mounted on
aufsroot 11T 4.7T 6.3T 43% /
udev 7.9G 4.0K 7.9G 1% /dev
tmpfs 1.6G 1.3M 1.6G 1% /run
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 7.9G 108K 7.9G 1% /run/shm
none 100M 160K 100M 1% /run/user
/dev/md127 5.5T 3.3T 1.9T 64% /media/grassdata
deptdisk.pas.rochester.edu:/home/ehansen 11T 4.7T 6.3T 43% /home/ehansen