Meeting Update 09/08/2014
Over the past few weeks I've spent my time shuffling around data and organizing things to make visualization not… impossible? So now we have all of the data together in their respective directories, shared between bamboodata and grassdata. Although one can use Erica's directories in grassdata as "home base" as I made sym-links to the bamboodata directories. So everything seems like it is stored with its applicable files on grassdata.
Current Disk Usage:
/bamboodata/ | 86% |
/grassdata/ | 89% |
Now we're doing post processing on the data, along with resolution testing. We have updated our wikipage (see: CFRunStatsHighRes).
I am also studying for the physics GRE, and given my study/work schedule I might not show up for group meetings. I'll still post updates.
Running Jobs
- Currently:
- Bluestreak (BS):
- Post processing (PP) on our chombos (reservation, for B1S60 and B10S15 256 nodes). Once those two runs are done, we'll do the other two. Eventually we'll need to throw the NoShear case into the mix (see 2.).
- Convergence testing at AMR level 3 (pending in Standard queue for both shear 60 cases). Once these jobs are done, we'll do the other two if necessary.
- Stampede:
- Running the NoShear case on there. Have done approximately 3 restarts. Current restart began 187, so this should be the last restart there. After each restart completes and I restart the next one, I tranfer files back to our local directories. Once we have all the files, I'll transfer them to BS to do PP.
- Bluestreak (BS):
- Future:
- PP on BS of NoShear data.
- PP for B10S30 and B10S60 on BS.
- Convergence Tests for the rest of the data sets.
Tranferring Files
- Once all PP on BS is done, transfer everything back to local directories.
- Tranferring NoShear to BS and local machines.
Visualization
- Erica submitted the proposal to the Vista Collaboratory Contest this past Friday.
- Visualization on BS using x2go.
Computing Resources & Local Machines
Noticed as I am scp-ing data on the CollidingFlows runs that some of the typical machines I used are starting to become full. I also do visualizations on Clover or my own machine, however they can be quite hard on my own graphics card. This has led to talk to Baowei and others about potentially looking into developing a new machine quote (see: https://astrobear.pas.rochester.edu/trac/blog/bliu02072012).
Given that the quote is 6 months old, that the price is not the same anymore. It has been suggested to me that someone in the group should e-mail Sean Kesavan <sean@asacomputers.com> and ask him to give up an updated quote or change it to meet our requirements (referencing the quote number on the quotation). I also think these are the specs for Bamboo (https://www.pas.rochester.edu/~bliu/ComputingResources/ASA_Computers.pdf).
In my opinion it might be a good idea to look into more disk space/storage and also get a new machine for visualization. In the mean time I took a look at our local machines and who is using them.
Commands I am using:
*du -sh * 2>&1 | grep -v "du: cannot"
prints the disk spaced used for each user by printing standard error to standard out.
*df -h
inside of the directory to see disk spaced used (along with %) and space that is available.
alfalfadata
The Disk Space Used
DS Used | User / Directory |
63G | bliu |
13G | ckrau |
4.6G | ehansen |
227G | erica |
422G | johannjc |
16K | lost+found |
1.1T | martinhe |
4.0K | modulecalls |
20M | nordhaus |
1.3T | shuleli |
300M | test.dd |
3.5G | trac_backup |
The Disk Space Available
Filesystem | Size | Used | Avail | Use% | Mounted on |
aufsroot | 11T | 4.7T | 6.3T | 43% | / |
udev | 7.9G | 4.0K | 7.9G | 1% | /dev |
tmpfs | 1.6G | 1.3M | 1.6G | 1% | /run |
none | 4.0K | 0 | 4.0K | 0% | /sys/fs/cgroup |
none | 5.0M | 0 | 5.0M | 0% | /run/lock |
none | 7.9G | 108K | 7.9G | 1% | /run/shm |
none | 100M | 160K | 100M | 1% | /run/user |
/dev/md127 | 5.5T | 3.3T | 1.9T | 64% | /media/grassdata |
deptdisk.pas.rochester.edu:/home/ehansen | 11T | 4.7T | 6.3T | 43% | /home/ehansen |
deptdisk.pas.rochester.edu:/home/shuleli | 11T | 4.7T | 6.3T | 43% | /home/shuleli |
alfalfa.pas.rochester.edu:/alfalfadata | 5.3T | 4.3T | 802G | 85% | /mnt/net/alfalfadata |
bamboodata
The Disk Space Used
DS Used | User / Directory |
1.2T | bliu |
4.0K | brookskj |
6.0G | chaig |
1.6G | Defense.zip |
8.0K | ehansen |
580G | erica |
1.1T | johannjc |
16K | lost+found |
2.6T | madams |
1.2T | martinhe |
21M | rsarkis |
4.9G | ScalingTest |
3.6T | shuleli |
The Disk Space Available
Filesystem | Size | Used | Avail | Use% | Mounted on |
aufsroot | 11T | 4.7T | 6.3T | 43% | / |
udev | 7.9G | 4.0K | 7.9G | 1% | /dev |
tmpfs | 1.6G | 1.3M | 1.6G | 1% | /run |
none | 4.0K | 0 | 4.0K | 0% | /sys/fs/cgroup |
none | 5.0M | 0 | 5.0M | 0% | /run/lock |
none | 7.9G | 108K | 7.9G | 1% | /run/shm |
none | 100M | 160K | 100M | 1% | /run/user |
/dev/md127 | 5.5T | 3.3T | 1.9T | 64% | /media/grassdata |
deptdisk.pas.rochester.edu:/home/ehansen | 11T | 4.7T | 6.3T | 43% | /home/ehansen |
deptdisk.pas.rochester.edu:/home/shuleli | 11T | 4.7T | 6.3T | 43% | /home/shuleli |
alfalfa.pas.rochester.edu:/alfalfadata | 5.3T | 4.3T | 802G | 85% | /mnt/net/alfalfadata |
bamboo.pas.rochester.edu:/bamboodata | 13T | 11T | 1.2T | 91% | /mnt/net/bamboodata |
botwindata
The Disk Space Used
DS Used | User / Directory |
4.0K | bliu |
673G | johannjc |
904K | orda |
3.4G | repositories |
4.0K | repositories_backup.s |
11G | scrambler_tests |
2.6G | tests |
65G | TO_CLEAN |
75G | trac |
4.0K | trac_backup.s |
7.2G | trac_dev |
4.0K | trac.wsgi |
1.8M | www |
The Disk Space Available
Filesystem | Size | Used | Avail | Use% | Mounted on |
aufsroot | 11T | 4.7T | 6.3T | 43% | / |
udev | 7.9G | 4.0K | 7.9G | 1% | /dev |
tmpfs | 1.6G | 1.3M | 1.6G | 1% | /run |
none | 4.0K | 0 | 4.0K | 0% | /sys/fs/cgroup |
none | 5.0M | 0 | 5.0M | 0% | /run/lock |
none | 7.9G | 108K | 7.9G | 1% | /run/shm |
none | 100M | 160K | 100M | 1% | /run/user |
/dev/md127 | 5.5T | 3.3T | 1.9T | 64% | /media/grassdata |
deptdisk.pas.rochester.edu:/home/ehansen | 11T | 4.7T | 6.3T | 43% | /home/ehansen |
deptdisk.pas.rochester.edu:/home/shuleli | 11T | 4.7T | 6.3T | 43% | /home/shuleli |
alfalfa.pas.rochester.edu:/alfalfadata | 5.3T | 4.3T | 802G | 85% | /mnt/net/alfalfadata |
bamboo.pas.rochester.edu:/bamboodata | 13T | 11T | 1.2T | 91% | /mnt/net/bamboodata |
botwin.pas.rochester.edu:/botwindata | 890G | 838G | 6.3G | 100% | /mnt/net/botwindata |
cloverdata
The Disk Space Used
DS Used | User / Directory |
28M | afrank |
368K | ameer |
72K | aquillen |
4.0K | array.f90 |
4.0K | awhitbe2 |
20K | BasicDisk |
137G | bcc |
31M | blin |
9.7M | bliu |
1.7G | bobbylc |
3.3G | chaig |
2.6M | clover |
37M | cruggier |
0 | data |
851M | DB |
4.0K | devsrc |
107M | edmonpp |
72G | ehansen |
301G | erica |
1019M | iminchev |
448K | jnp1 |
3.0T | johannjc |
14M | langao |
154M | laskimr |
2.9M | lijoimc |
3.9G | local |
16K | lost+found |
221M | madams |
13G | martinhe |
8.0K | MegaSAS.log |
124K | mendygral |
37M | MHDWaves |
13M | mitran |
648M | moortgat |
852M | munson |
3.0G | noyesma |
20K | odat1 |
4.0K | old_accounts |
45G | pvarni |
292K | raid |
3.3G | repositories |
5.8G | repositories_backup |
140M | rge21 |
2.9G | rsarkis |
192K | ryan |
126G | scrambler_tests |
2.1T | shuleli |
16M | spearssj |
0 | test |
54M | test3 |
0 | test.me |
840M | tests |
355M | tests130 |
231M | tests_old |
452K | tkneen |
14G | trac |
27G | trac_backup |
60K | wma |
1.8M | www |
The Disk Space Available
Filesystem | Size | Used | Avail | Use% | Mounted on |
aufsroot | 11T | 4.7T | 6.3T | 43% | / |
udev | 7.9G | 4.0K | 7.9G | 1% | /dev |
tmpfs | 1.6G | 1.3M | 1.6G | 1% | /run |
none | 4.0K | 0 | 4.0K | 0% | /sys/fs/cgroup |
none | 5.0M | 0 | 5.0M | 0% | /run/lock |
none | 7.9G | 108K | 7.9G | 1% | /run/shm |
none | 100M | 160K | 100M | 1% | /run/user |
/dev/md127 | 5.5T | 3.3T | 1.9T | 64% | /media/grassdata |
deptdisk.pas.rochester.edu:/home/ehansen | 11T | 4.7T | 6.3T | 43% | /home/ehansen |
deptdisk.pas.rochester.edu:/home/shuleli | 11T | 4.7T | 6.3T | 43% | /home/shuleli |
alfalfa.pas.rochester.edu:/alfalfadata | 5.3T | 4.3T | 802G | 85% | /mnt/net/alfalfadata |
clover.pas.rochester.edu:/cloverdata | 11T | 5.8T | 4.5T | 57% | /mnt/net/cloverdata |
grassdata
The Disk Space Used
DS Used | User / Directory |
184M | afrank |
4.0K | bshroyer |
711M | bshroyerdata |
8.5M | chaig |
4.0K | cruggier |
27M | data4out.out |
127M | data5out.out |
4.0K | ehansen |
1.4T | erica |
4.0K | eschroe3 |
2.1G | ferreira |
3.2G | fnauman |
33G | grass_swapfile |
29G | jnp1 |
119G | johannjc |
4.0K | johannjc:~ |
4.0K | langao |
4.0K | laskimr |
16K | lost+found |
310G | madams |
656G | martinhe |
12M | munson |
1.4G | noyesma |
112M | root |
762G | shuleli |
1.4G | slucchin |
4.0K | tdennis |
1001M | test.dd |
4.0K | testlog |
403M | tkneen |
46M | xguan |
The Disk Space Available
Filesystem | Size | Used | Avail | Use% | Mounted on |
aufsroot | 11T | 4.7T | 6.3T | 43% | / |
udev | 7.9G | 4.0K | 7.9G | 1% | /dev |
tmpfs | 1.6G | 1.3M | 1.6G | 1% | /run |
none | 4.0K | 0 | 4.0K | 0% | /sys/fs/cgroup |
none | 5.0M | 0 | 5.0M | 0% | /run/lock |
none | 7.9G | 108K | 7.9G | 1% | /run/shm |
none | 100M | 160K | 100M | 1% | /run/user |
/dev/md127 | 5.5T | 3.3T | 1.9T | 64% | /media/grassdata |
deptdisk.pas.rochester.edu:/home/ehansen | 11T | 4.7T | 6.3T | 43% | /home/ehansen |