Changes between Version 19 and Version 20 of FrequentQuestions/LinuxFu

Timestamp:

08/16/11 11:25:19 (13 years ago)

Author:

Jonathan

Comment:

—

FrequentQuestions/LinuxFu

@@ -179 +179 @@
  == The Modules package ==
 See the [wiki:Modules] page.
+
+ == Scaling Tests scripts ==
+
+[[CollapsibleStart(Weak Scaling Scripts)]]
+  For weak scaling we need to specify the dimensions of each problem and the number of processors we want to use.  Then separate directories are created along with customized job scripts which are then submitted to the queue.  The job scripts are customized by echoing modified PBS directives along with any other necessary variables needed by the pbs script (in this case nproc, base_resx, and base_resy)
+
+{{{
+#!/bin/bash
+NP=(256 128 64 32 16 8)
+base_resx=(724 512 362 256 180 128)
+base_resy=(724 512 362 256 180 128)
+for (( j=0;j<${#NP[@]};j++)); do
+  nproc=${NP[j]}
+  mkdir $nproc
+  cp *.data $nproc
+  cd $nproc
+  mkdir out
+  nodes=`expr $nproc / 8`
+  echo "#!/bin/bash" > scrambler.pbs
+  echo "#PBS -q debug" >> scrambler.pbs
+  echo "#PBS -l nodes=$nodes:ppn=8,pvmem=1000mb,walltime=1:00:00" >> scrambler.pbs
+  echo "#PBS -N weakscalingtest-$nproc" >> scrambler.pbs
+  echo "nProcs=$nproc" >> scrambler.pbs
+  echo "base_resx=${base_resx[j]}" >> scrambler.pbs
+  echo "base_resy=${base_resy[j]}" >> scrambler.pbs
+  cat ../scrambler.pbs >> scrambler.pbs
+#  qsub scrambler.pbs
+  cd ..
+done
+}}}
+
+This then adds the following
+{{{
+#!/bin/bash
+#PBS -q debug
+#PBS -l nodes=1:ppn=8,pvmem=1000mb,walltime=1:00:00
+#PBS -N weakscalingtest-8
+nProcs=8
+base_resx=128
+base_resy=128
+}}}
+to the beginning of the default scrambler.pbs script
+{{{
+echo "==========="
+echo "Running on:"
+cat $PBS_NODEFILE
+echo "==========="
+
+cd $PBS_O_WORKDIR
+
+f=(.25 .5 .75)
+maxlevel=(0 1 2 3 4)
+threaded=(-1 0)
+mv data.out data.out.old
+../subst.s  Gmx $base_resx,$base_resy,1 global.data
+../subst.s  domain%mGlobal 1,1,1,$base_resx,$base_resy,1 global.data
+
+for (( l=0;l<${#maxlevel[@]};l++)); do
+  ../subst.s MaxLevel ${maxlevel[l]} global.data
+  for (( k=0;k<${#threaded[@]};k++)); do
+    ../subst.s iThreaded ${threaded[k]} global.data
+    for (( i=0;i<${#f[@]};i++)); do
+      ../subst.s filling_fractions ${f[i]} problem.data
+      echo ${maxlevel[l]} ${threaded[k]} ${f[i]} ${NP[j]}
+      mpirun -n $nProcs ../astrobear > output.out
+      grep scale_data output.out >> data.out
+    done
+  done
+done
+}}}
+
+Then when the job runs, the scrambler.pbs script can modify the various data files using the subst.s script to swap out different parameters and peform the various different runs all with the same number of processors.  The only requirement is that the values of a namelist variable appear on the same line as the namelist variable.
+
+For example
+
+{{{
+Gmx = 64,64,1  !Base resolution
+}}}
+
+instead of 
+{{{
+Gmx  =  64,  ! cells in x
+        64,  ! cells in y
+         1,  ! cells in z
+}}}
+
+Here is the subst.s script
+{{{
+#!/bin/bash
+# "subst", a script that substitutes one pattern for
+# another in a file,
+# i.e., "subst Smith Jones letter.txt".
+ARGS=3
+E_BADARGS=65   # Wrong number of arguments passed to script.
+if [ $# -ne "$ARGS" ]
+# Test number of arguments to script (always a good idea).
+then
+  echo "Usage: `basename $0` variable new-value filename"
+  exit $E_BADARGS
+ fi
+
+ var_pattern=$1
+ new_value=$2
+
+ if [ -f "$3" ]
+ then
+     file_name=$3
+ else
+     echo "File \"$3\" does not exist."
+     exit $E_BADARGS
+ fi
+
+ # Here is where the heavy work gets done.
+newfile="$file_name""2"
+
+ sed -e "s/$var_pattern\(\s*=\s*\)\S*/$var_pattern\1$new_value/i" $file_name | uniq > $newfile
+mv $newfile $file_name
+
+ # I'm sure there is a better way then to pipe the output through unique but I got frustrated trying JJC
+ # 's' is, of course, the substitute command in sed,
+ # and /pattern/ invokes address matching.
+ # The (\s*=\s*\) matches (and saves) all of the white space on either side of the equals sign
+ # The \S* terminates at the first non-white space character which is presumably the end of the variable value
+ # Note for array definitions like qTolerance = 1e-3 1e-3 1e-3, this will replace only the first value.
+ # This can be avoided by editing your data files to remove white space in array declarations (ie qTolerance = 1e-3,1e-3,1e-3)
+
+ exit 0    # Successful invocation of the script returns 0.
+
+}}}
+
+[[CollapsibleEnd]]