wikiroute

networking recipes

User Tools

Site Tools

Trace:
compiled_matlab_on_igrida

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
compiled_matlab_on_igrida [2013/07/20 11:47] – samercompiled_matlab_on_igrida [2014/03/02 17:44] (current) – [Preparing the Scripts] samer
Line 8: Line 8:
 </code> </code>
  
-Welcome to the IGRIDA frontend! It is prohibited to start processes at this level, only interactive or batch OAR jobs are allowed.+When at the IGRIDA front-end, start an interactive job: 
 +<code bash> 
 +igrida-oar-frontend%oarsub -I 
 +</code>
  
 =====Preparing the Scripts===== =====Preparing the Scripts=====
  
-  module load matlab +Let us assume that your main MATLAB function is called ''main_foo''. This function has two input parameters and calls another function called ''sec_foo''. These two functions are stored in your home directory ''/udd/mylogin''. However, any output must me stored (for example in .mat files) on ''/temp_dd/igrida-fs1/mylogin/'' and never on your NFS home directory.  
-  /soft/matlab_hd/R2012b/bin/mcc -R -singleCompThread -m function_igrida.m function_igrida.m + 
 +<file bash main_foo.m> 
 +function main_foo(x,y) 
 +a=x+y; 
 +c=sec_foo(a,x); 
 +matFilename = sprintf('/temp_dd/igrida-fs1/mylogin/results.mat'); 
 +save(matFilename,'a','b','c'); 
 +</file> 
 + 
 +<file bash sec_foo.m> 
 +function c = sec_foo(a,x) 
 +c=a*x; 
 +</file> 
 + 
 +In order to compile your scripts, you can use the ''mcc'' compiler. Note that you should limit MATLAB to a single computational thread (by default, MATLAB makes use of the multithreading capabilities of the computer on which it is running). All your MATLAB scripts should be given as arguments starting with ''main_foo.m''. The syntax of the ''mcc'' command is given hereafter while in the interactive OAR session:
  
-As a good practice, it is recommended that you create a log directory, located in ''/temp_dd/igrida-fs-1/mylogin'', where the run will take place. In particular, your run outputs should never be written to your home directory, which would necessarily drive to an NFS crash at some time. For instance, type: 
 <code bash> <code bash>
-mkdir -p /temp_dd/igrida-fs1/mylogin/log/+igrida01-01%module load matlab 
 +igrida01-01%/soft/matlab_hd/R2012b/bin/mcc -R -singleCompThread -m main_foo.m sec_foo.m 
 </code> </code>
  
-Then, create an OAR launch script in your home directory ''/udd/mylogin/matlab-igrida-demo/''. In this script, you start by loading the Matlab module. You can also specify the number of cores and the wall time. Do not forget to redirect the output and the error of the OAR job. The launch script ends by calling your Matlab m-file.+Now, you have got an executable called main_foo that you test on the command line by typing: 
 +<code bash> 
 +igrida01-01%./main_foo 10 20 
 +</code> 
 +You can now exit the interactive OAR session and get back to the frontend. 
 +=====Preparing the OAR jobs===== 
 +Let us suppose that you need to run hundreds of simulations of your MATLAB compiled script. You can choose to modify your MATLAB scripts and add iterating loops (then recompile). Then, you can follow this [[matlab_script_on_igrida|tutorial]] to run your job. However, with this method, you launch a single OAR job and do not take full advantage of the computing facilities on IGRIDA. 
 +In order to run multiple jobs, you can write a launcher script as in ''oar_launcher.sh''. This script outputs a set of jobs in separated script files called job*.sh. In each job file, you call the compiled main_foo with incrementing parameters. Note that you may need to change the walltime and the number of cores. The launcher ends by submitting all the created jobs.
  
-<file bash oar-launch-script.sh>+<file bash oar_launcher.sh>
 #!/bin/bash #!/bin/bash
-source /etc/profile.d/modules.sh +for i in {1..10}; 
-module load matlab+do 
 + for j in {1..10}; 
 + do 
 + echo "#!/bin/bash " > job$i$j.sh 
 + echo "source /etc/profile.d/modules.sh" >> job$i$j.sh 
 + echo "module load matlab" >> job$i$j.sh 
 + echo "#OAR -l core=10,walltime=3:00:00" >> job$i$j.sh 
 + echo "#OAR -O /temp_dd/igrida-fs1/mylogin/log/job.%jobid%.output" >> job$i$j.sh 
 + echo "#OAR -E /temp_dd/igrida-fs1/mylogin/log/job.%jobid%.error" >> job$i$j.sh 
 + echo 'echo "My job was ran on these nodes:"' >> job$i$j.sh 
 + echo 'cat $OAR_NODEFILE' >> job$i$j.sh 
 + echo "#Setup MCR cache directory locally" >> job$i$j.sh 
 + echo 'export MCR_CACHE_ROOT=/tmp/mcr_cache_${USER}_OAR_JOBID_${OAR_JOBID}' >> job$i$j.sh 
 + echo 'mkdir -p $MCR_CACHE_ROOT' >> job$i$j.sh 
 + echo "#cd to your execution directory first" >> job$i$j.sh 
 + echo "cd /udd/mylogin/" >> job$i$j.sh 
 + echo "./main_foo $i $j" >> job$i$j.sh 
 + echo "#Remove temporary MCR cache directory" >> job$i$j.sh 
 + echo '/bin/rm -rf $MCR_CACHE_ROOT' >> job$i$j.sh 
 +  done; 
 +done;
  
-#OAR -l core=10,walltime=6:00:00 +for k in job*; 
-#OAR -O /temp_dd/igrida-fs1/mylogin/log/job.%jobid%.output +do 
-#OAR -E /temp_dd/igrida-fs1/mylogin/log/job.%jobid%.error +    chmod +x $k; 
-echo "My job was ran on these nodes:" +    oarsub -S $k; 
-cat $OAR_NODEFILE +done;
- +
-#Setup MCR cache directory locally +
-export MCR_CACHE_ROOT=/tmp/mcr_cache_${USER}_OAR_JOBID_${OAR_JOBID} +
-mkdir -p $MCR_CACHE_ROOT +
- +
-#cd to your execution directory first +
-cd /udd/mylogin/matlab-igrida +
-matlab -r matlab-demo +
- +
-#Remove temporary MCR cache directory +
-/bin/rm -rf $MCR_CACHE_ROOT+
 </file> </file>
- +  
-In your Matlab file ''matlab-demo.m'', all the output must be redirected to the ''/temp_dd/igrida-fs1/mylogin/'' filesystem. Here is a simple example: +Do not forget to create your log directory:
 <code bash> <code bash>
-a=1; +igrida-oar-frontend%mkdir -p /temp_dd/igrida-fs1/mylogin/log/
-b=a+1; +
-matFilename = sprintf('/temp_dd/igrida-fs1/mylogin/results.mat'); +
-save(matFilename,a,b);+
 </code> </code>
  
 Prepare you OAR launch script to be executed: Prepare you OAR launch script to be executed:
 <code bash> <code bash>
-igrida-oar-frontend%chmod +x oar-launch-script.sh+igrida-oar-frontend%chmod +x oar-launcher.sh
 </code> </code>
 +
 =====Launching the Job===== =====Launching the Job=====
-You can now submit eh job using the command ''oarsub'' as in the following:+You can now launch your jobs by simply calling your launcher script as in the following:
 <code bash> <code bash>
-igrida-oar-frontend%oarsub -S  oar-launch-script.sh+igrida-oar-frontend%./oar-launcher.sh
 [ADMISSION RULE] Modify resource description with type constraints [ADMISSION RULE] Modify resource description with type constraints
 [ADMISSION RULE] Job walltime greater than 4 hours, adding property duration_weight > 2 (large job). [ADMISSION RULE] Job walltime greater than 4 hours, adding property duration_weight > 2 (large job).
 Generate a job key... Generate a job key...
 OAR_JOB_ID=666666 OAR_JOB_ID=666666
 +[ADMISSION RULE] Modify resource description with type constraints
 +[ADMISSION RULE] Job walltime greater than 4 hours, adding property duration_weight > 2 (large job).
 +Generate a job key...
 +OAR_JOB_ID=777777
 </code> </code>
      
-Step back and watch your job being executed. +Step back and watch your jobs being successively created then executed. 
  
 =====Verifying the Output===== =====Verifying the Output=====
compiled_matlab_on_igrida.1374313622.txt.gz · Last modified: 2014/01/11 05:24 (external edit)