Using OPT on Eddie
OPT (the Optimization and Profiling Tool)is designed to help users improve the performance of parallel codes - for C, C++ and Fortran MPI applications. It has been integrated with batch queue system (SGE) on Eddie, instrumented MPI jobs will be submited to Sun Grid Engine.
The following steps show how to start to use OPT on eddie:
Step 1. Login with X11 authentication
ssh -X eddie.ecdf.ed.ac.uk
Step 2. Add MPI and OPT modules to your environment
module add openmpi/gcc module add opt
Step 3. Source OPT environment file
Add the following lines to $HOME/.bashrc if you haven't done this step before.
# Source OPT Env File
if [ -f /usr/local/Cluster-Apps/opt/1.2/opt/etc/opt.env ]; then
. /usr/local/Cluster-Apps/opt/1.2/opt/etc/opt.env
fi
Step 4. Compile MPI codes
Currently OPT on Eddie supports OpenMPI and MPICH
OpenMPI
You can either link your codes with shared libraries to get a smaller size or link with static labraries to acquire better performance.
mpicc -o hello-shared \ -g hello.c \ -I /usr/local/Cluster-Apps/opt/1.2/opt/include \ -L /usr/local/Cluster-Apps/opt/1.2/opt/lib -lopt-openmpi
mpicc -o hello-static \ -g hello.c \ -I /usr/local/Cluster-Apps/opt/1.2/opt/include \ -L /usr/local/Cluster-Apps/opt/1.2/opt/lib/static -lopt-openmpi \ -L /usr/local/Cluster-Apps/opt/1.2/opt-deps/papi/lib/static -lpapi -lstdc++ -ldl \ -L /usr/local/Cluster-Apps/opt/1.2/opt-deps/libunwind/lib -lunwind-x86_64 -lunwind
MPICH
Similar commands to build your code with MPICH.
mpicc -o hello-shared \ -g hello.c \ -I /usr/local/Cluster-Apps/opt/1.2/opt/include \ -L /usr/local/Cluster-Apps/opt/1.2/opt/lib -lopt-mpich -shlib
mpicc -o hello-static \ -g hello.c \ -I /usr/local/Cluster-Apps/opt/1.2/opt/include \ -L /usr/local/Cluster-Apps/opt/1.2/opt/lib/static -lopt-mpich \ -L /usr/local/Cluster-Apps/opt/1.2/opt-deps/papi/lib/static -lpapi -lstdc++ -ldl \ -L /usr/local/Cluster-Apps/opt/1.2/opt-deps/libunwind/lib -lunwind-x86_64 -lunwind
Note: MPICH shared library not supports Intel compilers.
Step 5. Launch OPT
optgui &
Step 6. Connect with OPT servers
Type eddie01 into the Host input box and click Add Host. Highlight the host in the list and click Connect. In the authentication dialog panel, enter your system username and db password. You should see the host connection state to be Connected after you clicking Ok
Note: You need apply for a DB account for using OPT on Eddie, contact Science Support science.support@ed.ac.uk
Step 7. Create an OPT job
Note: You have to make sure your opt work directory can be accessed by 'others'
chmod o+rx /your/opt/work/dir
Click the New button to bring up the OPT Job Configuration dialog. Enter job information as following example
Job name: hello Executable: /home/ywan/opt-test/hello-shared Number of processes: 8 Launch method: openmpi-gnu Launch Directory: /home/ywan/opt-test Output File: /home/ywan/opt-test/hello.out Error File: /home/ywan/opt-test/hello.err
Note:
1. You are allowed to use upto 64 processes to run your parallel job.
2. You can select your prefered <MPI>-<compiler> combination from Launch Method drop list. For example openmpi-gnu
Save this job and it will appear in job list with Created state.
Step 8. Launch OPT jobs
Select a created job from job list and click Launch button. Input you EASE password in the new dialog. Click Ok to launch this job. OPT will submit your job to grid engine encapsulated by a template script. All being well, The state of job should cycle through created, started, running, logging and finally completed.
Step 9. View results
When job in completed state, the performance results can be viewd via View button. You can also edit the job to re-run it or delete it from joblist.