Running DDT, MAP and PR jobs

Preparing your code | Client interface setup | Running DDT and MAP | Performance Reports

The Arm Forge tools, DDT and MAP, are installed on Cheyenne for debugging, profiling, and optimizing code.

Follow the procedures below to configure the DDT and MAP client interface on your local machine and then start your debugging and profiling jobs. We recommend using the client software to get the best performance, but the tools also run from the Cheyenne command line interface.

Performance Reports is another Arm tool for Cheyenne users. It summarizes the performance of HPC application runs. Get details and a sample batch script for generating reports below.

Your job's memory footprint increases when you use these tools, especially when you use MAP. If your job normally uses most of the memory on a node, you might need to distribute your cores across more nodes to avoid running out of memory.

Arm Forge was formerly known as Allinea Forge, and "allinea" remains in use in some cases.


Preparing your code

Include the -g option when you compile your code for debugging with DDT or profiling with MAP. See our Compiling code documentation for the compilation commands to use.

Do not move or remove the source code, binary, or executable files from the directory or directories in which you compiled them.

Review the following Cheyenne-specific documentation and the relevant Arm Forge user’s guide, which you can download from the Arm product documentation.


Client interface setup

The client software version that you use locally and the server version that you use on Cheyenne must be the same. We recommend using the latest version that is available on Cheyenne. Run module av allinea-forge to identify the latest version.

Procedure

  • Download the client software from the Arm HPC tools and libraries web page.
  • Install and start the client on your local machine.
  • From the “Remote Launch” menu (see image), select Configure.

Forge client home screen

  • Configure as shown in the following image. The configuration will apply to both DDT and MAP, so you only need to do it once.
  • Enter your username with @cheyenne.ucar.edu in the “Host Name” field.
  • Then, fill in the “Remote Installation Directory” field. You can copy the  text from here and change the version number to match the version you are using:
/glade/u/apps/ch/opt/allinea-forge/18.0.2/gnu/6.3.0

Leave the "Remote Script" field blank.

Forge configuration

  • Click OK.

Running DDT and MAP

Prepare a job script. Specify the regular queue and customize the script with your own project code, job name, and so on.

On the last line of your script, replace mpiexec_mpt with ddt --connect (or map --connect).

ddt --connect ./myjob_f

Submit your job when indicated below.

Procedure

  • Start the client interface on your local machine.

  • From the “Remote Launch” menu, select your personal host name.

Remote launch

  • When the following dialog box appears, authenticate by entering your YubiKey or CRYPTOCard token response.

Authentication

  • After logging in, you will be returned to your normal terminal window, where you can load the Forge module and other modules that you need. (Some users include the module load commands in their job scripts.)

module load allinea-forge/18.0.2
  • Submit your job script on your command line.

qsub myscript.bash
  • When your job starts, the Forge GUI will show that a “Reverse Connect Request” has been made. Accept the request to continue. 

Reverse connect

  • A “Run” window will open, displaying settings imported from your job script. Click Run.

  • The DDT (or MAP) window will open.

DDT screen

  • Quit when you’re finished so the license is available to other users.


Performance Reports

To generate a report on the performance of your program, submit a batch job to run it. You do not need to compile it with the -g debug option first.

Modify your batch script to load the allinea-reports module that you want to use and include perf-report as shown in the sample scripts below.

When your job runs, the output will include both text and HTML report files.

See the Arm product documentation for additional information.

Sample tcsh script

#!/bin/tcsh
#PBS -N prjob
#PBS -A project_code
#PBS -l walltime=01:00:00
#PBS -q regular
#PBS -j oe
#PBS -l select=2:ncpus=36:mpiprocs=36
#PBS -m abe
#PBS -M email_address

module load allinea-reports/18.0.2

setenv TMPDIR /glade/scratch/$USER/temp
mkdir -p $TMPDIR

### Run the executable
perf-report -mpi -n 72 ./executable_name.exe

Sample bash script

#!/bin/bash
#PBS -N prjob
#PBS -A project_code 
#PBS -l walltime=01:00:00 
#PBS -q regular #PBS -j oe 
#PBS -l select=2:ncpus=36:mpiprocs=36 
#PBS -m abe #PBS -M email_address

module load allinea-reports/18.0.2

export TMPDIR=/glade/scratch/$USER/temp
mkdir -p $TMPDIR

### Run the executable
perf-report -mpi -n 72 ./executable_name.exe