Compiling code

Compilation commands | GPU compilers | Changing compilers
Native commands | Where to compile

Users have access to several brands of compilers on Yellowstone, Geyser, and Caldera: Intel, PGI, PathScale, GNU, and NVIDIA.

The Intel compiler module is loaded by default when you log in. See Changing compilers below to use a different compiler.

Best practice: Before you compile, unload any modules that are not explicitly required or listed as dependencies for the program that you are compiling.


Compilation commands

After loading the compiler module that you want to use, identify and run the appropriate compilation wrapper command from the table below. (If your script already includes one of these generic MPI commands, there is no need to change it: mpif90, mpif77, ftn; mpicc, cc; mpiCC and CC.)

Also consider using the compiler's diagnostic flags to identify potential problems.

Compiler Language Commands for serial programs Commands for programs
using MPI
Flags to enable OpenMP
(for serial and MPI)
Intel (default) Fortran ifort foo.f90 mpif90 foo.f90 -openmp
  C icc foo.c mpicc foo.c -openmp
  C++ icpc foo.C mpiCC foo.C -openmp

Include these flags for best performance when you use the Intel compiler regardless of where you compile: -march=corei7 -axAVX

PGI Fortran pgfortran (or pgf90 or pgf95) foo.f90 mpipf90 foo.f90 -mp
  C pgcc foo.c mpipcc foo.c -mp
  C++ pgcpp (or pgCC) foo.C Not available** -mp
PathScale Fortran pathf90 foo.f90 mpief90 foo.f90 -mp
  C pathcc foo.c mpiecc foo.c -mp
  C++ pathCC foo.C Not available** -mp
GNU Fortran gfortran foo.f90 mpfort foo.f90 -fopenmp
  C gcc foo.c mpicc foo.c -fopenmp
  C++ g++foo.C mpiCC foo.C -fopenmp

GPU compilers

To compile CUDA code to run on the Geyser and Caldera clusters, use either NVIDIA's nvcc compiler command for CUDA C and C++ code, or use PGI for CUDA Fortran code.

See Using GPU compilers in the documentation for Geyser and Caldera for examples.

 


Changing compilers

Should you prefer to use a compiler other than the Intel default compiler, use module swap to make the change.

In this example, you are switching from Intel to PGI:

module swap intel pgi

When you load a compiler module, the system makes other compatible modules available. This helps you establish a working environment and avoid conflicts. If you need to link your program with a library*, use module load to load the library as in this example:

module load netcdf

Then, you can just invoke the desired compilation command without adding link options such as -l netcdf. Here's an example:

mpiifort foo.f90

You can learn more about how using modules helps you manage your environment on our Environment modules page.


Native commands

We recommend using the module wrapper commands described above. However, if you prefer to invoke the compilers directly, unload the NCAR default compiler wrapper environment by entering this on your command line:

module unload ncarcompilers

You can still use the environment variables that are set by the modules you have loaded.

Then, follow these examples of compiling a Fortran program when invoking the compilers directly. See the chart below for compiling C and C++ programs.

Intel compiler

ifort -o a.out $INC_NCAR program_name.f $LIB_NCAR

PGI compiler

pgfortran -o a.out $INC_NCAR program_name.f $LIB_NCAR

PathScale compiler

pathf90 -o a.out $INC_NCAR program_name.f $LIB_NCAR

GNU/gcc family of compilers

gfortran -o a.out $INC_NCAR program_name.f $LIB_NCAR

For your reference, the table below shows the native commands for each of the compilers for Fortran, C, and C++ programs.

Compiler Fortran C C++
Intel ifort foo.f90 icc foo.c icpc foo.C
PGI pgfortran (or pgf90, pgf95) foo.f90 pgcc foo.c pgcpp foo.C
PathScale pathf90 (or pathf95) foo.f90 pathcc foo.c pathCC foo.C
GNU gfortran foo.f90 gcc foo.c g++ foo.C

Where to compile

Differences between computing systems in the Yellowstone environment may affect where you choose to compile your code.

The CPU architecture of the Yellowstone and Caldera clusters and the CPU architecture of the Geyser cluster are different. Yellowstone and Caldera have Intel Sandy Bridge processors; Geyser has Intel Westmere processors.

Compile on Yellowstone if…

  • You want to aggressively optimize CPU performance.
  • You will run your code only on Yellowstone or Caldera.

The executable will run at the fastest possible speed on Yellowstone and Caldera. On Geyser, it may crash with an "Illegal Instruction" error at runtime.

Compiling on Yellowstone is restricted to the six login nodes (yslogin1-6); it is not permitted on the batch nodes.

Compile on Geyser if…

  • You want to ensure that your code will run on Geyser.

The executable compiled on Geyser also will run on Yellowstone or Caldera, but likely slower than if it were compiled on Yellowstone. To compile on Geyser, submit an interactive or batch job specifying the "geyser" queue.

 

* In addition to multiple compilers, CISL keeps available multiple versions of libraries to accommodate a wide range of users' needs. Rather than rely on the environment variable LD_LIBRARY_PATH to find the correct libraries dynamically, we encode library paths within the binaries when you build Executable and Linkable Format (ELF) executables. To do this, we use RPATH rather than LD_LIBRARY_PATH to set the necessary paths to shared libraries.

This enables your executable to work regardless of updates to new default versions of the various libraries; it doesn't have to search dynamically at run time to load them. It also means you don't need to worry about setting the variable or loading another module, greatly reducing the likelihood of runtime errors.

** MPI versions of the C++ PGI and PathScale compilers are not available.