Python

Loading a module | Using Python virtual environments

Multiple versions of Python are available on the Cheyenne cluster. CISL recommends loading one of the Python environment modules; if you invoke python without loading a module first, you will start an older version that SGI provides (Python 2.7.9) and is used mostly for system scripting and other administrative tasks.

To load the default module, which will provide a Python 2 environment for you to use, run this without specifying a version:

module load python

To load any other version, including 3.x variants, specify the module version.

module load python/3.6.2

Once you have loaded a module, you can identify the Python packages that CISL has custom built for the Cheyenne environment by running the module av command. Then, load the package(s) that you want. For example, load an optimized version of NumPy that uses the Intel Math Kernel Library as follows:

module load numpy/1.12.0

If a package that you want is not available as a module, you can install the package for yourself using pip, or request a custom build by emailing cislhelp@ucar.edu.


Using Python virtual environments

CISL recommends setting up a virtual environment if you want to use CISL-built Python packages along with packages that you install or develop. Once you set up your virtual environment and run the CISL utility called pypkgs import as shown below, you no longer need to load the Python modules each time you want to use them. The utility installs them into the virtual environment.

Follow these steps to construct a custom virtual environment:

  1. Load the desired modules, as in this example.

    module load python/2.7.13 numpy/1.12.0 scipy/0.18.1
  2. Create the Python virtual environment at a path of your choosing by running one of the following.

    For Python 2

    virtualenv --system-site-packages /glade/u/home/$USER/pyenvs/py2-scipy

    For Python 3

    python3 -m venv --system-site-packages /glade/u/home/$USER/pyenvs/py3-scipy
  3. Activate the virtual environment by running the source command appropriate to your shell.

    tcsh

    source /glade/u/home/$USER/pyenvs/py2-scipy/bin/activate.csh

    bash

    source /glade/u/home/$USER/pyenvs/py2-scipy/bin/activate
  4. Run pypkgs import to install the modules that you loaded in Step 1 into your virtual environment. (Some package developers do not support this. If that is the case with a package that you need, the utility will ask you if you wish to install it from the Python Package Index using pip.)

    pypkgs import
  5. With your virtual environment activated, run pip list to see which Python packages are available.

    pip list
    

    To install additional packages, use pip install.

At this point, you can unload all of the Python modules and continue to run in your virtual environment. To use the same environment in the future, simply activate it as in Step 3 and your installed packages will once again be available to you.

You can install additional packages any time your virtual environment is active. Use the procedures in Step 4 (if you have loaded additional modules) or in Step 5.