Installing Software

In addition to the software already available on Hopper, users can install custom software. Because Hopper is a shared system, users do not have root access and cannot install software using administrative commands as you might on a home computer. Instead, software must be installed locally (to a location within your /home or /projects or /groups directory). The following are some common methods of installing user software:

Using Source Install

Building from Source

Some software is availabe only as source code (often hosted on github or similar shared repositories). Building and installing software from source is an advanced topic for experienced users. To build software, first ensure that you have loaded software modules for any compilers and libraries required by the code (e.g. the command ml gnu12 will load the GCC 12.3 Compiler Suite). Then download the source code (often with a git clone or wget command). If a configure script is part of the software installation process, you can specify the target installation directory using

./configure --prefix=$HOME/target_directory

Here we provide an example of how to install a custom package using a source distribution. We are using the gnu common lisp source package for demonstration purposes. You can download the latest source files from here: http://gnu.mirror.vexxhost.com/gcl/. You may need to review the instructions on Uploading Data in order to get the tarball to the cluster. Then unpack the contents of the tarball somewhere inside your home directory with one of the following commands, depending on the package:

$ tar xvfz <package-tarball>.tgz
 - or -
$ tar xvfz <package-tarball>.tar.gz

Let's assume that you want to install this package in the following directory:

/home/$USER/packages/gcl-<version.number>. 

Make sure that directory has been created:

$ mkdir -p /home/$USER/packages/gcl-<version.number>

Then from the directory where you had unpacked the contents of the tarball, run the following three commands. Note that each of these commands is likely to generate a large amount of text. Keep you eyes on the output to make sure there are no errors.

$ ./configure prefix=/home/$USER/packages/gcl-<version.number>
$ make
$ make install

If the compilation goes through without any errors, you will find the gcl executable inside the /home/$USER/packages/gcl-<version.number>/bin/ directory. Now the only thing left to do is to setup the appropriate environment parameters such that the system recognizes the new gcl command. We do this by creating a custom module file which will contain the path information of the gnu common lisp executable file in your home directory. We need to load it before we want to use the gcl command. We shall go over how to create and load custom module files in the Creating and Loading Custom Modules section below. At this point you can safely remove the tarball, and the directory it created.

Python Packages

Python packages can be installed into a python virtual environment. This is the preferred method of installing python packages on all machines. For instructions on creating python virtual environments and installing packages using pip, see Python Virtual Environments Hopper.

Conda

Conda is a popular tool for managing and installing software. To search for software which can be installed using conda see Conda Forge, Bioconda, and Anaconda repositories. To install any conda software, see Conda Environments on Hopper

Running Docker/Singularity/Apptainer Containers Images

For software which is available as a Docker or Singularity/Apptainer Container Image, the software can be run using Singularity.