Environment Modules (EM)
What are Environment Modules (EMs)
The "Environment Module" system is a tool that allows you to control your own access to the different pieces of software that are currently installed on the cluster. This is important because we often have many different versions of the same software installed, and accessing the wrong version could create bugs that would be difficult to track down. EMs give you a way to access the software you need, while also avoiding any unnecessary conflicts.
In Unix, access to programs and libraries is generally controlled by setting a number of "environment variables". On many systems, users need to setup these variables themselves by editing certain scripts that get run automatically when the user logs in. Environment Modules simplify this process by allowing you to type some simple commands to query, load and unload software as you need it. To use a particular software package, you have to load the relevant module. Some info about EMs can be found in the wikipedia page : https://en.wikipedia.org/wiki/Environment_Modules_(software).
Flat vs Hierarchical Modules
Argo uses a flat module scheme that displays all module available on the system at all time. When
you search a package via module avail <package_name>, you will see modules matching
<package_name> regardless of their dependencies.
Hopper uses a hierarchical module scheme that displays only modules which are compatible with the particular
compiler and MPI library you have loaded at a given time to avoid incompatibilities. Therefore,
searching a package via module avail <package_name> will not necessarily show you all available
versions of that package. A more comprehensive way of searching for packages is using the module
spider <package_name> command. That will report all available packages matching <package_name>.
Initializing and Loading EMs
There are some standard software and libraries that are already installed for all user on the cluster. Users can see which of these are currently available (Note. Some modules may be removed and/or updated, it is important that you always check to make sure the software/library you want to use is available) by using the following command:
module avail
This will show:
----------------------------/cm/shared/modulefiles--------------------------
..........
.some package_1
.some package_2
.some more packages
The use.own is a special module, it enables the user to load their own
modules for software they have installed locally inside their home
directory. For more information about how to install software and create
modules for them please see the wiki page: Installing User Packages In Argo.
Users can load a specific module from the list of available modules by invoking:
module load module_name
You must load the respective module for the software or a library before
using it inside your application. You can load modules inside your
sbatch script by using the above command. You can check which modules
are currently loaded by:
module list
To get information about a particular module (such as the collection of environment variables that are set by that module) you can use:
module show module_name
You can unload a module by:
module unload module_name
To find more options type in:
module --help
Auto Loading Modules at Login
Users can make a module load automatically each time they login to the cluster using the following command:
module initadd module_name
Similarly one can remove a module from being auto loaded by:
module initrm module_name
You may also get more information about a particular module by invoking:
module whatis module_name
Note: Whatever modules you need to run your
application must also be loaded from your sbatch Slurm job submission
script which you submit to the Slurm scheduler in order to run your
application.
#SBATCH options
....
....
#load modules after this line
Base/ Default Modules
Argo
No modules are loaded by default at login on Argo unless you explicitly load them via your startup.
Hopper
Since modules available to you depend on the compiler and MPI library loaded in your environment, Hopper loads a set of default modules including a default compiler (gnu9/9.3.0) and MPI library (openmpi4/4.0.4).
$ module list
Currently Loaded Modules:
1) use.own 4) gnu9/9.3.0 7) hwloc/2.1.0
2) autotools 5) ucx/1.8.0 8) openmpi4/4.0.4
3) prun/2.0 6) libfabric/1.10.1 9) hosts/hopper
Warning: If you load modules or set up other environmental variables using your startup scripts on Argo, you will likely get errors and warning messages when logging into Hopper because those modules do not exist on Hopper with the same name. To avoid these issues, you can wrap some of the logic in your startup scripts to behave differently based on the cluster. Such logic would look like this:
# load the proper set of modules based on the cluster
export CLUSTER=`sacctmgr -n show cluster format=Cluster|xargs`
export CNODE=`hostname -s`
if [ ${CLUSTER} == "argo" ]; then
module load <ARGO_MODULE_NAME...>
source <ARGO_FILE...>
export ARGO_ENVIRONMENT=...
...
else [ ${CLUSTER} == "hopper" ]
Module Naming
Argo
Modules are generally named as <package_name>/<package_version> on Argo
Hopper
Depending on the source of the package, Lmod modules on Hopper can have longer names and aliases.
- Spack-built packages are named
<package_name>/<package_version>-<two-character-hash> - Some packages have useful aliases such as
<package_name>/<package_version>-<two-character-hash> (mixed-precision) - Some important packages such as compilers, MPI and math libraries have global aliases appearing at the top when you execute
module avail
Searching for modules
Info:
On Hopper, module spider searches the whole module tree to find matching
modules whereas module avail will only search modules built with your current compiler and MPI
library.
How you search for modules in Hopper is very different from Argo Cluster. Examples and demostration of the key differences are shown in the Argo vs Hopper page. Argo vs Hopper page
Basic LMod Usage on Hopper Cluster
The Lmod is only used on the Hopper Cluster.The table below summarizes the most commonly used LMod commands. Please note that you can use ml as an alias or shortcut to module.
| Module Command | Description |
|---|---|
| ml avail | List available modules |
| ml list | Show modules currently loaded |
| ml load/add package | Load a selected module* |
| ml +package | Load a selected module* |
| ml unload/rm package | Unload a previously loaded module |
| ml -package | Unload a previously loaded module |
| ml swap package1 package2 | Unload package1 and load package* |
| ml purge | Unload all loaded modules |
| ml reset | Reset loaded modules to system defaults |
| ml display/show package | Display the contents of a selected module |
| ml spider | List all modules (not just available ones) |
| ml spider package | Display description of a selected module |
| ml keyword key | Search for available modules by keyword |
| ml, module help | Display help, usage information for modules |
| ml use path | Add path to the MODULPATH (module search path) |
| ml unuse path | Remove path from the MODULPATH (module search path) |
Creating Custom Modules
Additional information on Customs modules is given on the User packages wiki page linked below: User packages wiki page
Reference
- Linux manpage: http://linux.die.net/man/1/module