OPENMX
OpenMX 3.9 is available on Hopper.
Installation
OpenMX 3.9 with path 3.9.6 was installed on Hopper with Intel One API compilers. To install: 1. Set up your shell environment:
module load OneAPI
module load fftw
source $SETVARS
- The installation instructions for the makefile to build from source were referenced from this page. Relevant extract:
In the makefile, CC, FC, and LIB as follows:
CC = mpiicc -O3 -xHOST -ip -no-prec-div -qopenmp -I${MKLROOT}/include -I${MKLROOT}/include/fftw
FC = mpiifort -O3 -xHOST -ip -no-prec-div -qopenmp
LIB= -L${MKLROOT}/lib/intel64 -lmkl_scalapack_lp64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lifcore -lmkl_blacs_intelmpi_lp64 -liomp5 -lpthread -lm -ldl
Then, for the installation you can perform as
make all
make install
Running OpenMX on Hopper
Slurm script for openmx in batch mode
#!/bin/bash
#SBATCH --job-name=openmx_test
#SBATCH --partition=normal
#SBATCH --constraint=intel
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=4
#SBATCH --cpus-per-task=12
#SBATCH --mem-per-cpu=3500M
#SBATCH --output=openmx_test-%j.out
#SBATCH --error=openmx_test-%j.err
## Load Modules
module load openmx
source $SETVARS --force
## Execute
#mpirun -np ${SLURM_NTASKS_PER_NODE} -runtest -nt ${SLURM_CPUS_PER_TASK}
mpirun -np 4 openmx -runtest -nt 1
cp runtest.result runtest.result.4.1
mpirun -np 2 openmx -runtest -nt 2
cp runtest.result runtest.result.2.2
mpirun -np 1 openmx -runtest -nt 4
cp runtest.result runtest.result.1.4
Run with
sbatch run.slurm
Results for different configurations of threads and tasks.
These tests are the same ones run in the install instruction pages. The cluster comes out a bit faster.
- Case 1: 4 MPI Tasks, 1 OMP threads
1 input_example/Benzene.dat Elapsed time(s)= 5.25 diff Utot= 0.000000000031 diff Force= 0.000000000003
2 input_example/C60.dat Elapsed time(s)= 24.66 diff Utot= 0.000000000000 diff Force= 0.000000000002
3 input_example/CO.dat Elapsed time(s)= 11.13 diff Utot= 0.000000000096 diff Force= 0.000000000277
4 input_example/Cr2.dat Elapsed time(s)= 8.84 diff Utot= 0.000000000943 diff Force= 0.000000000294
5 input_example/Crys-MnO.dat Elapsed time(s)= 23.13 diff Utot= 0.000000000003 diff Force= 0.000000000017
6 input_example/GaAs.dat Elapsed time(s)= 37.45 diff Utot= 0.000000000000 diff Force= 0.000000000000
7 input_example/Glycine.dat Elapsed time(s)= 6.60 diff Utot= 0.000000000001 diff Force= 0.000000000000
8 input_example/Graphite4.dat Elapsed time(s)= 3.98 diff Utot= 0.000000000027 diff Force= 0.000000000152
9 input_example/H2O-EF.dat Elapsed time(s)= 5.17 diff Utot= 0.000000000000 diff Force= 0.000000000002
10 input_example/H2O.dat Elapsed time(s)= 3.85 diff Utot= 0.000000000000 diff Force= 0.000000000011
11 input_example/HMn.dat Elapsed time(s)= 15.29 diff Utot= 0.000000000131 diff Force= 0.000000000021
12 input_example/Methane.dat Elapsed time(s)= 3.31 diff Utot= 0.000000000009 diff Force= 0.000000000001
13 input_example/Mol_MnO.dat Elapsed time(s)= 9.24 diff Utot= 0.000000000370 diff Force= 0.000000000085
14 input_example/Ndia2.dat Elapsed time(s)= 5.42 diff Utot= 0.000000000001 diff Force= 0.000000000000
Total elapsed time (s) 163.32
- Case 2: 2 MPI Tasks, 2 OMP threads
1 input_example/Benzene.dat Elapsed time(s)= 5.29 diff Utot= 0.000000000019 diff Force= 0.000000000006
2 input_example/C60.dat Elapsed time(s)= 28.42 diff Utot= 0.000000000004 diff Force= 0.000000000001
3 input_example/CO.dat Elapsed time(s)= 12.50 diff Utot= 0.000000000095 diff Force= 0.000000002442
4 input_example/Cr2.dat Elapsed time(s)= 8.24 diff Utot= 0.000000000979 diff Force= 0.000000000039
5 input_example/Crys-MnO.dat Elapsed time(s)= 33.84 diff Utot= 0.000000000002 diff Force= 0.000000000004
6 input_example/GaAs.dat Elapsed time(s)= 49.37 diff Utot= 0.000000000010 diff Force= 0.000000000000
7 input_example/Glycine.dat Elapsed time(s)= 6.32 diff Utot= 0.000000000001 diff Force= 0.000000000000
8 input_example/Graphite4.dat Elapsed time(s)= 4.18 diff Utot= 0.000000000006 diff Force= 0.000000000008
9 input_example/H2O-EF.dat Elapsed time(s)= 4.85 diff Utot= 0.000000000000 diff Force= 0.000000000002
10 input_example/H2O.dat Elapsed time(s)= 3.69 diff Utot= 0.000000000001 diff Force= 0.000000000394
11 input_example/HMn.dat Elapsed time(s)= 16.55 diff Utot= 0.000000000151 diff Force= 0.000000000021
12 input_example/Methane.dat Elapsed time(s)= 2.91 diff Utot= 0.000000000006 diff Force= 0.000000000001
13 input_example/Mol_MnO.dat Elapsed time(s)= 9.21 diff Utot= 0.000000000316 diff Force= 0.000000000083
14 input_example/Ndia2.dat Elapsed time(s)= 3.71 diff Utot= 0.000000000001 diff Force= 0.000000000000
Total elapsed time (s) 189.09
- Case 3: 1 MPI Task, 4 OMP threads
1 input_example/Benzene.dat Elapsed time(s)= 6.49 diff Utot= 0.000000000008 diff Force= 0.000000000009
2 input_example/C60.dat Elapsed time(s)= 29.55 diff Utot= 0.000000000009 diff Force= 0.000000000004
3 input_example/CO.dat Elapsed time(s)= 12.79 diff Utot= 0.000000000026 diff Force= 0.000000006976
4 input_example/Cr2.dat Elapsed time(s)= 10.15 diff Utot= 0.000000000928 diff Force= 0.000000000144
5 input_example/Crys-MnO.dat Elapsed time(s)= 56.56 diff Utot= 0.000000000001 diff Force= 0.000000000055
6 input_example/GaAs.dat Elapsed time(s)= 85.88 diff Utot= 0.000000000009 diff Force= 0.000000000002
7 input_example/Glycine.dat Elapsed time(s)= 8.25 diff Utot= 0.000000000001 diff Force= 0.000000000000
8 input_example/Graphite4.dat Elapsed time(s)= 3.98 diff Utot= 0.000000000001 diff Force= 0.000000000031
9 input_example/H2O-EF.dat Elapsed time(s)= 5.74 diff Utot= 0.000000000000 diff Force= 0.000000000004
10 input_example/H2O.dat Elapsed time(s)= 4.54 diff Utot= 0.000000000001 diff Force= 0.000000000371
11 input_example/HMn.dat Elapsed time(s)= 21.78 diff Utot= 0.000000000055 diff Force= 0.000000000022
12 input_example/Methane.dat Elapsed time(s)= 3.22 diff Utot= 0.000000000002 diff Force= 0.000000000000
13 input_example/Mol_MnO.dat Elapsed time(s)= 12.39 diff Utot= 0.000000000059 diff Force= 0.000000000136
14 input_example/Ndia2.dat Elapsed time(s)= 3.93 diff Utot= 0.000000000000 diff Force= 0.000000000000
Total elapsed time (s) 265.26