Running QE in MareNostrum5
In MN5, the compilation (not the version) of Quantum ESPRESSO is different than what I have worked with in TRUBA/ARF. Here, this is a MPI+OpenMP compilation of QE. So, when we load the quantumespresso/7.3.1 in GPP partition unlike the previous MPI compilation, now it shows:
Parallel version (MPI & OpenMP), running on 1120 processor cores
Number of MPI processes: 1120
Threads/MPI process: 1
ACC partition, not in GPP partition.
Pseudopotentials
For everyone's use, I put some pseudopotentials in the shared project storage: /gpfs/projects/etur37/pseudo/. Depending on what types of pseudopotentials one wants to use, the pseudo_dir can be set as:
pseudo_dir = "/gpfs/projects/etur37/pseudo/sg15/"
pseudo_dir = "/gpfs/projects/etur37/pseudo/sg15/pswfc/"
pseudo_dir = "/gpfs/projects/etur37/pseudo/sssp_efficiency/"
pseudo_dir = "/gpfs/projects/etur37/pseudo/sssp_precision/"
pseudo_dir = "/gpfs/projects/etur37/pseudo/gbrv/"
/gpfs/projects/etur37/pseudo/readme.txt for the explanation.
Hello world!
Previously, we have done an 8-atoms GaAs scf calculation using QE on 112 cores. The calculation took 20 seconds but the job was around 1 minute 30 seconds long. We didn't follow the suggested practice and ran the calculation from the /gpfs/home/ filesystem. Now I am going to try a little large system (64-atoms pristine ZnS) which should be done in a few minutes on around hundread cores, since in ARF, it took 1m 14s using 220 cores (2 nodes). Here is our 64-atoms ZnS input file (with converged parameters):
Click to see the full input file: scf.in
&CONTROL
calculation = 'scf'
outdir = './out/'
prefix = 'pure_ZnS'
pseudo_dir = "/gpfs/projects/etur37/pseudo/sg15/"
verbosity = 'high'
disk_io = 'low'
/
&SYSTEM
ecutwfc = 125
ibrav = 1
a = 10.893517000000000
nat = 64
ntyp = 2
occupations = 'fixed'
nspin = 2
tot_magnetization = 0.00
/
&ELECTRONS
conv_thr = 1d-08
/
ATOMIC_SPECIES
Zn 65.38 Zn_ONCV_PBE-1.2.upf
S 32.065 S_ONCV_PBE-1.2.upf
ATOMIC_POSITIONS crystal
Zn 0.0000000000 0.0000000000 0.0000000000
Zn 0.0000000000 0.2500000000 0.2500000000
Zn 0.2500000000 0.0000000000 0.2500000000
Zn 0.2500000000 0.2500000000 0.0000000000
S 0.1250000000 0.3750000000 0.3750000000
S 0.1250000000 0.1250000000 0.1250000000
S 0.3750000000 0.3750000000 0.1250000000
S 0.3750000000 0.1250000000 0.3750000000
Zn 0.0000000000 0.0000000000 0.5000000000
Zn 0.0000000000 0.2500000000 0.7500000000
Zn 0.2500000000 0.0000000000 0.7500000000
Zn 0.2500000000 0.2500000000 0.5000000000
S 0.1250000000 0.3750000000 0.8750000000
S 0.1250000000 0.1250000000 0.6250000000
S 0.3750000000 0.3750000000 0.6250000000
S 0.3750000000 0.1250000000 0.8750000000
Zn 0.0000000000 0.5000000000 0.0000000000
Zn 0.0000000000 0.7500000000 0.2500000000
Zn 0.2500000000 0.5000000000 0.2500000000
Zn 0.2500000000 0.7500000000 0.0000000000
S 0.1250000000 0.8750000000 0.3750000000
S 0.1250000000 0.6250000000 0.1250000000
S 0.3750000000 0.8750000000 0.1250000000
S 0.3750000000 0.6250000000 0.3750000000
Zn 0.0000000000 0.5000000000 0.5000000000
Zn 0.0000000000 0.7500000000 0.7500000000
Zn 0.2500000000 0.5000000000 0.7500000000
Zn 0.2500000000 0.7500000000 0.5000000000
S 0.1250000000 0.8750000000 0.8750000000
S 0.1250000000 0.6250000000 0.6250000000
S 0.3750000000 0.8750000000 0.6250000000
S 0.3750000000 0.6250000000 0.8750000000
Zn 0.5000000000 0.0000000000 0.0000000000
Zn 0.5000000000 0.2500000000 0.2500000000
Zn 0.7500000000 0.0000000000 0.2500000000
Zn 0.7500000000 0.2500000000 0.0000000000
S 0.6250000000 0.3750000000 0.3750000000
S 0.6250000000 0.1250000000 0.1250000000
S 0.8750000000 0.3750000000 0.1250000000
S 0.8750000000 0.1250000000 0.3750000000
Zn 0.5000000000 0.0000000000 0.5000000000
Zn 0.5000000000 0.2500000000 0.7500000000
Zn 0.7500000000 0.0000000000 0.7500000000
Zn 0.7500000000 0.2500000000 0.5000000000
S 0.6250000000 0.3750000000 0.8750000000
S 0.6250000000 0.1250000000 0.6250000000
S 0.8750000000 0.3750000000 0.6250000000
S 0.8750000000 0.1250000000 0.8750000000
Zn 0.5000000000 0.5000000000 0.0000000000
Zn 0.5000000000 0.7500000000 0.2500000000
Zn 0.7500000000 0.5000000000 0.2500000000
Zn 0.7500000000 0.7500000000 0.0000000000
S 0.6250000000 0.8750000000 0.3750000000
S 0.6250000000 0.6250000000 0.1250000000
S 0.8750000000 0.8750000000 0.1250000000
S 0.8750000000 0.6250000000 0.3750000000
Zn 0.5000000000 0.5000000000 0.5000000000
Zn 0.5000000000 0.7500000000 0.7500000000
Zn 0.7500000000 0.5000000000 0.7500000000
Zn 0.7500000000 0.7500000000 0.5000000000
S 0.6250000000 0.8750000000 0.8750000000
S 0.6250000000 0.6250000000 0.6250000000
S 0.8750000000 0.8750000000 0.6250000000
S 0.8750000000 0.6250000000 0.8750000000
K_POINTS automatic
4 4 4 0 0 0
We use the following job.sh file to perform the calculation.
#!/bin/bash
#SBATCH --job-name=test
#SBATCH --account=etur37
#SBATCH --partition=gpp
#SBATCH --qos=gp_ehpc
#SBATCH --ntasks=112
#SBATCH --nodes=1
#SBATCH --cpus-per-task=1
#SBATCH --time=0-01:00:00
#SBATCH --output=slurm-%j.sout
#SBATCH --error=slurm-%j.serror
#SBATCH --mail-type=ALL
#SBATCH --mail-user=abdul.muhaymin@bilkent.edu.tr
module load fftw/3.3.10
module load quantumespresso/7.3.1
srun pw.x -i scf.in > scf.out
The calculation ran successfully.