This is a short help file on using the parallel electronic-structure code "NWChem" on systems at the Centre for Advanced Computing. This software uses MPI as a message-passing system and is (in principle) able to run on an arbitrary number of processors. Its ability to perform a very broad spectrum of molecular-structure calculations, ranging from CI to ab-initio molecular dynamics, makes it an interesting alternative to the standard electronic structure code Gaussian.
NWChem is an electronic-structure code that is suitable to perform complex calculations on molecular structure. It was specifically designed to perform well on high-performance parallel computers. The installation on the SunFire cluster of HPCVL employs the MPI message passing package for parallel execution.
NWChem allows, among others, the following calculations to be performed:
For a more complete list, see the official NWChem homepage and click on "capabilities".
Location of the program and setup
The NWChem program is located in the directory /opt/nwchem/bin. To access it, you have to use the usepackage command
which will set you up automatically.
Running NWChem from a command line
Like other electronic-structure programs, NWChem is run by supplying an input file that defines the system on which to perform a calculation (usually a molecule, or a group of molecules), and the method to use (i.e., the level of calculation, such as "Hartree-Fock", the basis set, and other details of the computation).
The variety of possible calculations is great, and so is the complexity of systems. It is impossible for us here to explain the format that a NWChem input file needs to have. This is explained in the User's Manual which is available online.
Here, we provide a simple sample input file which should be given the file extension .nw.
start h2o title "H2O, cc-pVDZ basis, SCF optimized geometry" geometry units au H 0.0000000000 1.4140780900 -1.1031626600 H 0.0000000000 -1.4140780900 -1.1031626600 O 0.0000000000 0.0000000000 -0.0080100000 end basis H library cc-pVDZ O library cc-pVDZ end scf thresh 1.0e-8 end task scf
This extension may be omitted when calling the program. NWChem creates typically a whole array of output files that are documented in the User's Manual. A general log is displayed on the console, and may be saved in a file by redirecting the standard output:
nwchem sample > sample.log
where we assume that your input file is called sample.nw and you want to save the log to a file sample.log.
Note that this is just the basic syntax of the program call. In practise you will use a parallel environment to execute the program (see next section). In fact, executing NWChem by just typing the above line will run it in serial mode.
NWChem is inherently parallelized and designed to scale well on a multi-processor machine or a cluster. The underlying parallel system is MPI (Message Passing Interface) which is a commonly available standard that runs on many platforms. Consult our MPI help file and follow some of the links in there if you want to have more information about MPI. Even if you want to use only one processor for your NWChem run (which sometimes is the best solution, particularly for smaller computations), you have to submit the program to a parallel environment. On our clusters, the relevant command is mpirun:
mpirun -np 8 nwchem sample > sample.log
This will run your sample.nw input file on eight processors. Note that you are only allowed to run NWChem this way for small test systems! For any production jobs, you have to submit the task to the scheduler (see next section.).
Submitting (parallel) NWChem jobs
NWChem jobs are to be submitted on the SW (Linux) systems via the GridEngine, which is a load-balancing software. To obtain details, read our GridEngine FAQ . For an NWChem batch job, this means that rather than issuing the command in the previous section directly, you wrap it into a GridEngine batch script.
Here is an example for such a batch script:
#$ -S /bin/bash #$ -q abaqus.q #$ -l qname=abaqus.q #$ -o sample.out #$ -e STD.err #$ -M hpcXXXX@localhost #$ -m be #$ -V #$ -cwd #$ -pe dist.pe 4 mpirun -np $NSLOTS nwchem sample
This script needs to be altered by explicitly replacing the entries that differ in your case. We suggest you use it as a template for all your NWChem runs. For details, consult our Scheduler Help File.
Note that there is no need in this script to redirect the standard output via the > operator. Instead, you define where the output goes to the GridEngine via the "#$ -o" command. In our case, we send it to a file called sample.out.
Email notification is set up through the "#$ -M" line. In the above example you need to replace XXXX by the actual 4 digits in your username, and place a file ".forward" with your email address into your home directory.
In the example we are executing with 4 processes. To choose a different number, alter the "#$ -pe" line in the script. For this example script to work you need to have set up the calling shell through the "use nwchem" command because the above script inherits all the environment settings (due to the "#$ -V" option).
The script is submitted to the GridEngine by typing
The advantage to submit jobs via a load balancing software is that the software will automatically find the resources required and put the job onto a set of processors that have a low load. This will help executing the job faster. Production jobs on our cluster must be submitted using GridEngine from a login node (sflogin0 or swlogin1), and executed under GE's control on the Linux production nodes without any need for you to log in.
NWChem is obtainable free of charge from the Pacific Northwest National Laboratory. To obtain your own copy, go here. NWChem is ditributed under an Open Source Educational Community License. Like with other software, HPCVL requires users who want to use NWChem, to read this agreement, and sign a statement that they have done so and will abide by its terms. You can fax the signed statement to (613) 533-2015 or scan/email it to firstname.lastname@example.org. You will then be included in a Unix user group that has access to the NWChem executables.
NWChem is a very complex software package, and requires practice to be used efficiently. We cannot explain it use in any detail here.