Difference between revisions of "HowTo:ansys"

From CAC Wiki
Jump to: navigation, search
(On Frontenac)
 
(24 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
= ANSYS Mechanical =
 
= ANSYS Mechanical =
This is a help file on using the Mechanical Engineering structural code "ANSYS Mechanical" on our systems. This software is only licensed for academic researchers who have prior training. The software is only made available to persons who belong to a specific Unix group. See details below.
+
 
 +
'''Important Note: This software is provided on the basis of a "hosting" model. This means that users who want to run this software on our cluster will have to supply a license that supports the run.'''
 +
 
 +
This is a help file on using the Mechanical Engineering structural code "ANSYS Mechanical" on our systems. This software requires a user-supplied license. The software is only made available to persons who belong to a specific Unix group. See details below.
  
 
{|  style="border-spacing: 8px;"
 
{|  style="border-spacing: 8px;"
Line 12: Line 15:
 
== Version ==  
 
== Version ==  
  
The most current version on our systems is Ansys-18.
+
The most current version on our systems is Ansys/ext2020R1. To see other versions, use '''Module spider ansys'''
  
 
== Location and Access ==
 
== Location and Access ==
Line 18: Line 21:
 
ANSYS Mechanical runs under the Linux operating system.  
 
ANSYS Mechanical runs under the Linux operating system.  
  
The program is located in /opt/fluent/ on swlogin1 ("SW cluster") and in /global/software/ansys on caclogin01 ("Frontenac").
+
The program is located in /global/software/ansys.
  
To use it, you have to be a trained University User. It is furthermore required that you [http://www.ansys.com/terms.htm read our licensing terms], and [http://cac.queensu.ca/wp-content/files/cac-ansys-statement.pdf sign a statement]. We will confirm your statement, and you will then be made a member of a Unix group fluent (so called for "historical" reasons), which enables you to run the software. [mailto:cac.help@queensu.ca Contact us] if you are in doubt of whether you qualify to run ANSYS on our system.
+
To use it, you have to be covered by a user-supplied license. It is furthermore required that you [http://cac.queensu.ca/wp-content/files/cac-ansys-statement.pdf sign a statement]. We will confirm the statement, and you will then be made a member of a Unix group fluent (so called for "historical" reasons), which enables you to run the software. [mailto:cac.help@queensu.ca Contact us] if you are in doubt of whether you qualify to run ANSYS on our system or if you are looking for options for access to a license.
  
 
== Licensing ==
 
== Licensing ==
  
The ANSYS license is "seat limited" and "process limited". At present, there are the following licensing limits on our systems:
+
To use ANSYS, you have to provide us with a license. The software is only accessible to users who are covered by such a license and are members of the "fluent" Posix group (so called for historical reasons). To be included in that group you need to complete a statement that you can [https://cac.queensu.ca/wp-content/files/cac-fluent-statement.pdf download here]. Note that prior inclusion in the fluent group is now void as the licensing terms for the software have changed. If you are in doubt, please contact us at [mailto:"cac.help@queensu.ca" cac.help@queensu.ca].
  
<pre>25 program runs plus 512 parallel processes</pre>  
+
The license is "seat limited" and "process limited".
 +
 
 +
<pre>The number of seats and process available to you depends on the license under which you are covered</pre>  
  
i.e. at most 25 separate sessions can be run simultaneously (serial or parallel). Each of these sessions can run up to 4 processes for a total of 100. In addition, it is possible to run up to 512 "parallel only" processes in total. One scenario would be 24 users have 24 process parallel jobs running, and another one with 36, thus using up all available resources. Not that the seats are shared with the CFD software Fluent.
 
 
|}
 
|}
  
Line 36: Line 40:
 
== Running ANSYS ==
 
== Running ANSYS ==
 
=== Setup ===
 
=== Setup ===
==== On the SW cluster ====
 
 
The setup for ANSYS on the SW cluster is done via '''usepackage'''. Simply type:
 
<pre>use ansys</pre>
 
on the workup node (swlogin1) or include this command in your setup (.bash_profile) file. This will set up the default (Ansys 18) version. Other versions can be setup through specification of the proper version, for instance
 
<pre>use ansys17</pre>
 
to set up the "Ansys 17.0" version.
 
 
==== On Frontenac ====
 
 
 
The setup for ANSYS on Frontenac is done via '''module'''. Type:
 
The setup for ANSYS on Frontenac is done via '''module'''. Type:
 
<pre>
 
<pre>
 
module purge --force
 
module purge --force
module load ansys
+
module load ansys/ext181
 +
export ANSYSLMD_LICENSE_FILE={port}@{license server address}
 
</pre>
 
</pre>
on the workup node (caclogin01) or include these commands in your setup (.bash_profile) file. Note that this is "purging" the present setup which may make the shell in which this done unusable for running other software.
+
on the workup node or include these commands in your setup (.bash_profile) file. Note that this is "purging" the present setup which may make the shell in which this done unusable for running other software. The setting of the environment variable ANSYSLMD_LICENSE_FILE points the system to the correct license server and port. This information is dependent on the user (substitute the information in the curly brackets).
  
 
==== Note ====
 
==== Note ====
  
You have to be in the '''fluent''' Unix group for this to work on either system, as access permissions prevent general users from accessing ANSYS software such as Fluent.  
+
You have to be in the '''fluent''' Unix group for this to work on either system, as access permissions prevent general users from accessing ANSYS software.
 
+
=== Interactive runs ===
+
 
+
(...under construction...)
+
  
 
=== Batch runs ===
 
=== Batch runs ===
  
ANSYS can (and usually must) be run in '''batch mode'''. Since you likely have access to Fluent on your local machines, most interactive work should be done there, whereas the computationally intensive runs can be executed on a parallel system such as ours. For this, data and commands are written into a text file written in '''ANSYS Parametric Design Language''' (APDL) which is used to specify the system and describe the Analysis to be performed.
+
ANSYS can (and usually must) be run in '''batch mode'''. Since you likely have access to ANSYS on your local machines, most interactive work should be done there, whereas the computationally intensive runs can be executed on a parallel system such as ours. For this, data and commands are written into a text file written in '''ANSYS Parametric Design Language''' (APDL) which is used to specify the system and describe the Analysis to be performed.
 
Here is the top of an input file (the full file is too long to be displayed here):
 
Here is the top of an input file (the full file is too long to be displayed here):
  
 
<pre>
 
<pre>
 
/prep7
 
/prep7
ET,1,SOLID185, ,2
+
ET,1,SOLID185, ,2
 
MP,EX,1,70e09
 
MP,EX,1,70e09
 
MP,EY,1,70e09
 
MP,EY,1,70e09
Line 105: Line 96:
 
=== Production runs ===
 
=== Production runs ===
  
To submit a production job on our clusters, '''you must use the scheduler'''. To obtain details, read through [[HowTo:Scheduler|Sun Grid Engine]] (SW cluster) or [[SLURM|SLURM]] (Frontenac). Production jobs that are run without scheduler will be terminated by the system administrator.
+
To submit a production job on our clusters, '''you must use the scheduler'''. To obtain details, read through the wiki page for [[SLURM|SLURM]] (Frontenac). Production jobs that are run without scheduler will be terminated by the system administrator.
 
+
==== On the SW cluster ====
+
 
+
For an ANSYS production job, the batch command is "wrapped" into a Grid Engine script that looks like this:
+
 
+
<pre>
+
#! /bin/bash
+
#$ -S /bin/bash
+
#$ -V
+
#$ -cwd
+
#$ -m be
+
#$ -M my_email@service.com
+
#$ -o STD.out
+
#$ -e STD.err
+
#$ -pe shm.pe 8
+
. /opt/fluent/ansys-18.1/setup_64bit.sh
+
ansys181 -np $NSLOTS -b -i testsys.txt -o test.out -j test
+
</pre>
+
 
+
Here we are running a case "testsys.txt" using 8 processors on a parallel machine. The output and any error messages from the system are re-directed to files called "STD.out" and "STD.err", respectively. Email notification is handled by the "#$ -m" and "#$ -M" lines.
+
 
+
Many Fluent jobs that you run on our machines are likely to be large, so you are likely to want to run things in parallel.
+
ANSYS offers several options to execute the solver in a parallel environment, i.e. on several CPU's simultaneously. presently the default option for such runs is shared memory.
+
 
+
To take advantage of the parallel capabilities, you have to call the program with additional command line options that specify the details of your parallel run, "-np" followed by the number of processors you want to employ. The number of processors "8" specified in our example script appears only once, after
+
 
+
<pre>#$ -pe shm.pe</pre>
+
 
+
which is where you let the Grid Engine know how many processors to allocate to run the program. The internal environment variable '''NSLOTS''' will automatically be set to this value and can then be used in the command line.
+
 
+
It is also necessary to source a setup file called '''setup_64bit.sh'''. This will set various environment variables and enable the program to properly interact with Grid Engine.
+
 
+
All processes are allocated within a single node. This means that the size of the job is restricted by the number of cores on a node. Once the script has been adapted (let's call it "ansys.sh"), it can be submitted to the Gridengine by
+
 
+
<pre>qsub ansys.sh</pre>
+
 
+
from the login node. Note that the job will appear as a parallel job on the Grid Engine's "qstat" or "qmon" commands.
+
 
+
==== On Frontenac ====
+
  
 
On Frontenac, the scheduler in use is SLURM. Here is a SLURM example script of an ANSYS production job:
 
On Frontenac, the scheduler in use is SLURM. Here is a SLURM example script of an ANSYS production job:
Line 150: Line 102:
 
<pre>
 
<pre>
 
#!/bin/bash
 
#!/bin/bash
#SBATCH --job-name=ANSYS_test
+
#SBATCH --job-name=ansys-test
#SBATCH -c 8
+
#SBATCH --mail-type=ALL
#SBATCH -t 00:30
+
#SBATCH --mail-user={email address}
#SBATCH --mem=1000
+
#SBATCH --output=STD.out
 +
#SBATCH --error=STD.err
 +
#SBATCH --nodes=1
 +
#SBATCH --ntasks=1
 +
#SBATCH --cpus-per-task=8
 +
#SBATCH --time=00:30
 +
#SBATCH --mem=1G
 
module purge --force
 
module purge --force
module load ansys
+
module load ansys/ext181
 +
export ANSYSLMD_LICENSE_FILE={port}@{license server address}
 
ansys181 -np $SLURM_CPUS_PER_TASK -b -i testsys.txt -o test.out -j test
 
ansys181 -np $SLURM_CPUS_PER_TASK -b -i testsys.txt -o test.out -j test
 
</pre>
 
</pre>
Line 161: Line 120:
 
Here we are running the file "testsys.txt" using 8 processors on a parallel machine. The output and any error messages from the system are re-directed to a file called "slurm-XXXXX.out" (where XXXXX is the job number).
 
Here we are running the file "testsys.txt" using 8 processors on a parallel machine. The output and any error messages from the system are re-directed to a file called "slurm-XXXXX.out" (where XXXXX is the job number).
  
The -t option is used to specify a time limit. If it is omitted you are assigned a default limit. It is best to specify this limit, and choose it to be slightly longer than the largest expected execution time. This will make the job harder to schedule, but it will ensure that the job is not terminated before it finishes. Note that time limits are "hard", i.e. the job will be stopped when it exceeds its limit. This is necessary to make efficient scheduling possible.
+
The --time option is used to specify a time limit. If it is omitted you are assigned a default limit. It is best to specify this limit, and choose it to be slightly longer than the largest expected execution time. This will make the job harder to schedule, but it will ensure that the job is not terminated before it finishes. Note that time limits are "hard", i.e. the job will be stopped when it exceeds its limit. This is necessary to make efficient scheduling possible.
  
 
The --mem option is used to specify a memory limit. If it is omitted you are assigned a default limit. It is best to specify this limit, and choose it to be slightly larger than the largest expected memory usage. This will make the job harder to schedule, but it will ensure that the job is not for exceeding its memory allocation. Note that memory limits are "hard", i.e. the job will be stopped if it exceeds its allocated memory. This enable efficient memory allocation.
 
The --mem option is used to specify a memory limit. If it is omitted you are assigned a default limit. It is best to specify this limit, and choose it to be slightly larger than the largest expected memory usage. This will make the job harder to schedule, but it will ensure that the job is not for exceeding its memory allocation. Note that memory limits are "hard", i.e. the job will be stopped if it exceeds its allocated memory. This enable efficient memory allocation.
Line 167: Line 126:
 
Parallel jobs of longer runtime should only be run in batch using SLURM. The number of processors "8" specified in our example script appears only once, in
 
Parallel jobs of longer runtime should only be run in batch using SLURM. The number of processors "8" specified in our example script appears only once, in
  
<pre>#SBATCH -c 8</pre>
+
<pre>#SBATCH --cpus-per-task=8</pre>
  
 
which is where you let SLURM know how many processors to allocate to run the program. The internal environment variable '''SLURM_CPUS_PER_TASK''' will automatically be set to this value and can then be used in the ansys command line.
 
which is where you let SLURM know how many processors to allocate to run the program. The internal environment variable '''SLURM_CPUS_PER_TASK''' will automatically be set to this value and can then be used in the ansys command line.
Line 186: Line 145:
 
ANSYS Mechanical is a complex software, and requires some practice to be used efficiently. We can not explain it use in any detail here.
 
ANSYS Mechanical is a complex software, and requires some practice to be used efficiently. We can not explain it use in any detail here.
  
The documentation for Fluent can be access from inside the program GUI.
+
The documentation for ANSYS can be access from inside the program GUI (WorkBench).
  
The documentation is subject to the same license terms as the software itself, i.e. you have to be signed up as a Fluent user in order to access it.
+
The documentation is subject to the same license terms as the software itself, i.e. you have to be signed up as a user in order to access it.
  
 
If you are experiencing trouble running a batch command script, check carefully if the sequence of commands is exactly in sync with the program. This might mean typing them in interactively as a test. If you have problems that you cannot resolve through the documentation, contact user support by sending [mailto:cac.help@queensu.ca email to cac.help@queensu.ca].
 
If you are experiencing trouble running a batch command script, check carefully if the sequence of commands is exactly in sync with the program. This might mean typing them in interactively as a test. If you have problems that you cannot resolve through the documentation, contact user support by sending [mailto:cac.help@queensu.ca email to cac.help@queensu.ca].
 
|}
 
|}

Latest revision as of 18:59, 3 February 2021

ANSYS Mechanical

Important Note: This software is provided on the basis of a "hosting" model. This means that users who want to run this software on our cluster will have to supply a license that supports the run.

This is a help file on using the Mechanical Engineering structural code "ANSYS Mechanical" on our systems. This software requires a user-supplied license. The software is only made available to persons who belong to a specific Unix group. See details below.

What is ANSYS Mechanical ?

ANSYS Mechanical is a Mechanical Engineering software that uses finite element analysis (FEA) for structural analysis. It covers a large range of applications ranging from geometry preparation to optimization. You can model advanced materials, complex environmental loadings and industry-specific requirements in areas such as offshore hydrodynamics and layered composite materials.

It can be used interactively and supplies a graphical user interface. It can also run in batch mode, if the required time for solving a problem is too long for interactive use. The latter situation is the standard if you are using it on CAC machines.

Version

The most current version on our systems is Ansys/ext2020R1. To see other versions, use Module spider ansys

Location and Access

ANSYS Mechanical runs under the Linux operating system.

The program is located in /global/software/ansys.

To use it, you have to be covered by a user-supplied license. It is furthermore required that you sign a statement. We will confirm the statement, and you will then be made a member of a Unix group fluent (so called for "historical" reasons), which enables you to run the software. Contact us if you are in doubt of whether you qualify to run ANSYS on our system or if you are looking for options for access to a license.

Licensing

To use ANSYS, you have to provide us with a license. The software is only accessible to users who are covered by such a license and are members of the "fluent" Posix group (so called for historical reasons). To be included in that group you need to complete a statement that you can download here. Note that prior inclusion in the fluent group is now void as the licensing terms for the software have changed. If you are in doubt, please contact us at [mailto:"cac.help@queensu.ca" cac.help@queensu.ca].

The license is "seat limited" and "process limited".

The number of seats and process available to you depends on the license under which you are covered

Running ANSYS

Setup

The setup for ANSYS on Frontenac is done via module. Type:

module purge --force
module load ansys/ext181
export ANSYSLMD_LICENSE_FILE={port}@{license server address}

on the workup node or include these commands in your setup (.bash_profile) file. Note that this is "purging" the present setup which may make the shell in which this done unusable for running other software. The setting of the environment variable ANSYSLMD_LICENSE_FILE points the system to the correct license server and port. This information is dependent on the user (substitute the information in the curly brackets).

Note

You have to be in the fluent Unix group for this to work on either system, as access permissions prevent general users from accessing ANSYS software.

Batch runs

ANSYS can (and usually must) be run in batch mode. Since you likely have access to ANSYS on your local machines, most interactive work should be done there, whereas the computationally intensive runs can be executed on a parallel system such as ours. For this, data and commands are written into a text file written in ANSYS Parametric Design Language (APDL) which is used to specify the system and describe the Analysis to be performed. Here is the top of an input file (the full file is too long to be displayed here):

/prep7
ET,1,SOLID185, ,2
MP,EX,1,70e09
MP,EY,1,70e09
MP,EZ,1,60e09
MP,NUXY,1,0.33
MP,NUYZ,1,0.30
MP,NUXZ,1,0.30
MP,GXY,1,26.5e09
MP,GYZ,1,22e09
MP,GXZ,1,22e09
*AFUN,DEG
thetax=0.000000
thetay=0.000000
fx= 0
fy= -1000
fz= 0
fxp= fx
fyp= fy*(cos(thetax)) - fz*(sin(thetax))
fzp= fy*(sin(thetax)) + fz*(cos(thetax))

[...]

Let's call this file "testsys.txt". The analysis can now be performed by calling ANSYS directly from the command line

ansys181 -b -i testsys.txt

In this case, output is sent to the screen, and output files are given the default name "file.*". No further input from the user is required. Once everything works you could submit this job into the background (using bash) by typing

ansys181 -b -i testsys.txt > test.out 2>&1 & 

This would redirect standard output and standard error to test.out. The point is that ANSYS is run non-interactively this way, i.e. we can use the same technique to submit a production job to the scheduler, as shown in the next section.

Production runs

To submit a production job on our clusters, you must use the scheduler. To obtain details, read through the wiki page for SLURM (Frontenac). Production jobs that are run without scheduler will be terminated by the system administrator.

On Frontenac, the scheduler in use is SLURM. Here is a SLURM example script of an ANSYS production job:

#!/bin/bash
#SBATCH --job-name=ansys-test
#SBATCH --mail-type=ALL
#SBATCH --mail-user={email address}
#SBATCH --output=STD.out
#SBATCH --error=STD.err
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=8
#SBATCH --time=00:30
#SBATCH --mem=1G
module purge --force
module load ansys/ext181
export ANSYSLMD_LICENSE_FILE={port}@{license server address}
ansys181 -np $SLURM_CPUS_PER_TASK -b -i testsys.txt -o test.out -j test

Here we are running the file "testsys.txt" using 8 processors on a parallel machine. The output and any error messages from the system are re-directed to a file called "slurm-XXXXX.out" (where XXXXX is the job number).

The --time option is used to specify a time limit. If it is omitted you are assigned a default limit. It is best to specify this limit, and choose it to be slightly longer than the largest expected execution time. This will make the job harder to schedule, but it will ensure that the job is not terminated before it finishes. Note that time limits are "hard", i.e. the job will be stopped when it exceeds its limit. This is necessary to make efficient scheduling possible.

The --mem option is used to specify a memory limit. If it is omitted you are assigned a default limit. It is best to specify this limit, and choose it to be slightly larger than the largest expected memory usage. This will make the job harder to schedule, but it will ensure that the job is not for exceeding its memory allocation. Note that memory limits are "hard", i.e. the job will be stopped if it exceeds its allocated memory. This enable efficient memory allocation.

Parallel jobs of longer runtime should only be run in batch using SLURM. The number of processors "8" specified in our example script appears only once, in

#SBATCH --cpus-per-task=8

which is where you let SLURM know how many processors to allocate to run the program. The internal environment variable SLURM_CPUS_PER_TASK will automatically be set to this value and can then be used in the ansys command line.

All processes are allocated within a single node. This means that the size of the job is restricted by the number of cores on a node. Once the script has been adapted (let's call it "ansys.sh"), it can be submitted to SLURM by

sbatch ansys.sh

from the login node. Note that the job will appear as a parallel job on the "squeue" command.

Further Help

ANSYS Mechanical is a complex software, and requires some practice to be used efficiently. We can not explain it use in any detail here.

The documentation for ANSYS can be access from inside the program GUI (WorkBench).

The documentation is subject to the same license terms as the software itself, i.e. you have to be signed up as a user in order to access it.

If you are experiencing trouble running a batch command script, check carefully if the sequence of commands is exactly in sync with the program. This might mean typing them in interactively as a test. If you have problems that you cannot resolve through the documentation, contact user support by sending email to cac.help@queensu.ca.