Serial and Parallel ANSYS Mechanical APDL Simulations

ANSYS-APDL-Macro-PeDALThere are times when you want to study the effects of varying parameters.  If you have an existing MAPDL script that is parameterized, the following procedure will allow you to easily run many variations in an organized manner. 

Let’s assume a parameterized MAPDL macro called build_solve that does something you want to simulate many times and has 2 variables called power and scale which are set with argument 1 and 2 respectively.  Running this macro with the classic interface, with power=30 and scale=2.5 would look like this:

build_solve,30,2.5

Next, create a MAPDL macro to launch all of the simulations.  This script could be named control.mac.  The first thing to do here is to create arrays of your parameters and assign values to them.  This example will vary power and scale.  Here are the arrays of values that will be passed to build_solve:

*dim,power,array,4

power(1)=10,20,40,80

*dim,scale,array,6

scale(1)=1,2,3,5,10,20

Most of the control.mac commands will be put inside of nested *do loops.  There will be a *do loop for each of parameters being varied.

*do,ii,1,4

*do,jj,1,6

Next, use *cfopen to set up the arguments to be passed to build_solve.  Each time through the *do loops will create a new run1.mac

*cfopen,run1,mac

  a=power(ii)

  b=scale(jj)

  *vwrite,a,b

  build_solve,%G,%G

*cfclose

One of the key features of this approach is to run anywhere and build directories below the working directory.  Use the /inquire command to store the current directory name.

/INQUIRE,dir_,DIRECTORY

Use *cfopen to create a string that will be used for the directory name.  By using the variables as part of the string, the directories will have unique names.  A time or date stamp could also be included in this string.  This macro is executed immediately to create the string dirnam for use in the commands subsequently.

*dim,power,array,4

*cfopen,temp1,mac

*vwrite,a,b

dirnam='power_%G_scale_%G'

*cfclose

/input,temp1,mac

Eventually, the resulting directory structure will look something like the image below.  Each directory will contain a separate simulation with the arguments of power and scale set respectively.

mapdl-script-automation

The last *cfopen creates a windows batch file which will (when executed)

  1. Create the new directory

  2. Copy all of the macro files from the working directory into the new directory (including run1.mac)

  3. Change into the new directory using CD

  4. Launch ansys in batch mode, in this case using a gpu and 12 cpus, using the run1.mac input and outputting to f.out

  5. Change back to the working directory (ready to do it all again)

The code for the windows batch file is:

*cfopen,rfile,bat

*vwrite,dir_(1),dirnam

MKDIR "%C\%S"

*vwrite,dir_(1),dirnam

COPY *.mac "%C\%S"

*vwrite,dir_(1),dirnam

CD "%C\%S"

*vwrite,

"C:\Program Files\ANSYS Inc\v150\ansys\bin\winx64\ansys150" -b -acc nvidia -np 12 -i run1.mac -o f.out

*vwrite,dir_(1)

CD "%C"

*cfclose

The last step is to run the windows batch file.  /sys is used to make this system call.  If the simulation is not well parallelized and you have enough licenses available, run the simulations in low priority mode immediately.  This will launch all of your simulations in parallel:

  • /sys,start /b /low rfile.bat

If the model is well parallelized (in other words, it will use your system’s gpu/cpus/RAM efficiently) or you only have 1 license available, launch the batch files in high priority mode and use the /wait option which will insure that windows waits for the job to finish before launching the next simulation.

  • /sys,start /b /high /wait rfile.bat

You can download and view the examples control.mac and build_solve.mac from this zip file: build_solve-control-macros.zip

Some Tips on Configuring RSM as a User

rsm1If you’re not familiar with it, RSM is the ANSYS Remote Solve Manager.  In short, it allows you to submit solutions from various ANSYS tools so they can be solved remotely, such as on a compute cluster, remote number cruncher, or perhaps just another computer that isn’t being used very much.  Note that there is no additional licensing or installation is required (other than perhaps ANSYS HPC licensing to take advantage of multiple cores).  RSM is installed automatically when ANSYS is installed; it just needs to be configured to be activated.

According to PC Revive, in version 14.5 and 15.0, there is a nicely documented Setup Wizard that helps with the setup and configuration of RSM on compute servers.  This setup wizard as well as the rest of the RSM documentation in the ANSYS Help does a great job of explaining RSM and what must be done to setup and configure it.  This Focus entry assumes that your crack IT staff has installed RSM on your compute machine(s) and has decided where the Compute Server will be (can be on your local machine or on your ‘number cruncher’ or on a different machine).  So, our focus here is on what needs to be done as a user to send your solutions off to the remote solver using RSM.

As an example, we have RSM 15.0 configured with the Compute Server on a remote computer named cs3a. The first time running RSM, using Start > All Programs > ANSYS 15.0 > Remote Solve Manager > RSM 15.0, we get the window shown here:

rsm2

Notice that it only shows our local machine (My Computer) and nothing about the actual remote computer on which we want to solve.

Therefore, we need to add the information on our cluster node which contains the compute server.

To do this, click on Tools > Options.  This is the resulting window.  Notice the Add button at lower left is grayed out:

rsm3

What it’s waiting for us to do is type in the name of our desired remote computer, like this:

rsm4

Now that a new name has been typed in the Name field, the Add button is active.  After clicking Add, we get this:

rsm5

After clicking OK, we will now see that the new remote computer has been added in the RSM window:

rsm6

The next step is to set your login password for accessing this computer.  Right click on the new hostname in the RSM window in the tree at left, and select Set Password.

rsma

Then enter your network login and password information in the resulting window:

rsmb

If your accounts are fully setup, at this point you can run a test by right clicking on the localhost item in the tree under the remote computer name and selecting Test Server:

rsm7

If the test is successful, you will see that the test job completed with a green checkmark on the folder icon in the upper right portion of the RSM window:

rsm8

If your login is not configured properly, you will likely get an error like this one shown below.  Notice that the upper right portion now states that the job has failed and there is a red X rather than a green checkmark on the folder icon.  By clicking on the job in the upper right panel, we can see the job log in the lower right panel.  In this case, it says that the login failed due to an incorrect password.

rsm9

The fix for the password problem is to ensure that the correct login is being accessed by RSM on the remote computer.  This is done from the RSM window by right clicking on the remote computer name and selecting Accounts.

rsm10

If your account and/or password are different on the remote computer than they are on your local machine, you will need to establish an alternate account so that RSM knows to use the correct login on the remote computer.  Right click on your account in the Accounts pane, and select Add Alternate Account:

rsm11

Enter your username and password for the remote computer in the resulting window.  Next, we need to associate that login with localhost on the remote computer.  This is down by checking the localhost box in the Compute Servers pane, like this:

rsm12

Another problem we have seen is that the user doesn’t have permission for ANSYS to write to the default solve directory on the remote computer.  In that case, the test job log will have an error like this:

rsm13

This fix in this case is to establish a solve directory manually, first by creating one on the remote computer, if needed, and second by specifying that RSM use that directory rather than the default.  The second step is accomplished in the RSM window via right clicking on the localhost item for the remote computer, then selecting Properties.  On the General tab, you should be able to change the Working Directory Location to User Specified, then enter the desired directory location as shown in the image below.  If that option is greyed out, either your password for the remote machine has not been entered correctly, or you are not part of the admin group on the remote computer.  In the case of the latter, either your RSM administrator has to do it for you, or you have to be granted the admin access.

rsm14

At this point, if the test server runs have completed successfully you should be ready to try a real solution using RSM.  We’ll use Mechanical to show how it’s done.  In the Mechanical editor, click on Tools > Solve Process Settings.  Here we will need to specify the remote computer and queue we’ll be using for the solution.  Click on the Add Remote button:

rsm15

In the resulting Rename Solve Process Settings button, type in a name for your remote solve option that makes sense to you.  We called ours RemoteSolve1.  This new option will now show up on the left side of the Solve Process Settings window:

rsm16

The next step is to type in the name of the Solve Manager over on the right side.  In our case, the Solve Manager is on computer cs3a.  Any queues that are available to RSM for this Solve Manager will show in the Queue field, after a brief period of time to make the connection.  In our case, the only queue is a local queue on cs3a.

rsm17

We are now ready to solve our Mechanical model remotely, using RSM.  Instead of clicking the Solve button in Mechanical, we will click on the drop down arrow to the right of the solve button.  From the dropdown, we select the remote solve option we created, RemoteSolve1:

rsm18

Assuming the solution completes with no errors, this job will show up in the RSM window with a status of Finished when it is done.

rsm19

The final step in this case is to download the results from the remote computer back to the client machine.  In the Mechanical editor, this is done by right clicking on the Solution branch and selecting Get Results as shown below.  Also note that you can monitor a nonlinear solution via Solution Information.  You’ll just need to right click during solution to have a snapshot of the nonlinear diagnostics brought back from the remote computer.

rsm20

We hope this helps with the setup and utilization of RSM from a user perspective.  There are other options and applications for RSM that we didn’t discuss, but hopefully this is useful for those needing to get ‘over the hump’ in using RSM.