Running an APDL Command Snippet for Every Load Step in ANSYS Mechanical

This week went by very fast, and I never got time to do the more advanced article on tables to follow up on last week’s article.  So I was going to give up till someone stopped by my office to ask a question and I thought my simple and clever answer would make a nice quick, but useful, posting.

What he need to do was apply fairly complicated loading over multiple substeps. Do some *get’s, calculate some stuff, then apply a load. I immediately thought of a trick we used in the early days of ANSYS Mechanical (before it was called ANSYS Mechanical) where we would put in a script that redefined the solve command as nall (*abbr,solve,nall).  You then used your own code to do the solves.

This made us feel very smart and clever.

However, something in the corner of my brain was saying “dumb and silly.” So I fired up 14.0 and realized that my brain was right, you don’t have to trick ANSYS Mechanical any more. The developers now allow you  to specify load steps and such for preprocessing command objects.  I should know this because I did a seminar on APDL Command Objects. .

Darn no article for this week, it was already covered. 

But just to make sure I looked through the PowerPoint and found that the ability wasn’t covered. Yipee! I have an article, now to stretch it out make it look important!

If you insert a command object into your model setup:


You end up with a Details view like so:


Under definition you can set “Step Selection Mode”  This simply lets you determine if the APDL code in the command object is applied every load step (All), at the first (First), at the last (Last), or if the command object is only applied to a specific load step number (By Number). 

  If your complicated loading/modification to your model is the same commands for every load step, pick All and enter your commands. If it varies by load step in some way, you have two choices. You can write a set of commands for each load step, or you can write a macro that uses a *get,nmstp,active,,solu,ncmls and then use logic to figure out what you need to do.

So, pretty simple, but it opens up a lot of possibilities when you need to do some simple tweaks during a multi-step solve.

There, now I don’t feel like a looser for not doing an article this week.

What Every User Should Know About Tables in ANSYS Mechanical APDL

I was having a discussion with a user who is very experienced with a FEA tool other than ANSYS. He wanted to define some properties with respect to time and his rotational speed and wanted to know how hard it would be to write a custom routine in ANSYS to do that.  I immediately explained that it was not that hard, as long as you have the right compiler. Then I realized that you did not need to compile any code. Unlike that other software, we have tables in ANSYS and you can use those to interpolate data relative to some other value.  And, now that it is Friday morning and we still do not have a FOCUS article for the week, I thought it would be a good time to review the basics of Tables in APDL.

ANSYS Mechanical (Workbench) users, do not leave and go back to reading TMZ.  This is useful to you as well because you can use code snippets in ANSYS Mechanical to define some very sophisticated loads without ever getting into Mechanical APDL.  One of those cool powerful things you can do in Mechanical. In fact, if you look at the input file that ANSYS Mechanical sends to the solver, it is often full of tables that ANSYS Mechanical makes.

This article will just touch on the simple aspects of tables that every user should know. There is a lot more you can do with tables, but we will save that for future articles.

The Basics

The APDL language has three type of parameters: variables (single numbers or 8 character strings), arrays, and tables.  variables and arrays are just like variables and arrays in most programming languages.  But tables are unique in that the indices are real numbers rather than integers.  And when you refer to a value in a table the program does a linear interpolation between the numbers you supply to get the value at that location. Think of it as a graph where instead of points you have a line:


What makes it even better, is that the table can be multiple dimensions. So you can make a value dependent on a location in 3D space, val(x,y,z), space and time, val(x,y,z,t), or even some input you need to use, val(x,y,z,t,myVal). 

You get an interpolated value by simply using it in a formula:

x = 2.4
y = 1.66
z = 23.5
frc = frc_tbl(x,y,z)

Also, many commands in ANSYS Mechanical APDL take a table as an argument. And the solver will input proper index values at solve time.  The simples example of this is a nodal force using the F command:


For each substep, the solver will interpolate a force value for node 47 in the X direction based the value of things like time, frequency, position, temp, and such for the current substep. More on how to define what values to use as an index below.

One other key thing to know about tables before we get into the details is the way they work.  What ANSYS does is take an actual array, and add a 0 column, row or plane to the array. So instead of going from 1 to 10, the array goes from 0 to 10.  And the index values are stored in this 0 row, column, or plane.  So to see the values using the *STATUS command, you have to tell it to start listing at 0 with the IMIN, JMIN, or KMIN arguments:


Another good way to look at a table is using the *VEDIT (Parameters->Array Parameters->Define/Edit->Edit.)


Defining a Table

The hardest part of defining a table is defining the index and values.  Arrays are simple, you just define you r size with a *DIM and then supply a value for each integer index value:

val(1) = 12.4,15.6,18.5,12.4,12.4,5,3.2





For tables, you need three steps: define the table with *DIM, define the indices (the axes), then provide the values.

val(1) = 12.4,15.6,18.5,12.4,12.4,5,3.2

Note that in the *DIM command, you have to specify that this is a table with TABLE.  For array’s you can say ARRAY or leave it blank, because ARRAY is the default. 

The next command, *TAXIS, is the important one. It does not call several cabs to pick you up.  It defines the Table AXIS… get it, TAXIS.  The first argument is the name of the table you want to fill, with the index put in for the row you want to start on, and the second is the index you want to fill.  Then you give the actual values.

Finally we supply the actual values just like in an array, but each value corresponds to the index values specified in the *TAXIS command.

The above example results in:




This example is for a 1D table. A 3D table works the same, you just need to define 3 axes and 3 columns of values.  Note that the first argument on the *TAXIS command is the column number that the axis refers to.

temptab(1,1,1) = 10,100,10
temptab(1,2,1) = 12,150,10
temptab(1,3,1) = 10,90,7
temptab(1,1,2) = 12,120,12
temptab(1,2,2) = 15,180,15
temptab(1,3,2) = 17,90,12
temptab(1,1,3) = 20,200,20
temptab(1,2,3) = 22,250,20
temptab(1,3,3) = 20,290,27

This will produce:


If you find the code a bit confusing, we recommend that you use Parameters->Array Parameters->Define/Edit->Add… to create your tables, then look at the log file to get the commands.

Using Tables with Loads

The real value of tables is their use with commands that accept a table as a value, and this is usually some sort of load.  The help for a given command will tell you if it takes a table as an argument. If it does not, simply put the command in a *do-loop.

When you specify a table, you need to tell the command interpreter that it is a table and not variable or a string by placing the command in side percent signs:  f,frc_nodes ,fx ,%fx_load%.

But how do you tell the solver what solver values your indices refer to?  If you want the force applied based on the X,Y,Z position of the nodes in the component frc_nodes, you need to say which column in your table is X, Y, and Z.  You do that with the *DIM command:

*DIM, Par, Type, IMAX, JMAX, KMAX, Var1, Var2, Var3, CSYSID

Var1, Var2, and Var3 are predefined keywords that are called Primary Variables.  The possible values are:

FREQ Frequency
X X location of entity
Y y location of entity
Z z location of entity
TEMP Temperature
SECTOR Cyclic sector number

So to define loads that vary with Z position and time you would do:

val(1,1) = 12.4,17,10.5
val(1,2) =8.3,7.4,12.8

At each substep the program will go through each node in FRCNDS and get its Z location and the current time and interpolate a force and apply it to that node.  I do not know about you, but I think that is pretty slick.

The obvious next question is how do I deal with a different coordinate system?  The last argument on *DIM is CSYSID. You supply a local coordinate system number here and the program will use that coordinate system to figure out the position of the entity it is applying a load to.

Other Stuff

We strongly recommend you look at the *DIM and *TAXIS commands in the help. Also read section 3.10 of the ANSYS Parametric Design Language Guide, specifically the section on Table Type Array Parameters.

Some other things you can do with tables that we hope to cover in the future, but that you can also figure out on you own using the help are:

  • You can create 4 and 5 dimension tables using TAB4 and TAB5 for TYPE in the *DIM command.  Other things need to be done as well, it gets complicated. You can find an example at:

    // Basic Analysis Guide // 2. Loading // 2.5. Applying Loads

  • You can nest tables inside tables.  So you can have one table calculate a value based on primary values (X,Y,Z, FREQ, TIME, etc) then use the result of that interpolation to interpolate from another table.   One use of this would be if a load varies with rotational velocity, and rotational velocity varies with time.  So you define an RPM table that is dependent on TIME, then use the table to get a load.  (maybe that will be next weeks article…)

And we will finish with a picture of a really cool table and bench I found on line:  I want to get this for my patio.

smooth 4 660x396

An Ocean in the Desert: What is AQWA?

Maybe you’ve seen AQWA show up in the list of ANSYS products to be installed on your computer, or maybe you’ve seen it as a topic in the ANSYS Help System and you otherwise wondered about it.  Is it related to Aqua Man or perhaps Aqua Lung?  Not at all.  If you are not familiar with it, AQWA is a tool for simulating wave and current as well as wind loads on marine vessels and structures.

Truthfully, although we are located in the Sonoran Desert, we have some great lakes within a short drive of Phoenix, let alone Tempe Town Lake just a few miles from PADT, and lakes Powell and Mead up on our northern and western borders with Utah and Nevada offer miles and miles of incredible scenery.  However, most of our rivers are dammed for water storage and irrigation so it’s not uncommon to see river beds with no water in them for most of the year.  A standard joke is that Arizona is the place where rivers and bodies of water are not associated with each other.


The bottom of the Salt River, aka Rio Salado, which flows through Phoenix (Sometimes)


The Nav System Knows it as a River

Yes, it does rain in Phoenix.  Just not often.  This storm caused over an inch of rain in some places:


This is more of a typical day.  No rain in sight:


You may be wondering why we have an interest in AQWA here in the Sonoran Desert.  The short answer is that we support an organization working on alternative energy sources, including offshore wave power. As a result, we’ve had to become familiar with the ANSYS AQWA suite of tools.

Interaction Between Waves and a Floating Structure

Incident Waves on a Moving Ship:


At version 14.0, the main part of AQWA, hydrodynamic diffraction, is integrated with ANSYS Workbench.  In a hydrodynamic diffraction analysis, we are calculating the response on our structure due to incoming and receding waves.  The effects on the water due to its interaction with the structure are also included.  It’s also possible to perform a hydrodynamic time response analysis within the Workbench framework.  Interaction with permanent structures such as piers and breakwaters can be included as well.

The use of Workbench means we can use ANSYS DesignModeler to construct and edit our geometry, including a slice at the waterline and formation of a multi-body part, both of which are needed for AQWA.  It is somewhat integrated with Workbench Mechanical, in that it’s possible to map pressure and inertia loads from AQWA into Mechanical for a detailed structural analysis, but it’s somewhat of a manual process currently.  For those familiar with the ANSYS APDL command language, it’s fairly straightforward.

Geometry Split at the Water Line in DesignModeler:


AQWA can also be run in stand-alone mode, which opens up additional capabilities while still taking advantage of DesignModeler surface geometry or another source.  The various modules have creative names such as AQWA-Line, AQWA-Librium, AQWA-Fer, AQWA-Drift, and AQWA-Naut.  Besides hydrodynamic diffraction, one can look at the effects of mooring lines, cables and tethers and varying wave and wind loads on structures.  Special elements are included which facilitate the simulation of ‘stingers’ or articulated trusses which are used to connect underwater piping to floating vessels.  Fenders can also be modeled to allow for interaction between floating and floating or floating and permanent structures that may come into or out of contact.

Geometry of the structures modeled consists of surface models which are meshed with shell elements.  Meshes must be relatively coarse compared to what most of us who normally perform structural analyses are used to.  Small geometric details usually need to be omitted to keep the mesh sizes down, but that’s not really a problem as we are trying to accurately predict interaction between the fluid and the structures, not stress concentrations or other localized results within AQWA.

In addition to plots and animations of interacting waves and pressure distributions, many other results quantities are available including cable forces, RAO’s (response amplitude operators), drift coefficients, shear force and bending moments, and other quantities used in the industry. These quantities are useful to those developing sea-going vessels as well as for those in the oil and gas industry involved in developing Floating Production, Storage, and Offloading (FPSO) structures and Tension Leg Platforms (TLP’s), etc.

Forces on Two Mooring Lines vs. Time:


Floating Structure Lateral Position vs. Time:


If any of this looks like a tool that would be useful for the types of projects you work on, by all means contact your local ANSYS provider for more information.  You’ll want to plan on taking the AQWA training class, which we have found very useful.

One of the lakes along the Salt River.  Indeed there is water in the desert, if no ocean:


Solution Information: Monitoring your Solves in ANSYS Mechanical

There is a folder with a big fat exclamation point on the top of the Solutions branch in ANSYS Mechanical. It is called “Solution Information.”  Most users click on it after their run is done and maybe look at the output from the ANSYS Mechanical APDL solve. 


And it is very handy to check your output when your job is done:


But this feature has an exclamation point on its folder icon for a reason! It is a very useful tool! While doing tech support we have found that users often do not take advantage of the information displayed here, and if they did so their ANSYS Mechanical experiences would be more efficient and even more enjoyable.

Solver Output

When you click on the Solution Information branch in Solution, the graphics window turns to the worksheet window and you see Solver Output.  This is the jobname.out file that ANSYS Mechanical APDL creates as it solves, and it is full of useful information.  The window updates at a user defined interval, the default of 2.5 seconds seems to work well. 

It is a good idea, even for a static run, to watch this window as things solve.  It tells you where the solver is in the solver process, shows any warnings that might pop up, and lists the key information about your model and solver settings.

At first the information may seem a bit overwhelming. But give it time, study it, understand what each piece of information is and what it is telling you.  Users who watch and understand the Solver Output when they solve understand their models better, and debug problems much faster.

Non-Linear Graphs

The down side of the solver listing is that it is a text file. Text files are great for showing information at a certain portions of your run, but are not so great for comparing multiple points.  But graphs are.  And the same command can be changed to show all sorts of useful information about non-linear runs.


The list of available graphs varies depending on what type of solve you are doing.  The most common values to look at are the Force Convergence for structural. 


Take a look at this image.  As you can see there are two graphs. The top one plots the convergence information you want to see vs. the number of iterations.  The bottom graph shows time vs cumulative iterations.  Notice that there is data being graphed, force convergence and criteria in this case.  But there are also vertical dashed lines.  These give you feedback on what events happen and at what iteration they occurred.  Mostly they tell you that a substep or a load step converged. 

You can watch these non-linear graphs while your model is running, or after the run to see what actually happened.  I like to watch them as I solve because, honestly, it seems like the runs go faster. You find yourself watching that magenta line go up and down hoping it will go under the light blue line, and cheering when it does.

Take a look at the other types of non-linear graphs you can view and think about the impact of the data towards your run.  As an example, if you see a lot of vertical lines indicating that you are bisecting, then you should look at setting up more substeps on each loadstep.  The same thing if you see the convergence taking a long time.  Such information can not only help converge a model that is having problems, but it can help you set up future runs such that they converge faster.

If you click on an item that there is no data for, you get a nice “No data to display” message. You will also get this before the data is available.


Result Tracker

When you click on the Solution Information branch you will notice a “Solution Information” ribbon bar show up at the top of the window. 


This allows you to define information you want to track while solving, things like displacement, gap on contacts, or energy.  The values update as the problem is being solved, providing some nice insight into what your model is doing. 


To use it, RMB on Solution Information or pull down the Result Tracker menu.  Only results that are applicable to your solve will be available.  Once you pick the ones you want, you need to specify the geometry you want to apply it to, if applicable. Usually it needs to be a vertex or a contact/joint. Do this in the normal way then fill out the rest of the detail view. As an example, if we want the deflection on the corner of an object, you pick the vertex on the corner then specify the axis you want the information in.

Note: You can not add a result tracker object during or after a solve, you have to do it up before you solve. So do not think about using this capability as a probe.

You can select as many of the objects as you want and plot them all at the same time, which is very handy. Once the run is done, you can save the graph as an image or export the data to a comma delimited file or as an Excel file.

We recommend you always set up a Result Tracker for tricky contact pairs and for any significant deflection that tells you a lot about your model.

FE Connection Visibility

The last feature in the Solution Information branch is really not solution information, but it kind of is.  If you click on the branch you will notice a “FE Connection Visibility” area in the detail view. You will also find a “Graphics” tab under the Worksheet tab:


You use these tools to see things like beams, constraints and springs that are added to your model before the solve in ANSYS Mechanical APDL.  So they can only really be seen as post processing entities.  By default, all types are shown. But you can change the selection under “Display” in the details view to just show specific ones.  You can also change the thickness of the display and do lines or points. 

What is especially important about this is that beams, Constraint Equations and springs that you add in code snippets will also show up.

To see the entities, simply click on the Graphics tab:image

Conclusions and Recommendations

It is probably true that you could use ANSYS mechanical your whole career and never use this feature. but your career will be much more stressful and much less enjoyable than if just make their use part of your normal everyday process.  It can save hours of trial and error debugging.

Plus, the reality is that while you are solving you could watch Justin Bieber videos on YouTube, or you can watch your model converge.  That sounds like a much better way to go… watching the model convergence, of course… yea… cause Justin Bieber is lame… and… well, you know.


Surface Projection Based Contact Detection

The ANSYS Mechanical APDL solver has a lot of contact options.  KEYOPT(12) this and KEYOPT(4) that.  I’m sure we all read Chapter 3.9 of the Contact Technology Guide to understand and remind ourselves on all these options.  I know here at PADT we always do so on the third Tuesday of every month… OK, maybe not.

But as fate would have it I was looking something up in there the other month and came across one of the newer options: Surface Projection Based Contact.  It is on of those little options that can make a big difference, so I thought I would put it on the list of potential Focus articles, and it just bubbled to the top.

Contact and Contact Detection

The way contact works in an FEA program is that you define a contact and a target. The program goes in and calculates if a point on the contact surface is near, touching, or inside the surface or point that is the target. If they are touching or inside forces are calculated acting on the contact points in the normal and tangential (friction) directions.  You use a slew of options to determine what that algorithm looks like. But hey all need to know what the gap or penetration is between the contact and target surfaces.

What is important for this article is how the program chooses those contact points to look at and, if needed, calculate forces on.

The default (KEYOPT(4) =0) is to use the gauss points on the elements.  Here is a great illustration from the online help:


Gauss points are good because there are more per element than nodes and calculations are done at the gauss points anyhow. The down side is that if you have a lot of curvature or even a corner, the true surface of your contacting part can penetrate into the target.

For those times when you do need to control things at the nodes, you can use KEYOPT(4) = 1 or 2. 


The down side is that the program now has to do a lot more calculations to make things happen at the nodes, and it has to calculate normals for the contact or target surface. Plus there are fewer points.  But the worst thing that can happen is node drop off if your mesh is not refined enough:


KEYOPT(4) = 3

Sometimes using points for contact results can cause problems. The node penetration or drop off being good example. it can also be an issue if there is a lot of “overhang” on an element.  So the developers at ANSYS have added a third option, the Surface Projection Based Contact Method, or KEYOPT(4) = 3.  What the method does is look at the element faces on each side of the pair, contact and target, and calculate the union of each.  If we look at the following example you can see it.  The green mesh are the contact surface elements, the cyan mesh is the target.  If we put them on top of each other, each unique area becomes a surface projected contact area, shown in the third image.


 image             image


So instead of calculating penetration/gap at each integration point or node, it calculates an average value for each overlapping area.  This can really make a big difference when you have the contact hanging off of the target, as shown in this illustration:



So the three big advantages of this approach are:

  1. In most cases it provides a more accurate calculation of contact traction and stresses for the underlying elements.
  2. It is less sensitive to which side you designate as the target and which ones you designate as the contact surface. This is because you end up with the same projected areas.
  3. Moment equilibrium is satisfied if there is an offset between the target and contact surface and friction is turned on.
  4. Contact forces do not “jump” when a contact node slides off of the edge of a target surface.


No surprise, it is not perfect or it would be the new default. The disadvantages are:

  1. The method is computationally more expensive because you usually have more contact areas then you would have contact points with the other method, especially if the target mesh is refined giving you more areas.
  2. The penetration or gab is an average value so when a model has corner or edge contact, the nodes on the edge or corner will penetrate into the target surface.  This is similar to what you see with the gauss point detection method. More refinement usually solves this.
  3. Free thermal surfaces don’t work with it. (What is a free thermal surface, I forgot as well.  It is a feature in thermal contact, Chapter 7.1 of the Contact Technology Guide. It is now on the Focus to-do articles list)
  4. You can not use a primitive for a rigid target. This makes sense if you think about it. If you use a primitive there are no element faces to find the overlap on.
  5. 3D contact and target elements can not have partially dropped midside nodes.  They either all have to be there or none.

ANSYS Mechanical

Although we have been talking about how things work in the solver, and the KEYOPTs used in Mechanical APDL, you can still access this through ANSYS Mechanical.  Any contact par will have an Advanced portion of its details view. In there you can access most of the KEYOPTs, including the Detection method, which is KEYOPT(4).


And here is the drop down for Detection Method. The last one, Nodal-Projection Normal from Contact is actually Surface Based Contact Projection:




The best way to get a feel for this is to find a couple of models you have already run with one of the other contact methods and run them with this option turned on and take a look at run time per iteration,  number of iterations for convergence and the resulting stresses in the contact area.    If you have flat, fairly refined contact/target meshes with no overhanging, you should just see things take longer.  But if you do have a curvy surface, a corner, or a coarse mesh you should see better performance and accuracy.

Webinar Files: Moisture Diffusion Modeling with ANSYS at R14 and Beyond

On May 24, 2012 Matt Sutton gave a well attended webinar on the new moisture diffusion modeling capabilities in ANSYS Mechanical APDL at R14.


Although we didn’t get a successful recording for this one (Matt has been chastised and done his penance for forgetting to turn on the recording…) we do have a PDF of the PowerPoint and a copy of the sample macro he used:

Coordinating Coordinate Systems in ANSYS Mechanical

Coordinate systems are one of those things that are fundamental to Finite Element Analysis, but that most of us do not think about a lot.  They are there, but some users never fiddle with them. And some users are constantly futzing around with them.  We thought it would be a good idea to do a quick review of how they work in ANSYS Mechanical.  We will also go over the basics for  Mechanical APDL (MADL) in case you need to work with snippets.

Why Coordinate Systems Matter

ANSYS cares a lot about coordinate systems because they allow the program to solve in a standard, global, Cartesian system while allowing loads, constraints, material directions, layer information, beam sections, joints, result values, and a whole slew of other important aspects of the model to be specified in unique coordinate systems. This avoids making the user do coordinate system transformations.  At solve time, everything gets converted.

Since everyone reading this is an engineer, I’m going to assume that everyone already knows what a coordinate system is.

Coordinate Systems in DesignModeler and CAD Tools

It is important to start at the beginning.  Design Modeler and all CAD packages I’m aware of allow you to define some sort of coordinate system. Usually just Cartesian.  In workbench you can import those coordinate systems into ANSYS Mechanical by clicking “Import Coordinate Systems” from the “Advanced Geometry Options” properties for the Geometry cell in you systems.

For DesignModeler, there is an extra step.  Even if you turn on the import properties you need to dell DM which coordinate systems you want imported.  But first, be aware that there is no “coordinate system” entity in DM.  Instead it has planes, which is a coordinate system where you draw on the Z-normal plane. 

To make these available in ANSYS Mechanical, you need to scroll down to the bottom of the details for the planes you want converted over to coordinate systems, and set “Export Coordinate System” to Yes.

The following three images show setting it in DM, setting the property on the Project page, and how it shows up in Mechanical:

image  image  image

Creating Coordinate Systems in ANSYS Mechanical

In ANSYS Mechanical, coordinate systems reside in the Model Tree between Geometry and Connections.  Once you define a coordinate system it becomes available for use with any other object that can use a coordinate system.  This allows you to define it once, and then use it many times.

You always get a Global Cartesian coordinate system, called Global Coordinate System.  It is Cartesian, has an ID of 0, and sits at 0,0,0.  You can not change any of these values. Any imported coordinate systems will show up underneath the global.

To create a coordinate system you Right Mouse Button (RMB) on the Coordinate Systems branch and Insert->Coordinate System.  Or, when you click on the branch you also get a Coordinate System toolbar:


Click on the three color triad icon and a new system will be inserted.

Let’s look at each of the options in the details view.  But note before you go there, that the first set of group define a starting location and orientation, then you apply transformations in the last detail group in order to modify those locations.

  • Definition Group: This group specifies the type and MAPDL number for the Coordinate System
    • Type:  You can have the default Cartesian or Cylindrical here. The resulting coordinate system triads show up on your model like so.  As you can see, Z is the rotational axis, Y is tangential an X is radial.
      image   image
    • Coordinate System: In my opinion this should say Coordinate System ID because this detail lets you decide if you want ANSYS Mechanical to assign the number that MAPDL will use, or if you will.  Program Controlled is the default and is fine in most cases.  If you need to wrote a snippet to work with a coordinate system then you should change it to “Manual” and Coordinate System ID will show up.  Set it to any number over 11.


  • Origin Group:  This group defines where the center of the coordinate system is.
    • Define By:  You can specify a Geometry Selection or Global Coordinates. 
      • Geometry Selection: The cool thing about using Geometry Selection is that as you update your CAD model, the origin will shift with it.  As with any geometry specification, you click on a surface, line, vertex or a collection of these.  Mechanical will calculate the geometric center of the entity of entities that you picked and place the origin at that centroid.  It will also shows the position in the global coordinate system below your geometry selection, but you can not change them.
      • Global Coordinates: Here you simply put in an X, Y, and Z value in one of two ways. The easiest is to just type them in.  Or, you click “Click to Change” for Location and pick the coordinate picker image icon and move your mouse over your model.  Mechanical calculates the point under the cursor (surface closest to camera) and displays it. When you click it will create a little blue cross on the geometry.  Choose apply on Location an it will enter that point in as the origin.  Kind of cool, if a bit inaccurate…
  • Principal Axis Group: You need to tell Mechanical how to orient the coordinate system. By default it will align with the global.  But you can use Axis and Define By to specify that any of the three axis are aligned with a global axis, or with a piece of geometry.  Aligning with geometry is very useful because this is how you get coordinate systems aligned with your geometry. And when your geometry updates, that coordinate system aligns with the updated geometry.  This is especially useful when specifying a coordinate system in a cylinder because you can pick the cylinder face for your Z axis and it will move with the cylinder.
    Note that you can not specify align with Global –Z. If you want to do that you need to align with Z and use the transformation below to flip that.
    One option for “Define By” is “Fixed Vector” This uses the current orientation but disassociates it from the geometry.
  • Orientation about Principal Axis Group:  One point and a vector does not a coordinate system define. You have to specify an orientation around that principal axis.  You do that just like the how you specify that principal axis.  Define a global X, Y, or Z or a piece of geometry.
  • Direction Vectors Group:  These show the vectors for X, Y, and Z.  You can’t change them (I wish you could) and they are not a parameter. But they are useful.
  • Transformations Group:  This area allows you to stack offsets, rotations, and mirrors.
    You use the Coordinate System Toolbar to insert transformations into this group.  They are executed in order from top to bottom and the resulting orientation and position are shown in Directional Vectors and Transformed Configuration.
    There is a “Move Transform Up” and “Move Transform Down” icon as well in the toolbar to move the transformations around. There is also a delete to remove one.
    Note: When you click on a transformation in the list, the coordinate system on your model is shown AT THAT STEP, not at the final position.  This always confuses me.  So make your change, then click on the last step to see it.

Using Coordinate Systems

This is the easiest part. You simply choose one of your defined coordinate systems from a dropdown list when you create an object that is dependent on a coordinate system.  Usually this is when you can define a value based Components rather than on geometry:



Do note that you can also use coordinate systems to transform directional result values.  Simply pick the Coordinate system from the dropdown list.  This is especially important when looking at hoop or radial stresses in a cylindrical part.

Coordinate Systems in ANSYS Mechanical APDL

Coordinate systems are huge in MAPDL.  Nodes have them, elements have them, sections have them. Plus you can make a coordinate system active and every command you execute is done in that active coordinate system, and converted for you to the global. Very powerful.

If you do a search in help on “Coordinate System” you get hundreds of hits in the MAPDL manual.  Way too much to go over here.  We do recommend that you start with:

Mechanical APDL // Modeling and Meshing Guide // 3. Coordinate Systems // 3.1. Global and Local Coordinate Systems

It explains the types, the math, and the commands needed.  Read that, then move on to 3.3, 3.4, and 3.5 which talk about nodal, element and result coordinate systems.

Some key things every user should know are:

  1. All coordinate systems are defined by a number.  0-10 are reserved by MAPDL for its use.  Users can do 11 or higher.
  2. MAPDL has 6 default coordinate systems:image
    0 = Cartesian
    1 is cylindrical down the Z axis
    2 is Spherical
    4 is Cartesian, same as 0
    5 is Cylindrical down the Y axis.
    6 is Cylindrical down the X axis (not shown)
    I have no idea what happened to 3 or why 4 is the same as 0.
  3. When you change the active coordinate system with CSYS, all commands that involve coordinates get transformed into that coordinate system.  So:
    Actually makes a node at 12.5,9.8481,1.7365 in the global coordinate system.
  4. You can show local coordinate system with /psymb,csys,1


  5. You can list your coordinate system definitions with CSLIST:


  6. Only Cartesian and cylindrical are supported in ANSYS Mechanical, so if you need to use spherical or Toroidal you need to use snippets


Make sure you understand how Mechanical is using coordinate systems by bringing your models up in MAPDL.  Look at your nodes and see if they are rotated and how.  Check the coordinate systems with a CSLIST. Make sure you feel comfortable, don’t take it for granted.

Writing Text files with *VWRITE

A very common need in the world of ANSYS FEA simulation is to write text to a text file from within Mechanical APDL. Sometimes you are running in MAPDL, sometimes you are using ANSYS Mechanical but you still need to write stuff out using APDL with a code snippet. The way most people do that is with *VWRITE. 

Originally written to write out data in arrays, it is a very flexible and powerful command that can be used to write pretty much any type of formatted output. Something that every ANSYS user should have in their back pocket.

The Command

*VWRITE, Par1, Par2, Par3, Par4, Par5, Par6, Par7, Par8, Par9, Par10, Par11, Par12, Par13, Par14, Par15, Par16, Par17, Par18, Par19

Looks pretty simple right, just *vwrite and list what you want printed. But there is a lot more to this command.

A Lot More

First off you need to open up a file to write to.  You have a couple of options.

  1. *CFOPEN,fname, ext, –, Loc
    This opens the specified file for writing with *cfwrite and *vwrite commands.  This is the preferred method.
  2. /output,fname, ext, –, Loc
    By default *VWRITE output to standard output – the jobname.out (batch) file or the command window (interactive). So if you use /output you can redirect to a file instead of the *.out or screen.  We don’t recommend this because other stuff might get written as well to the file.

Now you have a place to write to, next you need to use *VWRITE to write. *VWRITE is a unique command because it actually uses two lines.  The first contains *VWRITE and a list of parameters and/or arrays to write and the second contains a format statement.  We will cover the first line first, and the format second.

Parameter Arguments for *VWRITE

As you can see from the command, you can have up to 19 parameters listed on a *VWRITE command.  PAR1 through PAR19 can be array, scalar, or character parameters. They can also be a constant. This is where the real flexibility comes in.  You can do something like (just look at the *VWRITE line, we will talk about the rest further on):

   1: adiv = ' | '

   2: *dim,nds, ,10

   3: *dim,temps,,10

   4: *vfill,nds(1),ramp,1,1

   5: *vfill,temps(1),rand,70,1500

   6: *cfopen,vw1.out

   7: *VWRITE,'Temp: ',nds(1),temps(1),adiv, 'TREF: ',70

   8: (A6,F8.0,g16.8,A3,A6,F10.4)

   9: *cfclose

This mixes characters, arrays, and constants in one command.  As output you get:

Temp:       1.   429.56308     | TREF:    70.0000
Temp: 2. 263.55403 | TREF: 70.0000
Temp: 3. 1482.8411 | TREF: 70.0000
Temp: 4. 605.95819 | TREF: 70.0000
Temp: 5. 782.33391 | TREF: 70.0000
Temp: 6. 1301.1332 | TREF: 70.0000
Temp: 7. 1119.4253 | TREF: 70.0000
Temp: 8. 202.87298 | TREF: 70.0000
Temp: 9. 1053.4121 | TREF: 70.0000
Temp: 10. 805.71033 | TREF: 70.0000

Array Parameters

The first thing you will notice is no *do loop.  If you supply an array parameter, *vwrite loops on the parameter from the given index (1 in this case) to the end of the array.  But if you don’t want the whole array written, you can control by placing *VLEN and/or *VMASK in front of the *VWRITE command:

  • *VLEN,nrow,ninc
    This will only write out nrow times, skipping based on ninc (defaults to 1)
    • As an example, if you want to write just the fourth value in array A() you would do:
  • *VMASK,Par
    You make a mask array of 0’s and 1 that is the same size as your array, and supply it to *VMASK.  *VWRITE will only write out values for your array if the mask array is not zero for the same index.

You can have a multiple dimensions on your array. *VWRITE only increments the first index. Say you specify X, Y, and Z coordinates in an array call xyz.  It would look like:


String Parameters

Being an older program, you are limited in what you can do with character parameters.  You are limited to 8 characters. So you just use a long string parameter several times and increment the index by 8:

   1: *dim,mystring,string,80

   2: mystring(1) = 'This is a very long sentance'

   3: *cfopen,vw2.out

   4: *VWRITE,mystring(1), mystring(9), mystring(17), mystring(25), mystring(33)

   5: (5A) 

   6: *cfclose

Kind of hokey, but it works.


Sigh.  This is the one thing that I’m not fond of in *VWRITE. The original command did not support outputting integer values.  That is because the FORTRAN I descriptor was not supported, and ANSYS stores everything as an integer anyhow. But people needed to output integer values so they took the ‘C’ format routines for *MSG and made them work with *VWRITE. So you can do a %I.  See the section on ‘C’ formatting below for more information on this.

Close the File

Before you can do anything with the file you create you need to close it.  Not to hard: *CFCLOSE does the trick.

Other Stuff you Need to Know

Don’t put a blank in there.  If you do, *vwrite stops looking at parameters.  So if you need a blank in your file, put in a space ‘ ‘ or use the X FORTRAN descriptor.

Be aware of *CFWRITE as well.  It is a way to write APDL commands to a file. If what you want to do is have your macro write a macro, *CFWRITE is better because you don’t have to do format statements. And those can be a pain when you need commas.

If your arrays are of different lengths, *VWRITE will loop for the length of the longest array. Any shorter arrays will be replaced with zeros for number arrays and blanks for character/string arrays.

You can not use *VWRITE by pasting or typing into the command line.  You have to read it from a file.


The key, and the difficult part of using *VWRITE is the format statement. We recommend that you use the FORTRAN formatting when you are writing out large amounts of columnar data, and use the ‘C’ format if you are writing out text rich information for a report or to inform the user of something.

Many users today may not even know what a FORTRAN statement looks like.  A good place to look is:

Just remember that you can’t use the Integer (I) format.  The list directed format (*) also does not work.  If you are new to it also remember everything goes in parenthesis and it has to fit on one line.  It does not have to start in column 8 (if you think that is funny, you are old)

As to ‘C’ formatting, you have a lot more options but we have found that the behavior is not as consistent between Linux and windows as the FORTRAN.  But if you are more comfortable with ‘C’, do that. Not all of ‘C’ formatting works in APDL, but what does is actually documented under the *MSG command. 

Making it Work

We always recommend you work out your *VWRITE issues in a small macro that you can run over and over again as you work out the formatting.  If you are writing a snippet for ANSYS Mechanical. Go ahead and save your model, bring it up in MAPDL, then work on your *VWRITE statement till you get it right.

Some other useful suggestions are:

  • Keep things simple. Don’t try and format a novel or an HTML page with *VWRITE.  You probably could, but there are better tools for that.
  • Make sure you understand arrays, how they work in APDL, and how you have yours dimensioned.
  • Get familiar with *VMASK and *VLEN. They are useful

Turning on Beta Features in Workbench

It is a busy week and I’m in between meetings for about 30 minutes, just enough time for a very short Focus posting for the week.  So, I thought I would share something I had to remember for the first time in a long time: How to access beta features in ANSYS Workbench.

First off a word of warning:  Beta features are beta features. They are capabilities in the software that have either not finished testing, are not fully documented, or that have a known issue.  They therefore must be used AT YOUR OWN RISK!!!!   If you find a bug or a problem, report it to your technical support provider, they need that feedback. But don’t call up indignant because it is not working the way you want it, or because the documentation is non-existent.  It is a beta feature.

Set the Option

Not too difficult.  From the Project Schematic page go to the tools menu and select Options.


Now in the Options dialog click on Appearance in the tree on the left.  You will not see Beta Options.  Scroll down and near the bottom there are a bunch of check boxes. Check “Beta Options”image

Now, in your project toolbar you should see (Beta) next to the exposed beta functions:


This will also impact any beta features, if any, in the workbench native applications:  Parameter manager, Engineering Data, or DesignXplorer.

That is it. I promised short.  Off to another meeting.


Webinar Information: Constraint Equation Primer

Here are the files from the webinar held on Friday, April 27, 2012: A Constraint Equation Primer: How to Tie Degrees of Freedom Together


Link to webinar recording:

No models or anything on this one.

To GPU or Not, This is No Longer The Question…

imageExecutive Summary

Without losing you quickly due to fabulous marketing and product information. I thought it prudent if I get to the answer of my question as quickly as possible. What was the question? Should I invest in a companion processor and an ANSYS HPC Pack? Yes, now is the time.

  • If your current workstation is beginning to show its age and you are unable to purchase new hardware.
  • Get your critical results fast. It is painful waiting upwards of 50 hours for your solution to solve.
  • Assign a dollar value to your current frustration level.
  • Current pricing for Nvidia TESLA C2075 is around $2500. Current pricing for the NVidia Quadro 6000 is around $5,000.

“Keeping It Real”

Matt Sutton our Lead Software Development Engineer at PADT, Inc. was solving a very large ANSYS MAPDL acoustic ultrasound wave propagation model. The model labored over 12 hours to solve on Matt’s older Dell Precision 690 8 core workstation. We loaded the model onto our CUBE HVPC w8i with the NVidia Tesla C2075 and it solved the model in 50 minutes. That is a 15x speedup for Matt’s particular model!



I started off with our benchmark assault using an industry standard ANSYS benchmark for HPC. This is a benchmark that NVidia requested we use for testing our TESLA C2075. The next benchmark that I used was an internal PADT, Inc. modal coupler benchmark.

Nvidia V14sp-5 ANSYS R14 Benchmark Matrix (Sparse solver, 2100k)


CUBE HVPC – w12a-GPU – 64GB







Config A: Win v14 SMP In-Core

Config A: Win v14 SMP In-Core & GPU

Config A: Win v14 DIST In-Core

Config A: Win v14 DIST & GPU

Config A: Linux v14 SMP In-Core

Config A: Linux v14 SMP In-Core & GPU

Config A: Linux v14 DIST In-Core

Config A: Linux v14 DIST & GPU



  2 1388.20 463.40 1425.50 294.40 1324.40 466.30 1312.30 292.10      
  4 870.00 447.20 855.00 232.40 817.00 422.00 975.40 221.80      
  6 670.60 440.20 588.00 245.10 625.50 394.00 601.60 208.10      
  8 594.30 439.10 605.00 222.00 480.10 383.90 542.30 181.10      
  10 539.50 426.90 435.00 283.40 491.30 392.80 396.90 220.80      
  12 538.40 437.10 402.20 211.10 480.10 394.80 343.80 183.10     2x
CUBE HVPC – w8i-GPU – 64GB CONFIG B                
64GB # Cores Config B: Win v14 SMP In-Core Config B: Win v14 SMP In-Core & GPU Config B: Win v14 DIST In-Core Config B: Win v14 DIST & GPU Config B: Linux v14 SMP In-Core Config B: Linux v14 SMP In-Core & GPU Config B: Linux v14 DIST In-Core Config B: Linux v14 DIST & GPU      
  2 1078.50 361.00 1111.90 235.70 1036.00 350.20 1072.90 250.60      
  4 645.50 330.70 652.30 185.50 608.10 312.00 790.90 193.20      
  6 494.30 322.70 458.30 233.70 464.90 303.20 502.70 178.70      
  8 438.50 328.30 462.20 230.40 406.10 304.80 451.20 166.00     2.5x
CUBE HVPC – w16i-GPU – 128GB


128 GB # Cores Config C: Linux v14 SMP In-Core & GPU Config C: Linux v14 DIST & GPU                  
  2 296.6 208.10                  
  4 254.4 160.70                  
  6 254.2 164.20                  
  8 239.9 138.20                  
  10 238.6 159.70                  
  12 246.3 129.60                  
  14 237.6 129.1                  
  16 248.9 130.5                  


CUBE HVPC – PADT, Inc. – Coupling Modal Benchmark (PCG solver, ~1 Million DOF, 50 modes)



ANSYS 13.0

Shared Memory Parallel INTEL XEON 2×6 @3.47GHz /144GB of RAM w12i-GPU
  Processors Time Spent Computing Solution (secs) Date/Individuals Initials:
  2 5416.4 11/7/2011 – DRJM
  10+GPU 1914.2 (incore) 11/8/2011 – DRJM
  12+GPU 1946 (incore) 11/8/2011 – DRJM



Shared Memory Parallel INTEL XEON 2×4 @2.8GHz /64GB of RAM w8i-GPU
  6+GPU 3659.4 (out of core) 4/11/12 – DRJM
  8+GPU 3686.7 (out of core) 4/11/12 – DRJM



Shared Memory Parallel INTEL XEON e5-2690 2×8 @2.9GHz /128GB of RAM W16i-GPU
  14+GPU 2113 (incore) 4/18/12 – DRJM
  16+GPU 1533.9 (incore) 4/18/12 – DRJM



Summary, Debates, Controversy & Conclusions

One question that I hear often is this: “What operating systems is faster. Linux or Windows?”. My typical response will begin with “Well, it depends…” However, what the data illustrates with the two independent CPU workstations as well as Operating Systems. The ANSYS benchmarks were all performed in the same relative time frame. The only exception was the ANSYS modal analysis benchmark.

Are you ready for the answer? Here it is…yes Linux is faster than windows! Even if you come from the AMD CPU or INTEL CPU side of the tracks. Linux is faster! No big discovery right? I know we all knew this already but we were maybe afraid to ask by Operating system as well as CPU manufacture how much? Well with our 2.1 million degree of freedom Nvidia benchmark. Ummm, not very much as the data clearly indicates. However once again, the Linux based OS does give you a system performance advantage! If you use AMD or INTEL it is still faster. The next question that I hear next is: “What processor is faster AMD or INTEL?”. My typical response will begin with “Well, it depends…”

You did buy the NVidia TESLA C2075 GPU, okay this is great news! However, you figured it best that you keep it safe, stay the same, and continue to use Shared Memory Parallel solve method. As you ponder further on the speed up values. You will see once again that the best results are going to be found on the Linux Operating System and when you choose to solve in Distributed Memory Parallel mode. The AMD based CPU had the best speedup values when comparing the workstation against the two solve modes with a 2.2x’s speed up.

Lets begin to unpack this data even more and see if you can come up with your own judgments. This is where things might start to get controversial…so hang on.

Windows, Linux, AMD or INTEL

Time Spent Computing The Solution

  • ANSYS R14 Distributed Memory Parallel Results: (Incore)
    • LINUX 64-bit:
      • One minute forty-seven seconds faster on 2x AMD CPU (343.80 seconds vs. 451.20 seconds)
    • Windows 7 Professional 64-bit:
      • One minute faster on 2x AMD CPU (402.20 seconds vs. 462.20 seconds).
  • ANSYS R14 Shared Memory Parallel Results: (Incore)
    • LINUX 64-bit
      • One minute fourteen seconds faster on 2x INTEL XEON CPU (406.10 seconds vs. 480.10 seconds)
    • Windows 7 Professional 64-bit is sixty seconds faster solve time on AMD based CPU
      • One minute forty seconds faster on 2x INTEL XEON CPU (438.50 vs. 538.40 seconds)

The Nvidia TESLA C2075 GPU and ANSYS R14 – GPU to the rescue!

  • ANSYS R14 Distributed Memory Parallel with Nvidia TESLA C2075 GPU Assist Results
    • LINUX 64-bit:
      • Seventeen seconds faster on 2x INTEL XEON CPU (166 seconds vs. 183.10 seconds)
      • 129.1 seconds was the fastest overall solve time achieved for any operating system on this benchmark!
      • 2 x INTEL XEON E5-2690 – fourteen cores with ANSYS R14 DMP w/GPU assist
    • Windows 7 Professional 64-bit:
      • Nineteen seconds faster on 2x AMD CPU (211.10 seconds vs. 230.40 seconds).
      • 211 seconds was the fastest solve time achieved using Windows for this benchmark!
      • 2 x INTEL XEON X5560 – eight cores with ANSYS R14 DMP w/GPU assist
  • ANSYS R14 Shared Memory Parallel with Nvidia TESLA C2075 GPU Assist Results
    • LINUX 64-bit:
      • Seventeen seconds faster on 2x INTEL XEON CPU (166 seconds vs. 183.10 seconds)
      • Config C: 237.6 seconds was the fastest overall solve time achieved for any operating system on this benchmark!
      • 2 x INTEL XEON E5-2690 – fourteen cores with ANSYS R14 SMP w/GPU assist
    • Windows 7 Professional 64-bit:
      • Nineteen seconds faster on 2x AMD CPU (211.10 seconds vs. 230.40 seconds).
      • 211 seconds was the fastest solve time achieved using Windows for this benchmark!
      • 2 x INTEL XEON X5560 – eight cores with ANSYS R14 SMP w/GPU assist

ANSYS R14 Distributed Memory Parallel with GPU vs. ANSYS R14 Shared Memory Parallel with GPU speed up results:

Distributed Memory Parallel (DMP) or Shared Memory Parallel (SMP)

  • 2 x AMD Opteron 4184 based workstation vs. “DMP or SMP”
    • 2.2x’s speedup on Linux Operating System (183.10 seconds vs. 394.80 seconds)
    • 2x’s speedup on Windows Operating System (211.10 seconds vs. 437.10 seconds)
  • 2 x INTEL XEON E5-2690 based CPU vs. “DMP or SMP”
    • 1.9x’s speedup on Linux Operating System (129.41 seconds vs. 237.6 seconds)

With or Without GPU? ANSYS R14 Distributed Memory Parallel speed up results

Distributed Memory Parallel (DMP) with GPU vs. Distributed Memory Parallel without GPU

  • AMD Opteron 4184 based workstation
    • 2x’s speedup Linux Operating System (183.10 seconds vs. 343.80 seconds)
    • 2x’s speedup on Windows Operating System (211.10 seconds vs. 402.2 seconds)
  • INTEL XEON x5560 based workstation
    • 2.5x’s speedup on Linux Operating System (166 seconds vs. 451.20 seconds)
    • 1.4x’s speedup on Windows Operating System (230.40 seconds vs. 462.20 seconds)

Some Notes:

  • ANSYS Base License – unlocks up to 2 CPU Cores
  • ANSYS HPC Pack – unlocks up to 8 CPU Cores and GPU
  • The total amount of system RAM you have affects your Distributed solve times. A minimum of 48GB of RAM is recommended.
  • The processing speed of your CPU affects your Shared Memory Parallel solve times.
  • Model limits for direct will depend on largest front sizes
    • 6M DOF for 6GB Tesla C2075 and Quadro 6000
  • Model limits for iterative PCG and JCG
    • 3M DOF for 6GB Tesla C2075 and Quadro 6000



Hardware Specs of Workstations:


The AMD® based CUBE HVPC w12a FEA Simulation Workstation:

  • CPU: 2x AMD Opteron 4184 (2.8GHz Ghz 6 core)
  • GPU: NVIDIA TESLA C2075 Companion Processor
  • RAM: 64GB DDR3 1333Mhz ECC
  • HDD: (os and apps): 450GB WD Velociraptor 10k
  • HDD: (working directory): 6 x 1TB WD RE4 drives using LSI 2008 RAID 0
  • OS: Windows 7 Professional 64-bit, Linux 64-bit
  • OTHER: ANSYS R14, latest NVIDIA TESLA Drivers

(Here is a picture of Sam with two full
tower CUBE HVPC workstations getting
ready to ship out!)


The INTEL® based CUBE HVPC w8i FEA Simulation Workstation:

  • CPU: 2x INTEL XEON x5560 (2.8GHz 4 core)
  • GPU: NVIDIA TESLA C2075 Companion Processor
  • RAM: 64GB DDR3 1333Mhz ECC
  • HDD: (os and apps): 146GB SAS 15k
  • HDD: (working directory): 3 x 73GB SAS 15k – LSI RAID 0
  • OS: Windows 7 Professional 64-bit, Linux 64-bit
  • OTHER: ANSYS R14, latest NVIDIA TESLA Drivers


The INTEL® based CUBE HVPC w16i-GPU FEA Server:

  • CPU: 2x INTEL XEON e5-2690 (2.9GHz 8 core)
  • RAM: 128GB DDR3 1600 MHz ECC
  • HDD: (os and apps): 300GB SATA III 10k WD Velociraptor
  • HDD: (working directory): 3 x 600GB SATA III 10k WD
  • OS: Linux 64-bit
  • OTHER: ANSYS R14, latest NVIDIA TESLA Drivers


The INTEL® based CUBE HVPC w12i-GPU FEA Simulation Workstation: image

  • CPU: 2x INTEL XEON x5690 (3.47GHz 6 core)
  • RAM: 144GB DDR3 1333Mhz ECC
  • HDD: (os and apps): 256GB SSD SATA III
  • HDD: (working directory): 4 x 600GB SAS2 15k – LSI RAID 0
  • OS: Windows 7 Professional 64-bit


Happy 10th Birthday: The Focus

image_thumb1Don’t you hate it when you miss someone’s birthday?  I was looking up an old article in the Focus and noticed that the first issue was published on January, 13th, 2002. 

Happy Belated Birthday!

It was sometime in 2001 that Rod Scholl pointed out that there was no good ANSYS newsletter out there.  People would do one or two issues then get busy and never put out any more, or only publish once in a while.  So we decided that we would not only do a newsletter, but that we would keep it up and publish  on a regular basis. The first issue came out as an HTML email on January 13th of 2002. 


The First Issue of The Focus

And Rod was instrumental in keeping us on track and making sure that with it. Since then we published 74 actual newsletters before switching to a blog format in 2010.  Just before this one goes up on the blog, we will have published 59 The Focus articles.

Thank you to everyone who subscribes to The Focus and reads our postings, rates us, and sends us such great comments and questions.  

Here is to 10 more years!

Files for Webinar: A POST26 Primer

imageWe just finished our webinar for 4/12/2012 on the basics of the POST26 Time History Post Processor.  As promised, here are the files used for examples in the webinar, as well as the PowerPoint:

Tower-of-Test (ToT) test model, APDL:

Tower-of-Test (ToT) test model, ANSYS Mechanical with APDL Snippets:

tower-of-test-transient-2013.wbpz (note this is an R15 file.  See why here.)


imagePowerPoint Presentation:

A recording of this webinar will be available on the following site after 4/13/2012:

Click on  PADT ANSYS Webinar Series to see all recordings.

Some Revolutionary HPC Talk: 208 Cores+896GB < $60k, GPU’s, & ANSYS Distributed

imageThe last couple of weeks a bunch of stuff has gone on in the area of High Performance computing, or HPC, so we thought we would throw it all into one Focus posting and share some things we have learned, along with some advice, with the greater ANSYS user community.

There is a little bit of a revolution going on in both the FEA and CFD simulation side of things amongst users of ANSYS, Inc. products.  For a while now customers with large numbers of users and big nasty problems to solve have been buying lots of parallel licenses and big monster clusters. The size of problems that these firms, mostly turbomachinery and aerospace, have been growing and growing. And even more so for CFD jobs.   But also FEA for HFSS and ANSYS Mechanical/Mechanical APDL.  That is where the revolution started.

But where it is gaining momentum, where the greater impact is being seen on how simulation is used around the world, is with the smaller customers.  People with one to five seats of ANSYS products.  In the past they were happy with their two “included” Mechanical shared memory parallel tasks.  Or they might spring for 3 or 4 CFD parallel licenses.  But as 4, 6, and 8 core CPU chips become mainstream, even on the desktop, and as ANSYS delivers greater value from parallel, we are seeing more and more people invest in high performance computing. And they are getting a quick return on that investment.

Affordable High Value Hardware

208 Cores + 869 GB + 25 TB + Infiniband + Mobile Rack = $58k = HOT DAMN!

Yes, this is a commercial for PADT’s CUBE machines.  ( Even if you would rather be an ALGOR user than purchase hardware from a lowly ANSYS Chanel Partner, keep reading. Even if you would rather go to an ANSYS meeting at HQ in the winter than brave asking your IT department if you can buy a machine not made by a major computer manufacturer, keep reading.

Because what we do with CUBE hardware is something you can do yourself, or that you can pressure your name brand hardware provider into doing.

We just got a very large CFD benchmark job for a customer.  They want multiple runs on a piece of “rotating machinery” to generate performance curves.  Lots of runs, and each run can take up to 4 or 5 days on one of our 32 core machines.  So we put together a 208 core cluster.  And we maxed out the RAM on each one to get to just under 900 GB. Here are the details:

Cores: 208 Total
    3 servers x48 2.3 GHz cores each server
    2 servers x32 3.0 GHz cores each server
RAM: 896 GB
    3 servers  x128GB DDR3 1333MHz ECC RAM each server
    2 servers  x256GB DDR3 1600MHz ECC RAM each server
DATA Array:  ~25TB
Interconnect: 5 x Infiniband 40Gbps QDR
Infiniband Switch: 8 port 40Gbps QDR
Mobile Departmental Cluster Rack – 12U

All of this cost us around $58,000 if you add up what we spent on various parts over the past year or so.  That much horsepower for $58,000.  If you look at your hourly burden rate and the impact of schedule slip on project cost, spending $60k on hardware has a quick payback. 

You do need to purchase parallel licenses. But if you go with this type of hardware configuration what you save over a high-priced solution will go a long way towards paying for those licenses.  Even if you do spend $150k on hardware, your payback with the hardware and the license is still pretty quick.

Now this is old hardware (six months to a year –  dinosaur bones).  How much would a mini-cluster departmental server cost now and what would it have inside:

Cores: 320 Total
5 servers x64 2.3 GHz cores each server
RAM: 2.56 TB
5 servers x512 GB DDR3 RAM each server
DATA Array: ~50TB
Interconnect: 5 x Infiniband 40Gbps QDR
Infiniband Switch: 8 port 40Gbps QDR
Mobile Departmental Cluster Rack – 12U

The cost?  around $80,000.  That is $250/core.  Now you need big problems to take advantage of that many cores.  If your problems are not that big, just drop the number of servers in the mini-cluster.  And drop the price proportionally. 

Same if you are a Mechanical user.  The matrices in FEA just don’t scale in parallel like they do for CFD, so a 300+ core machine won’t be 300 times faster. It might even be slower than say 32 cores.  But the cost drop is the same.  See below for some numbers.

Bottom line, hardware cost is now in a range where you will see payback quickly in increased productivity.



We think they should have some 80’s era super model
draped over the front like those Ferrari posters we
had in our dorm rooms.

For you Mechanical/Mechanical APDL users, let’s talk GPU’s.

We just got an NVIDIA TESLA C2075 GPU.  We are not done testing, but our basic results show that no matter how we couple it with cores and solvers, it speeds things up.  Anywhere from 3 times faster to 50% faster depending on the problem, shared vs. distributed memory, and how many cores we throw in with the GPU.  

This is a fundamental problem with answering the question “How much faster?” because it depends a lot on the problem and the hardware. You need to try it on your problems on your hardware. But we feel comfortable in saying if you buy an HPC pack and run on 8 cores with a GPU, the GPU should double your speed relative to just running on 8 cores.  It could even run faster on some problems. 

For some, that is a disappointment.  “The GPU has hundreds of processors, why isn’t it 10 or 50 times faster?”  Well, getting the problem broken up and running on all of those processors takes time.  But still, twice as fast for between $2,000 to $3,000 is a bargain. I don’t know what your burden rate is but it doesn’t take very many hours of saved run time to recover that cost.  And there is no additional license cost because you get a GPU license with an HPC pack.

Plus, at R14 the solver supports a GPU with distributed ANSYS, so even more improvements.  Add to that support for the unsymmetrical or damped Eigensolvers and general GPU performance increases at R14.

PADT’s advice? If you are running ANSYS Mechanical or Mechanical APDL get the HPC Pack and a GPU along with a 12 core machine with gobs of RAM (PADT’s 12 core AMD system with 64GB or RAM and 3TB of disk costs only $6,300 without the GPU, $8,500 with).  You can solve on 8 cores and play Angry Birds on the remaining 4.

Distributed ANSYS

For a long time many of users out there have avoided distributed ANSYS. It was hard to install, and unless you had the right problem you didn’t see much of a benefit for many of the early releases. Shared Memory Parallel, or SMP, was dirt easy – get an HPC license and tell it to run on 8 cores and go.

Well at R14 of ANSYS Mechanical APDL it is time to go distributed.  First off they make the install much easier.  To be honest, we found that this was the biggest deterrent for many small company users.

Second, at R14 a lot more things are supported in distributed ANSYS. This has been going on for some time so most of what people use is supported. At this release they added subspace eigensolving, TRANS, INFINI and PLANE121/230 elements (electrostatics), and SURF251/252. 

Some “issues” have been fixed like restart robustness and you now have control on when and how multiple restart files are combined after the solve. 

All and all, if you have R14, you are solving mechanical problems, and you have an HPC pack, you should be using distributed most of the time.


We get a ton of questions from people about what they should buy and how much.  And every situation is different. But for small and medium sized companies, the HPC revolution is here and everyone should be budgeting for taking advantage of HPC:

    • At least one HPC pack
    • Some new faster/cheaper multicore hardware (CUBE anyone?)
    • A GPU. 

STOP!  I know you were scrolling down to the comments section to write some tirade about how ANSYS, Inc overcharges for parallel, how it is on a moral equivalence with drowning puppies, and how much more reasonable competitor A, B, or C is with parallel costs.  Let me save you the time.

HPC delivers huge value.  Big productivity improvements.  And it does not write itself. It is not an enhancement to existing software.  Scores of developers are going into solver code and implementing complex logic that allows efficiency with older hardware, shared memory, distributed memory, and GPU’s. It has to be written, tested, fixed, tested again, and back and forth every release.  That value is real, and there is a cost associated with it.

And the competitors pricing model? The only thing they can do to differentiate themselves is charge nothing or very little. They have not put the effort or don’t have the expertise to deliver the kind of parallel performance that the ANSYS, Inc. solvers do.  So they give it away to compete.  Trust me, they are not being nice because they like you. They have the same business drivers as ANSYS, Inc.  They price the way they do because they did not incur as much cost and they know if they charged for it you would have no reason to use their solvers.

ANSYS users of the world unit!  Load your multicore hardware with HPC packs, feed it with a GPU, and join the revolution!