Combining ANSYS Simulation with HPC

Engineering simulation has become much more prevalent in engineering organizations than it was even 5 years ago.  Commercial tools have gotten significantly easier to use whether you are looking at tools embedded within CAD programs or the standalone flagship analysis tools.  The driving force behind these changes are to ultimately let engineers and companies understand their design quicker with more fidelity than before.

Engineering simulation is one of those cliché items where everyone says “We want more!”  Engineers want to analyze bigger problems, more complex problems and even do large scale design of experiments with hundreds of design variations – and they want these results instantaneously.   They want to be able to quickly understand their designs and design trends and be able to make changes accordingly so then can get their products optimized and to the market quicker.

ANSYS, Inc. spends a significant amount of R&D in helping customers get their results quicker and a large component of that development is High Performance Computing, or HPC.  This technology allows engineers to solve their structural, fluid and/or electromagnetic analyses across multiple processors and even across multiple computing machines.  Engineers can leverage HPC on laptops, workstations, clusters and even full data centers.

PADT is fortunate to be working with Nimbix, a High Performance Computing Platform that easily allowed us to quickly iterate through different models with various cores specified.  It was seamless, easy to use, and FAST!

Let’s take a look at four problems: Rubber Seal FEA, Large Tractor Axle Model, Quadrocopter CFD model and a Large Exhaust CFD model.  These problems cover a nice spectrum of analysis size and complexity. The CAD files are included in the link below.

Click here to download geometry files that were used in the following benchmarks

TRACTOR AXLE FEA

This model has several parts all with contact defined and has 51 bolts that have pretension defined.  A very large but not overly complex FEA problem.  As you can see from the results, even by utilizing 8 cores you can triple your analysis throughput for a work day.  This leads to more designs being analyzed and validated which gives engineers the results they need quicker.

SUMMARY

  • 58 Parts
  • 51 x Bolts with Pretension
  • Gaskets
  • 928K Elements, 1.6M Nodes

Cores

Elapsed Time
[s]

Estimated Models Per 8 [hours]

2

14,525

2

4

9,710

3

8

5,233

6

16 4,009

7

 

RUBBER SEAL FEA

The rubber seal is actually a relatively small size problem, but quite complex.  Not only does it need full hyperelastic material properties defined with large strain effects included, it also includes a leakage test.  This will pressurize any exposed areas of the seal.  This will of course cause some deformation which will lead to more leaked surfaces and so on.  It basically because a pressure advancing solution.

From the results, again you can see the number of models that can be analyzed in the same time frame is signifcantly more.  This model was already under an hour, even with the large nonlinearity, and with HPC it was down to less than half an hour.

SUMMARY

  • 6 Parts
  • Mooney Rivlin Hyperelastic Material
  • Seal Leakage with Advancing Pressure Load
  • Frictional Contact
  • Large Deformation
  • 42K Elements, 58K Nodes

Cores

Elapsed Time
[s]
Estimated Models Per 8 [hours]

2

3,353

9

4

2,489

12

8 1,795

16

 


QUADROCOPTER DRONE CFD

The drone model is a half symmetry model that includes 2 rotating domains to account for the propellers.  This was ran as a steady state simulation using ANSYS Fluent.  Simply utilizing 8 cores will let you solve 3 designs versus 1.

SUMMARY

  • Multiple Rotating Domains
  • 2M Elements, 1.4M Nodes

Cores

Elapsed Time
[hours]
Speedup

2

2.1

1

4

1.2

1.8

6

0.8

2.6

8 0.7

3

 

EXHAUST CFD

The exhaust model is a huge model with 33 million elements with several complicated flow passages and turbulence.  This is a model that would take over a week to run using 1 core but with HPC on a decent workstation you can get that down to 1 day.  Leveraging more HPC hardware resources such as a cluster or using a cloud computing platform like Nimbix will see that drop to 3 hours.  Imagine getting results that used to take over 1 week that now will only take a few hours.  You’ll notice that this model scaled linearly up to 128 cores.  In many CFD simulations the more hardware resources and HPC technology you throw at it, the faster it will run.

SUMMARY

  • K-omega SST Turbulence
  • Multi-Domain
  • 33M Elements, 7M Nodes

Cores

Elapsed Time
[hours]
Speedup

16

26.8

1

32

13.0

2.1

64

6.8

3.9

96

4.3

6.2

128 3.3

8.2

As seen from the results leveraging HPC technology can be hugely advantageous.  Many simulation tools out there do not fully leverage solving on multiple computing machines or even multiple cores.  ANSYS does and the value is easily a given.  HPC makes large complex simulation more practical as a part of the design process timeline.  It allows for greater throughput of design investigations leading to better fidelity and more information to the engineer to develop an optimized part quicker.

If you’re interested in learning more about how ANSYS leverages HPC or if you’d like to know more about NIMIBX, the cloud computing platform that PADT leverages, please reach out to me at manoj@padtinc.com

 

Activating Hyperdrive in ANSYS Simulations

punch-it-chewie-ansysWith PADT and the rest of the world getting ready to pile into dark rooms to watch a saga that we’ve been waiting for 10 years to see, I figured I’d take this opportunity to address a common, yet simple, question that we get:

“How do I turn on HPC to use multiple cores when running an analysis?”

For those that don’t know, ANSYS spends a significant amount of resources into making the various solvers it has utilize multiple CPU processors more efficiently than before.  By default, depending on the solver, you are able to use between 1-2 cores without needing HPC licenses.

With the utilization of HPC licenses, users can unlock hyperdrive in ANSYS.  If you are equipped with HPC licenses it’s just a matter of where to look for each of the ANSYS products to activate it.

ANSYS Mechanical

Whether or not you are performing a structural, thermal or explicit simulation the process to activate multiple cores is identical.

  1. Go to Tools > Solve Process Settings
  2. The Solve Process Settings Window will pop up
  3. Click on Advanced to open up the Advanced Settings window
  4. You will see an option for Max number of utilized cores
  5. Simply change the value to your desired core count
  6. You will see below an option to allow for GPU acceleration (if your computer is equipped with the appropriate hardware)
  7. Select the GPU type from the dropdown and choose how many GPUs you want to utilize
  8. Click Ok and close
hyperdrive-ansys-f01
Go the proper settings dialog
hyperdrive-ansys-f02
Choose Advanced…
hyperdrive-ansys-f03
Specify the number of cores to use

Distributed Solve in ANSYS Mechanical

One other thing you’ll notice in the Advanced Settings Window is the option to turn “Distributed” On or Off using the checkbox.

In many cases Distributing a solution can be significantly faster than the opposite (Shared Memory Parallel).  It requires that MPI be configured properly (PADT can help guide you through those steps).  Please see this article by Eric Miller that references GPU usage and Distributed solve in ANSYS Mechanical

hyperdrive-ansys-f04
Turn on Distributed Solve if MPI is Configured

ANSYS Fluent

Whether launching Fluent through Workbench or standalone you will first see the Fluent Launcher window.  It has several options regarding the project.

  1. Under the Processing Options you will see 2 options: Serial and Parallel
  2. Simply select Parallel and you will see 2 new dropdowns
  3. The first dropdown lets you select the number of processes (equal to the number of cores) to use in not only during Fluent’s calculations but also during pre-processing as well
Default Settings in Fluent Launch Window
Default Settings in Fluent Launch Window
Options When Parallel is Picked
Options When Parallel is Picked

ANSYS CFX

For CFX simulations through Workbench, the option to activate HPC exists in the Solution Manager

  1. Open the CFX Solver Manager
  2. You will see a dropdown for Run Mode
  3. Rather than the default “Serial” option choose from one of the available “Parallel” options.
  4. For example, if running on the same machine select Platform MPI Local Parallel
  5. Once selected in the section below you will see the name of the computer and a column called Partitions
  6. Simply type the desired number of cores under the Partitions column and then either click “Save Settings” or “Start Run”
Change the Run Mode
Change the Run Mode
Specify number of cores for each machine
Specify number of cores for each machine

ANSYS Electronics Desktop/HFSS/Maxwell

Regardless of which electromagnetic solver you are using: HFSS or Maxwell you can access the ability to change the number of cores by going to the HPC and Analysis Options.

  1. Go to Tools > Options > HPC and Analysis Options.
  2. In the window that pops up you will see a summary of the HPC configuration
  3. Click on Edit and you will see a column for Tasks and a column for Cores.
  4. Tasks relate to job distribution utilizing Optimetrics and DSO licenses
  5. To simply increase the number of cores you want to run the simulation on, change the cores column to your desired value
  6. Click OK on all windows
hyperdrive-ansys-f09
Select the proper settings dialog
hyperdrive-ansys-f10
Select Edit to change the configuration
Specify Tasks and Cores
Specify Tasks and Cores

There you have it.  That’s how easy it is to turn on Hyperdrive in the flagship ANSYS products to advance your simulations and get to your endpoint faster than before.

If you have any questions or would like to discuss the possibility of upgrading your ship with Hyperdrive (HPC capabilities) please feel free to call us at 1-800-293-PADT or email us at support@padtinc.com.

Video Tips: Parallel Part by Part Meshing in ANSYS v15.0

This video shows you a new capability in ANSYS v15.0 that allows multiple parts to be simultaneously meshed on multiple CPU cores…with no additional licenses required!