When the going gets tough, the tough use ANSYS for CFD Meshing

If you do CFD simulations then you know the struggle that is involved in meshing. It is a fine balance of accuracy, speed, and ease of set up. If you have complex geometry, large assemblies, or any difficulty meshing then this blog article is for you.

Why should I spend time making a good mesh?

The mesh is arguably one of the most important parts of any simulation set up. A good mesh can solve significantly faster and provide more accurate results. Conversely, a poor mesh can make the simulation have inaccurate results and be slow to converge or not converge at all. If you have done any simulation then you know that hitting the solve button can feel like rolling the dice if you don’t have a robust meshing tool.

When is it going to matter?

A good mesh is going to matter on a Friday afternoon when you need to get the simulation started before you leave for the weekend because it takes two days to run and you need to deliver results on Monday but you are up against the clock because you have to get to your kid’s soccer game by 5pm and the mesh keeps crashing.

A poor mesh can do more than just reorganizing you’re social agenda. A poor mesh can drastically change results like pressure drop in an internal flow passage or drag over a body. If you go into that meeting on Monday and tell your boss that the new design is going to perform 10% better than the previous design – you need to be confident that the design is 10% better not 10% worse.

What should I do when I need to create a good mesh?

If you’re the poor soul reading this on a Friday afternoon because you are trying to frantically fix you’re mesh so you can get your simulation running before the weekend – I pity you. Continue reading for my proprietary step by step approach titled “How to get you’re CFD mesh back on track!” (Patent pending).

Step 1) Know your tools

ANSYS has been developing its meshing technology since the beginning of time (not really but almost) – it’s no surprise that its meshing algorithms are the best in the business. In ANSYS you have a large number of tools at your disposal, know how to use them.

The first tool in your toolbox is the ANSYS automatic meshing technology. It is able to predictively apply settings for your part to get the most accurate automatic mesh possible. It has gotten so good that the automatic mesh is a great place to start for any preliminary simulations. If you want to get into the details, ANSYS meshing has two main groups of mesh settings – Global Meshing Parameters and Local Meshing Parameters. Global mesh parameters are great for getting a good mesh on the entire model without going into detailed mesh settings for each part.

But when you do have to add detailed meshing settings on a part by part basis then local mesh settings won’t let you down.

Step 2) Know your physics

What is your primary result of interest? Drag? Pressure drop? Max velocity? Stagnation? If you can quantify what you are most interested in then you can work to refine the mesh in that region so as to capture the physics accurately. ANSYS allows you set local sizing parameters on bodies, faces, lines, and regions which allow you to get the most accurate mesh possible but without having to use a fine mesh on the entire part.

Step 3) Know your mesh quality statistics

Mesh quality statistics can be a good way to gauge the health of your mesh. They are not a foolproof method for creating a mesh that will be accurate but you will be able to get an idea of how well it will converge. In ANSYS meshing there is a number of mesh quality statistics at your fingertips. A quick and easy way to check your mesh is to look at the Minimum Orthogonal Quality statistic and make sure it is greater than 0.1 and Maximum Skewness is less than 0.95.

Step 4) Know your uncertainty

Every test, simulation, design, process etc… has uncertainty. The goal of engineering is to reduce that uncertainty. In simulation meshing is always a source of uncertainty but it can be minimized by creating high quality meshes that accurately model the actual physical process. To reduce the uncertainty in meshing we can perform what is called a mesh refinement study. Using the concept of limits we can say that in the limit of the mesh elements getting infinitely small than the results will asymptotically approach the exact solution. In the graph below it can be seen that as the number of elements in the model are increased from 500 – 1.5million the result of interest approached the dotted line which we can assume is close to the exact solution.

By completing a mesh refinement study as shown above you can be confident that the mesh you have created is accurately capturing the physics you are modeling because you can quantify the uncertainty.

If you currently just skip over the meshing part of your CFD analysis thinking that it’s good enough or if your current meshing tool doesn’t give you any more details than just a green check mark or a red X then it’s time dig into the details of meshing and start creating high quality meshes that you can count on.

For more info about advanced meshing techniques in ANSYS – see this PDF presentation that is a compilation of ANSYS training material on the meshing subject.

Advanced Techniques in ANSYS Meshing_Blog

If you still haven’t figured out how to get your mesh to solve and its 5pm on Friday see below*

*Common pitfalls and mistakes for CFD meshing:

  • Choose your turbulence model wisely and make sure your mesh meets the quality metrics for that model.
  • Make sure you don’t have boundary conditions near an area of flow recirculation. If you are getting reverse flows at the boundary then you need to move your boundary conditions further away from the feature that’s causing the flow to swirl in and out of the boundary.

ANSYS 17.2 Executable Paths on Linux


ansys-linux-penguin-1When running on a machine with a Linux operating system, it is not uncommon for users to want to run from the command line or with a shell script. To do this you need to know where the actual executable files are located. Based on a request from a customer, we have tried to coalesce the major ANSYS product executables that can be run via command line on Linux into a single list:

ANSYS Workbench (Includes ANSYS Mechanical, Fluent, CFX, Polyflow, Icepak, Autodyn, Composite PrepPost, DesignXplorer, DesignModeler, etc.):

/ansys_inc/v172/Framework/bin/Linux64/runwb2

ANSYS Mechanical APDL, a.k.a. ANSYS ‘classic’:

/ansys_inc/v172/ansys/bin/launcher172 (brings up the MAPDL launcher menu)
/ansys_inc/v172/ansys/bin/mapdl (launches ANSYS MAPDL)

CFX Standalone:

/ansys_inc/v172/CFX/bin/cfx5

Autodyn Standalone:

/ansys_inc/v172/autodyn/bin/autodyn172

Note: A required argument for Autodyn is –I {ident-name}

Fluent Standalone (Fluent Launcher):

/ansys_inc/v172/fluent/bin/fluent

Icepak Standalone:

/ansys_inc/v172/Icepak/bin/icepak

Polyflow Standalone:

/ansys_inc/v172/polyflow/bin/polyflow/polyflow < my.dat

Chemkin:

/ansys_inc/v172/reaction/chemkinpro.linuxx8664/bin/chemkinpro_setup.ksh

Forte:

/ansys_inc/v172/reaction/forte.linuxx8664/bin/forte.sh

TGRID:

/ansys_inc/v172/tgrid/bin/tgrid

ANSYS Electronics Desktop (for Ansoft tools, e.g. Maxwell, HFSS)

/ansys_inc/v172/AnsysEM/AnsysEM17.2/Linux64/ansysedt

SIWave:

/ansys_inc/v172/AnsysEM/AnsysEM17.2/Linux64/siwave

Video Tips: Fluid Volume Extraction

This video shows a really quick and easy way to extract a fluid domain from a structural model without having to do any Boolean subtract operations.

CoresOnDemand: Helping Engineers Do Their Magic

CoresOnDemand-Logo-120hEngineers Do Magic

In the world of simulation there are two facts of life. First, the deadline of “yesterday would be good” is not too uncommon. Funding deadlines, product roll-out dates, as well as unexpected project requirements are all reliable sources for last minute changes. Engineers are required to do quality work and deliver reliable results in limited time and resources. In essence perform sorcery.

af-01

Second, the size and complexity of models can vary wildly. Anything from fasteners and gaskets to complete systems or structures can be in the pipeline. Engineers can be looking at any combination of hundreds of variables that impact the resources required for a successful simulation.

Required CPU cores, RAM per core, interconnect speeds, available disk space, operating system and ANSYS version all vary depending on the model files, simulation type, size, run-time and target date for the results.

Engineers usually do magic. But sometimes limited time or resources that are out of reach can delay on-time delivery of project tasks.

At PADT, We Can Help

PADT Inc. has been nostrils deep in engineering services and simulation products for over 20 years. We know engineering, we know how to simulate engineering and we know ANSYS very well. To address the challenges our customers are facing, in 2015 PADT introduced CoresOnDemand to the engineering community.

af-02

CoresOnDemand offers the combination of our proven CUBE cluster, ANSYS simulation tools and the PADT experience and support as an on demand simulation resource. By focusing on the specific needs of ANSYS users, CoresOnDemand was built to deliver performance and flexibility for the full range of applications. Specifics about the clusters and their configurations can be found at CoresOnDemand.com.

CoresOnDemand is a high performance computing environment purpose built to help customers address numerical simulation needs that require compute power that isn’t available or that is needed on a temporary basis.

Call Us We’re Nice

CoresOnDemand is a new service in the world of on-demand computing. Prospective customers just need to give us a call or send us an inquiry here to get all of their questions answered. The engineers behind CoresOnDemand have a deep understanding of the ANSYS tools and distributed computing and are able to asses and properly size a compute environment that matches the needed resources.

Call us we’re nice!

Two Halves of the Nutshell

The process for executing a lease on a CoresOnDemand cluster is quite straight forward. There are two parts to a lease:

PART 1: How many cores & how long is the lease for?

By working with the PADT engineers – and possibly benchmarking their models – customers can set a realistic estimate on how many cores are required and how long their models need to run on the CoresOnDemand clusters. Normally, leases are in one-week blocks with incentives for longer or regular lease requirements.

Clusters are leased in one-week blocks, but we’re flexible.

Part 2: How will ANSYS be licensed?

An ANSYS license is required in order to run on the CoresOnDemand environment.  A license lease can be generated by contacting any ANSYS channel partner. PADT can generate license leases in Arizona, Colorado, New Mexico, Utah & Nevada. Licenses can also be borrowed from the customer’s existing license pool.

An ANSYS license may be leased from an ANSYS channel partner or borrowed from customer’s existing license pool.

Using the Cluster

Once the CoresOnDemand team has completed the cluster setup and user creation (takes a couple of hours for most cases), customers can login and begin using the cluster. The CoresOnDemand clusters allow customers to use the connection method they are comfortable with. All connections to CoresOnDemand are encrypted and are protected by a firewall and an isolated network environment.

Step 1: Transfer files to the cluster:

Files can be transferred to the cluster using Secure Copy Protocol which creates an encrypted tunnel for copying files. A graphical tool is also available for Windows users (& it’s freeJ). Also, larger files can be loaded to the cluster manually by sending a DVD, Blu-ray disk or external storage device to PADT. The CoresOnDemand team will mount the volume and can assist in the copying of data.

Step 2: Connect to the cluster and start jobs

Customers can connect to the cluster through an SSH connection. This is the most basic interface where users can launch interactive or batch processing jobs on the cluster. SSH is secure, fast and very stable. The downside of SSH is that is has limited graphical capabilities.

Another option is to use the Nice Software Desktop Cloud Visualization (DCV) interface. DCV provides enhanced interactive 2D/3D access over a standard network. It enables users to access the cluster from anywhere on virtually any device with a screen and an internet connection. The main advantage of DCV is the ability to start interactive ANSYS jobs and monitor them without the need for a continuous connection. For example, a user can connect from his laptop to launch the job and later use his iPad to monitor the progress.

af-04

Figure 1. 12 Million cell model simulated on CoresOnDemand

The CoresOnDemand environment also has the Torque resource manager implemented where customers can submit multiple jobs to a job queue and run them in sequence without any manual intervention.

Customers can use SCP or ship external storage to get data on the cluster. SSH or DCV can be used to access the cluster. Batch, interactive or Torque scheduler can be used to submit and monitor jobs.

All Done?

Once the simulation runs are completed customers usually choose one of two methods to transfer data back. First is to download the results over the internet using SCP (mentioned earlier) or have external media shipped back (External media can be encrypted if needed).

After the customer receives the data and confirms that all useful data was recovered from the cluster, CoresOnDemand engineers re-image the cluster to remove all user data, user accounts and logs. This marks the end of the lease engagement and customers can rest assured that CoresOnDemand is available to help…and it’s pretty fast too.

At the end of the lease customers can download their data or have it shipped on external media. The cluster is later re-imaged and all user data, accounts & logs are also deleted in preparation for the next customer.

CoresOnDemand-Advert-Rect-360w

CFX Expression Language–Part 5: Using CEL for Solution Monitoring

In four previous entries we introduced CFX Expression Language, CEL:

After a break to teach some ANSYS classes in beautiful northern Utah we’re back to conclude our series on CFX Expression Language.

image

In our fifth and final installment on CEL we will discuss the use of CEL in monitoring items of interest while the CFX solution is progressing. Back in the first installment in this series, we showed how to create expressions for results quantities in CFD Post. By creating expressions in similar fashion for results quantities in CFX Pre, we can use these expressions to monitor items during solution.

Here is an example. In CFX Pre we have defined three expressions which are really extracting and operating on results data.

forceX1 = resulting force on one face of the structure
forceX2 =resulting force on another face of the structure
fdiffx = the difference between these two values or the net force acting on the structure

This shows these three expressions in the CFX Pre outline tree:

image

Still within CFX Pre, click on Insert > Solver > Output Control. That will add an Output Control tab on the left side of the CFX Pre window. Click on the Monitor tab, expand Monitor Points and Expressions, and then click on the button near the right of the window below that to add a monitor point:

image

Set the Option to Expression and in the Expression Value box left click then right click to select from your list of defined expressions:

image

The CFX Pre tree will now have one or more Monitor Points listed under Output Control:

image

When we initiate the solution, these user-defined monitor points will be available for real time display in addition to the normal Momentum and Mass and Turbulent kinetic energy monitors. All we need to do is click on the User Points tab during solution to view our expressions as monitors. In the example shown below, Monitor 1 is forceX1, Monitor 2 is forceX2, and Monitor 3 is fdiffx, the difference between the first two quantities. These could have been renamed during their definition to make it easier to understand the monitor plot. Here is a snapshot of the quantities being monitored during the solution:

image

So, as we have seen in the last 5 CFX Expression Language blog entries, CEL exposes a lot of powerful capability to CFX users (and to Fluent users within CFD Post). In this case we have seen how to add additional items to monitor during the solution process. The advantages of this are to help us determine on the fly if the solution is progressing as expected and to give us an idea of the values of certain results quantities before the solution is fully completed.

We certainly hope you now have a better understanding for how CEL can be used to increase the capabilities and useful information available from CFX and CFD Post.

CFX Expression Language – Part 1: Accessing CFD Simulation Information in CFX (and FLUENT)

This week we are presenting an introduction to CFX Expression Language. If you’re not familiar with CFX, it is one of the two CFD tools available from ANSYS, Inc., the other being Fluent. CFX has been part of the ANSYS family of engineering tools since 2003. It is relatively easy to use and can be run stand-alone or tightly integrated with other ANSYS products within ANSYS Workbench. We have some general information on CFX available at this link.

CFX Expression Language, or CEL, is the scripting language that allows us to define inputs as variables, capture outputs as variables, and perform operations on those variables. Through the use of CEL we can be more efficient in our CFD runs and better capture results that we need. With CEL we can access and manipulate information without needing to recompile code or access separate routines besides the main CFX applications.

Also note that since CEL can be used in CFD Post, it is useful for postprocessing FLUENT solutions in addition to CFX, since CFD Post is common to both CFX and FLUENT. There are some things to be aware of regarding FLUENT In CFD Post. This link in to the ANSYS 14.5 Help system explains it:

// User’s Guide :: 0 // 7. CFD-Post File Menu // 7.15. File Types Used and Produced by CFD-Post // 7.15.10. Limitations with FLUENT Files

If you are a user of APDL, ANSYS Parametric Design Language, what I have written above about CEL should look familiar. One difference, though, is that while Mechanical APDL is dimensionless, CFX is not. Therefore, CEL definitions contain units where appropriate.

CEL is typically used in CFX-Pre and CFD-Post. A handy editor is available to assist in the definition of the expressions. Most of the activity is enabled by right clicking.

Virtually any quantity in CFX that requires a value input can make use of CEL, including boundary conditions and material properties. CEL can also be used to access and enhance results information. Expressions defined in CEL can be used in design point studies in ANSYS Workbench, either as input or output parameters.

So, what kind of things can you do in an expression? In addition to accessing simulation information and storing it as a variable, you can manipulate values using operators such as add, subtract, multiply, divide, and raise to a power. You can also use built-in functions such as sine, cosine, tangent and other trig functions, exponent, log, square root, absolute value, minimum, maximum, etc.

There are many predefined values, including some common CFD constants such as pi, the universal gas constant, and Avogadro’s number. The available options are different in CFX pre vs. CFD Post, with relevant choices for each.

In CFX Pre, expressions are accessed by double clicking on Expressions in the tree. That takes you to the expression editor, as shown here:

image

Notice how units are defined for each expression, but they can be mixed if desired.

Regarding CFD Post, the example below shows three expressions defined in CFD Post. The expressions within the box are user-defined. The other expressions listed are setup automatically.

The values for forceX1 and forceX2 are calculated by extracting X-direction forces on two different surfaces. The surface names were defined in ANSYS Meshing in this case, as Named Selections. The value fdiffx is calculated by subtracting forceX1 from forceX2. The resulting value, fdiffx, has been specified as an output parameter in Workbench; hence the P-> symbol next to the name.

clip_image004

New expressions are created by right-clicking in the Expressions tab. The new expression value is given a name, then the definition is input, typically by right clicking and selecting from the menus of available quantities, like this:

clip_image006

The location of application for an expression can also be selected by right clicking:

clip_image008

So we’ve got our variables defined using CEL. Now what? Here are some things we can do with CEL variables:

1. Use them as inputs such as material properties or boundary condition values in CFX. If we are running multiple cases, it is typically much easier to define quantities that we want to vary this way. The values can then be changes in the Expression Window, or if defined as a parameter in Workbench, in the parameters view as part of a parameter study.

2. Use them for reporting results quantities of interest, such as forces at a desired location.

3. Use them as input or output parameters in a design point study or design optimization.

Hopefully this brief introduction gives you a glimpse at the power of CEL. In a future article we will look at using CEL for more advanced functionality, such as applying ramped or time varying boundary conditions, using IF statements, and monitoring expression values during solution.

Monster in the Closet: PADT Goes Live with 512 Core HVPC CUBE Cluster

imageThere is a closet in the back of PADT’s product development lab. It does not store empty boxes, old files, or obsolete hardware.  Within that closet is a monster.  Not the sort of monster that scares little children at night.  No, this is a monster that puts fear into the heart of those who try to paint high performance computing as a difficult and expensive task only to be undertaking by those who are in the priesthood.  It makes salespeople who earn fat commissions by selling consulting services and unnecessary add-ons quake in fear.

This closet holds PADT’s latest upgrade to our compute infrastructure: a 512 core CUBE HVPC Cluster.  No data center, no special consultants, no expensive add-ons. Just 512 cores chugging away at solving FLUENT and CFX problems, and pumping a large amount of heat up into the ceiling.

Here are the specifics:

CUBE C512 Columbia Class Cluster

  • 512 AMD 2.4GHz Cores (in 8 nodes, 4 sockets per node, 16 cores per socket)
  • 2TB RAM (256 GB per node of DDR3 1600 ECC RAM)
  • Raid Controller Card (1 per node)
  • 24TB Data Disk Space (3TB per node of SAS2 15k drives in RAID0)
  • Infiniband (8 Port switch, 40 Gbps)
  • 52 Port GIGE switch connected to 2 GIGE ports per node
  • 42 U Rack with thermal convection ducting (chimney)
  • Keyboard, monitor, mouse in drawer
  • CENTOS (switching to RedHat soon)

We built this system with CFD simulation in mind.  The original goal was to provide a proof of concept to expand our CUBE HVPC offering, showing that you can create a cluster of this size, with very good speed, for a price that small and medium sized companies can afford.  We also needed a way to run large problems for benchmarks in support of our ANSYS sales efforts and to provide faster technical support our FLUENT and CFX customers.  We already have a growing queue of benchmarks waiting to get into the machine.

The image above is the glamour shot.  Here is what it looks like in the closet:

image

Keeping with our theme of High Value Performance Computing we stuck it into this closet that was built for telephone equipment and networking equipment back at the turn of the century when Motorola had this suite.  We were able to fit a modern rack in next to an old rack that was in there. We then used the included duct to push the air up into our ceiling space and moved the A/C ducting to duct right into the front of the units.  We did need to keep the flow going into the rack instead of into the area under the networking and telephone switches, so we used an old video game poster:

image
Anyone remember Ratchet and Clank? 
Best PS2 games ever.

It works well and adds a little color to the closet.

So far our testing has shown some great numbers. Not the fastest cluster out there, but if you look at the cost, it offers incredible performance.   You could add a drive array over Infiniband, faster chips, and some redundant power. And it will run faster and more reliably, but it will cost much more.  We are cheap so we like this solution.

Oh yea, with the parts from our old CFD cluster and some new bits, we will be building a smaller mini-cluster using INTEL chips, a GPU or two, and a ton of fast disk and RAM as our FEA cluster.  Look for an update on that in a couple of months.

Interested in getting a cluster like this for you computing pleasure?  A system configured like this one will run about $150,000 (video game poster is extra). Visit our CUBE page to learn more or just shoot an email to sales@padtinc.com.  Don’t worry, we don’t sell these with sales people, someone from IT will get back with you.