Happy Holidays From PADT to the ANSYS Community

aa1aa2aa3aa4

Well, another year is coming to an end and we are about to enter the Holiday Season.  It is a difficult time for many of us, leaving our computers at work, not being allowed to  log in from home, and being forced to interact with our families.  But after  a long and challenging year, even CAE elves need a break.

We want to take some time to offer a special thanks to all of you out there who read The Focus.  You have stuck with us through the transition from Newsletter to Blog, through all the bad puns, and most of all the ‘B’ movie references.  We do appreciate all the kinds words and comments that we receive.

We hope that all of you have a a fantastic holiday we wish you safe and prosperous simulation in the new year.

SnowManCFD1

Finding the Location of a Value in a Common Block – Again

CMNDIF

Every once in a while you need to do something fairly tedious and you know you’ve written a tool to do it, but you can’t find it. That happened to Matt today when he was trying to remember an easy way to locate an important variable that gets stored in one of the common blocks that ANSYS uses.  We both kind of had an tickle in the back of our heads that one of us wrote something a while back but we couldn’t find it.  “It was in the Focus… search… no luck. 

Then I checked www.XANSYS.org and boom, there it was.  Once we found the name there, we found the focus article in Issue 58 from 2007.  Then I found it on Sheldon’s ANSYS.net

Turns out, it is all over the place… we just couldn’t find it on our own hard drive.  Aging sucks, but now we have Google.  Anyhow, we figured since we forgot about it, other ANSYS MAPDL customization types may also have want a reminder.

And the Composite World Rejoiced: FiberSIM and ANSYS ACP Hook-Up

An early Christmas for those of us who use ANSYS with composites.  ANSYS and VISTAGY just announced that they will offer bi-directional linkage between FiberSIM and ANSYS through ANSYS Composite PrepPost.  The key statement from the press release:

ANSYS, Inc. … , and VISTAGY, Inc., … , have taken a major step in closing the loop between composites design and analysis by integrating ANSYS®13.0 Composite PrepPost and FiberSIM® 2010 composites engineering software. This bi-directional integration, enabled by FiberSIM’s Analysis Interface, allows customers to exchange detailed lay-up information between design and analysis quickly and accurately. As a result, users can optimize parts, assemblies and structures more efficiently.

Read the whole press release at:

http://www.businesswire.com/news/home/20101214005251/en/ANSYS-VISTAGY-Integrate-ANSYS-Composite-PrepPost-FiberSIM

Looks like there will be a Webinar on February 3: http://events.vistagy.com/vistagy-ansys-webcast.

If you are interested, contact your ANSYS Sales professional.   No word on when it will be released, but our guess is after the February webcast.

Quick Note: Nice Article on GPU Usage at R13 with ANSYS Mechanical Solvers

image_thumbI was chatting with a  product manager at ANSYS, Inc. about HPC. We got on the topic of GPU’s and he pointed out that there is a good article on the ANSYS Advantage magazine website on this very topic.

We get a ton of questions all the time about how GPU’s work and the kind of performance you can get from them.  This article goes into some good detail and has benchmarking data on R13.  Worth a read!

http://www.ansys.com/magazine/issues/vol4-iss2-2010/AN-Advantage-GPU.pdf

HPC on the Cheap: 48 cores for $11k

imageEvery once in a while computer hardware takes a big step forward, either in performance or lower cost.  Recently AMD, Intel’s biggest rival in CPU chips, and Super Micro Computers, a maker of high-performance motherboards and other hardware, have both come out with products that have changed the landscape of affordable High Performance computing forever.  About a month ago PADT put together a single system with 48 cores, 128 GB SDRAM, and 3TB of disk for just over $11,000.  And it is not too good to be true, the machine is fast and reliable.

We were able to get this type of price performance by leveraging the 12 Core server processors coupled with a motherboard that supports four chips: 4×12 = 48, something that only exists in the AMD world. We kind of stumbled upon this when our IT guy thought it would be good to see what AMD is up to.  We have been buying and using Intel based systems for a while now since they leapfrogged AMD the last time.  Once he started hunting around he found that AMD still supported four chips on a board, and that they had working 12 core chips.  So he went off to New Egg and built a shopping cart.  It came up at around $10,000 because we already had some of the hardware we needed.  Next came his hardest task, convincing management (me) that it wasn’t too good to be true.

After some research we found that others were benchmarking this chip and finding good performance on parallel tasks.  It is a slower clock speed than the top Intel chips, but because you can fit so many cores in one box, you get better performance for your money on applications that are parallel – like CFD.  After pulling the trigger on the order and sweating a bit the order showed up:

clip_image002

An Early Christmas at PADT:
A Barebones 1U Box, 4 Parts (48 cores), 32 SDRAM Sticks, and 3 Hard Drives

This created a bit of a buzz in the office, almost as big as when one of our customers brought his Tesla over for a test drive.  For those of us who started our career running on million dollar “super computers” with the same power as my cell phone, this was a pretty cool site to see.

The Box:

Here is what we purchased:

4 AMD Opteron 6174 Magny-Cours 2.2 GHZ 12 Core Server Processors 48 Core
32 4GB DDR3 Kingston 240 pin Registered SDRAM 128 GB RAM
3 1TB Western Digital RE3  7200 RPM, SATA 3.0 Hard Drives 3 TB Disk
1 80GB Intel X18-M Solid State Drive 80 GB / Drive
1 Supermicro AS-1042G-TF 1U Barebones Server 1 Box

We put it together, loaded it with CentOS and booted it up.  We got a little silly when we saw the boot screen:

clip_image002[4] 

None of us had seen a boot screen with a count of 48, nor one with that much RAM on a single system.  And we had to keep telling ourselves that it was only $11,000.

We had no issues with drivers or software.  ANSYS FLUENT, CFX and Mechanical APDL loaded up just fine and ran smoothly.

The Benchmarks:

The first thing we did is run a customer problem that was given to us as a test for a sale. The sales was being held up because we couldn’t get time on our existing systems to run it.  No surprise, it ran fast.  As we upped the number of CPU’s we could see that for a medium sized CFD problem, it was scaling well to 24 cores, then tapered off above that.  Once that problem was done we ran the ANSYS standard benchmarks to see how larger problems went.

First we ran the CFX benchmarks. They use something called “Speedup” which is simply the run time on one core divided by the run time on N cores. Or, how many times faster it is.

imageimage
imageimage

As you can see, you get the usual variation from problem to problem.  The Indy Car run actually did better than parallel, indicating that there is some sort of bottleneck with one processor and something about that problem that this system likes.  They other cases did well against a much more expensive IBM XEON system with a faster clock speed CPU.

Next, we looked at the performance on FLUENT.  They must have a bigger staff for benchmarking because we had a lot more machines to compare to.  But they use a different scale called a “Rating” It is 24 hours divided by the run time in hours on N Cores.  So a 1 would mean it runs in 1 day and 256 means it runs in 5.625 minutes.

imageimage

imageimage

Our takeaway from these studies is that 1) the system is pretty linear on large problems, and 2) that although it is slower than other systems, it is a lot less money, sometimes less than 25% as much.

The next thing to look at is what the speed is per core between Nehalem and AMD based systems.  For the data we have on FLUENT problems we compared a 2.9GHz Intel Nehalem based system to our 2.2GHz AMD system. These other systems have faster drives and cost 3 to 4 times as much.  What we found was that our AMD system was anywhere from 52% to 65% of the speed of the faster clock speed Intel chips.

Remember, to get to 48 cores on the Intel platform you will need six systems or blades, because each system or blade can only support two chips with four cores per chip, so 8 cores per board.  Let’s say the Intel chips are twice as fast.  So to get the same performance you would need 24 cores, or three 8 core boxes.  Then you need Infiniband with a switch and three cables.  So now you are talking three to four times the cost of the 48 core single box AMD system.

imageDon’t Forget Power

One of the knocks against the last time AMD had really competitive chips was heat and power consumption.  Well, these guys run cool and low power. We found that the power consumption maxed at 800 watts and average 760. When you consider that there are 48 cores chugging away at 100%, that is pretty impressive.  17watts per core!

  • Max Watts reached during test: 800
    • Watt usage settled down to around: 761
  • Number of hours for usage test: 167
  • Max Volts: 116 Volts
  • Amps: 6.60
  • $ rate used per/kWh: $0.11
  • Costs per/kWh:
    • Hour: $0.08
    • Day: $1.98
    • Week: $13.89
    • Month: $59.55
    • Year: $724
  • 200 Watts per part (CPU)
    • 17 Watts per core

Other Things to Consider

This particular configuration is very exciting, and it also opens up a lot of doors to other options and combinations.  The power of this system comes from parallel processing, so you need to consider the cost of adding parallel solves. Right now ANSYS, Inc. has a licensing option called HPC packs that are perfect for this application.  If you buy one pack, you can run on 8 cores, 2 packs is good for 32 cores, and 3 packs allow you to run on 128.

This doesn’t line up to well with 12 cores per chip.  But AMD does make a chip with 8 cores that run at a slightly higher clock speed (2.4 GHz vs. 2.2 GHz) that also cost less ($750ea vs. $1299ea)  So you could buy two HPC packs for 32 cores and then put together a system for less than $10,000.

Or, if that is not enough punch, you could also upgrade the motherboard to the next level up and purchase two barebones server boxes.  This allows you to use Supermicro’s Infiniband card to connect two boxes together over Infiniband without a hub.  Now you are talking 96 cores connected with a high speed interconnect, 256 GB of RAM, and all for under $27,000.

The last thing to consider is the fact that AMD has announced that they will be releasing a 16 core server chip to be widely available in 3Q11.  These will work with the motherboards we are talking about here, so you can upgrade your machines to have 4 more core per chip – so if you invest in a 96 core dual box system now, you can upgrade it to 128 cores next year.  Not bad.

Right for You?

After the initial excitement dies down for this much horsepower for this little of an investment, most users will ask themselves if this type of system is right for them.  You need to think it over and ask yourself some questions:

  1. Will I benefit from that much parallel?
    Remember, a single core actually runs slower than a top of the line 2.9GHz Intel system.  If you run Mechanical or have small problems, you may be better off paying for fewer, faster cores.
  2. Can I build my own system?
    We have the luxury at PADT of not having a corporate IT department that would freak out over a home-built system.  We also have an IT department with the skills to put systems together.  Run this idea by your IT before you go to far.  You can purchase systems with similar guts from HP and Dell, but a large bit of the cost savings go away when you do that.
  3. Do I have the Parallel Task Licenses?
    See what parallel you already own and contact your ANSYS sales person to see what it would cost to add more, especially HPC Packs.

If your problems are large and have long run times, this hardware and the needed software licenses will pay for themselves very quickly.  For us, a job that was running over 5 days now finishes in half a day.  That is huge as far as being able to either up the problem size to get more accuracy or getting results to our customers much faster.

Interested in Getting a System from PADT?

Several of our customers have expressed interest in purchasing our leasing one of these low-cost cluster systems from PADT. So We are offering a purchase or a lease on custom systems if you are located in Arizona, New Mexico, Colorado, Utah or Nevada.  Shoot
sales@padtinc.com an e-mail if you would like more information.  If you are outside of PADT’s ANSYS sales territory, we are more than happy to answer any questions you may have via e-mail to help you put your own system together, but we just don’t have the bandwidth to support outside of our area.

General Update on the Blog and News

Well, we are now a couple of weeks into this blog experiment and it is going well.  As part of being more "bloggy" and less "newslettery" we are going to stick in little updates and news items like this.

First off, thanks to a few dedicated users who requested a better way to print out the articles. We found a chunk of code that we added in to the templates for the postings and you should now see a "Print" option down in the lower right corner (with Permalink and comments) of each posting.  this brings the posting up in a separate window that is more printer friendly.  Just print this from your browser and you have hard copy.

The next thing we are going to work on are comments.  We had it turned on for about two days when we went live and received some incredible offers to visit sights where we could: 1) increase our "size" in a few weeks, 2) obtain various drugs very cheaply, 3) meet a very nice girl that wants to show off her new webcam, and 4) help the widow of a former Nigerian minister of the interior get some money back from where the a bank is holding it.  So we turned off commenting. We are looking at a couple of open source tools that have spam filters for commenting and will hook up to one of those to get comments working.

In the area of news, we are very happy to report that our last seminar on Geometry Tools in ANSYS, Inc. Products had 145 people signed up and about 90 attendees.  We’ll post upcoming seminars in the right column of this blog, or you can subscribe to receive invites by using the same subscription service that controls your e-mail about this blog.  Click the arrow to the right to manage your subscriptions.

Thanks!

Learning about blogging

Update: 4:50,

For any of you who are blogging or thinking of it, I just tried out the free tool from Microsoft: Windows Live Writer.  It works very well.  I will try the next couple of articles that way.

– Eric

 

Well, it seems that it is a bad idea to paste from MS Word into the Blog editing tool.  Funny enough, you can view such an entry on Firefox or Chrome, but it doesn’t work on MS Internet Explorer ?!?!

So this wednesday’s post has been fixed with a new version that was pasted as text then formated in the blog editor.  Live and learn.

 

– Eric Miller

FE Modeler Part 1: Translating Meshes with ANSYS FE Modeler

Ever get thrown an old NASTRAN model or been asked to convert your model into ABAQUS? Did you spend hours scouring the internet for a free translator?  Did you know that ANSYS software came with a translator for FLUENT, CFX, ABAQUS, NASTRAN , STL files and a host of other formats?  Well it does and it comes free with most products. It is one of the least known jewels in the ANSYS product family.

FE Modeler is a module developed to handle some of the mesh based capabilities found in Mechanical APDL that don’t really fit into the paradigm of what is now called ANSYS Mechanical.  Over the years it has grown to be a very useful tool for translating models, reviewing meshes, morphing meshes, and even converting meshes into geometry that can be re-meshed.  In this article we will talk about the translators and quality tools and will address the morphing and geometry-from-mesh tools once R13 comes out in November.
The first thing to know is how to get to this useful tool.  On the workbench page, it is in Toolbox under Component Systems with the name Finite Element Modeler.  Figure 1 shows where it is located.

Figure 1:  Location of FE Modeler Tool

There are a couple of ways to use the tool.  The first is to drag it onto a mesh that already exists in your project.  This is the best way to proceed if you meshed with Workbench meshing or a system that uses Workbench Meshing (Mechanical, FLUENT, CFX, etc…). You can also connect it to the Setup block on an ANSYS Mechanical system.  You can of course connect to an FE Modeler system by right clicking on a mesh and choosing “Transfer Too New… -> Finite Element Modeler” Figure 2 shows some examples of what it looks like once you have connected.

Figure 2: Connecting to a Workbench Mesh or Model

If you are not starting with a Workbench related mesh or model, you can still use the tool.  This is the most common method for reading in NASTRAN or ABAQUS meshes.  Simply drag and drop the Finite Element Modeler System to a blank spot on the Project Schematic.  Once there you can specify your input file in one of two ways: double-click on the Model (brute force, take charge, throw caution to the wind approach) or right click on the Model and choose “Add Input Mesh” (take your time, make sure it is right, no risks approach).  And yes, it says “Add” because you can specify multiple meshes, an added bonus. Figure 3 shows an example of what this will look like.

Figure 3: Stand Alone FE Modeler Systems


When you are in the browser dialog you can see the various formats that are supported (Figure 4).  These are also summarized in Table 1:

Figure 4: Input Options

Table 1: Supported Input File Summary

ABAQUS (*.inp) MAPDL (*.cdb)
CFX (*.def, *.res) NASTRAN (*.bdf, *.dat, *.nas)
ANSYS WB Meshes (*.cmdb, *.meshdat) Mechanical (*.dsdb, *.mechdat)
Fluent (*.msh,*.cas) STL (*.stl)
ICEM CFD (*.uns)

Now you have a mesh defined.  The next step is to read it in to FE Modeler.  You can do this by double clicking on the model or RMB->Edit.  The program will now read in your file, and display a nice animated spinning gear to keep you occupied.   The numbers in feedback in the Import Summary (the default view) also update as the file is read.Once in FE Modeler you will see a pretty standard layout for a Workbench application.  The tree on the left, Details view on the lower left, and a graphics window.  To start with Import Summary will be selected in the tree and a description of what was read in is shown.  There is a lot of useful information in this view. Take some time to look at each table and see if it makes sense.  Probably the most important table is Table 4.  It shows feedback from the import.  If the reader ran into any entities it didn’t recognize or any lines it could not read, you will see feedback here.  This is important because there often are not one-to-one mappings between programs so some entities will not read in.  You will be able to see those in this table.  Figure 5 shows the output from reading in a basic test model from a NASTRAN file.


Figure 5: Typical Feedback from Input

Now, if you want to see your model, you can click on other branches in the tree.  The Element Types allows you to view by element topology and Bodies will show the contents of a given file.  You can also interrogate the mesh, selecting nodes, external faces, or elements and viewing their position. Take a look at the icon bar, it is pretty standard for Workbench and everything is self explanatory. You can see mesh metrics by choosing Insert from the menu and then picking Mesh Metrics.  Once it is in the tree, click on it and change the options in the details view.  Figure 6 shows the test model and some quality metrics.

Figure 6: Mesh Metrics

The last step is the best part, writing out in the new format that you want.  Find the “Target System” drop down at the top of the icon bar, and choose from Mechanical APDL, ABAQUS, NASTRAN, and STL.  Then select “Generate Data” in the model tree and the program will create an output file in the format you want.  This can take a while for a large model.

Figure 7: ABAQUS Output

If you are an expert in the program you are writing to, you can check this file out and see what is in there.  If you like what you see, or don’t care and just want your output file, click on the “Write Solver File” button on the top icon bar.  Specify a file name, and you are done.
There is one last important thing to mention. If you want to control your import a little, go back to the project page and click on the model.  RMB and choose “Manage Input Meshes”. This will then bring up the outline for the schematic and you will be able to set options for each file you specified for input. (Figure 8) For most files the only things you can change are units, how to group bodies and how to number things.

Figure 8: Options for Inputting of Files

Nothing too complicated, it does what it does and it does it fast.  To learn more play with it and read the help.  We hope you find this hidden tool as useful as we have in the past.


Welcome to the New “The Focus”

In January of 2002, Rod Scholl did what a lot of people talk about doing but never actually do.  He published a newsletter for ANSYS users.  Called “The Focus” it was created with a very simple concept: users want a consistent and frequent publication with short, useful articles that will help them use ANSYS better. Tips and Tricks on a regular basis.  Short and to the point was the main purpose, which is why it was called The Focus.  The second concept was also evident in the fact that “The Focus” masthead was a blurred image. We wanted this publication to be fun and poke fun at itself, PADT, and nerds, we mean ANSYS users, in general.

Since then we have published 74 unique issues.  That is 93 months since that first issue, and with 74 issues we met our goal of being almost monthly! Rod really deserves a lot of credit for making sure that we did not do what most companies do when they start a newsletter – put out a few issues then get too busy to continue and let it die.  As of today, we have just over 1,950 registered subscribers, and there is no way to tell how many users around the world access the archives without subscribing. 

But, as time went by we started to make bigger and fancier issues, and took longer and longer between publishings.  We moved to a PDF file that looked fancy but took a lot of time to lay out and publish.  We basically drifted away from that initial concept of short, useful articles that come out often.  We also started feeling like dinosaurs in that we were using a “magazine like” format.  It was time to go 2002 and move to a blog instead of a newsletter.  And here it is, welcome.

The switch has one thing that bothers us.  We now publish articles one at a time, shooting for four a month, hopefully resulting in one a week.  This is great for users but we can no longer number issues.  So there will be no Issue 75, which is sad. Instead, we will count years. The Focus is almost 8 years old now, maybe our grand children will be around for the 75th year?

We have published our first technical article with this welcome.  All 74 previous issues are also available as linked PDF files, so you can search from this site.  If you are a subscriber, you will continue to get e-mails for a while, but we will be moving to RSS feeds (use the subscribe button at the top of this page) or the Newsletter function on this blog.

Thank you to everyone that has passed along their good wishes to us over the years, that is what keeps us going.  And thank you to everyone at PADT who has contributed to this publication and who will contribute in the future.

We hope you all continue to find it useful.

 – PADT’s Technical Support Staff

 


Banishing the Bad Geometry Blues with Design Modeler

(NOTE: This article is a reprint from the final PDF version of The Focus, Issue 74.  We used it to prove out the new blog format.)

We have all been there before.  You are poised in front of your computer, you just read in your geometry and you throw on a quick mesh to see what you have and how much effort this will take.  Next thing you know you are getting errors or red lines are showing up on your geometry.  You got bad geometry and now you need to clean it up.

There are a lot of options available including: 1) going back to the original CAD and fixing that, 2) using a repair tool like CADFix, 3) using some intermediate geometry tool that is good at repair like SpaceClaim or IronCAE, or 4) meshing in ICEM CFD which is much more forgiving with bad geometry.  All of these work and may be the way you have dealt with this situation in the past.  But you should also be aware that ANSYS DesignModeler has an array of tools purpose build to help you solve this problem.

The first things to try are the options when you read your geometry in to DesignModeler.  Figure 1 shows the appropriate details view.

Figure 1: Import Options

You can have the program simplify geometry, simplify topology, heal the body or clean the body.  It can also try and replace any missing geometry.  By the way, the options available change based upon where your geometry came from so if you do not see all of them, that is normal. Heal geometry is on by default, as is clean.  Sometimes you may not want them on.  Play with simplifying as well.  Sometimes modifying these options can clean everything up.

If the automatic stuff does not work , it is time to look at the tools available to you in DM.  The first to look at is the “Small Entity Search” Just as the name implies, it goes out and identifies where you have small entities and gives you their size.  You access the command a little differently in that it is not something you insert into the tree.  Go to Tools > Analysis Tools > Small Entity Search. This will bring up the Details View shown in Figure 2.

Figure 2: Small Entities

Pick one or more bodies you want checked and set the options, although the defaults are usually good.  Then, this is the strange part, change the option next to “Go!” to “Yes” It will do the search and present the results down below.  Click on any item listed and you will be able to see it on the screen.  This should help you understand where you might have problems before you go in and start fixing things.

Our favorite repair tool in DM is the Merge tool.  It takes edges or surfaces that are connected without any sharp corners and merges them into new smooth entities.  Figure 3 shows a typical example for edges and Figure 4 shows the same for surfaces. 

Figure 3: Merge Edges

Figure 4: Merge Faces

You can do an automatic merge or pick entities that you want merged.  The automatic is a great way to get rid of small slivers without having to hunt them down.  This tool is also useful for simplifying your geometry in order to get a better mesh.  Figure 5 shows how you can really clean up a fillet and the suction side of a turbine blade. 

Figure 5: Merge Faces Ceanup

Sometimes you just want to de-feature your model by deleting holes, fillets, bosses, etc…  You can do most of that with the Edge and Face Delete commands.  They can be found under Tools and they work as you would expect. You simple identify the face or edge you want removed and the program takes it out and heals the solid.  Figures 6 through 8 show examples.  We find that this is the most efficient and controlled way to disfeature a model and get rid of tiny geometry that is causing issues.

Figure 6: Delete Faces to Remove Fillets

Figure 7: Remove Edges

Figure 8: Remove Faces to Remove Features

If, after using the tools mentioned above, you still have the bad geometry blues you can get down and dirty with a set of repair tools that address the most common  issues.  They can all be found under Tools-> Repair.  The names are pretty self-explanatory.  There are tools for removing slivers, spikes, small edges, small faces, seams, holes, and sharp angled surfaces.  Most of them work the same way – you can specify a size and any features under that size get cleaned, or you can pick on geometry. The remaining figures show examples of the various options.

Figure 9: Repair Tools

One small side note before we finish, most of the options available in the menu can be displayed as icons in the tool bars if you go to Tools->Options->Toolbars. If you are doing a lot of repairs, we recommend that you add the tools you are using to the toolbars.  Another important thing that users should know is that you can repair geometry and save it as an ANSYS ANF file, the native file format for ANSYS Mechanical APDL. So even if you are not meshing and pre-processing in Workbench, you can still use this tool to clean up your geometry.
After spending years repairing bad geometry we are thrilled to have these tools in our main software product, removing the need to jump out to other programs.  Taking the time to learn these tools, when they work and when they don’t and understanding their options has paid off in chasing away our bad geometry blues.