Caps and Limits on Hardware Resources in Microsoft Windows

Sometime around 3am last October I found myself beating my head up against a server rack. I was frustrated with trying to figure out what was limiting my server hardware. I was aware of a couple limits that Microsoft had placed into its OS software. However, I had no idea how far reaching the limits were. I figured it would be best if I had a better understanding of these hardware limits. So I researched the caps that are placed on the hardware by two of the most popular Operating Systems on the planet: Microsoft Windows 7, Windows Server 2008 R2 editions and REHL.

I have compiled this easy to read superlative analysis of my facts and findings. Read on, I know you are curious to find out what I’ve uncovered.


“They capped and limited US!"

Microsoft Windows Operating Systems

·        Windows 7

a.     Professional / Enterprise / Ultimate

                                                              i.      Processor: 2 Socket limit (many cores)

1.     Core limits:

a.     64-bit: 256 max quantity of cores in 1 physical processor

b.     32-bit: 32 cores max quantity of cores in 1 physical processor

                                                            ii.      RAM: 192 GB limit to amount of accessible

b.     Home Premium

                                                              i.      RAM: 16GB

c.      Home Basic

                                                              i.      RAM: 8GB

d.     Starter Edition

                                                              i.      RAM: 2 GB


·        Windows Server 2008

a.      Standard & R2

                                                              i.      Processor: 4 socket limit – (many cores)

1.      (4 – Parts x 12core) = 48 cores

                                                            ii.      RAM: 32 GB

·        Windows Server 2008 R2 Foundation  (R2 releases are 64-bit only)

                                                          iii.      RAM: 128 GB

·        HPC Edition 2008 R2 (R2 releases are 64-bit only)

                                                         iv.      RAM: 128 GB

·        Windows Server 2008 R2 Datacenter (R2 releases are 64-bit only)

a.     Processor: 8 socket limit

b.     RAM: 2TB

·        Windows Server 2008 R2 Enterprise (R2 releases are 64-bit only)

a.     Processor: 8 socket limit

b.     RAM: 2TB

Red Hat Enterprise Linux – 64-bit

·         Red Hat defines a logical CPU as any schedulable entity. So every core/thread in a multi-core/thread processor is a logical CPU

·         This information is by Product default.  Not the maximums of a fully licensed/subscribed REHL product.

a.     Desktop

                                                              i.      Processor: 1-2 CPU

                                                            ii.      RAM: 64 GB

b.     Basic

                                                              i.      Processor: 1-2 CPU

                                                            ii.      RAM: 16 GB

c.      Enterprise

                                                              i.      Processor: 1-8 CPU

1.     RAM: 64 GB

d.     *** Red Hat Enterprise Linux: Red Hat would be happy to create custom subscriptions with yearly fees for other configurations to fit your specific environment. Please contact Red Hat to check on costs.


AMD Opteron Magny-Cours Processor Wins InfoWorld Best CPU (Parallel Processing)

It was nice to see that the chip PADT has been using to build very inexpensive compute clusters just got recognition from an industry leading magazine:

Read the InfoWorld article here.

When we talk about these systems some folks look at us like we are a bit insane, “AMD can’t be a better solution than Intel”  It is good to see that some experts are agreeing with us.  Makes us feel less lonely out there waving the Opteron flag.

ANSYS CFX: Going 2D in CFX

If you have had the chance to work with CFX, most likely you have been working only with three-dimensional models. You usually receive a CAD model or create your own geometry and then use one of the many meshing tools available in ANSYS to mesh your 3D geometry. However, what if you are tasked to solve a simple two-dimensional model? Without thinking twice about it you create a 2D planar or axisymmetric geometry and generate a mesh. Unfortunately, when you try to read this simple 2D mesh into CFX you get an error while trying to load it. Unlike FLUENT, CFX does not have a separate 2D and 3D solver and doesn’t read in simple 2D meshes. This may first appear as a big problem but the work-around is rather simple. This article will give you the information needed to solve “2D” problems in CFX.

Since CFX is naturally a 3D solver, using a planar or axisymmetric 2D mesh is not possible. Instead, you will need to create a thin 3D volume or a thin 3D wedge, respectively. You can call these thin geometries “quasi-2D.” For example, consider the 2D planar geometry shown in Fig. 1 that represents a channel with flow entering through the boundary on the left and exiting on the right boundary. The upper and lower boundaries are no-slip walls. To solve this type of problem in CFX, you need to give the planar geometry some thickness in the 3rd dimension.


Figure 1: Simple 2D planar geometry representing a channel.

The first question is how thick or thin should you make the volume. You need to consider the quality of the cells so creating a thin volume usually works well. For a 2D planar geometry, a thickness of approximately 1/100th the length of the largest dimension in the model generally provides a nice mesh with good quality cells. Depending on possible smaller features in the geometry, this rule-of-thumb value can be adjusted. Thus, when you create your geometry, you can build a volume with the proper thickness or simply extrude an existing 2D planar model.

Figure 2 shows the channel geometry from Fig. 1 extruded in the 3rd dimension using the rule-of-thumb value from above. Note that if you create an extremely thin volume it may be difficult to select and to see the thin side areas for naming the boundaries. A very thin model is also not necessary and may affect the quality of the cells. The CFX solver is very robust so the thickness can vary quite a bit without any problems; however, it is best to stay consistent and using the suggestions outlined in this article are recommended.


Figure 2: Quasi-2D thin volume representing a channel.

Since the model is a 2D representation, you can just use a single cell thickness in the 3rd dimension. You can mesh the 2D face the same way you would mesh a regular 2D planar model. Once meshed and loaded into CFX, you can easily apply your inlet boundary, outlet boundary and upper and lower wall boundaries. The question now is what to do with the faces in the 3rd dimension. In the example above and with any geometry that represents a 2D planar model you will need to use symmetry boundaries.

Creating a “quasi-2D” model for a simple 2D planar geometry is easy enough. What about creating a model for a 2D axisymmetric geometry? In this case, we need to extrude a wedge. Again, the question is how many degrees should the wedge be extruded.


Figure 3: Quasi-2D thin wedge representing an axisymmetric pipe.

For a 2D axisymmetric geometry, a wedge having a thickness of about 5 degrees works just fine. An example of a 5 degree extrusion for an axisymmetric geometry representing a pipe is shown in Fig. 3. Once again, you only need a single cell in the 3rd dimension. There is no need for any special meshing near the axis. You can mesh the planar face just as you would a normal 2D geometry and take into consideration the need to resolve gradients and other important features in the flow. After loading the mesh into CFX, you apply your inlet boundary, outlet boundary and the outer wall boundary. There is no need to apply any boundary at the axis. There are now two options for the boundary condition on the faces of the 3rd dimension. If there is no flow in the 3rd direction, then you apply a symmetry boundary just as is done in the 2D planar case. If you are trying to model an axisymmetric problem with swirl, then you will need to apply a periodic boundary condition on the faces. Remember to use a periodic boundary only if you have swirling flow.



This article shows that solving a “2D” model in CFX is relatively straightforward. Except for the step of having to create a thin volume or wedge and applying the extra boundary conditions (symmetry or periodic) as specified in the article, setting up these types of models is the same as any other 2D solver and the results are basically identical. A helpful hint that may prove useful is as follows. The CFX solver requires that symmetry boundaries be planar to within a certain tolerance and sometimes a symmetry boundary mesh fails to meet this tolerance. This will require that you go back and remesh the model and ensure that the symmetry planes are essentially flat. However, if the solver continues to complain, you can change the offending symmetry boundary to a wall with a slip condition. The resulting solution will be the same as using a symmetry boundary.

Questions, comments and suggestions for future thermal/CFD FOCUS articles can be sent to J. Luis Rosales at the following email address:

IT Tips: Setting Up FreeNX for Remote Access to Linux Machines from Windows

How To Install FreeNX with Key-based authentication on CentOS 5.5 in a Windows Platform Environment


In this article I will show you how to install and configured one of our more popular remote access programs here at PADT, Inc.  The software that I will be showing you how to install is called FreeNX.  FreeNX is the opensource version of a product distributed by the company called NOMACHINE,  Another program that the Analysts have struggled with over the years and still is a favorite with some of the Analysts here at PADT, Inc. is a program called CYGWIN However, the analysts seem to prefer the fast interface and overall robustness of the FREENX server.

Within this guide I will attempt to break the install down into two components so that Network Administrators and IT Managers are able to have this up and running in 30 minutes. For my installation I used the How To install NX Server using FreeNX guide freely provided by the CentOS wiki site as a reference.

Of course it’s okay to install NX server on FreeNX using the graphical user interface


1.       Install FREENX on your CentOS 5.5 server. My test NX server is a DELL Poweredge 1950 server, that has an install of CentOS 5.5 64-bit edition.

a.       Console installation

                                                               i.      Login as root console

                                                             ii.      Click Applications à Add/Remove Software

                                                            iii.      You will see the following dialogue box.

                                                           iv.      clip_image002[3]_thumb[2]

                                                             v.      On The left had side scroll down the list until you see CentOS extras. On the right window select FreeNX and NX. Then select Apply. A screen shot is shown below.

                                                           vi.      clip_image004[3]_thumb[1]

                                                          vii.      Allow the dependencies check to run. When you see the image that looks like the one below prompting to Cancel or Continue. Select Continue

                                                        viii.      clip_image006[3]_thumb[1]

                                                           ix.      Next, Allow the system to download and install the packages.

                                                             x.      clip_image008[3]_thumb[1]

                                                           xi.      When it has completed the checkmark on FreeNX and NX will be selected and you can now close out the package manager.

STEP 2  – SERVER: Terminal Session command line changes for FreeNX Key-based authentication

1.       You may have thought you could get away with a GUI only installation of FreeNX but not this time.

2.       Click Applications à Accessories à Terminal

a.       clip_image010[3]_thumb[1]

3.       For my server example I choose to use vi as my text editor. However you may choose to use whatever text editor you may like. I guarantee any Linux user will immediately be impressed if you open up a terminal session and start editing the file using vi.

First lets go over of my most used vi commands: vi basics for Windows Network Administrators

                                                                           i.      i = insert text

                                                                         ii.      d = delete text

                                                                        iii.      d[space bar] = please delete the one piece of text to the right.

                                                                       iv.      dd = opps did I just delete the entire line?

                                                                         v.      :q! = Eek! get me out of here now I screwed up the file really bad. No I don’t want to save the file right now!

                                                                       vi.       :wq! = Yes save the file right now

                                                                      vii.      For further research and light reading please go here:

Editing the sshd_config file using vi

b.      Within your open terminal window type:

c.       cd /etc/ssh/


d.      Login as root. Because you are the Network Administrator or IT Manager. Hopefully you are already logged in as root

e.      Next type, vi sshd_config (reference or print out David’s most used vi commands first!)

f.        clip_image014[3]_thumb[1]

4.       Now don’t panic…using your down-arrow key tap down until you get to the area that reads Password Authentication. Remove the # mark by pressing (gently) d then the space bar. This will remove the # (remark) comment.

5.       Per the documentation modify if you have disallowed ssh password cleartext tunneled passwords. You will need to make the PasswordAuthentication  line to look like the above highlighted text.

6.       Next, add the following line below the text PasswordAuthetication yes

                                                                          i.      Within vi press i  

1.       This will put you into INSERT mode.

2.       Add the text AllowUsers nx

3.       Add any additional users similar to how I have it above.

a.       :ie.  AllowUsers nx userid1 userid2 Ansys, Inc.

4.       Now that you have that text added you have the vi basics for Windows Network Administrators command lists from above press. :wq!

Configure the NX database to allow passthrough authentication.

Editing the node.conf file using vi

7.       Next, we need to edit the node.conf file within the /etc/nxserver/ folder

a.       You should be back to your terminal session

b.      Within your open terminal window type:

c.       cd /etc/nxserver/

                                                                           i.      clip_image016[3]_thumb[1]

                                                                         ii.      vi node.conf

1.       With your vi editing session open tap down to the place in the file that reads  ## Authentication / Security directives

a.       clip_image018[3]_thumb[1]

b.      Un-remark # the ENABLE_PASSDB_ AUTHENTICATION =”0” and modified to read as the one highlighted above.

                                                                                                                                                   i.      ENABLE_PASSDB_AUTHENTICATION=”1”

Add your nx users to the NX Database

Add yourself to the nxserver database.

Suppose your username is ansys

·         [root@ben1]# nxserver –adduser ansys

·         NX> 100 NXSERVER – Version 1.5.0-60 OS (GPL)

·         NX> 1000 NXNODE – Version 1.5.0-60 OS (GPL)

·         NX> 716 Public key added to: /home/ ansys /.ssh/authorized_keys2

·         NX> 1001 Bye.

·         NX> 999 Bye

Assign a password for the userid’s.

Add your nx server password to the NX Database

·         [root@ben1]# nxserver –passwd ansys

·         NX> 100 NXSERVER – Version 1.5.0-60 OS (GPL)

·         New password:

·         Password changed.

·         NX> 999 Bye

Verify that you have added userid1on the AllowUsers line in the /etc/ssh/sshd_config file: and then reload sshd:

·         In your Terminal session type:

·         service sshd reload


·         clip_image020[3]_thumb[1]



·         You can download FreeNX windows client from here and install. Follow the instructions.

Key points to remember on your Windows Client Installation:


·         Follow the excellent instructions provided by NOMACHINE – However.

·         I suggest that you change Desktop to GNOME,

o    Ansys, Inc., Inc. installations prefers the GNOME desktop. Also increase your bandwidth slider to you current network connection.

·         I choose:

o   Host: your NX Server

o    LAN

o   Display of 1024×768 (or all available)

·         clip_image022[3]_thumb clip_image024[3]_thumb


The critical piece to finish the installation is to copy and paste the client key from the nxserver into your Windows Client install.

·         Located under the GENERAL tab (see image above) Click Key… and delete the key within your client installation.

o   Paste in the key from your new nx server install

o   Next, click Save.

·         To locate your nxserver key copy the text out of the file located in the directory:

o   /etc/nxserver/

§  vi client.id_dsa.key

§  copy the text out

o   client.id_dsa.key – Copy all of the text from this file and paste it into the Key.. file on your Windows Client installation.

§  As root user – highlight the text and Copy then paste into your Client

§  Again the location of the client.id_dsa.key is below

o   clip_image026[3]_thumb[1]






New Year, New Articles, And New Features on The Focus Blog

Sorry about the multiple posts of the customization articles.  We are still figuring out the most efficient way to post things to this blog.  LiveWriter seems the best, but we have trouble editing a post or publishing with formating.  Still learning.

We also upgraded to BlogEngine V2.0.  Things seem more stable and a bit faster.  

And lastly, we hooked the blog up to  This is a community sight for blog comments that does spam filtering and controls access a bit.  We hope it will keep the offers for a very special private video chat from Amber, a bored student, to a minimum.  

The article buffer is filling up.  Look for information on a nice tool for logging on to Linux machines from Windows (NX), more information on using FE Modeler, and some useful CFD hints and tips in the comming weeks.

ANSYS Mechanical Scripting: HOWTO Part 3

Where We Left Off Last Time…

In the second part of this series I left you with a recipe for discovering ANSYS Mechanical functionality by using entities in the GUI itself to help guide your search through the ANSYS source code that is shipped with every ANSYS Mechanical installation.  This technique is the first tool in our toolbox.  In this blog post, I’ll introduce you to a couple of the internal ANSYS Mechanical data structures that are associated with geometry in particular.  I’ll simply be introducing them in this blog without any discussion of how one might determine additional functionality associated with these structures.  In a future blog post, I’ll step you through the use of the most powerful tool available in our toolbox: the Visual Studio Script Debugger.

Introducing the ANSYS Mechanical Geometry Selection Manager and B-Rep Geometry Data Structure

CAD geometry is represented inside a computer using a Boundary Representation (B-Rep) data structure.  This data structure falls into a category of data structures computer scientists like to call graphs.   In general, a graph is a structure comprised of nodes connected to each other with arcs.  A road map is a type of graph where cities represent nodes and the roads connecting them represent arcs.  Sites like facebook and linkedin are giant graphs where people are represented as nodes and friendships or acquaintances are represented as arcs.   As you can see, graphs are pretty general and you can represent lots of structured relationships with them.

In CAD geometry, the nodes in our graph are comprised of various topological entities like vertices, edges, faces, shells and bodies.  The arcs are the incident relationships that exist between these entities.  So, for example, an edge will have a start vertex and an end vertex.  These relationships can be represented by links between the particular edge and its start vertex or end vertex.  Likewise, however, the particular start vertex of one edge may be the end vertex of another edge.  So, this relationship can be represented by links between a given vertex and all of the edges to which it is attached.  There are similar relationships between edges and faces.  A series of edges will form the boundary for a given face and a series of faces might share a common edge.  Shells represent a volume that is defined by a set of connected faces. (i.e faces that share common edges)  A body is comprised of one or more shells.  Why isn’t a body the same as a shell?  Well, it turns out that bodies with internal holes are more conveniently represented as comprising of two or more shells.  One shell represents the outermost envelope and the other shells represent the internal voids.   How, then, does one know if they are inside or outside of a body?  Well, the faces that comprise a shell each have an associated outward normal vector that points to the “outside” of the body in question.  The normal vector sense is defined using an ordering on the bounding edges of a given face.  Typically as you traverse the list of edges comprising the boundary of a given face, you are actually walking counter-clockwise around the face.  Using this assumption, one can determine an outward normal.

Lets discuss some of the specifics of the ANSYS Mechanical B-Rep.  First, an individual B-Rep structure is associated with each part in a Mechanical session.  So, if you have 15 parts you will have 15 different B-Rep structures.  Second, each entity (vertex, edge, face, body) within a given B-Rep has an associated unique ID. (This ID is called the TopoID, or Topological ID)  This ID is just an unsigned integer value, with one small twist.  The top two bits of the ID encode the entity’s topological type.  That is, if you stripped off the top two bits, you would get one of the following four combinations:

Top Two Bits

Decimal Value

Topological Type

00 0 Vertex
01 1 Edge
10 2 Face
11 3 Body

The lower 30 bits actually encode the unique id for a given vertex, edge, face or body.  So, what that means is that ANSYS Mechanical is limited to geometry with fewer than roughly one billion vertices, edges, etc… If you have a model with more than that, perhaps a bit of geometry simplification is in order.

To access a given topological entity within a B-Rep object, one uses the following type of function call (Note, I haven’t introduced where the brep_object below comes from…  Bear with me, we’ll get to it soon.)

var topo_object = brep_object.Cell(topo_id);

The Cell function accepts a valid topoID and returns the associated object.  The returned object is a rich object and has numerous fields that obviously differ depending on the topological type.  So, for example, an edge object will have a StartVertex and EndVertex field that contains references to its start and end vertices in particular.  However, all topological objects have one field in common.  They all contain an ID field.  So the following code would supply the topolD of the start vertex associated with a given edge:

var brep_object = /*Get B-Rep from somewhere */
var edge_id = /* Get Edge ID from somewhere*/
var edge = brep_object.Cell(edge_id);
var start_vertex_id = edge.StartVertex.Id;

You might gather that one can get the actual start vertex object with a subsequent line like:

var start_vertex = brep_object.Cell(start_vertex_id);

Now, each vertex object is going to contain a list of edges that are attached to this vertex.  So, if we needed all the edges attached to the original edge for some reason, we could use this technique with the start vertex and the end vertex. 

Walking the B-Rep data structure is perhaps interesting at best, but it currently doesn’t do anything for us.  What we need is to be able to select some piece of geometry, or pieces of geometry.  Enter the ANSYS Mechanical Selection Manager.  This object is closely associated with the B-Rep data structures.  Before we discuss this object in some detail, consider starting each and every one of your scripts with the following code:

//  The script code object is a reference to the javascript engine embedded
//  inside the mechanical application.  This object has all of the functions
//  that are defined in the ANSYS mechanical javascript files
var SC = DS.Script;

//  The selection manager object is a reference to the ANSYS mechanical 
//  geometry selection tool.  This is a complex object that contains 
//  references to the currently selected set of geometric entities.  More
//  importantly, it contains back pointers to entity ids within the 
//  Mechanical BREP structure.
var SM =;

This code introduces two global variables into your script.  The first object contains a reference to the actual javascript engine embedded in the Mechanical application.  This object is derived from the DS object, which represents the fundamental Mechanical application object.  You can think of the DS object as the mother of all objects.  The ScriptCode or SC object stores all of the javascript code that ships with the ANSYS mechanical product.  So, if you use the technique described in Part 2 of this series and find a particular function call, you would actually call it against this object.  So, for example, we found the function doPrototypeInsertCommandEditor() in Part 2.  If we were to actually call that function in our script, it would look like:


The second global variable shown above is the selection manager, or SM.  You can see that this object is a child of the script code object.   This object has numerous functions and properties and in a later post I hope to document as many as I can.  However, a great deal of mileage can be gained by understanding the following two function calls, and the following few properties.  I list them below in code form so you can see how they would typically be called inside a script:

/*  F U N C T I O N S   O F   T H E    S E L E C T I O N    M A N A G E R */
// Clear()  
// This function resets the selection manager so that no geometry is currently 
// selected.

// AddToSelection(partID, topoID, bShouldDraw)
// This function adds a particular topological entity (vertex, edge, face, body) 
// to the currently selected set. The first argument specifies the particular
// part this entity is associated with.  You can think of this as referencing the
// particular BRep structure.  The second argument specifies the topological
// entities specific ID.  The final argument is a boolean specifying whether or
// not the selection manager should highlight the selected geometry in green.
SM.AddToSelection(partID, topoID, bShouldDraw);

/*  P R O P E R T I E S   O F   T H E    S E L E C T I O N    M A N A G E R */
// Vertices
// The number of currently selected vertices
var num_selected_vertices = SM.Vertices;

// Edges
// The number of currently selected edges.
var num_selected_edges = SM.Edges;

// Faces
// The number of currently selected faces.
var num_selected_faces = SM.Faces

// Parts
// The number of currently selected (bodies*)
// Note: I think this is where the SM and the BRep use different
// terms for a body.  However, I may be wrong.
var num_selected_bodies = SM.Parts

// SelectedCount
// The total number of selected entities.  (Sum of the above)
var num_selected = SM.SelectedCount;

// SelectedEntityTopoID(i)
// The ith selected entity's topological id.  Note you can use the
// top two bits of this id to determine the type of this selected
// object.  i goes from 1 to SelectedCount.
var selected_topoid = SM.SelectedEntityTopoID(i);

// SelectedPartID(i)
// The part id for the ith selected entity. For models with numerous
// parts this will tell you which part the currently selected entity is
// associated with.  You use this to reference the proper BRep structure.
// i goes from 1 to SelectedCount.
var selected_partid = SM.SelectedPartID(i);

// PartMgr
// The Part Manager (PartMgr) object is a child of the selection manager.
// It also has numerous methods and properties, however, the one
// property that I have found most useful is the PartById(id).
// PartMgr.PartById(id)
// This returns the particular part object associated with a given part id
var part = SM.PartMgr.PartById(partID);

// The part object returned by the PartMgr.PartById(partID) call has 
// associated with it a given BRep data structure.  Referencing this brep
// structure is how you get from the selection manager's selected entity
// id to a given piece of topology.  So for example, assume the user
// has selected one face in the model.  This is one way we can get to that
// face object.
var topo_id = SM.SelectedEntityTopoID(1);
var part_id = SM.SelectedPartID(1);
var part = SM.PartMgr.PartById(part_id);
var brep = part.BRep;
var face = brep.Cell(topo_id);

The above code represents a small portion of the interface of the ANSYS Mechanical selection manager and also the geometry B-Rep structure.  As you can see, the two structures are coupled through the notion of a part id, which selects a given B-Rep structure, and a selected entity id, which represents a piece of topology within the specified B-Rep structure.   Once a given topological entity is identified and retrieved from the B-Rep structure, one can navigate the connectivity information contained in the B-Rep to find other topological entities nearby.  By using a combination of the Clear() function and the AddToSelection(…) function associated with the selection manager, one can programmatically control which topological entities are currently selected. 

You can see this technique in action by perusing the Example in the previous posting.

An ANSYS Mechanical Scripting Example

Creating Unique Named Selections for Every Face in A Body.

I’ve put together a simple example of creating a unique named selection for each face associated with a selected set of bodies.  This example demonstrates the some of the techniques described in Part 3 of this series.  I recommend you read that post first to get a feel for the selection manager and ANSYS B-Rep data structures before returning to this post.

The unique name for each face is built up from a base name with the part ID and face ID appended to this name.  The function call used to create the named selection was taken directly from the ANSYS source code.  I used the technique described in Part 2 to find this function.  That is, I searched first for the string “Create Named Selection” and then for the string “ID_ComponentAdd” which led me to this XML code in dstoolbar.xml

   1:  <contextToolbar handle="NamedSelections">
   2:      <display id="2500">
   3:        <entry type="button" tipid="ID_ComponentAdd" icon="addselectiontolist">
   4:          <visibilityFilter name="idMeshing"/>
   5:          <visibilityFilter name="idMeshingAlbionSimAddin"/>
   6:          <visibilityFilter name="idAlbionSimAddin"/>
   7:          <actionCallback objectId="1" methodName="doAddComponent"/>
   8:          <visibilityCallback objectId="11" methodName="CanAddComponent"/>
   9:        </entry>

Which, as you can see on line 7, there is a method call named doAddComponent().  If we look into the call for this function, we see that it calls a function called addNamedSelection(bDialogFlag, specifiedName, criteria).  We can use this function to create a named selection with a programmatically generated name.

The flow of the example code is as follows:

  1. Check to see that at least one body is selected, if not, warn the user and quit.
  2. If one or more bodies are selected, build up a list of face ids and part ids associated with each body.
  3. Clear the selection manager so that nothing is selected.
  4. Iterate over the structure of part ids and face ids constructed in step 2 .
  5. For each face, create a unique name that is comprised of the part id and the face id concatenated to a base string.
  6. When finished, clear the selection manager so that nothing is selected.  This is sort of a clean up operation.  One enhancement might be to remember the bodies that were selected and clear out the faces from the selection manager and reselect these bodies so that the program is in the same state after this macro as it was when the macro was invoked.

Here is the code.  You can copy and paste this into a text file and save it somewhere on your hard drive.  The, select a body and choose Tools->Run Macro and navigate to the text file to see it work.

 *                                                      Phoenix Analysis And
 *                                                       Design Technologies
 *  This script is designed to be run from the Tools->Run Macro menu from
 *  within ANSYS Mechanical.  The purpose of the script is to create a named
 *  selection for each individual face within the currently selected body.  
 *  This script makes use of the addNamedSelection() ANSYS Mechanical function 
 *  to actually create the component once the selection has been made.

//  NOTE: The base name below is used as the first part of each named
//  selection name.  Change this to something more appropriate if you desire.
var basename = 'faces';

//  Here are a series of global variables that hold references to some
//  of the main objects in the mechanical application.

//  NOTE: The DS object is defined by the script engine ahead of time.  This
//  object references the main ANSYS Mechanical application

//  The script code object is a reference to the javascript engine embedded
//  inside the mechanical application.  This object has all of the functions
//  that are defined in the ANSYS mechanical javascript files
var SC = DS.Script;

//  The selection manager object is a reference to the ANSYS mechanical 
//  geometry selection tool.  This is a complex object that contains 
//  references to the currently selected set of geometric entities.  More
//  importantly, it contains back pointers to entity ids within the 
//  Mechanical BREP structure.
var SM =;

//  NOTE:  The ANSYS Mechanical BREP structure is a classical topological
//  data structure that defines the connectivity of the geometry within
//  the currently opened model.  This structure consists of lists of
//  vertices, edges, faces, shells, bodies and parts.  More importantly,
//  the connectivity between these entities is represented in the data
//  structure as "use" lists.  For example, a vertex contains a list
//  of edges to which it is attached.  Likewise, an edge contains a list
//  of faces to which it is attached, as well as a pointer to the start
//  and end vertices of the edge.  With this information, one can "walk"
//  the topology of the part by iterating over the various use lists.
//  For this macro, we will grab all of the faces attached to a given
//  body using the BREP structure as a guide.

//  These type constants are encoded into the top two bits of a given
//  brep entity id.

//  There is no concept of "main" in javascript, but it makes it easy to
//  find the entry point to the script.  There is a call to this function at
//  the bottom of this script
function main() {
    // First, see if we have a body selected.  If not, report to 
    // the user that a body must be selected first.
    if (SM.Parts > 0) {
        // Do the real work
    } else {
        SC.WBScript.Out('You must select a body before running the \
FacesFromBody.js macro.  Please select a body and re-run this \
macro.', true);

// This function simply picks off the top two bits of a given topo id
// and returns the decimal value [0, 1, 2 or 3]  This corresponds to
// the topological types:
//      BODY_TOPO_TYPE = 3
//      FACE_TOPO_TYPE = 2
//      EDGE_TOPO_TYPE = 1
//      VERT_TOPO_TYPE = 0
function classify_entity_type(topo_id) {
    // The top two bits store the topological entity type within the topo
    // id value
    var type = topo_id >>> 30;
    return type;

// This function creates a named selection of all the faces associated with
// the currently selected body(s).  
// Pre Condition:   At least one body must be selected.
// Post Condition:  The selection manager will be cleared and a new named
//                  selection will exist for each face in the body.  The
//                  named selection will be of the form:
//                  basename_partid_faceid
function create_faces_ns_from_body() {
    // See structure definition below.
    var face_id_map = new Array(SM.Parts);

    // First we want to iterate over the selected parts and create
    // a list of all of the face ids for each selected part
    for (var i = 1; i <= SM.SelectedCount; i++) {
        var topo_id = SM.SelectedEntityTopoID(i);
        var part_id = SM.SelectedPartID(i);

        // Make sure this is a body.  If not, just continue around
        if (classify_entity_type(topo_id) != BODY_TOPO_TYPE) {
        var part = SM.PartMgr.PartById(part_id);
        var brep = part.BRep;
        // Pull the body object out of the BRep structure.  The call
        // for this is the Cell function with the topological id for
        // the body of interest
        body = brep.Cell(topo_id);
        // This array will be used to hold a list of face ids for a given
        // body.
        var face_ids = new Array();
        // These are the actual face objects associated with this body
        var faces = body.Faces;
        // Now store all of the face ids in our face id list
        for (var f = 1; f <= faces.Count; f++) {
        // Now that we have the face ids, put them into an array of structures
        // that each look like the following:
        // |-------|
        // |   0   |-> Part ID
        // |-------|
        // |-------|
        // |   1   |-> List of face ids for this part ID
        // |-------|
        face_id_map[i - 1] = new Array(2);
        face_id_map[i - 1][0] = part_id;
        face_id_map[i - 1][1] =
            face_ids.slice(0, face_ids.length); // Slice creates a copy of the array

    // Now that we've built up our datastructure of face ids, we need to select them all
    var name = null;
    for (var i = 0; i < face_id_map.length; i++) {
        var part_id = face_id_map[i][0];
        var face_ids = face_id_map[i][1];
        for (var j = 0; j < face_ids.length; j++) {
            // Create a unique name based on the part id and face id
            name = basename + '_' + part_id.toString() + '_' + face_ids[j].toString();
            SM.AddToSelection(part_id, face_ids[j], false);
            // Create the component
            SC.addNamedSelection(false, name, SC.id_NS_UnknownMultiCriterion);           
    // Clear out the selection manager

// Make it happen...

Happy Holidays From PADT to the ANSYS Community


Well, another year is coming to an end and we are about to enter the Holiday Season.  It is a difficult time for many of us, leaving our computers at work, not being allowed to  log in from home, and being forced to interact with our families.  But after  a long and challenging year, even CAE elves need a break.

We want to take some time to offer a special thanks to all of you out there who read The Focus.  You have stuck with us through the transition from Newsletter to Blog, through all the bad puns, and most of all the ‘B’ movie references.  We do appreciate all the kinds words and comments that we receive.

We hope that all of you have a a fantastic holiday we wish you safe and prosperous simulation in the new year.


ANSYS Mechanical Scripting: HOWTO Part 2

Where We Left Off Last Time…

In the first part of this how to guide, I left you with a general description of the architecture of ANSYS Mechanical product.  If you haven’t read that post, I recommend that you at least skim through it to familiarize yourself with the structure of ANSYS Mechanical.   One thing we established is that customizing ANSYS Mechanical is not for the faint of heart, but at the same time, it is also not an impossible task.  There are two primary difficulties associated with Mechanical customization.  They are:

  1. ANSYS Mechanical was never truly designed to be customized, at not least by end users.
  2. There is no documentation of the internal API, so we’re left with deciphering the inner workings ourselves.

Nonetheless, the Mechanical application is architected as a collection of functional pieces written in C++ that are tied together with Javascript via a COM interface.  What that means for us is that most of the functionality of ANSYS Mechanical is exposed to us through an interface that can be scripted.  So, even though Mechanical may not have been explicitly designed to be scripted nor are the internals documented, yet we find that it is possible to script, and since the ANSYS developers have followed a consistent pattern, its internal structure is comprehensible by an outside agent.

A Recipe for Discovering ANSYS Mechanical Functionality

As I hinted in the introduction to this post, the ANSYS developers have followed a sound software development practice in the construction of the ANSYS Mechanical product.  The practice is this:  Find a particular pattern that solves a problem you currently face in the most general case.  So, lets describe the problem the ANSYS developers faced and then we will describe what I think was their solution.   The problem can be summarized as this:  How do you create a GUI that is easily extended to later include functionality that you don’t even know about today?  That is the problem from 50,000 ft.  Here are some more details about the problem.

  • Extensible means that I (the ANSYS Developer) may need to add / subtract menu items, toolbar buttons, menu groups, button groups at will as I add in more functionality over time.  I don’t want to have to rewrite lots of low level code every time I need to add a button, for example.
  • I already know we (ANSYS) have customers around the world who speak different languages.  I don’t want to maintain 10 different versions of ANSYS Mechanical for every language that I need to support.  It would be nice if I could separate the language from the rest of the code.
  • If I’m going to be adding buttons, menu items, etc… at will, I need to be able to describe what happens when the user presses the button, selects the menu, etc…  I don’t want to paint myself into a corner, so this needs to be as general as possible.

So, that’s the problem and perhaps you’re already seeing the solution in your head.  Here is what the folks at ANSYS came up with, which is a pretty standard pattern for constructing a flexible GUI.

  1. Push the actual GUI construction to the very last minute.  That is, you don’t hard code a GUI, rather you hard code a GUI builder engine.  The GUI builder engine reads in a human readable text file at run time that describes the GUI layout and then it dynamically creates the GUI on the fly every time the program launches.   Now what you have is a static executable that can dynamically morph its appearance to anything you want.
  2. This is a collorary to 1.  Since we want to push GUI construction to the very last minute, we therefore have to have the means to describe at runtime what should happen when the user presses a button or selects a menu item in the most general terms.  Hence we need a dynamic scripting language that isn’t interpreted until run time.  Note that the only real purpose of this language is to orchestrate at a high level what should happen.  The low level details of say meshing for example can all be constructed in a compiled language and contained in libraries as long as we can call into those libraries using this scripting language.  So, because of 1 we end up with a scriptable interface to Mechanical.
  3. We want to isolate language differences, i.e. English, French, Germany, etc… to as small a subset of files as possible.  So, the best programming technique for this problem is the old “extra level of indirection.”  That is, instead of using a literal string like “File” in the code that describes the menu item “File”, we use an identifier ID_File that is just a numeric constant.  Then, we construct a table that maps a string value to a numeric identifier.  So, maybe ID_File has a numeric value of 438.  In our table there would be an entry of: 438, “File”.  Likewise, anywhere in the actual GUI that we need the string “File”, we substitute the constant ID_File in its place.  Now, lets say we want a French version of ANSYS Mechanical, what do we do?  We simply translate the string table entries into French and create a new string table file.  At run time, we read in the French string table and voilà, we have a French version of ANSYS Mechanical.

So, that is the approach the ANSYS developers took to solve the problem of creating an easily extensible GUI.  The benefit for us is that we get a scripting language to boot, and if we “run the engine backwards” we get an easy algorithm for figuring out what is going on under the hood in Mechanical.  So, here is the recipe for determining functionality within ANSYS Mechanical.

Main Ingredients

  1. Good Text Search Tool (I use Slickedit.  An added benefit is if it can search entire directories recursively)
  2. Patience


  1. Locate the desired functionality in the GUI itself.
  2. Search the file dsstringtable.xml for the menu string, or toolbar button description string
  3. Note the corresponding ID_*** string ID within the string table file.
  4. Search for the ID_*** in dscontextmenu.xml or similar GUI structure file.
  5. When you find the ID_***, note the do*** command listed in the section associated with that GUI element’s action callback.
  6. Recursively search the aisol, or DesignSpace directory for the corresponding do*** command.

Could it possibly be simpler?

Example Use of the Above Recipe

I want to add a command object to a particular body in the tree, and eventually I want to do it programmatically.

Step 1: Locate the Functionality in the GUI Itself

Below is a screen shot of the context menu hierarchy one navigates through to insert a command object on a body in the tree.  So, we see that the string we are looking for is “Commands”


Step 2: Search dsstringtable.xml for the Menu String: “Commands”

Note that the string tables are located underneath a language directory structure.  If you haven’t currently guessed based on the content of this blog post, we’re using the string table located under Language\en-us\xml.  When we perform the search we find the following:


Step 3: Note the Corresponding ID_*** in the String Table

You can see from the image above that the ID associated with the string Commands is ID_CommandEditorDefaultName.  Sounds plausible.

Step 4: Search for ID_CommandEditorDefaultName in dscontextmenu.xml

Note, that the structure of the ANSYS Mechanical GUI is described in a series of xml files as mentioned in the sections above.  The file dscontextmenu.xml is one such file that shockingly describes the layout of the various context menus within the program.  So, lets search for the string ID_CommandEditorDefaultName in that file and see what we find.  Here are the results of that search:


Note the structure of this file.  There are various entries that represent individual menu items within a given context menu.  For each entry there are a series of action callbacks that have an associated methodName attribute.  By deduction we can assume that when the user clicks on this menu item, the listed javascript functions are called to perform the menu item’s action.  You’ll note too that there is a visibilityCallback as well.  Those wacky ANSYS developers thought of everything…

Now, one of the action callbacks is called “doPrototypeInsertCommandEditor”.  If that doesn’t sound like a function that will insert a command editor on a prototype, I don’t know what does.  Let’s see if we can find it.

Step 5: Search for doPrototypeInsertCommandEditor in the aisol or Designspace Directories

Now we search inside the javascript (.js) files for a function called doPrototypeInsertCommandEditor() and see what we can find.  Inside the file DSMenuScript.js we find this function definition:


Well, what do you know?  There is the 21 line function that gets called whenever you insert a command object onto a body in tree.   In the intervening time between this post and the next, spend some time staring at this code.  There are some fundamental aspects of the architecture of ANSYS Mechanical that appear even in this short function.  Now, I know your probably thinking to yourself, this is all fine and dandy, but what if I don’t just want to insert a command object.  What if I want the command object to have some APDL in it?  Furthermore, what do all those other variables, etc… really mean.  Lastly, how do I even call this blasted function?  Stay tuned… there is much, much more to come.

Finding the Location of a Value in a Common Block – Again


Every once in a while you need to do something fairly tedious and you know you’ve written a tool to do it, but you can’t find it. That happened to Matt today when he was trying to remember an easy way to locate an important variable that gets stored in one of the common blocks that ANSYS uses.  We both kind of had an tickle in the back of our heads that one of us wrote something a while back but we couldn’t find it.  “It was in the Focus… search… no luck. 

Then I checked and boom, there it was.  Once we found the name there, we found the focus article in Issue 58 from 2007.  Then I found it on Sheldon’s

Turns out, it is all over the place… we just couldn’t find it on our own hard drive.  Aging sucks, but now we have Google.  Anyhow, we figured since we forgot about it, other ANSYS MAPDL customization types may also have want a reminder.

And the Composite World Rejoiced: FiberSIM and ANSYS ACP Hook-Up

An early Christmas for those of us who use ANSYS with composites.  ANSYS and VISTAGY just announced that they will offer bi-directional linkage between FiberSIM and ANSYS through ANSYS Composite PrepPost.  The key statement from the press release:

ANSYS, Inc. … , and VISTAGY, Inc., … , have taken a major step in closing the loop between composites design and analysis by integrating ANSYS®13.0 Composite PrepPost and FiberSIM® 2010 composites engineering software. This bi-directional integration, enabled by FiberSIM’s Analysis Interface, allows customers to exchange detailed lay-up information between design and analysis quickly and accurately. As a result, users can optimize parts, assemblies and structures more efficiently.

Read the whole press release at:

Looks like there will be a Webinar on February 3:

If you are interested, contact your ANSYS Sales professional.   No word on when it will be released, but our guess is after the February webcast.

Quick Note: Nice Article on GPU Usage at R13 with ANSYS Mechanical Solvers

image_thumbI was chatting with a  product manager at ANSYS, Inc. about HPC. We got on the topic of GPU’s and he pointed out that there is a good article on the ANSYS Advantage magazine website on this very topic.

We get a ton of questions all the time about how GPU’s work and the kind of performance you can get from them.  This article goes into some good detail and has benchmarking data on R13.  Worth a read!

Licensing Tips and Tricks, R13 Version

image_thumb31To many users, licensing is transparent and is managed by a system administrator pretty much behind the scenes. For users at smaller companies, license management may be part of your tasks as an ANSYS user. No matter where you fall on that spectrum, user, user-administrator, or administrator of many products including ANSYS, this article will hopefully have some beneficial information. First, we’ll explain some basics and then look at some problem scenarios and how to resolve them.

But before we get into the nitty gritty please take a second to hear our own very special musical take on licensing:

I’ve Got those Licensing Blues

The Basics

Why licensing at all? Licensing is how ANSYS, Inc. makes sure that only those paying for software can use the software. Just like you wouldn’t walk out of Ray’s Music Exchange without paying for a guitar, you wouldn’t want folks using ANSYS without paying for their product.



ANSYS uses FLEXlm software to administer licensing of the ANSYS products. However, unlike other applications that also use FLEXlm, the ANSYS license manager has an additional component, the ANSYS Licensing Interconnect. The ANSYS Licensing Interconnect was new at version 12.0 and was previously discussed here:

The recommended way to install the ANSYS license manager is to install FLEXlm and the ANSYS Licensing Interconnect together. The ANSYS licenses are then administered with the ANSLIC_ADMIN Utility. To access this utility on Windows license servers, the menu pick would be

Start > All Programs > ANSYS, Inc. License Manager > Server ANSLIC_ADMIN Utility.

On Linux, the path is

/ansys_inc/shared_files/licensing/lic_admin and the executable is anslic_admin.


FLEXlm and the ANSYS Licensing Interconnect can also be installed separately so that one can administer the FLEXlm portion using tools like FLEXnet Manager, but the ANSYS Licensing Interconnect still needs to be administered using the ANSLIC_ADMIN Utility.

Once we launch the ANSLIC_ADMIN Utility, the first things we want to check is the status box at the lower left. There are three components to check, the Licensing Interconnect, the Licensing Interconnect Monitor, and FLEXlm. Ideally the status for all three should be ‘running’, like this:


If all three are listed as ‘not running’, the first thing to try is to click on the Start the ANSYS, Inc. License Manager button. If the license file is properly installed and all else is right, the status of each of the three components should change to ‘running’.

Once the status of all three is ‘running’, it’s a good idea to verify the license status by clicking on the View Status/Diagnostic Options button, then Display the License Status. If all is well, the resulting listing should show that the license server is UP, and there are licenses available. The client machine should now be able to check out ANSYS licenses, again if all is well.

Useful Tools

Besides the ANSLIC_ADMIN Utility, a couple of useful tools are the license.log file and the ansyslic_server.log file in the ANSYS licensing directory. These files often provide clues to problem situations. The ANSLIC_ADMIN Utility can be used to view these files with the View Status/Diagnostic Options button, or they can be opened in a text editor. Another useful tool is the troubleshooting section in the ANSYS Licensing Guide in the ANSYS Help.

Problem Scenario 1: License Server will not start.

A couple of things can cause this. First, if you have not installed the license file properly. To install a license file, use the Run the License Wizard button or the Install the License File button in the ANSLIC_ADMIN Utility. Second, you are using an expired license file (for leased products, for example), or the hostname or flexid in the license file are wrong. To check it, click on the Register License Server Machine Information button in ANSLIC_ADMIN and compare the results with the info in the license file. It should match. If you have a dynamic IP address, you’ll need the license file to be tied to the hard driver serial number rather than the IP address.

Problem Scenario 2: License Manager starts, but FLEXlm doesn’t start.

In this case, you get a message in the ANSLIC_ADMIN Utility stating not all of the licensing components have started, and it may take a several minutes for everything to work properly. The status portion of the ANSLIC_ADMIN Utility will show that the interconnect is up and the interconnect monitor is up, but FLEXlm is down. The license.log file may show messages like this:

0:01:18 (lmgrd) The TCP port number in the license, 1055, is already in use.
0:01:18 (lmgrd) Possible causes:
0:01:18 (lmgrd) 1) The license server manager (lmgrd) is already running for this license.
0:01:18 (lmgrd) 2) The OS has not “cleared” this port since lmgrd died.
0:01:18 (lmgrd) 3) Another process is using this port number (unlikely).
0:01:18 (lmgrd) Solutions:
0:01:18 (lmgrd) 1) Make sure lmgrd and all vendor daemons for this
0:01:18 (lmgrd) license are not running.
0:01:18 (lmgrd) 2) You may have to wait for the OS to clear this port.
0:01:18 (lmgrd) Retrying for about 5 more minutes
0:01:36 (lmgrd) Still trying…
0:01:54 (lmgrd) Still trying…


In this case, the port may clear on it’s own, or you may need to change the port number from the default of 1055 to something else. This is done by adding a port number to the VENDOR line in the license file, see the ANSYS Licensing Guide for more info. Or, in the worst case, you’ll need to restart the license server computer to free up the needed port.

Problem Scenario 3: The license is up and running, but the client can’t connect to the license server

The first thing to check is that the client machine (which can also be the server) is properly pointing to the license server, even if they are the same computer. To test this, on the client machine (if it’s different from the server machine), run the Client ANSLIC_ADMIN Utility (On Windows: Start > All Programs > ANSYS 13.0 > ANSYS Client Licensing > Client ANSLIC_ADMIN Utility). Click on the Specify the License Server Machine button. Make sure the port numbers and server specification (usually the hostname) are correct. If they are not, take corrective action.

If the above is correct, the second thing to check is if the client can ‘see’ the server on the network. A quick way to test that is to ‘ping’ the server from a command window or shell. Type

ping hostname

in a command window/shell where ‘hostname’ is the name or IP address of the server machine. It should report back that bytes of data have been received from the server. If not, take corrective action, which may be to resolve network problems.

The third thing to check is that the ports and licensing executables can pass through any firewalls. The default port numbers that need to pass are 1055 and 2325. The server machine should allow lmgrd.exe, ansyslmd.exe, ansysli_server.exe, ansysli_monitor.exe, and ansysli_client.exe to pass through any firewall. The client machine should also allow ansysli_client.exe through any firewall.

Most problems in scenario three are resolved by one of those three steps.

Certainly there are other problems that can crop up with licensing. For those we suggest you review the ANSYS Licensing Guide in the Help, or contact your ANSYS support provider.

Hopefully this information is helpful and will keep the users from pursuing you, you won’t feel like the target of a manhunt, and you won’t get Those Licensing Blues.


HPC on the Cheap: 48 cores for $11k

imageEvery once in a while computer hardware takes a big step forward, either in performance or lower cost.  Recently AMD, Intel’s biggest rival in CPU chips, and Super Micro Computers, a maker of high-performance motherboards and other hardware, have both come out with products that have changed the landscape of affordable High Performance computing forever.  About a month ago PADT put together a single system with 48 cores, 128 GB SDRAM, and 3TB of disk for just over $11,000.  And it is not too good to be true, the machine is fast and reliable.

We were able to get this type of price performance by leveraging the 12 Core server processors coupled with a motherboard that supports four chips: 4×12 = 48, something that only exists in the AMD world. We kind of stumbled upon this when our IT guy thought it would be good to see what AMD is up to.  We have been buying and using Intel based systems for a while now since they leapfrogged AMD the last time.  Once he started hunting around he found that AMD still supported four chips on a board, and that they had working 12 core chips.  So he went off to New Egg and built a shopping cart.  It came up at around $10,000 because we already had some of the hardware we needed.  Next came his hardest task, convincing management (me) that it wasn’t too good to be true.

After some research we found that others were benchmarking this chip and finding good performance on parallel tasks.  It is a slower clock speed than the top Intel chips, but because you can fit so many cores in one box, you get better performance for your money on applications that are parallel – like CFD.  After pulling the trigger on the order and sweating a bit the order showed up:


An Early Christmas at PADT:
A Barebones 1U Box, 4 Parts (48 cores), 32 SDRAM Sticks, and 3 Hard Drives

This created a bit of a buzz in the office, almost as big as when one of our customers brought his Tesla over for a test drive.  For those of us who started our career running on million dollar “super computers” with the same power as my cell phone, this was a pretty cool site to see.

The Box:

Here is what we purchased:

4 AMD Opteron 6174 Magny-Cours 2.2 GHZ 12 Core Server Processors 48 Core
32 4GB DDR3 Kingston 240 pin Registered SDRAM 128 GB RAM
3 1TB Western Digital RE3  7200 RPM, SATA 3.0 Hard Drives 3 TB Disk
1 80GB Intel X18-M Solid State Drive 80 GB / Drive
1 Supermicro AS-1042G-TF 1U Barebones Server 1 Box

We put it together, loaded it with CentOS and booted it up.  We got a little silly when we saw the boot screen:


None of us had seen a boot screen with a count of 48, nor one with that much RAM on a single system.  And we had to keep telling ourselves that it was only $11,000.

We had no issues with drivers or software.  ANSYS FLUENT, CFX and Mechanical APDL loaded up just fine and ran smoothly.

The Benchmarks:

The first thing we did is run a customer problem that was given to us as a test for a sale. The sales was being held up because we couldn’t get time on our existing systems to run it.  No surprise, it ran fast.  As we upped the number of CPU’s we could see that for a medium sized CFD problem, it was scaling well to 24 cores, then tapered off above that.  Once that problem was done we ran the ANSYS standard benchmarks to see how larger problems went.

First we ran the CFX benchmarks. They use something called “Speedup” which is simply the run time on one core divided by the run time on N cores. Or, how many times faster it is.


As you can see, you get the usual variation from problem to problem.  The Indy Car run actually did better than parallel, indicating that there is some sort of bottleneck with one processor and something about that problem that this system likes.  They other cases did well against a much more expensive IBM XEON system with a faster clock speed CPU.

Next, we looked at the performance on FLUENT.  They must have a bigger staff for benchmarking because we had a lot more machines to compare to.  But they use a different scale called a “Rating” It is 24 hours divided by the run time in hours on N Cores.  So a 1 would mean it runs in 1 day and 256 means it runs in 5.625 minutes.



Our takeaway from these studies is that 1) the system is pretty linear on large problems, and 2) that although it is slower than other systems, it is a lot less money, sometimes less than 25% as much.

The next thing to look at is what the speed is per core between Nehalem and AMD based systems.  For the data we have on FLUENT problems we compared a 2.9GHz Intel Nehalem based system to our 2.2GHz AMD system. These other systems have faster drives and cost 3 to 4 times as much.  What we found was that our AMD system was anywhere from 52% to 65% of the speed of the faster clock speed Intel chips.

Remember, to get to 48 cores on the Intel platform you will need six systems or blades, because each system or blade can only support two chips with four cores per chip, so 8 cores per board.  Let’s say the Intel chips are twice as fast.  So to get the same performance you would need 24 cores, or three 8 core boxes.  Then you need Infiniband with a switch and three cables.  So now you are talking three to four times the cost of the 48 core single box AMD system.

imageDon’t Forget Power

One of the knocks against the last time AMD had really competitive chips was heat and power consumption.  Well, these guys run cool and low power. We found that the power consumption maxed at 800 watts and average 760. When you consider that there are 48 cores chugging away at 100%, that is pretty impressive.  17watts per core!

  • Max Watts reached during test: 800
    • Watt usage settled down to around: 761
  • Number of hours for usage test: 167
  • Max Volts: 116 Volts
  • Amps: 6.60
  • $ rate used per/kWh: $0.11
  • Costs per/kWh:
    • Hour: $0.08
    • Day: $1.98
    • Week: $13.89
    • Month: $59.55
    • Year: $724
  • 200 Watts per part (CPU)
    • 17 Watts per core

Other Things to Consider

This particular configuration is very exciting, and it also opens up a lot of doors to other options and combinations.  The power of this system comes from parallel processing, so you need to consider the cost of adding parallel solves. Right now ANSYS, Inc. has a licensing option called HPC packs that are perfect for this application.  If you buy one pack, you can run on 8 cores, 2 packs is good for 32 cores, and 3 packs allow you to run on 128.

This doesn’t line up to well with 12 cores per chip.  But AMD does make a chip with 8 cores that run at a slightly higher clock speed (2.4 GHz vs. 2.2 GHz) that also cost less ($750ea vs. $1299ea)  So you could buy two HPC packs for 32 cores and then put together a system for less than $10,000.

Or, if that is not enough punch, you could also upgrade the motherboard to the next level up and purchase two barebones server boxes.  This allows you to use Supermicro’s Infiniband card to connect two boxes together over Infiniband without a hub.  Now you are talking 96 cores connected with a high speed interconnect, 256 GB of RAM, and all for under $27,000.

The last thing to consider is the fact that AMD has announced that they will be releasing a 16 core server chip to be widely available in 3Q11.  These will work with the motherboards we are talking about here, so you can upgrade your machines to have 4 more core per chip – so if you invest in a 96 core dual box system now, you can upgrade it to 128 cores next year.  Not bad.

Right for You?

After the initial excitement dies down for this much horsepower for this little of an investment, most users will ask themselves if this type of system is right for them.  You need to think it over and ask yourself some questions:

  1. Will I benefit from that much parallel?
    Remember, a single core actually runs slower than a top of the line 2.9GHz Intel system.  If you run Mechanical or have small problems, you may be better off paying for fewer, faster cores.
  2. Can I build my own system?
    We have the luxury at PADT of not having a corporate IT department that would freak out over a home-built system.  We also have an IT department with the skills to put systems together.  Run this idea by your IT before you go to far.  You can purchase systems with similar guts from HP and Dell, but a large bit of the cost savings go away when you do that.
  3. Do I have the Parallel Task Licenses?
    See what parallel you already own and contact your ANSYS sales person to see what it would cost to add more, especially HPC Packs.

If your problems are large and have long run times, this hardware and the needed software licenses will pay for themselves very quickly.  For us, a job that was running over 5 days now finishes in half a day.  That is huge as far as being able to either up the problem size to get more accuracy or getting results to our customers much faster.

Interested in Getting a System from PADT?

Several of our customers have expressed interest in purchasing our leasing one of these low-cost cluster systems from PADT. So We are offering a purchase or a lease on custom systems if you are located in Arizona, New Mexico, Colorado, Utah or Nevada.  Shoot an e-mail if you would like more information.  If you are outside of PADT’s ANSYS sales territory, we are more than happy to answer any questions you may have via e-mail to help you put your own system together, but we just don’t have the bandwidth to support outside of our area.

General Update on the Blog and News

Well, we are now a couple of weeks into this blog experiment and it is going well.  As part of being more “bloggy” and less “newslettery” we are going to stick in little updates and news items like this.

First off, thanks to a few dedicated users who requested a better way to print out the articles. We found a chunk of code that we added in to the templates for the postings and you should now see a “Print” option down in the lower right corner (with Permalink and comments) of each posting.  this brings the posting up in a separate window that is more printer friendly.  Just print this from your browser and you have hard copy. We are working closely with a SEO reseller agency to promote the website to the top of the ranking.

The next thing we are going to work on are comments.  We had it turned on for about two days when we went live and received some incredible offers to visit sights where we could: 1) increase our “size” in a few weeks, 2) obtain various drugs very cheaply, 3) meet a very nice girl that wants to show off her new webcam, and 4) help the widow of a former Nigerian minister of the interior get some money back from where the a bank is holding it.  So we turned off commenting. We are looking at a couple of open source tools that have spam filters for commenting and will hook up to one of those to get comments working.

In the area of news, we are very happy to report that our last seminar on Geometry Tools in ANSYS, Inc. Products had 145 people signed up and about 90 attendees.  We’ll post upcoming seminars in the right column of this blog, or you can subscribe to receive invites by using the same subscription service that controls your e-mail about this blog.  Click the arrow to the right to manage your subscriptions.