Spectre Side-Channel and Meltdown – How will living in this new reality affect the world of numerical simulation?

Literally, while I was sorting and running benchmarks and prepping the new benchmarks data originally titled. ANSYS Release 18.2 Ball Grid Array Benchmark information using two sixteen core INTEL® XEON® Gold 6130 CPU’s. I noticed that my news feeds had started to blow up with late breaking HPC news. The news as you may have guessed is the Spectre and Meltdown flaws that were recently published.

I thought to myself “Well this is just great the benchmarks that I just ran are no longer relevant.  My next thought was wait now I can show a real world example of exactly a percentage change. I have waited this long to run the ANSYS numerical simulation benchmarks on this new CPU architecture. I can wait a little longer to post my findings.” What now? Oh my more Late Breaking News! Research findings, Execution orders no barriers! Side channels used to get access to private address areas of the hardware! Wow this is a bad day. As I sat reading more news, then I drifted off daydreaming, then back to  my screen then the clock on the wall, great it is 2am already!, just go home…” Then thoughts immediate shifted and I was back thinking about indeed, how these hardware flaws impact the missing middle market? HPC numerical simulation!!! I dug in deep and pressed forward content with starting over on the benchmarks knowing after the patches released around Jan 9th will be a whole new world.

I decided to spare the ugly details related to the Spectre array bounds/brand prediction attack flaws. The out of order meltdown vulnerabilities! UGH! I seriously believe that someone has AI writing news articles written five or six different ways with each somehow saying the same thing. I also provide the links to the information and legal statements directly from a who’s who list of accountable parties:

Executive Summary:

  • * Remember every case is different so please do your run your own tests to verify how this new reality affects your hardware and software environment.*
    • Due to costs this machine has a single NVMe M.2 for the primary drive with a single 2TB SATA drive for its Mid-Term Storage area.
  • What was the impact for my benchmark?
    • Positive takeaway:
      • In all of the years of running the sp5 benchmark. I recorded the fastest benchmark time using this CUBE w32s, dual INTEL® XEON® Gold 6130 CPU workstation.
      • Using all thirty two cores 125.7 seconds for Solution Time (Time Spent Computing Solution).
        • Next, Coming in at 135.7 seconds the Solution Time metric after running the OS patches is my second fastest data point for the ANSYS sp5 benchmark.
          • ANSYS sp5 benchmark data – PADT, Inc. Currently from 2005 until this time.
      • The Solution Times continued to solve faster with each bump in cores.
      • Performance per dollar was maximized in this configuration.
    • Depending on number of cores used that I used for the ANSYS sp5 benchmark. I give the actual data below showing the percentage differences before and after:
      • Largest percentage difference:
        • Solution Time: -9.81% using four CPU cores.
        • Total Time: -7.87% using two CPU cores.
  • The need to turn the security screws down within your corporate enterprise network is now.
  • A rogue malicious agent needs to be on the inside of your corporate network to execute any sort of crafted attack. Much of these details are outlined in the Project Zero abstract.
  • Pay extra attention to just who you let on your internal network.
    • I reiterate the recommendations of many security professionals that you should already be restricting your internal company network and workstations to employee use. If you are not sure ask again.
  1. Spectre flaw:
    1. INTEL, ARM & AMD CPU’s are affected by the Spectre array bounds hardware attacks.
  2. Meltdown flaw:
    1. INTEL CPU’s and some ARM high performance CPU’s are affected by the “easier to exploit” Meltdown vulnerability.

I am also interested to see how continued insertion of code barriers and changed memory mappings affect my gaming performance. Haha! No, I am just kidding my numerical simulation performance benchmarks.

Clarifications & Definitions:

  • Unpatched Benchmark Data – No mitigation patches from Microsoft and NVidia addressing the Spectre and Meltdown flaws have been applied to the Windows 10 Professional OS running on the CUBE w32s that I use in this benchmark.
  • Patched Benchmark Data – I installed the batch of patches released by Microsoft as well as the NVDIA graphics card driver update released by NVIDIA addressing. NVIDIA indicates in their advisory that “their hardware their GPU hardware is not affected but they are updating their drivers to help mitigate the CPU security issue.” Huh? Installing now…
  • Solution Time – The amount of time in seconds that the CPU’s spent computing the solution. “The Time Spent Computing Solution”
  • Total Time – Total time in seconds that the entire process took. How the solve felt to the user also known as wall clock time.

The CUBE machine that I used in this ANSYS Test Case represent a fine balance based on price, performance and ANSYS HPC licenses used.

  • CUBE w32s, INTEL® XEON® Gold 6130 CPU, 128GB’s DDR4-2667MHz (1Rx4) ECC REG DIMM, Windows 10 Professional, ANSYS Release 18.2, INTEL MPI 5.0.1.3, 32 Total Cores, NVIDIA QUADRO P4000, Samsung EVO 960 Pro NVMe M.2, Toshiba 2TB 7200 RPM SATA 3 Drive.
  • Other notables, are you still paying attention?
    • My Supermicro X11Dai-N BIOS Settings:
      • BIOS Version: 2.0a
      • Execute Disable Bit: DISABLE
      • Hyper threading: ON
      • Intel Virtualization Technology: DISABLE
      • Core Enabled: 0
      • Power Technology: CUSTOM
      • Energy Performance Tuning: DISABLE
      • Energy performance BIAS setting: PERFORMANCE
      • P-State Coordination: HW_ALL
      • Package C-State Limit: C0/C1 State
      • CPU C3 Report: DISABLE
      • CPU C6 Report: DISABLE
      • Enhanced Halt State: DISABLE
    • With a read performance of up to 3,200MB/s and write performance of up to 1,900 MB/s using the Samsung NVMe M.2 drive was to tempting to pass up as my solve and temp solve area location. The bandwidth from the little feller was to impressive and continued to impress throughout the numerical simulation benchmarks.

My first overall impressions of this configuration is Wow! this workstation is fast, quiet and as you will see number crunches its way right on through to being my fastest documented workstation benchmark in this class. This extremely challenging and I/O intensive ANSYS benchmark is no match for this solver! Thumbs up and cheers to happy solving!

  • Cube w32s by PADT, Inc. ANSYS Release 18.2 FEA Benchmark
  • BGA (V18sp-5)
  • Transient nonlinear structural analysis of a electronic ball grid arrary
  • Analysis Type: Static Nonlinear Structural
  • Number of Degrees of Freedom: 6,000,000
  • Matrix: Symmetic

It Is All About The Data:

Benchmark data related to Pre and Post Spectre and Meltdown industry software patches on the CUBE w32s.

Table 1 – ANSYS sp5 Benchmark  – UnPatched Windows 10 Professional

ANSYS sp5 Benchmark  – Unpatched Windows 10 Professinal for Spectre and Meltdown hardware vulnerability – CUBE w32s
CPUs Solution Time Total Time
2 631.3 671
4 366.8 422
8 216 259
12 193 235
16 144.3 185
20 143.9 187
24 131.9 175
28 137.4 185
31 142.4 185
32 125.7 171
Apples to Apples, meltdown, spectre, ANSYS numerical simulation benchmark data
ANSYS Release 18.2 – SP5 Benchmark – Unpatched Windows 10 Professional CUBE w32s Solution and Total Time Values

Table 1.1 – ANSYS sp5 Benchmark  – Patched Windows 10 Professional

ANSYS sp5 Benchmark  – Patched Windows 10 Professional – CUBE w32s
CPUs Solution Time Total Time
2 683 726
4 405.5 446
8 235.8 277
12 209.2 251
16 148.8 191
20 145.7 189
24 136.3 182
28 138.7 186
31 134.6 179
32 135.7 179
Apples to Apples, meltdown, spectre, ANSYS numerical simulation benchmark data
ANSYS Release 18.2 – SP5 Benchmark – Patched Windows 10 Professional for the Sprectre and Meltdown hardware flaw – Solution And Total Time Values

Table 2 – ANSYS sp5 Benchmark  – The Before and After In Percentage Difference.

Percentage Difference – Not Patched vs. Patched for Sprectre, Meltdown
Solution Time Total Time
-7.94 -7.87
-9.81 -5.53
-8.34 -6.72
-7.57 -6.58
-2.73 -3.19
-1.09 -1.06
-2.87 -3.92
-0.81 -0.54
4.76 3.30
-6.74 -4.57

Fig 2.a

Percentage of impact for this example. Negative value means in this example. The patched Windows 10 Professional CUBE w32s is taking a performance hit.
Percentage of impact for this example. Negative value means “performance hit” in this example. Notice a very interesting blip of positive percentage at 31 cores. A patched CUBE w32s Windows 10 Professional for Sprectre and Meltdown hardware vulnerability. The data from this Windows 10 Professional CUBE w32s INTEL® XEON® Gold 6130 CPU is showing an impact related to the patches.

FIg 2.b

Percentage of impact for this example. Negative value means in this example. The patched Windows 10 Professional CUBE w32s is taking a performance hit.
Percentage of impact for this example. Negative value means there is some sort of impact. The patched Windows 10 Professional CUBE w32s will feel longer to solve by looking at the clock on the wall.
CUBE w32s in action - January 2018
CUBE w32s in action – January 2018

Please contact your local ANSYS Software Sales Representative for more information on purchasing ANSYS HPC Packs. You too may be able to speed up your solve times by unlocking more compute power!

What the heck is a CUBE? For more information regarding our Numerical Simulation workstations and clusters please contact our CUBE Hardware Sales Representative at SALES@PADTINC.COM

Designed, tested and configured within your budget. We are happy to help and to listen to your specific needs.

CUBE w32s in action - January 2018
CUBE w32s in action – January 2018

Distributed ANSYS 18.1 with the SP-5 Benchmark using an INTEL 1.6TB NVMe

I recently had a chance to run a series of benchmarks on one of our latest CUBE numerical simulation workstations. I was amazed by the impressive benchmark numbers and wanted to share with you the details for the SP-5 benchmark using ANSYS 18.1. Hopefully this information will help you make the best decision the next time you need to upgrade your numerical simulation C Drive from whatever to now is the time to buy a Non-Volitile Memory Express drive. Total speedup using identical CUBE hardware, except for the INTEL DC P3700 NVMe drive @32 Cores is a 1.19x speedup!

  • Time Spent Computing Solution ANSYS SP-5 Benchmark
    • 161.7 seconds vs. 135.6 second
    • ANSYS 17.1 & ANSYS 18.1 Benchmarks

The link below is to a great article that I think will catch you up to speed regarding NVMe, PCIe and SSD Technology.

HDD Magazine hints NVME is coming, I say NVMe is already here…

CUBE w32iP Specifications (July 2017)

  • CUBE Mid-Tower Super Quiet Chassis (900W PS)
  • CPU: 32 INTEL Cores – 2 x INTEL e5-2697A V4 32c@2.6GHz/3.6GHz Turbo
  • OS: INTEL NVMe – 1 x 1.6TB INTEL Enterprise Class SSD
  • Mid-Term Storage: – 1 x 10TB Enterprise Class SATA 6Gbp/s, 256M, Helium sealed
  • RAM: 256GB DDR4-2400MHz LRDIMM RAM
  • GRAPHICS: NVIDIA QUADRO P6000 (24GB GDDR5X RAM)
  • MEDIA: DVD-RW/Audio 7.1 HD
  • Windows 10 Professional

Just how much faster the INTEL NVME drive performs over previously run ANSYS Benchmarks?

Check out the data for yourself:

  1. ANSYS 17.1 – SP-5 Benchmarks
  2. ANSYS Website
  3. HPC Advisory Council
  • ANSYS Benchmark Test Case Information.
  • ANSYS HPC Licensing Packs required for this benchmark
    • I used (2) HPC Packs to unlock all 32 cores.
  • 1.19x Total Speedup!
  • Please contact your local ANSYS Software Sales Representative for more information on purchasing ANSYS HPC Packs. You too may be able to speed up your solve times by unlocking additional compute power!
  • What is a CUBE? For more information regarding our Numerical Simulation workstations and clusters please contact our CUBE Hardware Sales Representative at SALES@PADTINC.COM
    • Designed, tested and configured within your budget. We are happy to help and to  listen to your specific needs.

ANSYS SP-5 Benchmark Details

BGA (V18sp-5)

Analysis Type Static Nonlinear Structural
Number of Degrees of Freedom 6,000,000
Equation Solver Sparse
Matrix Symmetric
 July 2017 TIME SPENT COMPUTING SOLUTION TOTAL CPU TIME FOR MAIN THREAD ELAPSED TIME
CUBE w32iP CUEB w32iP CUBE w32iP
# of Cores CUBE w32iP CUBE w32iP CUBE w32iP
2 1034.3 1073.7 1076
4 594.7 630.3 633
6 431.5 465.7 472
8 333.4 367.9 377
10 268.7 302.6 316
12 243.6 276.5 287
14 223 256.2 264
16 186.8 219.3 227
18 180 212.4 226
20 174.4 207.4 220
22 164.5 197.4 209
24 155.6 188.2 199
26 147.1 179.2 193
28 146.4 178.2 190
30 140.8 168.5 196
31 140.4 164 196
32 135.6 158.1 182
WO/GPU Acceleration WO/GPU Acceleration WO/GPU Acceleration

July 2017, drjm, PADT, Inc.

CUBE W32iP SP-5 Benchmark Graph

CUBE w32iP with INTEL DC P3700 1.6TB

Click Here for more information on the engineering simulation workstations and clusters designed in-house at PADT, Inc.. PADT, Inc. is happy to be a premier re-seller and dealer of Supermicro hardware.