MinGW-w64 vs. MSVC - Forum Open Cascade Technology

Benjamin Bihler
Fri, 04/22/2016 - 11:17

Forums:

Hi!

I was thrilled to hear that also the official version of Open CASCADE will support MinGW-w64 from version 7.0.0 on. Since we do software development both on Windows and on Linux computers, it is useful to use the same compiler on both systems (up to now we have used the Open CASCADE Community Edition to achieve this). I have been able to compile OCCT with MinGW-w64, also when the additional libraries FreeImage and Gl2Ps are enabled and I would like to share some of my experiences.

Our software source code was the same for both compilers. It uses OpenMP, therefore OpenMP has been activated for both compilers.

MSVC:
I have used Visual Studio 2015 which seems to be named also "Microsoft Visual Studio 14.0" and the compiler prints the version number 19.00.23506. I have compiled 64 bit executables by using "vcvarsall.bat amd64" before project setup and compilation. For compiling OCCT I have created NMake make files with CMake and I have stuck to the default compilation flags. Our code I have compiled with the compile flags proposed by qmake (the most important being probably "-O2").

MinGW-w64:
I have used MinGW-w64 4.2.0 (at least this is written in the file build-info.txt) which used g++ 5.2.0 as compiler. I have compiled OCCT with CMake and MSys Makefiles. Again I have stuck to the default compilation flags ("-O3") for OCCT and our code, but I have also checked the alternative "-O2".

Algorithm 1: Solve a second order ordinary differential equation of a B-Spline surface (many D2 evaluations).
Algorithm 2: Compute special curves over a B-Spline surface (projections, intersection computations and D0, D1, D2 evaluations).

I have ensured that the programs are really 64-bit programs by looking at the task manager. Then I have executed the algorithms times for each configuration, measured the execution time with std::chrono and computed the average execution time. Compilation times I have just measured once.

OCCT Compilation Times:

MSVC: 104 minutes
MinGW-w64 with "-O3": 137 minutes

Algorithm 1 Execution Time on a Flat Surface:

MSVC: 0.47 s
MinGW-w64 with "-O2": 0.31 s
MinGW-w64 with "-O3": 0.32 s

Algorithm 2 Execution Time on a Flat Surface:

MSVC: 2.50 s
MinGW-w64 with "-O2": 1.79 s
MinGW-w64 with "-O3": 1.78 s

Algorithm 1 Execution Time on a Curved Surface:

MSVC: 8.55 s
MinGW-w64 with "-O2": 7.77 s
MinGW-w64 with "-O3": 7.35 s

Algorithm 2 Execution Time on a Curved Surface:

MSVC: 34.68
MinGW-w64 with "-O2": 29.76 s
MinGW-w64 with "-O3": 29.43 s

OCCT DLL Sizes:

MSVC: 55.126.016 Bytes
MinGW-w64 with "-O2": 55.881.216 Bytes
MinGW-w64 with "-O3": 59.709.952 Bytes

Results:
MSVC is doing the compilation job significantly faster than MinGW-w64. The DLL sizes are comparable, if optimization is set to "-O2" for MinGW-w64, with "-O3" the DLLs from MinGW-w64 are larger. Binary files compiled with MinGW-w64 are performing significantly better than those compiled with MSVC.

I have tried to use link-time optimization with MinGW-w64, but I have failed. If anyone knows how to do that, I would greatly appreciate any hint.

Benjamin Bihler

Benjamin Bihler
Fri, 09/15/2017 - 19:17

Hi!

I have repeated my performance tests with the current Open CASCADE version, the current version of my algorithms and newer compiler versions, because I always want to keep an eye on performance. I have used MSVC Version 19.11.25507.1 and MinGW-w64 with g++ 7.1.0. Since I have invested quite some time I would like to share my results with you. This time I have not recorded the compilation times. I have done five measurements for every case and computed the average

execution times.

Algorithm 1 Execution on a Flat Surface

MSVC: 0.33 s
MinGW-w64 with "-O2": 0.21 s
MinGW-w64 with "-O3": 0.22 s

Algorithm 2 Execution on a Flat Surface

MSVC: 4.60 s
MinGW-w64 with "-O2": 1.44 s
MinGW-w64 with "-O3": 1.41 s

Algorithm 1 Execution Time on a Curved Surface

MSVC: 12.37 s
MinGW-w64 with "-O2": 7.91 s
MinGW-w64 with "-O3": 7.83 s

Algorithm 2 Execution Time on a Curved Surface

MSVC: 48.48 s
MinGW-w64 with "-O2": 21.00 s
MinGW-w64 with "-O3": 20.84 s

OCCT DLL Sizes:

MSVC: 63.379.456 Bytes
MinGW-w64 with "-O2": 61.389.914 Bytes
MinGW-w64 with "-O3": 61.408.346 Bytes

Conclusion

With g++ 7.1.0 it is recommendable to use "-O3". With g++ 5.2.0 this had a negative effect on the execution times. The code created by MSVC is so shockingly slow that I refuse to take my measurements seriously. I have used the default compilation paramters set by CMake (especially "-O2" for MSVC optimization and I have used the "x64 Native Tools Command Line" to compile 64bit binaries), but such a degradation when switching to a newer compiler version means probably that I have done something wrong. I am more familiar with g++ than with MSVC, therefore a MSVC expert would probably have been able to compile the code somehow in a better way than I have done it.

My next interest was whether link-time optimization gives better results. With MinGW-w64 I have chosen "-flto -Wl,-allow-multiple-definition" as additional compile and link flags and with MSVC I have chosen "/GL" as additional compile and "/LTCG" as additional link flag.

This is the outcome when enabling link-time optimization:

Algorithm 1 Execution on a Flat Surface

MSVC: 0.30 s
MinGW-w64 with "-O3": 0.22 s

Algorithm 2 Execution on a Flat Surface

MSVC: 4.45 s
MinGW-w64 with "-O3": 1.41 s

Algorithm 1 Execution Time on a Curved Surface

MSVC: 11.99 s
MinGW-w64 with "-O3": 8.04 s

Algorithm 2 Execution Time on a Curved Surface

MSVC: 47.63 s
MinGW-w64 with "-O3": 21.00 s

OCCT DLL Sizes:

MSVC: 61.395.968 Bytes
MinGW-w64 with "-O3": 53.915.226 Bytes

Conclusion

It is nice that the binaries become smaller with MinGW-w64 and link-time optimization, but the code also becomes a little bit slower in some cases. I am not sure yet whether it is the right choice for me. With MSVC both the binary sizes and the code execution times were reduced, but the effect is rather small.

Benjamin

Kirill Gavrilov
Sun, 09/17/2017 - 23:06

Nice comparison - numbers give an impression that trying GCC on Windows might worth additional efforts ;).

What about debugging experience - do you use GDB with some IDE (Qt Creator?) for debugging without problems/inconviences?
Debugging plugins within Within Code::Blocks are terrible comparing to VS...

Benjamin Bihler
Wed, 10/04/2017 - 10:47

I compile with -Og and debug with GDB inside Eclipse. This works quite well, but to be honest I have never debugged with VS, therefore I cannot really compare it.

Andrey BETENEV
Wed, 11/01/2017 - 07:37

Hello,

I have made my own measurements (on current master), comparing Visual Studio 2010 and 2017 and MinGw-w64 (build 4.3.3 with GCC 7.1.0).

It should be noted that MinGw-w64 can be built with several options; among them one that defines exception model has severe impact on both performance and ability to catch software signals (such as access violations).

In exception model SJLJ exceptions are implemented with setjmp() and longjmp() C functions. With this option, C signal handlers (see OSD::SetSignal()) do not work.
Exception model SEH is based on structured exceptions handling (similar to MSVC). With this option, OSD::SetSignal() can be used to convert C signals to C++ exceptions.

To compare performance, I have run all OCCT tests in sequential (single process) mode. The results are as follows:

Compiler       | Build time | DLL size | All tests run time | CPU diff, % |
----------------------------------------------------------------
MSVC 2010      |   29 min   | 63348 Kb |      6 h 04 min    |      0      |
MSVC 2017      |   30 min   | 62021 Kb |      5 h 55 min    |     -2%     |
Mingw-w64 SEH  |   30 min   | 61716 Kb |      6 h 07 min    |     +1%     |
Mingw-w64 SJLJ |   35 min   | 78516 Kb |      7 h 42 min    |    +38%     |

All builds are made with default options, using CMake for MinGw and genproj for MSVC. Among third-party components, MinGw uses only Freetype, Tcl/Tk, Freeimage, and Gl2Ps, while MSVC also uses TBB and VTK. Workstation has Intel i7-4790 CPU with 8 Gb RAM, SSD system disk and HDD data disk (where both OCCT and test results are located).

As it can be seen, average performance of MSVC 2010, 2017, and MinGw-w64 SEH is nearly the same. Though, many tests show essentially different CPU time between MinGw SEH and MSVC 2010 (mostly within +-50%), but these deviations go in both directions and have not much effect on the average. The most essential deviations (confirmed by manual tests) are:

CPU bugs modalg_5 bug24190: 90.578125 / 51.328125 [+76.47%]
CPU bugs modalg_5 bug24359: 9.609375 / 5.390625 [+78.26%]
CPU bugs iges bug28694_8: 10.703125 / 13.015625 [-17.77%]
CPU bugs modalg_5 bug24981: 5.3125 / 6.46875 [-17.87%]
CPU bugs modalg_7 bug28119_1: 9.625 / 20.4375 [-52.91%]
CPU perf fclasses bug24947: 0.09375 / 3.6875 [-97.46%]

Note that the latter test, perf fclasses bug24947, was made to measure performance of initialization of OCCT RTTI (by loading and unloading some toolkit multiple times). It is slow on MSVC 2010 and older GCC (below 4.3) because these compilers do not support safe initialization of static variables (see C++11 feature N2660) and thus we have to initialize all types on load time.

My conclusions are:

If MinGw is used, SEH build is needed to have good performance (and ability to catch signals).
Average performance of code built with MinGw and MSVC looks the same, though it can be different for particular algorithms or tasks.

Benjamin Bihler
Wed, 11/01/2017 - 13:36

Thank you for sharing your measurements. It is good to know that the average performance of code built with both compilers is quite the same - then the compiler choice is actually a matter of taste.

Benjamin Bihler
Thu, 11/02/2017 - 15:43

Andrey, you have stated which exceptions you have used with MinGW. Just out of interest - which threads have you used? Win32 threads or pthreads?

Andrey BETENEV
Thu, 11/02/2017 - 16:17

I have used Win32 threads in both cases

Mauro C
Fri, 12/01/2017 - 12:43

I am interested to compile OCC 7.2 with MinGw.
Could you share,step by step,the whole procedure?
Thanks in advance

Benjamin Bihler
Fri, 12/01/2017 - 12:53

It is almost as simple as compilation with MSVC. Just download the 3rd party components compiled with MinGW from the Open CASCADE Download Center that you need for your project (you refer to them within CMake).

I have installed MSYS2 for compilation and I have added MinGW64 to my path (both is probably not necessary, but it makes the build convenient). I run CMake from MSYS2 and I compile with mingw32-make.exe, because this allows parallel builds with -j (the default make.exe is buggy when doing parallel builds).

This is no step-by-step description, I know. If you have problems with the build, then please describe your steps and the problems that you have met.