Mon, 04/02/2012 - 17:53
Dear Contributors,
The release 6.5.3 of OCCT libraries has been planned for the end of March.
Taking into account the needs of our projects (SALOME first of all) we are going to delay the release by about two weeks; this time will be needed to complete testing and verification of the important fixes made for the projects.
Anyway, most of the changes are already certified and integrated, so we consider current version as ready for beta-testing, and invite you to test this version in your applications. If you had reported some issues in Mantis bug tracker, please check those having Verified status -- we assume they are fixed, and will appreciate to see your comments in the notes of the issue.
Please use the commit tagged "V6_5_3_beta1" in the Git master branch for this testing. If you do not have direct access to the repository, you may download the archive of sources from GitWeb (just click on the "snapshot" link).
For the users who are not ready to build from bare sources with WOK, we are going to prepare standard source package containing pre-processed sources and build tools. This package will be published on www.opencascade.org.
Andrey
Wed, 04/04/2012 - 18:37
This release is a maintenance one, thus it provides mostly bug fixes and improvements; we tried to avoid changes in the public API. However, some changes may affect your applications if you use low-level tools and classes. Here I will try to provide synthetic overview of major changes in this version.
1. General code clean-up.
Inspired by the works on the upgrade of the test system, systematic execution of tests on different platforms, and multiple bug reports from the community (special thanks to Roman Lygin, Pawel, and Denis Barbier for reporting issues and posting comments!), we have eliminated numerous hidden problems in the code related to incorrect memory usage (potential leaks, corruption, use of uninitialized or freed memory, non-portable data types) and broken program logic (such as wrong priority of operators).
An important part of this clean-up was redesign of internals of TKOpenGl driver (see #22819 and related issues). This redesign allowed us to bring the code to maintainable state and hopefully to resolve all memory-related issues reported previously. Note that some obsolete functions and rendering primitives (TriangleMesh, TriangleSet, Bezier, Polyline, Polygon, PolygonHoles, QuadrangleMesh, QuadrangleSet) have been removed; the applications that might have used internal functions provided by TKOpenGl or removed primitives will need to be updated.
2. Changes coming from projects (only highlights; details to come with the Release Notes):
3. Some changes that might need to be accounted for in existing applications
4. Changes in MS VC++ project files and environment settings
Though this is related more to WOK than to OCCT code, some changes happened to how environment is set and project files are generated on Windows (MSVC):
Please note that WOK for Windows package has been updated on Resources page.
Wed, 04/04/2012 - 21:26
Thanks Andrey and the team!
As mentioned on the .org forum* you have been doing great for a few last months, hope this will pay your business back.
Good to see this pre-release as an opportunity to give early feedback, the timeframe (<2wks) is too challenging. I personally don't feel it will be possible for me to return it that fast.
Anyway, immediate one - I am concerned of #3/MMGT_OPT=0 as a default option. It's good that you go into a re-entrant mode as default, and this is likely a driver to change from OCC allocator (MMGT_OPT=1). By why not the TBB allocator (MMGT_OPT=2) ? With defaulting to a OS allocator you create risk of losing performance and/or strong OS dependence. Have you conducted performance benchmarking to see the downgrade is reasonable ? My latest experiments (partially published here - http://software.intel.com/en-us/blogs/2011/04/04/tbb-adoption-in-cad-tec..., and shared with you then) reveal that the TBB allocator is competitive vs OS allocator in a single thread mode and significantly outperforms in a multi-threaded mode.
Roman
(*) P.S. It's ridiculous to see this artificial separation of forums. You or Daniel probably explained this reasoning to me but I keep forgetting as it's something unnatural anyway :-|
Thu, 04/05/2012 - 07:27
Hello Roman,
Thank you for the positive feedback!
You can find some results of our tests of different memory managers in attachments to Mantis issues #21961 and #22627. TBB is clearly faster than others and scales better for multiple threads, however for practical cases when only a few threads are used its gain as compared to the system one is quite moderate, 1-10%. Note that your tests reported in the blog above confirm this (chart for Windows Server 2008; for XP and server 2003 we have activated low-fragmentation heap which should make system manager to behave like on server 2008 and Windows Vista / 7).
At the same time, we have found that on real-world cases (loading big STEP model), application using TBB allocator occupies twice more memory than with system allocator (tested on Windows, see #22627). For us it is more important to have OCCT capable of working with big industrial data on systems with limited resources than to have it 10% faster.
After all, TBB is an option for OCCT and cannot be default at least for this reason. And the last but not least: this is not more than default, you can always change it for your application.
Andrey
Thu, 04/05/2012 - 09:13
Hi Andrey!
Thanks for offering your persepctive and reference to the trackers. A few further thoughts for you to consider:
1. Posted measurements do not refer to TBB version, so I assume you used the one in OCC (3.0), right ? If so, please consider that for several last releases (which were 12 since then!) TBB has been addressing the memory footprint issue in addition to continuous performance improvements. So re-measuring can affect the situation (and integrating newer TBB version would be good anyway).
2. Trade-off of memory footprint vs user's productivity. With memory being currently cheap, those who deal with large data sets take care to have 4+GB/core. So stress-testing 1GB Intel Core 2 Duo with 185MB STEP file resulting in system thrashing (see Excel file in #22627) is just not representative, imho.
User's productivity can be a much more important goal, and a few extra minutes can be a much more precious asset to waste than a few GB's.
3. Consider enabling TBB allocator (MMGT_OPT) as default when HAVE_TBB is set during compilation. Those who set HAVE_TBB explicitly know what they are doing ;-). This refers to the patch I previously sent you:
Standard_MMgrFactory::Standard_MMgrFactory() : myFMMgr(0)
{
...
bOptAlloc = atoi((var = getenv("MMGT_OPT" )) ? var :
#ifdef HAVE_TBB
"2"
#else
"0"
#endif
);
...
Hope above points will persuade you to at least consider this again. This is what Beta is about in the end of the day, right ;-) ?
Another thought to consider. As a long-term goal on your roadmap I would suggest that you get rid-off of environment variables as often the only way to influence the library behavior. There should also be API to do so.
Env vars are bad for OCC-based application developers who redistribute their apps and have no direct controls of their end-user environment (.bat files, environment in Windows can be nightmare). They are evil for *library* developers who have no control whatsoever.
Thanks again for giving a chance to speak up!
Roman
Thu, 04/05/2012 - 11:36
bOptAlloc = atoi((var = getenv("MMGT_OPT" )) ? var :
#ifdef HAVE_TBB
"2"
#else
"0"
#endif
);
Hello Roman, IMO this is a terrible idea: 1) user may get a precompiled library and thus does not know whether HAVE_TBB was set at build time, and 2) your patch makes it impossible to use the system allocator.
Thu, 04/05/2012 - 11:57
Hi Denis,
We are opening a pandora box ;-).
1. The way above is just to set a *default* value when nothing is explicitly defined. As long as you have MMGT_OPT explicitly defined nothing is changed.
2. In no way the developer (OCC user) have a way to recognize which allocator is used today. Calling getenv ("MMGT_OPT") does not count - it may change too late when allocator has already been set (upon the first call to Standard_MMgrFactory constructor)
Thus the suggested way is not worse comparing to existing behavior but it does address a case of efficient multi-threaded use.
3. The long-term target objective for OCC must be to provide a developer (OCC user) a way to set his/her own desired allocator (an instance of Standard_MMgrRoot subclass). Before OCC 6.5.0 the only way to set TBB allocator was to patch the OCC source code. Today, you cannot inject google or any other allocator, just to experiment.
Obviously that must be possible only during some very limited time duration during library initialization, but today this is just impossible - static objects start massively being created inside OCC before you have a chance to inject your allocator.
Until that general mechanism is provided there is no perfect solution.
Roman
Thu, 04/05/2012 - 12:05
The way above is just to set a *default* value when nothing is explicitly defined. As long as you have MMGT_OPT explicitly defined nothing is changed.
You are right, I misread your code, sorry.
Thu, 04/05/2012 - 12:27
Denis, nevermind. Good that we resolved it.
Let me also elaborate on #3 above, why I believe it is important.
As a rule OCC is used as one of the components in eventually a larger application built of other components. Today (or before 6.5.3 based on Andrey's announcement of a likely change of default behavior), OCC would hoard all allocated memory under its own allocator (MMGT_OPT=1). Other components might use theirs or default to OS allocator.
This creates extra footprint as OCC hoarded cache is not available to others. This happens in Salome today - most classes and many containers in SMESH module (at least) default to OS and thus increase a footprint. Perhaps VTK and Qt come with their own allocators so there is extra fragmentation and footprint. I have once posted this suggestion at the Salome forum (http://www.salome-platform.org/forum/forum_11/116863807) but it remains unanswered.
The way to mitigate private allocators is to have a shared resource manager. Ideally this would be an OS allocator which would just put other allocators out of business, but OS vendors address multi-threaded needs too slowly (hence TBB allocator is important). As a general way, setting an explicit allocator in OCC would address this interoperability need too. The challenge is how to efficiently do this.
HTH.
Roman
P.S. As we dive into a technical discussion, I now believe even more firmly that separation of forums (.org vs dev.) was a damn wrong decision. This discussion is equally important for both 'contributors' and 'users'. With this separation we lose one part of the audience who might have something to say.
Please consider this as part of Beta feedback on the portal itself.
Thu, 04/05/2012 - 12:30
PP.S. And yes, minimizing fragmentation would help address the need of working with large data sets Andrey put above as a vital goal.
Tue, 04/10/2012 - 16:23
Hello Developers,
I've been looking at the new OCCT release for a couple of days now and have some questions/remarks regarding the new structure of the TKOpenGl driver:
- Some of the samples shipped with OCCT do not compile anymore because of the removed rendering functionalities: User_Cylinder.cxx in mfcsample projects calls Graphic3d_Group::TriangleSet which does not exist in OCCT 6.5.3.
As a workaround I used Graphic3d_Group::Polygon but some of the old functionalities are gone that way (normal direction/color at vertex).
Is this feasible to register the issue + workaround using the bug tracker? Or maybe this has already been addressed in the release?
- With the old OpenGL API it was possible to create some nice visualization effects (e.g. setting normals and colors like in the User_Cylinder.cxx). As this possibility seems to be gone what is the alternative? Can/will this issue be addressed in the documentation? Can anyone give some hints on how to update the old code preserving its functionality?
Thanks
Pawel
Tue, 04/10/2012 - 17:06
Dear Pawel,
I suggest you have a look at the OCCT Visualization User's Guide, "Primitive Arrays" chapter. Since OCCT 6.5.3 using primitive arrays is the only recommended way to pass geometry to OCCT visualization code.
Primitive arrays support all options needed by an application in order to display geometry, normals, colors and textures. The main reason to switch to them is that they are hardware-accelerated on graphic systems that support vertex buffer objects (VBO).
Hopefully, you will find the primitive arrays API suitable for all your needs.
Note that Graphic3d_Group::Polygon() and some other old Graphic3d_Group methods are left for compatibility reasons only and emulated internally using primitive arrays.
Best regards,
Sergey
Tue, 04/10/2012 - 19:25
Hello Sergey,
yes, thank you for the hints!
Reading the Visualization User's Guide helped ;)
One more question: I have noticed that displaying lots of texts (Graphic3d_AspectText3d) slows the the view navigation a lot. I started investigating the issue and observed that is not the case for the OCCT 5.2 (haven't tried all the releases in between, only some). Would you know what the reason might be?
Anyway, if I have some more data I will report the issue using the bug tracker.
Thanks
Pawel
Fri, 04/20/2012 - 15:22
Hello Pawel,
We have not analyzed this point in depth yet, though we suspect FTGL of doing some non-optimal things during rendering. By the way, OCCT 5.x did not use FTGL.
Another problem that arises when the number of text labels drawn using Graphic3d_Group::Text() method becomes large is described in http://tracker.dev.opencascade.org/view.php?id=23069, note that the part related to text rendering will not be corrected in OCCT 6.5.3 release.
There is an intention to put some efforts to improve text rendering in the future OCCT releases. There is already a Mantis issue where the discussion has been started: http://tracker.dev.opencascade.org/view.php?id=23049
Therefore, we will be extremely thankful if you share the results of your investigations.
Best regards,
Sergey
Fri, 04/20/2012 - 15:41
Hi Sergey,
thanks for the comments.
I have made some profiling using the "AMD CodeAnalyst" to get the impression where there problem might be. That tool gave me a couple of hints where to look for but I didn't have time to investigate the issue in the past days.
I hope I can get back to that next week...
Pawel
Tue, 04/10/2012 - 17:17
Hello Pawel,
Thank you for reporting the problem! We are currently working on update of the samples, but as far as I know this issue has not been addressed yet. Thus if you wish to help us, reporting the issue in Mantis and providing your workaround through Git would be the perfect way to do so. You are welcome!
Andrey