Mon, 12/12/2011 - 18:56
Dear Contributors,
This thread is to present our plans to upgrade the tools provided by this site and OCCT itself to facilitate collaborative development of OCCT and improve the existing process.
The important notice to be given is that the process we are establishing is the one to be used by all contributors, both external and internal to OPEN CASCADE.
The general workflow based on processing each change as an issue in MantisBT issue tracking system, described in my previous post, remains as a basis. Further we will gradually improve the tools and associated process:
- Provide public access to the source code repository.
This is our next main goal. Currently we consider setting up a Git repository managed by Gitorious server, with which we expect to get account management integrated within the dev web site, convenient management of branches and merge requests, and a code review tool. Integration with the tracker is naturally planned as well.
Definitely there is a huge amount of work to be done. Apart from setting up the tools, we still have to revise the structure of our existing SVN repository and convert it to Git.
As soon as the Git repository is there, we will replace the current method to submit the code (by patches) by the one using branches (normally one branch per issue) and merge requests. The details are still to be elaborated, thus all suggestions and ideas are very welcome.
If everything goes as expected, we are planning to have it ready in March 2012.
- Include an automated tests suite into OCCT code base, and provide tools to execute it.
This is our second major goal. At the moment we have upgraded our internal toolset to a new platform based on Perl implementation of TAP (Test Everything Protocol), and are working on conversion of the database of test scripts to that platform. As we have about 18,000 test cases, this work will take a good while of time. We plan to include the new test tools with the first subset of test cases in the next release of OCCT, and complete it gradually during 2012.
It should be made clear that a considerable subset of tests (especially those in the Data Exchange module) involve confidential data models coming from our customers and thus these tests may not be made public. They will remain a part of our internal certification process continued to ensure overall non-regression of OCCT.
As soon as the tests are available in the OCCT, we will require all the contributors to ensure that tests run well before making a merge request.
And yes, the new test system keeps using DRAW test harness with test scripts being in Tcl.
- Continue improvements of the developer’s portal following the experience that we obtain together, and new ideas that we are expecting to get from you.
I suggest that in this thread we can discuss general points, and create dedicated threads for discussion of particular tools.
Andrey
Thu, 12/22/2011 - 02:25
What about using the fully integrated collaborative development portal Redmine (http://www.redmine.org/) ?
Redmine is widely used for open source project and the project is very active and supported by a large community.
This would provide a radical improvement to the tool (rather than gradual).
Redmine has many advantages over Mantis as it provides a wiki, a roadmap(version) management system, a forum , a strong integration with git and a editor that allows rich description and formating.
Because everything is integrated in one tools, links can be easily made between task, issues , wiki pages. Git commit can easily be linked (back and forth) to redmine issues.
Redmine comes also with a nice collection of useful plug-in.
There are also some tools to migrate mantis database to redmine, which will preserve the legacy.
Concerning TAP, Are you sure that adding yet an other script language to OCC is mandatory ? We already have dependency to Tk,Tcl python and so on. Perl is known to be a rather difficult language and it creates yet an other dependency.
Thu, 12/22/2011 - 11:11
When migrating from our previous bug tracker (ancient installation of Bugzilla) to MantisBT this summer, we considered Redmine among other alternatives (MantisBT, Bugzilla, JIRA). All have their own pros and cons, so we decided to use Mantis since it is simple and does all what we needed, and does that (almost) in a way we like (at least, get accustomed to). Mantis got wide use in our projects last years, and we preferred to work with the same system in this project, too.
Mantis as well as Redmine provides version management (see Roadmap and Change log), issues relationships, has plug-ins for Wiki and version control integration, and even for project management facilities such as time tracking. That is, the functionality it provides is quite comparable to that of Redmine. The difference is mostly in presentation and level of integration between components, and I agree that Redmine looks better in this aspect.
We are using Redmine internally, and personally I like it for the nice way it presents the data, and especially mail notifications. However, when trying to use it on more-or-less real cases, I found its behavior unpleasant. Not sure to recall all issues, but at least several times I have lost some text I tried to input in the issues because when I tried to save it, Redmine just logged me off. Another issue I have is that trying to connect to the current OCCT SVN repository, all what I get is message 404 "The entry or revision was not found in the repository". The same happens when I click on some user names (including my own) -- instead of viewing user profile, I see just an error message.
There are also some conceptual problems to consider if we adopt Redmine. First, it is tailored to running complex projects rather than plain bug tracking. What we have in OCCT is mostly bug tracking, and Redmine's emphasis on project management would likely be useless.
Next, we want to keep all the issues we have on OCCT in one tracker system, but at the same time we need to separate issues created by commercial customers, and restrict access to issues on per-customer basis (for confidentiality reasons). This is not usual practice, and neither Mantis nor Redmine provide such possibility readily. Now the solution we have found is that we mapped customers to (sub)projects in Mantis. This approach also renders 'normal' project management features useless.
Finally, we need to compare tools for Git integration and code review provided by Redmine and Gitorious, to see if one is much better than another for our needs. If we end up with using Gitorious, Git integration capabilities of Redmine will become out of interest.
That is, I do not exclude that one day we decide to switch to Redmine, but this move should be carefully analyzed before. The idea of using one tool is quite attractive, and we definitely will pay more attention to Redmine.
Thu, 12/22/2011 - 11:50
For the moment we use SVN for OCCT internally. IMHO, it satisfies the development needs, especially for code revisions. WebSVN provides convenient interface for developers and technical experts code revision.
For sure, moving on Git will require some efforts from our site and we have to compare potential benefits of this movement.
Thu, 12/22/2011 - 12:50
I think, offering a clean bugtracking interface (i.e. Redmine and not Mantis) could be better for the following reasons:
- creating pleasant image of OCC development to the outside world
- working with cleaner designs is more pleasant for developers
- the not needed features can be turned off
- the amount of information visible at one time is reduced, enhancing the experience as well
- design and usability matter :)
Internally, if OCC is already using Redmine, that could mean more uniform administration.
P.S. I'm in no way connected to Redmine, but using it.
Thu, 12/22/2011 - 11:24
We need to have some system for running tests and processing results anyway, and the choice is between developing our own tools from scratch, or using existing one. Our current (home-made) test system is based on Perl, and when we chosen to use TAP we decided to rely on Perl implementation, as it seems most convenient for us and native for TAP.
I suggest we shall wait until inclusion of this system in OCCT 6.5.3 release to see how it works, and then perhaps consider other alternatives.
Fri, 02/10/2012 - 11:09
Hello all,
This is to inform you on our current status of setting up public repository and tools around it.
After spending much time setting up and configuring Gitorious software, then trying to simulate real work with it, we decided to abandon this tool. The reason is that the functionality provided by it is quite limited and does not fit our needs well. The major points that we miss are possibility of managing access rights on the branch level, and integration with LDAP (even if for both problems some patches and tricks exist, they do not solve the problem completely).
Thus we decided on using Gitolite for managing repository, and integrate Git-related features into the existing tools we have. The contributors will add or edit SSH keys for accessing the repository via the user profile on the web site. We are going to provide GitWeb (or perhaps Cgit -- to be decided) access to view to the repository, and install Source Integration plugin in Mantis to display related Git commits inside the issues.
We shall start with single repository, where master branch will be populated by certified contributions only. The users will provide contributions via topic branches. The details are being elaborated and we will publish the document as soon as it is ready.
Following advise of community members, in parallel we are investigating capabilities of Redmine as a possible replacement for Mantis and GitWeb, promising more integrated solution and better functionality. We have successfully converted our issues from MantisBT to Redmine, and attached it to Git repository. It works, however we encounter several problems with its usability (list of issues, filtering and search), performance (listing big diffs), and lack of features we use in Mantis (e.g. inheritance of categories and version numbers in projects). We still hope that these problems can be solved or avoided. As possible switch to Redmine does not cancel the need of moving to Git, now we concentrate on setting up Git as first priority, and further investigation of Redmine is postponed.
Andrey