Delphi/C++ Team, Quality Central and Bug Prioritization
Quality Central is getting a lot of attention this week. I found out we had over *two thousand* QC reports in a ‘reported’ state. Yes, the system is expected to be maintained by Sysops and community members, but it’s starting to make CodeGear look like we do not care about our products. We really do 
So I wrote various thoughts in this developer network article, as well as a behind the scenes look at our bug prioritization criteria.
One of the biggest battles being a quality assurance manager is balancing time for feature development against time for bug fixing. In Delphi 2007, we allocated a large amount of time for bug fixing, and many of these fixes were shared with the recent C++Builder 2007 IDE.
To be fair, with all this effort from the development team on bug fixing - some of the new features in the works mentioned on the Delphi and C++ roadmap were pushed back.
Now we’re looking at Quality Central with a critical eye. We’ve been using it effectively for field tests, but that typically focuses on the current version - not all previous versions. The reason is mainly about use of time - we have Field Test Marshal’s who help prioritize bugs in a feature area, promote QC reports to the internal system and communicate with the development team on the most serious issues. That has been helping us a lot, but it’s not reflected in the backlog of reports in QC from previous releases.
Just this evening we made changes to Quality Central, including moving the Automated Incident Reports to their own section so we can more easily track manually entered reports. AIRs usually do not have steps or description that helps QA, but it’s the trends we data mine in AIRs that let us find and fix several memory leaks and crashes in the past year. They enhance our testing efforts.
So this is a great time to talk to use about Quality Central. I’m scheduling several meetings over the next couple weeks, and directed my QA team to look at 5 QC reports a day. Let’s see how effective taking numerous small bites off leads to solving that huge amount of ‘neglected’ reports.
June 14th, 2007 at 7:17 pm
I have to say that, after using QC for various things, that it’s time to take a hard look at it and think about perhaps moving to something more robust or open. Every time I have to fire up the QC client I cringe and the web front-end seems like it’s always a half step ahead or behind the client.
After having used a ton of bug reporting and tracking tools I’ve honestly found Trac to be right up there. It seems to have all the features you’d need with the right type of information and best of all you don’t have to maintain it. I use it both at my "day job" managing rather complex commerical pieces of software as well as my "night job", that being consulting and web dev.
It works great with subversion, has good reporting tools and a nice, clean, easy to use interface. I’m sure CodeGear would like to continue using what they have but it might be time for CG to bite the bullet and realize there are now better products out there and it would be easier to use on of those instead of half-maintaining an in-house custom system.
Hopefully I’ve planted at least a wee seed on using Trac
June 15th, 2007 at 5:07 am
Continue to focus on bugfixing first. Delphi 2007 and the recent Update 1 gives me hope that you are on the right way. Nobody needs a feature ritch application that is unusabe because of its pure quality.
June 15th, 2007 at 6:24 am
Great post Chris!
It’s nice to know that QC is getting some attention. It did feel like whatever we reported was simply not getting noticed. There’s also a large quantity of things reported long ago that have still not been looked at.
Scheduling a "delphi hour" type chat with the people on the Field Test might be a good idea.
> Continue to focus on bugfixing first.
Crashes, memory leaks and basic functionality failures, sure, but there comes a time where you product must evolve, even with its flaws, or it will stagnate and you will loose users.
June 15th, 2007 at 7:49 am
In my opinion QC is a total mess. I’m used to using bugzilla with open source projects, which will.
1. allow you to enter bugs easily.
2. allow you to search bugs easily.
3. allow you to receive emails on individual bugs.
4. allow others to see the bugs you enter so that they can add their ideas or similar experiences to.
5. Allow more then one file to be uploaded.
6. The sysop can direct bugs of a particular type to one person.
CG
1. Has two different types of clients, which allow different types of things. (emailing?)
2. Can search but can’t search for a particular string in all bugs. Search is much to directed.
3. Finally figured out how to turn on my email from QC. But I never get emails about my own bugs, yet if I check manually people have replied to them. This is STUPID.
4. Cannot see other field testers bugs.
5. Upload a file, the original one is lost.
I think if you spend 2 days putting bugzilla on your site, and another 2 days writing conversion code, you would have a much better system to work from.
June 15th, 2007 at 9:28 pm
>I found out we had over *two thousand* QC reports in a ‘reported’ state
huh? Hadnt u find out that??
ARE U SURE??
seems u just ignored those report..
June 16th, 2007 at 6:07 am
You really should give QC more attention.
Fortunately 2000 reports in "Reported" state doesn’t mean 2000 more bugs.
D2007 Update #1 fixed for example some of my reports, but they are still in the state "Reported". Furthermore if you don’t fix a bug it gets reported multiple times.
June 18th, 2007 at 8:45 am
Our use of Quality Central is defined here:
(http://dn.codegear.com/article/33888)
QualityCentral (QC) provides a community process for resolving, clarifying, and tracking quality issues regarding CodeGear’s products and services. CodeGear Developer Network (CDN) members can create bug reports and feature requests, view other members’ reports, and comment and vote on the reports most important to them. CodeGear personnel also participate in QualityCentral, providing a permanent link between CodeGear customers and the teams who produce CodeGear’s products.
It’s not the main bug repository the team uses, but we refer to it in several ways.
However, that doesn’t address the problem that the QC ‘backlog’ has been growing, and as mentioned by others - it’s not accurate. Several reports exist on already fixed issues, and some serious issues that really should be promoted haven’t yet.
A longer term solution will be merging the two database in some fashion.
June 18th, 2007 at 10:01 am
I agree, that there are a lot of bugs in QC, that are waiting for ages. It’s time to get the most annoying fixed. The easy to confuse XML mapper and the lack of good documentation for the Internet stuff like WebSnap and IntraWeb comes to my mind. It is a pitty that these things have such a steep learning curve. The new documentation system itself (as opposed to the .hlp files) is very slow to use and find most of the time no answers. I regulary look things up with Delphi 6 Help. The new help system destroys all productivity gains of the new Delphi 2007 IDE.