Pushing the Quality Central ‘clean up’
Hard numbers - a search this morning on Quality Central resulted in 3980 Reported issues for Delphi. This should include ’everything’ - all versions, all locales, feature requests, the works.
What’s interesting is the C++ number - 144. David Dean (was an active field tester, now a member of my team), and Leo Siefert seem to be the main two keeping that product clean in QC. Special kudos to those two since that was a HUGE amount of work. It also proves the actions of a few can really matter.
Knowing the problem you have, is half the problem in solving it. I’ve directed the QA team to look at at least 5 a day, putting a quick note in weekly status reports their findings, and looking at ways to find the nastiest bugs we need to fix NOW, fast.
A few of the items today:
1)
The IDE is sending Delphi AIR’s (Automated Incident Reports) into the Delphi.NET project. (QC 44970) - this absolutely has to get fixed.
2)
From a QA engineer on my team came this email:
—–
I’ve just searched QC, and I see at LEAST 200 (I limited my search to 200) automated incident reports with the phrase:
"Cannot change Visible in OnShow or OnHide"
the stack trace has :
Forms.Forms.TCustomForm.CMShowingChanged
DeskForm.DeskForm.TDeesktopForm.CMShowingChanged
on top.
—–
Good example of data mining to find a trend of an otherwise difficult to reproduce report.
We’ve passed this information on to R&D and will investigate to understand and reproduce as much as possible.
3)
Today I put a Post-It wall pad from 3M next to my office, to track daily the number of QC Reported bugs. A little visibility is a good thing.
June 27th, 2007 at 1:04 am
this is great news!!
but 3980 issues at 5 per day is 796 [working] days. how about putting some more staff on the QA team?
June 27th, 2007 at 9:05 am
I actually have expanded the QA team in the last year In addition we combined the International QA group with Core QA to avoid some duplication of effort. It’s why we can spend more time in QC now, and you should see more QA faces in field tests and working directly with customers.
We brought Reported down from 3980 to 3920 after one day, with several promotions into the internal system - the goal was 5 per person, not 5 for the team.
I also am directing QA team to write automated tests that will be re-usable in the future, and that’s a time consuming development effort. As the QA team reproduces bug reports, the goal is write an automated test to reproduce the issue and have it run daily in a test framework. This ensures two things : 1) it’s visible when it’s fixed and 2) once fixed, it’s immediately visible if it ever breaks again.
Hence, if QC reports are reproducible with detailed steps it lets the QA team go through them much faster, and will ensure we have proper regression testing in place.
July 2nd, 2007 at 7:36 pm
Yesterday I submitted a new issue to Quality Central (48365). Today, it is "Closed" with the resolution stated as "Test Case Error". There were no comments or votes added. What does that mean please? I am still experiencing the problem cited in that issue.
July 3rd, 2007 at 9:17 am
Typically comments should be added. This one never got to the internal bug tracking system. I can’t reproduce your problem, is your system clock set ahead a month? What OS are you using?
July 3rd, 2007 at 5:04 pm
Hi Chris, thanks for the response, but how about let’s re-open the Quality Central issue and discuss it there. The post to the blog is more related to the question of what game are you guys playing at stomping on submitted Quality Central issues in this manner? And to ask what does "Test Case Error" mean? Presumably it is something different to "Cannot Reproduce"?
Thanks,
Grant
July 3rd, 2007 at 5:37 pm
Test Case Error means the submission is invalid as submitted. Cannot Reproduce means that the issue seems valid, but cannot be reproduced with given steps. It may mean steps are incomplete, or more information is required from the submitter on the environment the issue was encountered in.
Quality Central should contain product related issues, 48365 describes a problem with date displayed in the Start Page for articles posted on DN. It seems to be an invalid report since a) it’s related to the website, not product (what do you see when you browse to the website directly?) and b) we can’t reproduce it and no other steps/information are provided to help us do so if in fact it is a start page logic error.
We’re not playing any ‘games’ with Quality Central. We will have to push back on reports we cannot reproduce, and some folks will need to work with us and give the information we need. The goal is to reproduce, then fix, issues in the product. And of course we’d like to target the most serious ones first.
July 3rd, 2007 at 5:52 pm
The games I’m referring to is closing my report without any feedback or reason. Here you have given me feedback and reason, which is great. It should have been in that report.
The problem I experienced was in the Start Page of the Delphi IDE. If that is caused by something else or not is a back-end concern. If I submitted in the wrong place, then CODEGEAR should move it, don’t close it without explanation.
Although you cannot reproduce it, this is likely because you have not looked at the news feeds from one of the "Codegear Developer Network".
If I had been given a chance in the QC process, we would have been able to figure that out together and solve the problem. By closing the issue CODEGEAR has gained nothing except drive me to frustration and loud typing.
Thanks again,
Grant
July 3rd, 2007 at 6:41 pm
Ok Grant, message understood. Feedback should be provided on any closed reports describing why the action was taken place.
I did look at the feeds, both from IDE and in standalone browser and do not see anything wrong with the dates. Hence my questions about system clock setting and OS may still apply. What timezone are you in, etc…?
And did you confirm that your browser shows a different time/date then the IDE embedded browser?
July 3rd, 2007 at 9:06 pm
I still don’t understand why we are doing this in a blog. This discussion belongs in the QC report - this is one of the points that I have been trying to make.
My aim in mentioning this issue on the blog was to raise awareness that there maybe be some inherent danger in using "number of closed reports" as a metric.
And that maybe there is some internal misunderstanding at Codegear about the if/when/hows of closing a report. I would be rather shocked if it turned out that due process was followed in this particular case.
However, here goes:
Steps to reproduce on two machines in our office (both Tokyo timezone):
1 Open Delphi2007 IDE
2 Click "News" under the "Codegear from Boland" banner
3 Click "Select Newsfeed"
4 From the dropdown menu choose "Codegear Developer Network" and then a submenu like "Delphi"
5 Look at the green text at the bottom of each posting, check for dates in the future, such as (copy-paste here:) "Posted on: Friday, 3 August 2007"
Other notes:
* I’m sorry, but I don’t know how to navigate to an equivalent page in Internet Explorer.
* My Windows clock has the correct local time/date for me.
* Not all feeds have this problem. For example, if I select the feed "QualityCentral" -> "Delphi", then there are no future dates.
* We have reproduced this on two machines.
Thank,
Grant
July 3rd, 2007 at 9:19 pm
I have found the feed in Internet Explorer:
http://dn.codegear.com/delphi/rss
The dates are correct in IE, wrong in the Delphi2007 IDE.
Sample Copy-Paste from IE:
Hotfix 13 for Borland Developer Studio 2006
Today, 4 July 2007, 10 hours ago
Abstract: Hotfix 13 for Borland Developer Studio 2006 is now available
Equivalent Copy-Paste from IDE:
Hotfix 13 for Borland Developer Studio 2006
Abstract: Hotfix 13 for Borland Developer Studio 2006 is now available
Posted on: Friday, 3 August 2007
Thanks,
Grant
July 4th, 2007 at 8:44 pm
hi,
i think i have blogged you enough about my stomped-on QC report. so i’ve pasted my info from here into that report. however, it’s still closed, which makes me uncertain as to whether anyone will read it ever again. do you have enough inclination yet to get someone to re-open it and resolve it properly?
thanks,
grant
July 5th, 2007 at 12:27 am
Grant, I’ll get it all sorted out on Friday once I’m back in office. Was just enjoying a pleasant July 4th
You have provided plenty of information - I’ll be reopening the issue, then looking at testing with different system time zone settings.
We can move discussions into QC. I really don’t mind these discussions on my blog, in email or in QC as long as we get the issue identified and resolved.
I’ll also work with the QA team about ensuring we give adequate feedback in QC when resolving reports. Thanks for bringing all this up and spending the time providing clear and concise information.
July 5th, 2007 at 12:53 am
that’s great news. thanks. happy july 4th.