Chris Pattinson

We’ve made a change to how we will handle “Needs More Info” and “Cannot Reproduce” reports in Quality Central.

When the Sysop or QA engineer reviewing the report decides on this resolution, we’ll put the report into a ‘Pending’ status. This means that the report could not be verified with given steps/information. Sysops and QA can query specifically on this status to revisit or update the information on the report.

This is more user friendly then ’closing’ a report, which seems to message the CodeGear no longer cares about the report. Before we had pending, we only had the option to close to send a message to the one submitting the report.

In any case, a report that comes back as “Needs More Info” or “Cannot Reproduce” should have a brief description of what was tested, or what information is required. If this does not happen, then QC users should contact me so I can contact the appropriate Sysop and/or QA engineer.

Share This | Email this page to a friend

Quality Central is seeing a fair amount of activity in the past week. To give some insight into ‘whats going on’ - daily the development managers meet and review the last day of reports into Quality Central, both public and private. We go through as many reports as we can, and try to either promote or push back depending on if a report has everything we need to act on it.

In addition the QA team has been spending time reviwing QC reports for their areas. Anyone who subscribes to the daily digest of changes should be seeing this activity directly.

So far, it’s been great - when we bounce a report back as Needs More Information, the QC email notifcation lets the submitter know of a change to their report, and typically I’ve been getting a reply within 24 hours. We’re also seeing a lot of attention to detail in reports, especially from the field test. Clear descriptions including expected and actual behavior, code samples, screenshots and detailed steps. This makes everyone’s job a lot easier - from QA to reproducing it and writing an automated test, to R&D fixing it and ensuring a unit test is developed if appropriate.

The backlog of older QC reports is still huge. So far we’re making slow progress to a net gain, but at this rate it will take several years to get through all reports. So it’s time to look for more interested community members who are willing to be SysOps. Basically need people to look at the older reports in an area they care about, close out or push back the ones that cannot be reproduced, and mark as Needs Attention ones that they can reproduce and need to get fixes ASAP.

Any volunteers interested in helping out? Reply here, or contact me directly cpattinson@codegear.com (and CC chris.pattinson@gmail.com )

Hey it’s an (relatively?) easy way to fame. We publish publicly the sysop stats so you can see who has been busy…

http://qc.codegear.com/stats/sysop.aspx

Share This | Email this page to a friend

We’ve been working on an update to Delphi for PHP.

This update contains more then 70 fixes and touch most areas of the product.

We’re looking for field testers to test the update. You do not need to already have Delphi for PHP, so this is also a good way to take a quick preview and work with the development team.

Click Here to Apply

Share This | Email this page to a friend

We’re going to run a contest during our Highlander field test. It should be fun and a good way to encourage the field testers to work closely with the development team. Call it an ‘Iron Chef - Coding’ event, if you will. And the ingredient to be served up is ….

The new ASP.NET Provider.

To enter, you’ll need to apply to the Highlander fieldtest.

Here are the general details:

ASP.NET Provider Application Contest

Task

The contest tasks you to build an application that demonstrates and exercises several of the new features under development in Highlander.

• Submitted application should have the following features as a minimum:
  • Application should exercise the ASP.NET provider
  • Application should compile
  • Application should be deployable
  • Application should have a project page with instructions to use/extend
    • This is an HTML page that will describe the project, how it can be used, and how it can be expanded.
  • Must be built using Highlander

Application should exercise (if applicable):

• Membership
• Roles
• Session state providers
• Profile - where you store information specific to a user (eg: shopping cart)

Example Solutions

• Template Library
  • File New | Blog
  • File New | E-Commerce
  • File New | Content Management System
  • Any sort of online database requiring user access, authorization, and authentication

Process

• Applicants must agree that all submissions become the property of CodeGear
• Bug reports via Quality Central
• Submissions via .binaries in single .zip to field test newsgroups
• The ASP.NET provider fieldtest newsgroup will be used for discussions and issues surrounding the project.
• Participants will have to sign NDA and participate in Field Test
• Submissions must be received by August 15th
  • Winners will be announced on 29 August
  • Top three may ship with product (Other options: CodeCentral)

Extra Points

"Extra Points" will be given for applications that:

• are Database Agnostic - multiple database usage (Interbase, MSSQL, Oracle, Sybase, DB2, Informix, MySQL)
• have a thorough project page
  • No need to be localized
• are skinnable or come with various themes
• are ready to be shipped in the product
• exercise the features thoroughly
• result in finding bugs in the Provider implementation
• demonstrate complex session state providers/roles/membership
• use of ATLAS components
• use the ECO provider

Judges

• Nick Hodges - Chairman, Mark Edington - R&D, Jim Tierney - R&D, Chris Pattinson - QA

Prizes

• TBD: T-Shirt for all participants that meet minimum criteria
• CDN article to recognize all participants who met minimum criteria
• Recognition in Shipped Product 
• Product Licenses

Share This | Email this page to a friend

We’re starting the field test for the next product release on our roadmap, code named Highlander. Some of the features we want feedback on are ASP.NET features, supporting .NET 2.0/3.0, consumption and creation of parameterized types (“generics“) in Delphi for .NET, Vista support, ECO IV and some very cool database features. Review the roadmap document for more information.

This release also includes Delphi and C++, which will undergo some additional bug fixing and changes - and we also want to ensure we do not regress any of that functionality as a result of feature work. So this release has a bit of everything for everybody. If you wish to apply, please fill out the simple application form here:

http://www.surveymonkey.com/s.aspx?sm=zOitB9WcYlX1htdU4J6ioQ_3d_3d

We really want active and vocal (in a constructive way, please) folks on our field test.

Among the topics of interest are the changes to Winforms. We’re aiming to balance development work so we can get to Unicode support and 64 bit support as rapidly as possible. So here is a good chance to talk to the product team on what you need from RAD Studio, and the features/fixes that matter most to you.

Share This | Email this page to a friend

Spent an hour with Nick Hodges, John Kaster, Calvin Tang, Jeff Wilsbacher and Mike Devery discussing Quality Central use in CodeGear.

We discussed a variety of topics, John also gave a walkthrough of the reporting QC provides, and we took a look at several QC clients. 

John also made a good point - QA should first be looking at the ‘Needs Attention’ reports in QC. I only saw 10 listed for Delphi, so really Quality Central was in better shape then we first realized. We also raised several feature requests and ideas to help Quality Central self-clean, such as if a Reported issues hasn’t been commented, voted or raised in a certain period of time (such as two years) the issue could be set to a type of  ‘Not Active’. And we looked at the voting system, seperating feature requests from defects, and John pointed me to Jeremy North’s client to use for searching/processing through QC reports: http://www.jed-software.com/qc_download.htm

Then we went through some of the reports that are available. QC reports show Sysop stats, user stats, projects stats, and a breakdown of the ’state’ of features areas in Quality Central.

http://qc.codegear.com/stats/

Share This | Email this page to a friend

Chris Hesik sent me an email this morning offering to show me the tools he’s using to data mine the AIR’s in Quality Central. Of course I agreed, and we’ll meet on Monday.

In addition he sent the following out on an issue he found using this tool to another R&D engineer.

—–

I’ve been doing some data mining of the AIR reports in QC, and for the Cogswell RTM build, the most frequent stack trace reported (12 instances out of 191 total reports) is the stack trace shown in QC report 47921.

The error is a List Index out of Bounds and the top of the stack is:

[20031735]{rtl100.bpl  } Classes.TList.Get (Line 2992, "commonClasses.pas" + 2) + $A
[20031685]{rtl100.bpl  } Classes.TList.Error (Line 2961, "commonClasses.pas" + 1) + $14
[20031735]{rtl100.bpl  } Classes.TList.Get (Line 2992, "commonClasses.pas" + 2) + $A
[20031716]{rtl100.bpl  } Classes.TList.First (Line 2986, "commonClasses.pas" + 0) + $2
[20A450E3]{coreide100.bpl} EditorForm.TEditWindow.WindowActivate (Line 2487, "EditorForm.pas" + 3) + $5
[20A44E72]{coreide100.bpl} EditorForm.TEditWindow.WMActivate (Line 2411, "EditorForm.pas" + 2) + $B

It appears this is related to the change you made in revision 5882 of EditorForm.pas.  The List Index out of bounds occurs when calling EditWindowList.First if there are no items in EditWindowList.  I’m guessing the simple fix is to add a check to make sure that EditWindowList.Count>0, but I thought I would run this by you first.  Can you review the code and the stack trace and see if this is the correct fix?

—–

It’s a great example of using Quality Central and AIR’s from the development side. If I’m lucky, I can nudge him to blog about it.

I do suggest anyone logging an AIR from the IDE include their developer network information - that way we can contact you if needed.

Share This | Email this page to a friend

Hard numbers - a search this morning on Quality Central resulted in 3980 Reported issues for Delphi.  This should include ’everything’ - all versions, all locales, feature requests, the works.

What’s interesting is the C++ number - 144. David Dean (was an active field tester, now a member of my team), and Leo Siefert seem to be the main two keeping that product clean in QC. Special kudos to those two since that was a HUGE amount of work. It also proves the actions of a few can really matter.

Knowing the problem you have, is half the problem in solving it. I’ve directed the QA team to look at at least 5 a day, putting a quick note in weekly status reports their findings, and looking at ways to find the nastiest bugs we need to fix NOW, fast.

A few of the items today:

 1)

The IDE is sending Delphi AIR’s (Automated Incident Reports) into the Delphi.NET project. (QC 44970) - this absolutely has to get fixed.

2) 

From a QA engineer on my team came this email:

—–

I’ve just searched QC, and I see at LEAST 200 (I limited my search to 200)   automated incident reports with the phrase:

"Cannot change Visible in OnShow or OnHide"

the stack trace has : 

Forms.Forms.TCustomForm.CMShowingChanged

DeskForm.DeskForm.TDeesktopForm.CMShowingChanged

on top.  

—–

Good example of data mining to find a trend of an otherwise difficult to reproduce report.

We’ve passed this information on to R&D and will investigate to understand and reproduce as much as possible.

3)

Today I put a Post-It wall pad from 3M next to my office, to track daily the number of QC Reported bugs. A little visibility is a good thing. 

Share This | Email this page to a friend

Quality Central is getting a lot of attention this week. I found out we had over *two thousand* QC reports in a ‘reported’ state. Yes, the system is expected to be maintained by Sysops and community members, but it’s starting to make CodeGear look like we do not care about our products. We really do

So I wrote various thoughts in this developer network article, as well as a behind the scenes look at our bug prioritization criteria.

One of the biggest battles being a quality assurance manager is balancing time for feature development against time for bug fixing. In Delphi 2007, we allocated a large amount of time for bug fixing, and many of these fixes were shared with the recent C++Builder 2007 IDE.

To be fair, with all this effort from the development team on bug fixing - some of the new features in the works mentioned on the Delphi and C++ roadmap were pushed back.

Now we’re looking at Quality Central with a critical eye. We’ve been using it effectively for field tests, but that typically focuses on the current version - not all previous versions. The reason is mainly about use of time - we have Field Test Marshal’s who help prioritize bugs in a feature area, promote QC reports to the internal system and communicate with the development team on the most serious issues. That has been helping us a lot, but it’s not reflected in the backlog of reports in QC from previous releases.

Just this evening we made changes to Quality Central, including moving the Automated Incident Reports to their own section so we can more easily track manually entered reports. AIRs usually do not have steps or description that helps QA, but it’s the trends we data mine in AIRs that let us find and fix several memory leaks and crashes in the past year. They enhance our testing efforts.

So this is a great time to talk to use about Quality Central. I’m scheduling several meetings over the next couple weeks, and directed my QA team to look at 5 QC reports a day. Let’s see how effective taking numerous small bites off leads to solving that huge amount of ‘neglected’ reports.

Share This | Email this page to a friend

We have an update for Delphi for PHP in the works, a more major update then the first one.

It’s ready for testing, and we’re looking for folks interested in testing this update to ensure it’s of good quality.

If you are interested, please fill out this simple form:

Share This | Email this page to a friend