Chris Pattinson

A basic business theme drilled into me about 16 years ago was the customer is always right. Now, there are some exceptions - but that golden rule implies if you produce products or services that your customers need and want, you’ll have a successful business.

This was something on the back of my mind during my four days in Cambridge at the Software Test and Performance conference.  What can CodeGear do better to ensure what is delivered speaks to what customers really want? Many things…

Below is a part of some thoughts that came about from the conference, one of the major benefits of attending was a copy of all the speaker notes, and many great references to processes used by other organizations to solve some of those annoying problems in testing - such as documentation of the test effort, understanding how to really be agile, and how to identify how different your assumed processes are from your real processes. 

Regardless - it all comes down to what is the net result for your users. Are you ensuring the team is productive and focused enough on doing what your customers really want? And how do you know if you are or aren’t on track? Are you getting high return of value on the activities you are assigning your team? Are you spending the right amount of effort on important/high value features and fixes? Are the project metrics you are using the right ones for your needs? (I’ll write more about metrics in the future- metrics are very powerful if used well. And can be a major time sink if not properly thought out)

Regardless, here is the tip of the iceberg and a tease for those interested:

ST and P Conference - Rough Notes

Improvement Themes

  Requirements

• Are they clearly understood by all stakeholders? 
• Have they been reviewed by potential customers, especially major ones?
• Are Business Requirements ‘What’ and Functional Requirements ‘How’ ?
• Do any requirements have implied requirements which may not be obvious? (OS, Platform, Bug levels, user acceptance, documentation, translation, etc…)
• To be truly agile, QA and Doc needs to be involved early as possible in requirements definition. A good requirement should be seen as a public document.

Scrum/Agile

• Requirements in Scrum
  • Business requirements should drive functional requirements
  • Business requirements should link to functional requirements, and vice versa (are we doing work that is valuable?)
  • Product Owner has a lot of work here as they represent the customer

• Agile - what is it really?
  • Scrum is a project management tool, does NOT make a team agile by itself
    • Is the goal of the scrum clearly defined?
    • Organizations that give a theme to a sprint find the teams focused during that sprint
  • To be Agile you need to know where you are, and where you need to be
    • This links closely with well defined business requirements
    • 90% done is still not ‘done’.
    • Truly agile means the entire team takes ownership of testing.
    • Development should be very intimate with QA testing and able to identify high value tests
    • QA should be very intimate with code changes
    • Testers in agile are basically test developers.
    • Test results need to be visible and clearly understood by all members of a scrum team
    • Understanding of test coverage (current and planned) shall be understood by all scrum team members (blending of developers and testers for this purpose)
  • Unit tests exercise API and input/outputs, however will miss the ‘big picture’. Test developers have to test for the ‘big picture’ and how the systems work together. (for example, two systems - one designed for meters, one designed for inches, can pass unit tests, but fail when asked to communicate together)

Customer Involvement

• We should encourage field testers to think as end users
• Provide customers who beta test with the exact shipping product, and see if it’s usable. If you need to write scenarios and how-to’s, it’s likely your product is hard to use or poorly documented
• Open Beta’s are a great way to get feedback on your general customer base. Make sure to include a survey that asks the questions you really need answers to, such as are the new features and quality compelling enough to buy.
• Think of the end experience. The functionality is about 1/3 of what the customer cares about in the product. Does it do what they want, reliably, and without them scratching their heads a lot.
• It’s ok to provide ’scenarios’ document as long as it’s content that will go into documentation. It should be seen as marketing materials more then testing materials, as you want customers to provide feedback on all areas of the product - not focused on specific scenarios.

Bug Fixing/Stabilization

• Learn from past project data - projects often follow similar ‘flows’.
  • If you needed 10 final builds to get your shipping candidate several projects in a row, PLAN for it for the future. Odds are you will need to do so again
• Bugs that were not fixed in the past, is not an excuse to not look at them now. Look at the reasons they were not fixed - a product that keeps deferring previous bugs only gets worse in quality to the customer
• Review find and fix rates, as well as the age of fixed reports. It is common that older defects take significantly longer development and test time to process then newer reports. Take your bug backlog and age into account in planning.

Share This | Email this page to a friend

This entry was posted on Tuesday, October 9th, 2007 at 10:52 am and is filed under General, Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.