Software Test and Performance Conference
This week, I’m on the road to sharpen and learn about current QA processes. CodeGear has made some great progress in the past few years in terms of functional, performance, stress and user acceptance testing - it’s a journey I never really expect to finish since technology will keep moving very rapidly!
So I’m in Boston, at the Software Test and Performance Conference. http://www.stpcon.com/
It’s a three day conference from Tuesday to Thursday, yesterday was an all day session on Measuring and Improving the Test Process. Much of what was discussed was very familiar - questions such as ‘How effective is your test system?’ , ‘Are the tests you are doing valuable?’, ‘Where are you weak, and where are you strong, in your testing organization?’.
Still, it was good to take a step back with a bunch of peers and just … discuss.
A few good ideas came out of yesterdays session - we measure a lot of testing data, but spend a lot of time sifting through the data manually. A logical next step is rolling up and summarizing test results, and sending out comparison results of day to day builds in email. Now, we’ve been prototyping an email notification system, providing the comparison of previous test results seems perfect to add into that email.
In addition - requirements testing and management had a lot of focus. And that’s a good idea - if you rush your requirements phase, you are building the wrong features, and testing the wrong features or designs. Even if your testing is highly effective, testing a bad design and verifying it meets the criteria of a bad design wouldn’t be an effective way to spend your engineers time. CodeGear seems to have a good grip on requirements management - however it’s an area that should always be invested in.
Another interesting concept raised was the ‘REAL process’ against the ‘assumed process’. For example, in planning the projects we do not allocate specific time for bug councils (where developers and managers discuss the backlog of defects and prioritize them) or for work in our beta/field tests. This is all real work, where if not planned can put strain on other commitments.
Today the conference opened with a keynote on performance testing by Scott Barber. Yes, even CodeGear needs to consider performance testing - we all want our developer tools to be super snappy, reliable, and handle large and complex projects. Last year we started the basics of performance testing - memory management, startup/shutdown times, and metrics for a large number of common tasks or handling very large projects. The results are actually noticeable in RADStudio 2007, where several areas were optimized and fixed, and we can monitor to see if other changes have side-effects.
So far, so good. I’m sneaking in this blog post between a good lunch - this mornings sessions were on user acceptance testing and test automation projects. This afternoon I’ll be digging into building quality into software earlier in the project cycle and managing software testing metrics. Been worthwhile so far, have to consider going to San Mateo in April for the 2008 conference 