Chris Pattinson

I’d like to give some props to Pawel for great work finding and summarizing excellent references for the latest Delphi 2009 and C++Buider 2009 products, including good starting places to understand the unicode changes.

http://blogs.codegear.com/pawelglowacki/2008/11/03/38527

He’s provided sections for Overview, New IDE features, Unicode support, E/R Studio information, New language features, changes in RTL/VCL., DataSnap and a whole lot more. Seems like ‘Must Read!’ to me for the best Delphi and C++Builder release so far

Share This | Email this page to a friend

Hi all,

 We’ve started the Delphi Prism Beta, which is testing of our new Delphi.NET solution. It’s pretty cool, I’m not supposed to say more until Nick officially announces the details.

 However if you are interested in testing out fully functional beta builds, go here to apply:

https://beta.embarcadero.com/callout/default.html?callid=6E2829BA76664D57ABD8F26046781EA5

 We’ll be adding folks periodically from those that apply. Anyone who has an interest in using Delphi syntax in Windows .NET applications should check it out and give us some feedback on what they think. And Delphi users who want to see if they can develop easily with .NET should also take a look.

 Cheers,
Chris

Share This | Email this page to a friend

I’ve been quiet on my blog for about a year, that was a result of work to transition the team from Borland to CodeGear then CodeGear to Embarcadero, work on improving our beta tests, and championing product quality. I kept telling myself I should be blogging more, which I’m sure is also true

The team did several ‘cool things’ for quality this year. We went through the ENTIRE bug backlog, reprioritized and retested literally thousands of reports, and identified the highest severity reports we wanted to fix for the Delphi 2009 and C++Builder 2009 release.

Then the team settled in to a focused period of feature development and bug fixing, and the biggest feature work revolved around the latest Unicode feature, Generics, the new Ribbon Controls and DataSnap.  There were a few dozen other features put into the mix as well and we struck a generally good balance of features and fixing.

What really helped the quality this release was full management support. Of course revenue is a pressure, but our new management team as a longer term outlook - they wanted a release for more then just a quarter. So when our field test responses were not looking as good as Rad Studio 2007, we extended the schedule for several more weeks of focused bug fixing - and I think the results paid off.

 Can we do more? Heck yes. And we’ll work on regular updates to both product and documentation, and keep an eye on what you folks are using most and the most serious issues to fix first.

We’re also totally revamping our beta test,  I’ll blog more about that in a week or two since we’ll have our next Delphi .NET product to give this new system (CenterCode Connect) a spin.

Back on topic - I’ve posted fix lists of older Quality Central issues fixed as part of ‘Tiburon’ - the Delphi 2009 and C++ Builder 2009 project.  You can find these articles here:

Delphi 2009 - http://dn.codegear.com/article/38714

C++ Builder 2009 -  http://dn.codegear.com/article/38715

Over 4000 fixes were made, though of course some of those were regressions introduced during development, but it’s still a darn solid number. And the bug find rate was much lower in the final weeks - the usual sign during software development testing is completed and you are ready to ship. We had a good understanding of what we were shipping this time, and already a list of more bugs to fix to take ‘great’ to ‘Wow! Amazing!’.

I’m going to be honest - Documentation isn’t up a level of great quality yet. Delphi 7 help was awesome, and really set a good bar. We’re building back from some business decisions in the past that really cut the quality out from the Documentation since D8, and it’s been a hard climb.

That being said, Documentation/Help *is* definitely better then RAD Studio 2007 with over two hundred fixes between Delphi and C++, especially in terms of speed and number of examples, yet not the great reference experience we want it to be. The team will keep working away on regular updates, and are looking to increase the number of staff to be able to do more in less time.  The R&D team, Nick, Alisdair and even QA such as David Dean have been stepping in to provide webinars and videos to help and I highly suggest a browse of development network in the product communities to see what I’m talking about.

For RadStudio we revamped our test automation tools, and we really leveraged those this year - about 20 000 test cases run against every build, with nice graphical reporting broken down by functional area let us ensure we constantly moved forward with product quality. Now we plan to extend those tests to run against our localized product to ensure our international customers get high quality releases with proper functionality. The international QA teams work really hard and do a good job - but it’s hard to cover that much of a product in manual testing every day!

 Overall, quality in the product is easier to track, and test coverage is increasing. I highly recommend tools from Automated QA for profiling and determing testing code coverage. I’m sure Mark Edington would agree that AQTime was directly responsible for identifying some really nasty performance issues and crash issues in the IDE, leading to D2009 being far more stable then previous releases. 

Hope you enjoy the results of this years work, and feel free to send me an email directly at cpattinson@codegear.com if you run into any bug that you think is bad and we should be looking at ASAP. Of course, I’d appreciate a Quality Central report and good steps

 Regards,

Chris

Share This | Email this page to a friend

Since we’ve just released the December 2007 update for RAD Studio 2007, Delphi 2007 and C++Builder 2007, this seems like an appropriate time to reflect on this years work. (see http://dn.codegear.com/article/37350 for details on the update). 

First a few statistics: Since January 1, 2007 a total of 7006 defects have been verified by QA as ‘Fixed’ or ‘Retest’ between Delphi, Delphi.NET and C++Builder. This excludes duplicate reports, test case errors, or ’as designed’ reports. That’s a whole lot of product loving, in my opinion. 1196 of these reports are from Quality Central, which personally I’m glad to see the community feedback being used to help prioritize bug fixing efforts. 

The team also released three major products, each having as much testing and fix time as main development - Delphi 2007, C++ Builder 2007 and RAD Studio 2007.  Each release provided additional fixes to the previous, though it certainly made for a complex install environment.So there is where we are planning to focus next. Install needs work, and though that may not directly help current customers, we want a reliable, efficient and simple installer to get our products up and running on your systems in as efficient a manner as possible. Stay tuned for more information, since I expect to put a call out for field testers in Q1 2008 for helping ‘beat up’ new version(s) of the installer to see how much we can improve it in terms of performance and reliability.

2008 will be an interesting year - Unicode is on the roadmap, and work has already started. We’ll also be looking for users with complex projects to join the field test to exercise their projects against the next release. Odds are we’ll be simpler with the install (no multiple small releases - a pity in one way since we got tons of core fixes that way, but on the other hand made for very complex upgrades/patches!), yet nothing is finalized yet. I’d like to thank the CodeGear development team for putting a really big focus on quality this year. We put considerably more ‘bake time’ into the past few releases then in the previous years, and I strongly believe this helped both the products and the developers who use them.  And I’d really like to thank the field testers and field test marshal’s who were vocal in a very constructive manner, and effectively used quality central to report issues, and help prioritize them.

We made some improvements already to Quality Central (such as increasing maximum number of votes considerably, and allowing users to vote on hundreds of reports!) and much more on the way. We’re working to centralize our bug reporting to Quality Central instead of using RAID (kudos to RAID to serving us so well for so long) mainly to avoid the team having to look in two places for information, and two systems makes it too easy to let things slip between the cracks. Internally, I have quite a few plans, and the QA team is already working very hard on them - running automated tests on localized builds (already run against German a few months ago, now need them automated daily), a quality dashboard with test results, test coverage and performance metrics, and integrating more real world projects into our test system. Reminds me, I’d like to thank several of our long term customers for submitting their business source code demonstrating performance and stability problems, leading to significant fixes in performance and nailing some nasty memory leaks.

 Let’s do more of that in 2008, and make it even a better year. 

Share This | Email this page to a friend

Quality Central is one of the best methods to formally communicate with our customers.

One interesting statistic - 10472 reports have been pushed into RAID (internal bug tracking system) from QC since inception, and ‘Closed’.  Closed can mean several things, so taking a slice of ‘Fixed’ and ‘Retest’ ones I see 5335.  That seems pretty good.

On the other hand, we still have several thousand open and reported issues. And many of these have 0 votes, and not all product areas have QA or Sysops reviewing them on a periodic basis (since we’re all human, and folks tend to focus on the areas they are best at.)

Internally, we’re still continuing the plan to move our internal bug system into QC, so that the internal engineers can work with the one system and make the use of time much more efficient - especially when researching for similar or duplicated issues, or cleaning up ‘old’ areas that haven’t had much attention, or have been gathering reports on deprecated technology. 

Of the Open issues in Quality Central, we’re looking to clean up as much as possible during the next year.  This will take time,  on average it takes 5 minutes of a bug council’s time to review one report.   3077 reports are ‘Open’ from Quality Central under the Delphi project (this includes internal field test reports). 2347 are ‘unique’ reports with the others being duplicates, and 511 of those are Feature Requests. 

The numbers may seem big, but in the context that over 10 000 have been addressed and 2358 of those since 1/1/2006 - it seems doable.

So then it comes to priorities- we want to tackle the best ‘bang for the buck’ issues. I’ve proposed in borland.public.bdn.qualitycentral some changes to maximum voting and weighing, to encourage more Sysop participation. We still have a large backlog of Reported, though we’re are finally making slow progress knocking that down.

Some interesting ideas and points have been raised. The goal is to make it clearer to the development team which issues are being run into by several people. I see a focus too much on the voting of one individual, which is fine if weighing is given slightly to an expert, but I personally believe it is critical to determine the exposure of a problem, to help prioritize it.  Now, that also means QC itself has to be friendly and usable by the community - plus have value and a feeling that time spent voting and putting in detailed reports is well worth the effort.

So for those visiting today, leave a comment on what you like/dislike/want/need/hate/love about Quality Central and what CodeGear behaviors you’d like to see demonstrated.

 I’ll start with some examples:

*Feedback on reports from CodeGear in a timely manner

*Action on serious reports in form of patches or hotfixes in a timely manner

*Action on top voted feature requests

*Dislike - seperation of internal and external bug tracking systems

Share This | Email this page to a friend

Every release, I get those release jitters when putting out that gold image to manufacturing. RAD Studio 2007 has been out for about a month and a half now, and looking in Quality Central and at various forums - overall feedback has been positive.

A few folks have run into issues with the installer - some history to this is we originally scoped out the technology for a major suite release in June, and late 2006 we decided to react to public survey feedback and release the highest demand personality first as Delphi 2007 instead.

That was cool, Delphi 2007 was released in February and feedback was quite strong. We aimed for backwards compatibility as much as possible and included the launch of a new installer.

Later in May we released C++Builder 2007, which was also provided major upgrade to Delphi 2007 users. Here was where we identified and fixed several issues in the Delphi 2007 installers. This process of iterative releases was new, the QA team was learning all the installation scenarios and one challenge was reacting to the fact what we shipped in Delphi 2007 impacted the upgrade experience for C++ Builder 2007.

What was a major positive was we had another general round of bug fixes for Delphi 2007 users.

So then came along the completion of the series - RAD Studio 2007, which added Delphi.NET, Blackfish SQL and ASP.NET 2.0. AND - another round of bug fixes. It has ‘everything’ including Delphi 2007 and C++Builder 2007.

In 2007 we’ve had over 6 500 bug fixes between the releases (over 1000 from QC), which is a new record for the product. We made progress with the help system, restored lost documentation and fixing many F1 links in the IDE. The core binaries are in the best shape they have ever been, which explains why folks have called RADStudio 2007 (or Delphi 2007 Update 3) the ‘Delphi has come back!’ release.

Now, I’m still a critical kind of guy who believes being realistic is important. We’re not done yet. To me this is just a good start - there is a lot more to do on the Doc side (some examples are back, not all), and the installer is dealing with a very complex set of scenarios. That relates to our product complexity and some internal processes that could be improved to make things less painful/difficult all around.

It’s easy to ‘blame’ InstallAware - but seriously, it’s difficult for any installer technology to deal with all the complexity we are putting on it. Some of the issues are on our side. Others are on the InstallAware side and they have been very helpful.

I’ve been spending dozens of hours talking and helping various customers with install issues. It’s less then when I did work on Kylix - now THAT was complex :)  I think it’s confusing to customers as to how to apply an update, and when things go wrong - what to do.

The worst thing to do when the Update 3 fails, is grab your original D2007/C++Builder 2007 DVDs. That appears to be where a few folks run into trouble. The best thing to do there is download the ZIP file from http://cc.codegear.com/item/25014 into a clean directory, then install via the setup.exe including in the .zip. As soon as you have those files, no longer use the original DVD/version 1 install files. I’ve helped point several folks to this method, and it appears to address the main remaining ‘confusion’. (And hey - we’re taking responsibility for making this confusing and looking to fix it)

A couple other common issues are ones detailed in this blog. Odds are if you are reading this post - you’ve read those. Support is great at dealing with any install problems, and are being great about letting me know of any new issues.

Internal QA and the field tests did not run into the problems of ‘wrong install procedure’ since they were educated to the required process. I consider that a lesson for the future - do an open beta test for future releases and catch usability/first experience issues before going live.

Regardless, I invite anyone who is still running into any install issues to send me an email (cpattinson@codegear.com). I also keep an eye on the public newsgroup borland.public.install.delphi .

Consider this an open thread for quality discussions - we DO take quality extremely seriously. If you have a defect or issue, log it in Quality Central , then feel free to talk about it here. We have great feedback in quality central, and are always reviewing and prioritizing for future work. This includes feature requests and fixes (hey - you noticed the top voted request, Unicode, is on our roadmap, right?)

Share This | Email this page to a friend

In the past few weeks, I’ve had several customers and field testers ask why we don’t talk in detail about any of the updates we are working on as we used. I can understand the confusion, it’s fairly frustrating on the development side too since the more folks we have jump in and join the field test to test any upcoming updates, generally the better the updates are.

There are folks who ‘blame SOX‘. Well, I’ve only read several articles on section 404 compliance so by no means an expert. Is seems at least partially true - the issues centers around revenue recognition and how promising/describing an upcoming update means the product you released was not what customers paid for. It appears one intent is to avoid bait and switch techniques, and customers are supposed to be protected.

Even Apple had some ‘interesting times’  adjusting to the new rules. Another article I ran into described examples and scenarios of Revenue Recognition for Software Products with Multiple Deliverables - however it’s unclear even in that description if patching a failure discovered prior to or after release impacts revenue recognition. I feel for our accountants, I really do.

It gets fairly complex at this point, but our financial boys rightfully are cautious and it can be extremely expensive to recalculate revenue. Plus, you can get in all sorts of trouble with the SEC , fines and costs associated with audits. Ouch!

So I asked around about what we could say about updates. Not much unfortunately. I can acknowledge we’re aware of issues, which we also do via Quality Central , just cannot describe what’s on our update lists, general schedules, and certainly no promises to fix any specific bugs. Kind of hard to instill customer confidence, right? Especially for CodeGear who has a big focus on developers, and showing that we care.

We can demonstrate with results, and thinking about it - we can talk more openly in the private fieldtest where those joining need to sign an NDA. For those who want to understand and see what’s in the works…

Click Here to apply to the RAD Studio fieldtest

Anyways, it’s something for those interested and want to peek in more directly. We do care and we really are on a quality focus. 

Share This | Email this page to a friend

A basic business theme drilled into me about 16 years ago was the customer is always right. Now, there are some exceptions - but that golden rule implies if you produce products or services that your customers need and want, you’ll have a successful business.

This was something on the back of my mind during my four days in Cambridge at the Software Test and Performance conference.  What can CodeGear do better to ensure what is delivered speaks to what customers really want? Many things…

Below is a part of some thoughts that came about from the conference, one of the major benefits of attending was a copy of all the speaker notes, and many great references to processes used by other organizations to solve some of those annoying problems in testing - such as documentation of the test effort, understanding how to really be agile, and how to identify how different your assumed processes are from your real processes. 

Regardless - it all comes down to what is the net result for your users. Are you ensuring the team is productive and focused enough on doing what your customers really want? And how do you know if you are or aren’t on track? Are you getting high return of value on the activities you are assigning your team? Are you spending the right amount of effort on important/high value features and fixes? Are the project metrics you are using the right ones for your needs? (I’ll write more about metrics in the future- metrics are very powerful if used well. And can be a major time sink if not properly thought out)

Regardless, here is the tip of the iceberg and a tease for those interested:

ST and P Conference - Rough Notes

Improvement Themes

  Requirements

• Are they clearly understood by all stakeholders? 
• Have they been reviewed by potential customers, especially major ones?
• Are Business Requirements ‘What’ and Functional Requirements ‘How’ ?
• Do any requirements have implied requirements which may not be obvious? (OS, Platform, Bug levels, user acceptance, documentation, translation, etc…)
• To be truly agile, QA and Doc needs to be involved early as possible in requirements definition. A good requirement should be seen as a public document.

Scrum/Agile

• Requirements in Scrum
  • Business requirements should drive functional requirements
  • Business requirements should link to functional requirements, and vice versa (are we doing work that is valuable?)
  • Product Owner has a lot of work here as they represent the customer

• Agile - what is it really?
  • Scrum is a project management tool, does NOT make a team agile by itself
    • Is the goal of the scrum clearly defined?
    • Organizations that give a theme to a sprint find the teams focused during that sprint
  • To be Agile you need to know where you are, and where you need to be
    • This links closely with well defined business requirements
    • 90% done is still not ‘done’.
    • Truly agile means the entire team takes ownership of testing.
    • Development should be very intimate with QA testing and able to identify high value tests
    • QA should be very intimate with code changes
    • Testers in agile are basically test developers.
    • Test results need to be visible and clearly understood by all members of a scrum team
    • Understanding of test coverage (current and planned) shall be understood by all scrum team members (blending of developers and testers for this purpose)
  • Unit tests exercise API and input/outputs, however will miss the ‘big picture’. Test developers have to test for the ‘big picture’ and how the systems work together. (for example, two systems - one designed for meters, one designed for inches, can pass unit tests, but fail when asked to communicate together)

Customer Involvement

• We should encourage field testers to think as end users
• Provide customers who beta test with the exact shipping product, and see if it’s usable. If you need to write scenarios and how-to’s, it’s likely your product is hard to use or poorly documented
• Open Beta’s are a great way to get feedback on your general customer base. Make sure to include a survey that asks the questions you really need answers to, such as are the new features and quality compelling enough to buy.
• Think of the end experience. The functionality is about 1/3 of what the customer cares about in the product. Does it do what they want, reliably, and without them scratching their heads a lot.
• It’s ok to provide ’scenarios’ document as long as it’s content that will go into documentation. It should be seen as marketing materials more then testing materials, as you want customers to provide feedback on all areas of the product - not focused on specific scenarios.

Bug Fixing/Stabilization

• Learn from past project data - projects often follow similar ‘flows’.
  • If you needed 10 final builds to get your shipping candidate several projects in a row, PLAN for it for the future. Odds are you will need to do so again
• Bugs that were not fixed in the past, is not an excuse to not look at them now. Look at the reasons they were not fixed - a product that keeps deferring previous bugs only gets worse in quality to the customer
• Review find and fix rates, as well as the age of fixed reports. It is common that older defects take significantly longer development and test time to process then newer reports. Take your bug backlog and age into account in planning.

Share This | Email this page to a friend

This week, I’m on the road to sharpen and learn about current QA processes. CodeGear has made some great progress in the past few years in terms of functional, performance, stress and user acceptance testing - it’s a journey I never really expect to finish since technology will keep moving very rapidly!

So I’m in Boston, at the Software Test and Performance Conference. http://www.stpcon.com/

It’s a three day conference from Tuesday to Thursday, yesterday was an all day session on Measuring and Improving the Test Process. Much of what was discussed was very familiar - questions such as ‘How effective is your test system?’ , ‘Are the tests you are doing valuable?’, ‘Where are you weak, and where are you strong, in your testing organization?’.

Still, it was good to take a step back with a bunch of peers and just … discuss.

A few good ideas came out of yesterdays session - we measure a lot of testing data, but spend a lot of time sifting through the data manually. A logical next step is rolling up and summarizing test results, and sending out comparison results of day to day builds in email. Now, we’ve been prototyping an email notification system, providing the comparison of previous test results seems perfect to add into that email.

In addition - requirements testing and management had a lot of focus. And that’s a good idea - if you rush your requirements phase, you are building the wrong features, and testing the wrong features or designs. Even if your testing is highly effective, testing a bad design and verifying it meets the criteria of a bad design wouldn’t be an effective way to spend your engineers time. CodeGear seems to have a good grip on requirements management - however it’s an area that should always be invested in.

Another interesting concept raised was the ‘REAL process’ against the ‘assumed process’. For example, in planning the projects we do not allocate specific time for bug councils (where developers and managers discuss the backlog of defects and prioritize them) or for work in our beta/field tests. This is all real work, where if not planned can put strain on other commitments.

Today the conference opened with a keynote on performance testing by Scott Barber. Yes, even CodeGear needs to consider performance testing - we all want our developer tools to be super snappy, reliable, and handle large and complex projects. Last year we started the basics of performance testing - memory management, startup/shutdown times, and metrics for a large number of common tasks or handling very large projects. The results are actually noticeable in RADStudio 2007, where several areas were optimized and fixed, and we can monitor to see if other changes have side-effects.

So far, so good. I’m sneaking in this blog post between a good lunch - this mornings sessions were on user acceptance testing and test automation projects. This afternoon I’ll be digging into building quality into software earlier in the project cycle and managing software testing metrics. Been worthwhile so far, have to consider going to San Mateo in April for the 2008 conference

Share This | Email this page to a friend

I wrote these instructions in response to a question from SA customers on how to safely install RadStudio 2007.  The instructions are mostly aimed at existing D2007 users.  The main difference between an Update is when you want to install the full studio, run the setup program and select the upgrade option (top option) in the UI dialog that appears.  Then you can enter your new serial number which will provide access to all the new features in RadStudio 2007. 

***Important Note***

Indy and DBExpress have ‘breaking changes’ in Update 3 and RADStudio
2007. I recommend reading up on Nick Hodges blog for a description of
those changes. http://blogs.codegear.com/nickhodges/2007/09/14/38940

For installing RADStudio 2007 or Update 3, I recommend the following
procedure:

1) Create a clean installation directory such as
C:\RADStudio2007_Install

2) Backup your registry.

Specifically :
HKEY_CURRENT_USER\Software\Borland
HKEY_LOCAL_MACHINE\SOFTWARE\Borland

This might be overkill, but with critical development is worth the
caution.

2) Download the setup program to this new directory

3) If this is an update, before running setup - terminate any running
applications or services that may be related to D2007/RADStudio (to
avoid the system needing to reboot, though if that happens just run the
setup program again)

3) Run the setup program from the new directory. If this is an update,
you can run it from the command line with the /upgrade switch to
simplify the UI and process.

Note: Some users encountered a problem with serial number here, which
would not happen with use of the /upgrade switch. It’s an issue related
to expired field test and trial keys that may be on your system. If you
run into this, cancel the install and then delete cglm.ini files
(recommend a system level search including hidden directories to be
thorough as they are in different locations depending on the OS/Locale)

4) Ensure all personalities are checked during install - keep in mind
this install is a re-install of existing, not of modification of
existing.

5) Select your features. You have the option here to install features.
Documentation and ECO are part of the main feature set, though have
their own installers.

6) When doing online updates or installs, you will be prompted with the
following message:

The RADStudio 2007 installer is unable to locate all the required files
to install RADStudio 2007. Please enter the location of these files
either by inserting the original DVD media, or selecting the location
where the files were previously downloaded.

This message was intended for installs where required files were
already present on the system.

For Updates of existing product and new installs - select the newly
created directory as the destination for the files, otherwise you will
run into a problem downloading the files.

7) Some users reported Help not installing correctly. This seems to be
on non-standard directories on drives other then C:\. To reinstall help
after installing the main product, run Help_Setup.exe which will have
been downloaded in the directory indicated in step 6. (Hence why I
recommend to put these files somewhere easy to find )

I’ll be keeping an eye out for any problems folks run into. The above
procedure should help for a smoother install experience. The big one is
to install from a clean directory and to download files to it - that
will help the most.

Share This | Email this page to a friend