Delphi Roadmap 2005 - 2006

Here are a few notes on our current plans for the next few Delphi product releases:

2005 - DeXter

The next release of the Delphi product family, code named “DeXter” will further advance developer productivity through enhancements in Code Insight, Code Completion, Code Templates, and refactoring across all the Delphi IDE’s language personalities.  Significant language and toolchain enhancements have been made to the Win32 Delphi compiler, including support for operator overloading syntax in Win32 and support for inlining compiler magic functions such as Length().  On the .NET side, ECO III has a plethora of new stuff, including property change triggers and state machine definition and management.

DeXter will include support for Win32 C++ programming, ala BCB, alongside Delphi for Win32, Delphi .NET and C#.  C++ coders will benefit from the host of IDE productivity tools developed since the BCB days, including Code Insight, Error Insight, and even refactoring.  And of course, VCL app development in C++!

Performance analysis has been a hot item for this release cycle.  You’ll notice dramatic improvements not only in the DeXter IDE startup times, but also in the runtime performance of your own Win32 Delphi apps!  Highly optimized routines from the FastCode project have helped, as well has an all new high performance non-blocking multithreaded Delphi memory manager. 

DeXter is currently scheduled to be completed and released this year (2005).  It will support development of Win32 VCL applications and .NET 1.x WinForms and VCL.NET applications.  The command line compiler will support building apps for .NET Compact Frameworks, though there will be no IDE designer support for CF in this release.

2006 - Highlander

The next Delphi product cycle after DeXter is code named “Highlander,” and will focus on updating the entire Delphi toolset to support .NET 2.0.  There will be ongoing work for Win32 C++ and Win32 Delphi, but the largest portion of new work in Highlander will be for .NET 2.0.  Delphi.Net language support for generic types, partial classes, and nullable types are pretty much a requirement in .NET 2.0.  We intend to have IDE design surfaces for .NET Compact Frameworks up and running in Highlander (using VCL.NET on CF), as well as support for developing and debugging 64 bit .NET apps written using WinForms and VCL.NET. 

The current plan is to release Highlander in calendar year 2006.  We’re considering releasing tech preview builds for download by registered Delphi customers shortly after .NET 2.0 is released, but that will depend largely upon how quickly we can get the raft of new language syntax nailed down, and upon when Microsoft gets around to actually shipping .NET 2.0.

As always, these plans are subject to change.

Deferals & Distractions

I apologize for the tardiness of this roadmap update.  This post was originally drafted in February but was put on hold pending closure of some delicate contract negotiations.  When those negotiations were successfully concluded (in May!), I was busy with other Borland business and this post drifted further into the depths of my outbox.  Labor Day found me on the on the brink of the DeXter crunch mode maelstrom, thinking that it might be a good idea to push this post on out there.  (gee, ya think?)  By that time, though, it was considered “too close“ to the planned DeXter “rolling thunder“ campaign, so I reluctantly (and cursing a blue streak) put it back on hold, per Borland request.  At last, the holds are released!  Fly, be free!

Share This | Email this page to a friend

Posted by Danny Thorpe archive on September 27th, 2005 under Product | 141 Comments »

Language Integrated Query (LINQ)

Language Integrated Query (LINQ) is clearly the hot topic of PDC this year.  Windows Vista dominated the keynotes and looked very pretty up there on the screen, but nothing gets developers salivating like language enhancements.  What is LINQ, really, and what does it mean for Delphi?

The founding premise behind LINQ’s many facets is that there is a huge gap between program logic written in a high level programming language and the data that it operates upon.  The more data you have to process, the farther away it is from your program. 

Think about it.  Why is it that you program using strong data types, objects, method calls, and optimized compiled code, and yet whenever you want to deal with any data at scale, your data access is reduced to text strings containing SQL query statements?  All your strong data types, objects, and optimized code get thrown out the window just when you need it most.  This model requires the programmer to be fluent in two different language syntaxes and two different type systems with different quirks and representations.  This model requires constant conversion between the data in the SQL domain and your program’s domain.

This gap has long been recognized, and spawned several diverse techniques to narrow the gap between program logic and large scale data storage.  SQL store procedures, for example, were created so that you could bring the code closer to the data.  But oh, what code you had to write to get that to work!

This is not an attack on SQL itself.  SQL is very expressive and powerful for specifying how you want to filter, organize, and view a large set of data.  The problem is that SQL is always ‘far away’ and alien to your programmers.  Programmers are forced to think about data in SQL terms but think about the application logic in terms of their preferred programming language.  That discontinuity is an opportunity for human mistakes and wasted execution cycles.

What if you could bring the expressiveness of SQL closer to your core application logic?  Not just physical distance, but conceptual distance as well?  What if you could perform set logic on large data in your preferred programming language, using the objects, data types, and variables of your normal code, without having to jump through translation hoops? 

That’s what LINQ aims to do.

I’ve been aware of the LINQ project within Microsoft for quite some time, but hadn’t heard any specifics about it in part because “it’s too early” and “it’s not fully baked yet.”  So I was a bit surprised to see it headlining at PDC this year.  I suspect there was some mad scrambling going on behind the scenes to prep the LINQ preview release for PDC.  I’m glad they did!

LINQ is a combination of language syntax and .NET framework infrastructure to support SQL-like query capabilities within high-level programming languages, operating on data in the programming languages’ domain.  Here’s the exciting part:  LINQ is independent of SQL.  LINQ style queries operate on any data construct that implements IEnumerable.  Sure, that includes ADO.NET data sets hooked up to SQL back-ends.  But it also means you can perform sophistocated queries and views of data stored in your program’s local lists, collections, and even arrays.  Want to sort that array of integers?  Slap up a query with an orderby clause.

Enormous Back-end Optimization Opportunities

Far from the acreage of anyone’s derriere, LINQ provides an abstraction that separates the concepts that programmers need to deal with from the back-end implementation details.  The power of Language Integrated Query is that the programmer specifies what he/she wants as output, but does not need to get tangled up in the specifics of exactly how that output is achieved.  That means the how can be be implemented and reimplemented a multitude of ways by very speciallized experts without affecting how everyday coders specify the what.

Different data sources may implement their own query operators that are specifically tuned to the way they represent data internally.  A SQL dataset will take the LINQ query and emit a SQL select statement to throw across the network to the SQL server.  A query against an in-memory array will spin through the array looking for entries that satisfy the where clause conditions.  (ECO object spaces, anyone?)

And what about those where clause conditions?  The where clause has to be evaluated for every element in the data source to decide which elements should appear in the result set.  That could be a lot of execution.  What if the LINQ query back-end were smart enough to recognize when the where clause is a locally isolated expression (no global state or side effects) and in such cases distribute the testing the data across multiple processors in multiple threads?  That would be incredibly powerful.

In this way, LINQ presents an enormous opportunity to put parallel computing in a package that everyday coders can use safely and easily.  Exploit those multicore processors!  Even grid computing is within reason.

And Now the Critics

LINQ does have weak points and criticisms.  When dealing with external SQL data, LINQ’s programming model uses compile-time data binding to associate data in a particular column of a particular table in the SQL database ‘over there’ with a particular data object ‘over here.’  You do this all the time in Delphi when you create field components to correspond to the data fields you want to use in your program.  However, hard binding to database schema that is defined outside your program is a risk. 

What happens when some bozo (even a well-meaning bozo) decides to change a field name or table name on the SQL server?  Your app breaks.  Many times in PDC sessions involving LINQ, people in the audience asked if there would be any way to define mapping tables to go between the compile-time hard bindings and the external database.  The answer was no, for now.

Another criticism of LINQ is that it brings the complexity of query construction right into the middle of your programming logic.  It may be very convenient to assemble a VW in your living room, but is that really where it belongs? 

I agree that there is ample opportunity for abuse here, but the risk of obfuscation is outweighed by the power of expression. 

Consider this against the VW metaphor:  Indoor plumbing vs the well-understood “mature” technology of the outhouse.  You have to leave the warmth and comfort of your house to use the outhouse.  You have different expectations of your outhouse experience (smell, view, decor) than of your ‘inhouse’ experience.  Ignoring the obvious data flow flaw in this metaphor (inflow vs outflow), the traditional external database model (SQL and others) is the outhouse.  LINQ proposes to replace your data outhouses with indoor plumbing. 

What were the objections to indoor plumbing when it was first introduced?  Smell, obviously.  Unreliable, leaky plumbing.  Fear of rats entering the house through the toilet.  (seriously)  And yet indoor plumbing was considered a decadent luxury even hundreds of years before Thomas Crapper popularlized (but did not invent) the P-trap flush toilet to solve the smell issue.

“I don’t want no stinkin’ database crap in my code!” 

If you’re writing programs that don’t deal with data, perhaps that’s a reasonable stance.  For the rest of us, we already have database ‘crap’ running all through our program code.  Perhaps the real frustration behind that stance is that dealing with external data has never been completely smooth or seamless. 

I know of whence I speak.  I made the statement above when I first learned of the decision to make database support the third pillar of the then embrionic Delphi project (visual application design, compiled code, integrated database support) some 17 years ago.  I had plenty of experience with dBase II and dBase III+, and I didn’t want to see Pascal adulterated by database stuff like Browse commands.  Eventually, I realized I was being a Luddite and accepted that database support (intelligent, elegant database support) was critical to Delphi’s success, because when you get right down to it data is what programming is all about. 

LINQ for Delphi?

In a word:  Yes. 

I was very interested to see if the PDC sessions would have any tangible content about where LINQ draws the line between proprietary language magic and system supported infrastructure that all languages can leverage. 

If LINQ were mostly a C# compiler trick, then that would be the end of the story.  No excitement, nothing to write home about.  But Anders Hejlsberg is no dummy.  While he is often labelled the C# language architect, his true task is to advance the .NET platform.  Advancing the platform requires rising above any particular language, including your own.

Anders never fails to deliver on content, and I am pleased to see that LINQ provides significant system infrastructure for languages to leverage in implementing their own support for LINQ style operations.  It’s also a great relief that LINQ’s infrastructure runs on .NET 2.0 - no new CLR to wrestle with.  Yet.

Make no mistake - there is an enormous amount of work that we need to do to the Delphi syntax, compiler, and tool chain just to get to the point where we can start playing with LINQ notions.  LINQ is clearly “next generation,” a child of “what could you do if you had a really good generics implementation under the hood” thinking.  LINQ’s infrastructure is heavily dependent upon generic types and new things similar to or built upon anonymous methods.  So, first things first:  Delphi generics.

What will Delphi’s LINQ syntax look like?  I don’t know yet.  I’d prefer something more elegant than SQL’s Cobol-like froms, wheres, and wherefores (which is the path C# 3.0 is taking), but some verbage is necessary to avoid falling into the Algol #$%!! pit.

It’s interesting to observe that Delphi already has a language construct to perform set logic (intersections, unions, etc) on bags of data:  Delphi set types.  The element type of a set is currently limited to ordinal types with 255 or fewer elements, but what if it weren’t?  What if you could manipulate your terabytes of SQL data using types like “set of (Person, Address, Phone);“?

 

Share This | Email this page to a friend

Posted by Danny Thorpe archive on September 15th, 2005 under Events, Industry, Research | 84 Comments »

Windows Vista: Clear, Confident, Connected

Clear, Confident, Connected.  These are the mantra of Microsoft’s Windows Vista, rolled out in this morning’s keynote addresses at PDC. 

Clear

Microsoft aims to make information clearer through slick UI presentation, data organization, and data slicing.  I’m not sure if having multiple video feeds playing on screen simultaneously necessarily creates greater clarity, but it certainly demos well.  “Virtual folders“ allow you to bring together all files of a particular type on your system into a single virtual folder.  I can appreciate the problem that people lose track of things buried in hierarchical folders, but how is dumping hundreds of files into a pile going to help you find the right slide show or the right photo from last year’s summer trip?  It’s information overload either way.

Confident

On the virus and malware front, Windows Vista is planned to provide more control over what applications are allowed to do to your system, based on the application itself as well as the profile of the currently logged in user.  New ESRB support should allow parents to control what kinds of software their children install and run on the PC.  This will require the cooperation of software vendors to place their ESRB ratings in the setup and software somehow, but it’s an interesting improvement in program control.

New countermeasures are in progress for IE7 (due to ship with Vista) that actively detect and block “phishing” attacks, where email or web sites present links that take you to fake web sites hoping to capture personal identity information, login passwords, or credit card info from unsuspecting visitors.  IE7 implements some rules to examine URLs for suspicious patterns common to phishing attacks - such as the use of a raw numeric IP address rather than a domain name.

The theme here is to bolster OS services to instil greater user confidence in performing everyday tasks safely. 

Connected

“People Near Me” is a new networking metaphor for Vista that attempts to be easier and safer than existing peer to peer networking  protocols.  It’s primarily for wireless networks, and requires opt-in by both parties.  The general idea is: two people sitting down at a table to go over a presentation or whatnot, and they want to transfer files between their machines, or just view a slide deck or application demo hosted by the speaker’s machine and viewed by others at the table.

Why Vista?

Why should developers start thinking about how to support Vista in their legacy and new applications?  Microsoft’s answer:  Vista will drive a lot of churn in the marketplace.  Vendors can ignore it or can ride it to drive application updates.

Share This | Email this page to a friend

Posted by Danny Thorpe archive on September 13th, 2005 under Events, Industry | 7 Comments »

Off to PDC

I’m off to Microsoft’s Professional Development Conference (PDC) this week in Los Angeles.  If you haven’t been to PDC, picture this:  take the US BorCon, multiply by 4 (or is it 10?  hard to count when they won’t sit still), and remove all non-company speakers. 

PDC’s greatest difference from BorCon is that PDC is exclusively a Microsoft show - third party speakers are extremely rare at PDC.  BorCon events typically have far more sessions presented by Borland community experts than by Borlanders.  You should see the shock and disbelief on MS faces when they first hear of this.  “You have non-employee speakers?  Egads!  Who allowed that to happen?“   That would probably be the largely non-employee BorCon Advisory Board.  “ Non-employees participate in the administration of your company event?!?“  Ya, it was sort of borne out of necessity (we don’t have the staff to do everything) but it has the great side benefit of making BorCon an inclusive community event.  People get this warm fuzzy feeling of inclusion when they’re invited to donate weeks of nights and weekends to help run BorCon.

PDC compares well with E3 (the Electronic Entertainment Expo):  similar scale of bodies packed into the hallways, similar geek factor (E3 - geek consumers / PDC - geek programmers), similar lodging shortages throughout Los Angeles, and same venue - the LA Convention Center.  Just take the E3 expo floor and replace all the Nintendo and Sony Playstation booths (booths?  There are cities smaller than these “booths“) with .NET and SQL stuff and you’ve got the PDC expo floor.  Scatter in a few references to TerraServer to add a toupee of respectability to Microsoft’s efforts to claim the server high ground.

In recent years, PDCs have proven to be rather pivotal events in the industry.  It was the 2001 PDC that provided enough ammunition to convince Borland to develop tools for the .NET platform.  Longhorn (real name: Windows Vista) and Avalon (real name: some boring acronym) will undoubtedly take much of the limelight this year.  What nuggets will I find between the highly scripted Powerpoint sessions?

Share This | Email this page to a friend

Posted by Danny Thorpe archive on September 12th, 2005 under Events, Industry | 3 Comments »

Unicode Identifiers Require Unicode Source

I’ve seen several bug reports lately complaining that the compiler barfs on Unicode identifiers in certain contexts.  In all cases, the bug reports were in error because the source files were not Unicode encoded text.

Here’s the complicated rule: 

If you want to represent Unicode characters in a source file, the source file needs to be saved in a format that can represent Unicode. 

Seems obvious, no?

Plain ASCII text won’t cut it.  UTF-8 or UCS-2 is what you need.  ASCII values above 127 in non-Unicode source files are interpreted based on the user’s current locale and charset/code page.  The interpretation of those bytes will be different depending on how your environment is set up at compile time.  We’ve tolerated this kind of ambiguity in string constants for years, but when adding support for unicode identifiers to the Delphi language I chose to disallow that ambiguity.  The Delphi compiler will accept Unicode alphanumeric characters outside the traditional Pascal identifier range of [’a’..’z',’A’..’Z',’0′..’9′,’_'] if and only if the source file is encoded in Unicode.

How can you tell if a source file is Unicode encoded?  Look for the invisible Byte Order Mark at the start of the file.  In UCS-2 big-endian encoding, the BOM is $FEFF, for little endian $FFFE.  In UTF-8 encoding, the BOM is $EB $BB $BF.  (The Unicode char FFFE is designated as a non-printing char)  Or, just open the file in Notepad (on NT systems) and see what text format it displays in the File: Save As dialog.

How can you convert a source file to a Unicode encoding?  Open it in the Delphi 2005 IDE, right click and select File Format, and then select the character encoding format you want.  I recommend UTF-8.  UTF-8 is compact and still (mostly) readable with ASCII tools.  If you choose UTF-8, your source file will be viewable in any old ASCII text viewer if you can ignore the three “garbage” characters at the start of the file.  If you use high-ASCII chars in your source file (such as umlaut chars) those will not be readable by a plain ASCII viewer.  They will of course be readable by a more modern UTF-8 / Unicode aware editor such as Notepad or the Delphi IDE.

 

Share This | Email this page to a friend

Posted by Danny Thorpe archive on September 2nd, 2005 under Product | 7 Comments »

Hell Freezing, Chuck Blogging

After years of protest (”I have nothing interesting to say”), Chuck Jazdzewski has joined the blogsphere.  Hooray! 

Now about those blinking guids…

Share This | Email this page to a friend

Posted by Danny Thorpe archive on August 5th, 2005 under Community | 6 Comments »

Delphi for .NET Compact Frameworks Technology Preview Available for Download

I’m pleased to announce that the Delphi for .NET Compact Framework Technology Preview compiler is now available for download by registered Delphi 2005 customers here:  http://www.borland.com/downloads/registered/download_delphi.html

With this technology preview compiler and the Delphi 2005 toolset, you can explore and experiment with creating Delphi applications for mobile devices such as PDAs and SmartPhones.  There’s no shortage of rough edges, but we feel the compiler modifications are far enough along to be useful to early adopters who welcome a bit of tinkering to break the monotony of a typical workday.  Download it and see for yourself!

Delphi applications compiled for CF can be copied to a CF.NET enabled device using a file browser and ActiveSync, and can be executed on the device by using the device’s file browser or by creating a shortcut that refers to the executable file’s location in the device’s local storage.

If you don’t yet have a CF ready PDA or Smartphone, you can still tinker with CF using the Microsoft device emulator, currently available as a free download from Microsoft.  Instructions are here:  http://msdn.microsoft.com/mobility/windowsmobile/downloads/emulatorpreview/default.aspx

Keep an eye on BDN for a new article on Delphi for CF development from David Clegg!

 

Share This | Email this page to a friend

Posted by Danny Thorpe archive on July 18th, 2005 under Research | 16 Comments »

Limited by Nonsense

Paul Spain writes in the ADUG mailing list:  “As the evolution of Borland’s Object Pascal is now effectively bound to the evolution of .NET’s CLS, Delphi will have to wait for Microsoft before we see these features appearing in our tool of choice…”

Bullshit.

Delphi fans unite:  Whenever someone makes that claim, feel free to grab them by the throat (or other anatomy), slam them into the wall, and challenge them to describe exactly where CLR implements virtual constructors, virtual class methods, sets, global procedures, global procedure pointers, unit initialization semantics, or any of the other dozens of Delphi language things that are implemented by Delphi for .NET above and beyond what CLR provides.

The Delphi language is as constrained by CLR as it is constrained by the Intel processor, or the Windows OS, or the Linux OS. Some things the environment provides for you.  The rest you have to do for yourself. The only difference between the possible and the actual is cost.

For example, we could implement a proprietary form of generic types in .NET 1.1 that could only be used by Delphi customers.  Why should we dump millions into that effort when it will be provided by the .NET 2.0 platform in a non-proprietary cross-language way?  That would be a collosal waste of Borland effort, which would be much better spent working on stuff that CLR is not going to provide. 

It’s not about what can be done, it’s always about what should be done.

 

Share This | Email this page to a friend

Posted by Danny Thorpe archive on June 22nd, 2005 under Community | 43 Comments »

The Intel Mac

Apple’s announcement that it will abandon PowerPC and build future Mac OSX machines on Intel chips has raised a few eyebrows.  How compatible will this Intel Mac hardware be with the well established PC platform?  What about Windows/Mac software portability?  Should Borland create a Delphi for the Intel Mac?

Apple’s G4 and G5 Macs are powerful 64 bit PPC systems.  I seriously doubt that Apple is in any way interested in 32 bit Intel x86.  Apple needs 64 bit to maintain parity.  AMD64 / Intel x64 is the only x86 related chipset that makes sense.

So is this Intel Mac really so similar to existing x86 systems that it is more within striking distance for Delphi than the PPC Mac?  Essentially, no.  The Mac is just as far from the current Delphi target architectures as it was prior to this Intel Mac announcement.  Delphi’s current target architectures are 32 bit x86 and .NET IL.  I expect this Intel Mac will be x64.  Kylix targets Linux, but Linux is very different from the Unix flavor behind OSX. 

Delphi for this Apple / Intel Mac would require as much work as any other new platform.  New codegen, new linker, new language / RTL binding to the OS, etc.  It’s still a possibility, but no closer than it was before.

For all the hoopla in the newsgroups and blogs about how this will affect Apple, Mac customers, and Windows customers, the question that intrigues me most is: why?  Apple speaks of hardware costs and power requirement problems, but that seems a little thin to me as a justification for turning your core business completely upside down.

Others have observed the timing of all this is remarkably close to Microsoft’s Longhorn OS release, but changing your hardware horse is a rather bizarre way to compete against a software release.  It will cost Apple a lot more to manage panic and market attrition in its hardware switch than it will cost Microsoft to upgrade and migrate Windows people from one Windows release to the next.

What’s pawing around in the high grass that would flush these two birds together?  Theories abound.  Whatever it is, it needs to be big enough to scare Apple and Intel into giving up decades of rivalry to work together against a greater common threat.

Share This | Email this page to a friend

Posted by Danny Thorpe archive on June 8th, 2005 under Industry | 45 Comments »

Software Patents Stifle Open Source?

In reading this article at uk.builder, one might get the impression that Borland is pulling a SCO scorched earth manuver by maniacally raping and pillaging Open Source advocates over the use of a structure exception handling (SEH) technique that Borland happens to have a patent on.  I find it particularly significant that neither the uk.builder reporter nor the Open Source advocates felt it necessary to obtain a comment from a Borland representative before airing their gripes.

Over on slashdot, this post posits that the reason GCC doesn’t support SEH is because of the Borland patent.

That’s rich, and typical of Slashdot.  Let’s take a look at the time lines here:  GCC has been around for centuries, without Windows-style SEH support.  In fact, GCC’s implementation of C++ exception handling produces so much code bloat (50 to 150% larger exes) that nobody enables GCC exception handling support unless they’re absolutely desperate or completely clueless.  

Next, Borland enters the Linux scene relatively recently, discovers that GCC can’t cut the mustard for SEH semantics and language interop found on other platforms (Windows) (let’s not harp on the code bloat issue), so Borland implements SEH on its own in its own compilers (with much less code bloat), at considerable expense (and no bloat).  Since this represents a fairly unique nugget of technology, Borland patents its SEH technique and adds it to the Borland Defense Fund.  (Remember that $100 million payment from Microsoft to Borland a few years ago?  Borland didn’t start that fight, but we certainly finished it.)

Whether software patents are the root of all evil or merely a device in our intellectual economy is really beside the point.  Patents suck, sure, but it’s the system that we’re given.  You can choose to abstain on principle, and then be walked all over by those who use the system against you.  Or, you can choose to defend yourself and use the system to make the schoolyard bullies regret bothering you. 

The existance of a patent does not categorically mean all software based on similar ideas open or otherwise is null and void.  Licensing IP to specific open source projects is not unheard of, even with no monetary interest.  I believe IBM has granted IP rights for some of its compression patents to imaging standards bodies.

If you’re seeking reassurance that some artifact won’t be used to sabotage a project, just ring the doorbell and ask.  There are much better ways to open a discussion than throwing a brick through the window.

 

Share This | Email this page to a friend

Posted by Danny Thorpe archive on May 12th, 2005 under Industry | 32 Comments »

Server Response from: dnrh1.codegear.com