Comments on: The Unicode Shift

By: Steven T. Cramer

Steven T. Cramer — Wed, 16 Apr 2008 21:38:01 +0000

Nice! If I could request that while rewriting compiler for 64bit if you could make it a dual pass compiler so we could simplify classes into units of their own vs huge units with tons of .inc files that would be awesome.

Delphi 1 was single pass for speed reasons, and it has just stayed every since. But now is a good time to fix that.

Thanks!!! I look forward to unicode support.

By: Mike P

Mike P — Wed, 09 Apr 2008 15:40:54 +0000

thanks Nick!

By: Nick Hodges

Nick Hodges — Wed, 09 Apr 2008 02:31:03 +0000

Mike –

In that case, you’ll likely want to explicity declare the type you want for passing into the DLL.

However, as shown above, your code will compile. The constant will be fine for use as you are using it.

Nick

By: Mike P

Mike P — Wed, 09 Apr 2008 00:20:31 +0000

thank you, Nick, for your response.

contrary to the way i put my example, it’s someone else’s dll (a VC++ DLL). (i should’ve mentioned that.)

thank you!
mp

By: Nick Hodges

Nick Hodges — Tue, 08 Apr 2008 21:40:26 +0000

Mike P –

That code will work just fine — depending on what your DLL is expecting. In that particular case, you’d likely have to recompile your DLL.

Nick

By: Charles Vinal

Charles Vinal — Tue, 08 Apr 2008 21:05:52 +0000

Nice job Nick - you sure get a tough audience. On a side note, have you seen how REST and Web Oriented Architecture are taking off? Good old web broker is awesome for this stuff and we have created a complete WOA based on it - powering over 100 commercial websites today. Make sure you keep web broker in later versions - heck, you might want to even enhance it a bit.

Keep up the good work.

Best Regards,

Charlie

By: Mike P

Mike P — Tue, 08 Apr 2008 17:50:44 +0000

hopefully this isn’t double-posted (the e-mail address was old).

what happens with string constants used in a construct like this:

const
scDLLCallName_ModuleLED=’ModuleLED’;

function MyModuleLED():integer; stdcall; external ‘MyDll.dll’ name scDLLCallName_ModuleLED;

do i then need to use typed constants and type it as AnsiString? or is this code broken in d2008?

By: Mike P

Mike P — Tue, 08 Apr 2008 17:50:08 +0000

what happens with string constants used in a construct like this:

const
scDLLCallName_ModuleLED=’ModuleLED’;

function MyModuleLED():integer; stdcall; external ‘MyDll.dll’ name scDLLCallName_ModuleLED;

do i then need to use typed constants and type it as AnsiString? or is this code broken in d2008?

By: Nick Hodges

Nick Hodges — Thu, 03 Apr 2008 00:06:49 +0000

TK –

If you are using WideString properties on your components, then you can keep on doing that. WideString isn’t going to change.

But you can explore using UnicodeString as desired/needed.

Nick

By: pcunite

pcunite — Tue, 01 Apr 2008 15:46:34 +0000

I am a user since Delphi 6. I look forward to USC2 (windows Unicode) support. Anyone thinking that they won’t need it is only writing software for backyard garages…

To help some people out with Unicode. An ANSI string is already in UTF8 Unicode format. Windows and now Tiburon will use USC2 (also called UTF-16) encoding. It will be easy for the Delphi compiler team to see your ANSI string and make it work correctly.

By: S@nne

S@nne — Mon, 31 Mar 2008 21:48:36 +0000

Hi Nick, thanks a lot for this great news. Looking very forward for a beta release so we all can start rewriting code. I rather have to rewrite more and have a stable and transparant Delphi release/code than endless backward compability and huge unclear code.

Is there any estimation when a beta relase or final release will be shipped? Or does the roadmap still fits?

Awesome you write a blog and great idea of the component writers to test it.

1. Unicode
2. Ease of use multi # core processing
3. 64 bit support

Regards

By: TK

TK — Mon, 31 Mar 2008 20:20:11 +0000

What to do if I use WideString properties in my controls? How to update for Tiburon? Make a new conditional directive and use the old-new "string" in Tiburon only?

By: Luigi D. Sandon

Luigi D. Sandon — Fri, 28 Mar 2008 13:50:10 +0000

Yes, I wrote "switch" meaning Delphi going Unicode only, not a compiler switch.
I see and understand some of the issues it will bring, especially in some markets, but IMHO Delphi would be doomed if it becomes a rearguard development tool only.
It could be very difficult to attract new developers (and even keeping some of the current ones) saying "we support Windows 95!" but not Unicode fully, or the like.
I mantained some 16bit applications for years (up to the year D5 or D6 was released, IIRC) because of some customers requests, and I did it using D1 - never thought I should have asked Borland that Delphi 5 or 6 should have supported 16bit applications or it would have been the "last Delphi ever".
If RAD 2007 is the last version supporting the older OSes, keep using it if you need to develop for them. Codegear could even ship it with newer releases as they did with Delphi 1.
But if new releases are exactly like the old ones, what future for Codegear??

PS: I read Joel’s article about IE8: IMHO he’s plainly wrong. Other sectors went under huge standardizations efforts (what’s ANSI or ISO for??), it’s time IT follows the same path. Why should we be different?
Would people like appliances using each a different voltage and plug to work? Or cars needing custom gasoline and tyres to run? Or each country using a different metric system? EU changed its currency in 2002, can’t people update their web pages?
Sometimes standards needs to be enforced, and kudos to MS if they are going to do it the "right way" even if it would break pages working in previous IE releases.
And I won’t complain if Delphi will break bad written applications.

By: DelphiUser

DelphiUser — Fri, 28 Mar 2008 09:17:26 +0000

Talking of two different things?

I understood Luigi to mean "Unicode switch" as in "changing to the Unicode world". Not "Switch" as in a compiler unicode on/off switch.

From Bauer’s articles it’s clearly an either/or thing, not something you add, like wide strings.

I also read Microsoft’s UnicoWS.dll/MSLU description (see link in Giel’s post.) Microsoft evidently faced the same issue.

I am hopeful that someone smarter than me looks into the UnicoWS dll and suggests some way of integrating it/wrapping it around Delphi Tiburon.

By: Vitaliy

Vitaliy — Thu, 27 Mar 2008 20:33:25 +0000

@Luigi
"I wonder if you read Bauer’s articles about the Unicode switch"

I clearly read all about this.
And as I see, most commenters agree about importance of such switch. According to other Codegear blog posts we won’t see any such switch because of some difficulties they try to explain to us (this is bad side of blogs, instead it’ll be better to try to solve this difficult problems rather then write here poems how you can’t solve it).
I’ll be Cassandra here, but if won’t see this switch in next Delphi version it’ll be last Delphi version ever.
Another issue is immediate revival of Turbo line as key products to gain new users (as I understand, Nick still strongly believes in students buying Delphi 2007 )

By: DelphiUser

DelphiUser — Thu, 27 Mar 2008 09:29:03 +0000

@C Johnson: "The problem as I see it in supporting people who use OSs that are now coming up on 10 years old is that they probably aren’t spending much money on software or software development."

Not all software is sold as stand-alone solutions. Much of what I do supports other hardware. Supporting older OSs will sell more hardware boxes.

I also provide in-house solutions, small single-purpose utilities. (I venture to say that Delphi’s main market IS small-to-mid applications). I bill hours of work. Supporting two code bases is going to be hard to explain to the client, esp. since it wasn’t necessary so far.

I was looking forward to the time-saving aspects of generics (C# does them well), but not at this cost.

Giel’s tip on UnicoWS.dll is a good one, but at first glance it looks like a Delphi interface will be needed first.

Isn’t there an open project that covers this?

By: Giel

Giel — Thu, 27 Mar 2008 09:28:25 +0000

@Ron:

By: Luigi D. Sandon

Luigi D. Sandon — Thu, 27 Mar 2008 08:56:59 +0000

@Vitaliy: I wonder if you read Bauer’s articles about the Unicode switch, especially this: http://blogs.codegear.com/abauer/2008/01/09/38845, read the paragraph "OMG!! All my code is going to break! I can’t handle this!!", please.

By: Ron Grove

Ron Grove — Thu, 27 Mar 2008 05:38:51 +0000

Glad I don’t sell anything to programmers. Good grief.

By: Vitaliy

Vitaliy — Thu, 27 Mar 2008 05:14:01 +0000

@Fred - main problem is that you are forced to make them Unicode.
This’ll kill many legacy apps written in Delphi.
And all this to "do it right". Joel is fully right here.
MS and Codegear guys went out of their minds.
reality will stop them, of course.
But for CG it’ll be too late and too hard.

By: Fred

Fred — Thu, 27 Mar 2008 04:12:04 +0000

you guys have it all wrong…. C++ ??? what’s the market here?? people who can understand cryptic programming languages….OR….people who can understand READABLE programming languages?? with all due respect to C++ and the curly braces languages - to get the "masses" programming (mind you…it’s STILL after all = A Business)….is something most people will be able to GRASP in alot less time. good luck learning C++ or the like….or and "easy to read" language. yes…"if then else" is ALOT easier to understand than { } { }….after all - most of us DO speak English! Pardon moi…I have yet to see a language written in spanish…french…or german….or any foreign language other than assembly. LOL!

anyway…back to topic - nobody’s answered MY question!:

So here’s a big question, then: Will D2007 compatible components run fine on D2008 if we do NOT need "unicode enabled components"?? Of if I want to run my apps "as is" without worrying about Unicode or am I forced to make it Unicode??

By: Luigi Sandon

Luigi Sandon — Wed, 26 Mar 2008 22:32:27 +0000

@Sean: define a Java/C# type language, please. Garbage collected? No headers? Single inheritance? What’s actually the difference between templates and generics? IMHO a Java/C# type language is one designed around its VM - while C++ is designed to run on several different architectures. Probably what’s really missing in C++ are properties.
What market could have a C++ -like language for Win32/64 that is not C++ (and can’t use the tons of code available for C++)?
People are learning and using language like Python or Ruby that have very different syntaxes from Java/C#. What’t the problem in learning ObjectPascal? Just because it’s not fashionable?

By: Sean

Sean — Wed, 26 Mar 2008 19:55:31 +0000

@Luigi
c++ is not a C#/Java type language. While it has some broad similarities in syntax it is fundamentally different in several areas (header files, memory management, generics, namespaces…).

By: David Howes

David Howes — Wed, 26 Mar 2008 18:47:33 +0000

Thanks for the info Nick. Couple of questions? You say that the string type is UTF16 by default, does that mean it can be set to UTF32 to avoid needing to use surrogate pairs? What else is supported, presumably utf8 is for compatibility reasons? Is the new string type reference counted? Is it always multi-byte? for example if a string of 8 bit Ascii chars is assigned to it, will it always still use a 16bit widechar for each char (assuming utf16)?

By: C Johnson

C Johnson — Wed, 26 Mar 2008 15:47:15 +0000

@Vitality -> Just because there are non unicode APIs doesn’t mean they aren’t just translating stubs for the unicode API leading to an inherent slowdown. In fact, NT OSes are layers of translating layers all over the code native API most of us never see or touch (which is why there are win32 & posix APIs for NT) - less of those layers has to be for the better.

Besides, chars on 32 bit processors are inheriently bad. Anything that gets it closer to 4 byte entities would reduce the hardware latencies involved with data not quad byte aligned. Strange but true.

@Giel -> The problem as I see it in supporting people who use OSs that are now coming up on 10 years old is that they probably aren’t spending much money on software or software development.

You could play the "stability" card, except that W2K and XP are both significantly more stable (even Vista), so I can only see someone staying with Win98 either to support a flawed legacy app they don’t want to pay to bring forward, or is too cheap to pay for a few OS licenses.

(ok, maybe they have a machine with legacy hardware they can’t or won’t update either, but to hold back all the machines in the environment for that sake… again, back to wondering how much money they are actually going to spend on new software)

Either way, I’m not sure there is a ROI case to make it worth wasting much effort chasing that crowd.

Besides, if you REALLY need to target old platforms, you can always use the tools for those platforms (thinking D3, D5)…

By: DelphiUser

DelphiUser — Wed, 26 Mar 2008 15:10:44 +0000

That looks interesting. Will look into it.

By: Giel

Giel — Wed, 26 Mar 2008 11:31:52 +0000

I don’t see major problems with moving code to Unicode. The only drawback for me is that the code would no longer run on win95/98/me. In theory this could be solved using MSLU (http://www.microsoft.com.nsatc.net/globaldev/handson/dev/mslu_announce.mspx).

MSLU consists of the Unicows.dll which contains Unicode wrappers around many Ansi windows routines. With VC++ you have to compile your app with Unicows.lib included. This takes care of linking unicode routines to the Unicows.dll instead of user32.dll etc. Haven;t looked at this file, but I think this would require some work by CodeGear to get this to work with Delphi (but maybe anyone in the know can do it).

By: Luigi D. Sandon

Luigi D. Sandon — Wed, 26 Mar 2008 09:42:23 +0000

@Vitaly: I disagree with you too About the Windows API (from MSDN): "New Windows applications should use Unicode to avoid the inconsistencies of varied code pages and for ease of localization." "Some newer functions support only Unicode versions."
Some unicode controls are available, but you are forced to use only them, and can’t take advantage of other, more advanced ones (DevExpress, for example). And without a full Unicode RTL, you cannot be sure your string isn’t passed to a function that will trash it. Translating the interface is not enough, you may need an interface in one language being able to display data in another outside its ANSI character set - easily.
Compatibility is important, but can’t hinder any improvement. Unicode *is* the future (and most of the present, already). Or improvements should be just new UI controls? .
I am awaiting 64bit eagerly too, but do you think "compatibility" won’t be an issue too?

@Brian A: there is already a C#/Java type language for Win32. It’s called C++ . Both Java and its clone C# are based upon it, and use the same syntax, more or less. Any reason to create another one?

By: Brian A

Brian A — Wed, 26 Mar 2008 05:26:25 +0000

Thanks Nick for the update - that is great news! Unlike others here, Unicode support is by far the #1 priority feature request for Delphi (after all, 64 bit adoption rates are pretty meager, at least until Microsoft releases a 64-bit only OS like they should have done with Vista).

BTW, priority #2 for us would be reliability and stability in the IDE (CodeInsight, etc)!

Last point - anyone at Codebase ever kick around a C#/Java type language Win32 product? (modern syntax, best of breed Win32 support). The Pascal language is by far the #1 thing that prevents widespread Delphi adoption amongst _new_ users. New, young developers cringe at writing Pascal versus C# or Java code. But those developers also cringe every time they compare their memory/cpu hogging .NET/JRE application to a native Win32 competitor.

By: DanB

DanB — Wed, 26 Mar 2008 03:27:05 +0000

Nick, I’m glad to hear the 3rd party vendors are being involved. I’m really keeping my fingers crossed that this transition goes as smoothly as you anticipate (though I still think it will cause great pain for many developers out there).

For those worried about paying again for 3rd party components: I’ll happily pay for unicode component upgrades. It’s the components that are no longer supported that I worry about. So open those wallets; it’s good to support the delphi ecosystem and besides, software really is just about the cheapest industry you could be in. For the cost of a computer and all the software you need to be a profitable company, you couldn’t even get started in most other industrys or even open a coffee shop for that matter.