Tiburon DataSnap enhancements
The new DataSnap 2009 Overview provides a more up to date overview of this technology.
Share This | Email this page to a friend
Posted by Steve Shaughnessy on July 20th, 2008 under Uncategorized |42 Responses to “Tiburon DataSnap enhancements”
Leave a Comment
You must be logged in to post a comment.
Server Response from: dnrh2.codegear.com
RSS Feed
July 20th, 2008 at 11:16 pm
Hi Steve,
Very very very nice :-). Some questions:
1. Be sure to bundle with Tiburon some examples.
2. TSQLServerMethod descends from TDataSet? (You have TDataGenerator.DataSetEqual(SQLServerMethod, DataSet); )
3. Can we have TDataGenerator (Or something similar…) bundled also in Tiburon? Imho, it will be used heavily in certain circumstances…
4. We’ll have the DataSnap’s source?
5. It works natively…, -and/or- do you have a solution now for… -and/or- do you consider (for the future) the multithreaded/async communication between peers? (I’m thinking especially, of course, at large chunks of data over slow connections).
6. A little bit OT: TClientDataSet was improved? (I have in mind the lack of speed and the dependence of Midas.Dll)
…and a 128bit question:
7. Tiburon is Unicode. Allen stated that there are transparent conversions between strings. IOW, having s1: AnsiString; s2: string; s1:=’foo’; s2:=s1; S2 will be ‘foo’ in Unicode. So far so good, and imho the (obvious) expected behavior. But this does mean that any TStringField (or descendants), TMemoField (or descendants & friends) which use .AsString (or .Value) will pump to any SQL DB backend Unicode data and _only_ Unicode data (regardless of db comm layer, being it DBX, IBX, BDE, 3rd party lib etc.)? IOW, having
fldName.Edit;
fldName.AsString:=’Στέφανος’;
fldName.DataSet.Post;
This will put the Greek ‘Steve’ (Stephanos) as Unicode in my WIN1253 db?
Or perhaps did you changed the above classes to handle specific encodings?
FTR, I’d rather expect to have TDataSet.Encoding: TEncoding which in case of ‘nil’ will point to app. codepage and TField.Encoding: TEncoding which in case of ‘nil’ will point to TDataSet.Encoding. This of course for string/memo/blob fields… Also an action in TDataSet’s popup menu which will ‘Set the [x] encoding for the fields…’ will be welcomed.
Thanks once again for your blog post and looking forward for your responses,
m. Th.
July 21st, 2008 at 12:17 am
Hi Steve,
very grat article!! Only one question is possible to use TClientDataSet inside an ASP.Net application write with Tiburon? Because in the past with D8 i tried it and i didn’t able to do that , there was an error like "error in threadin model".
bst rgds
ivan revelli
July 21st, 2008 at 9:13 am
Hi Steve,
I have a question: you wrote "We currently provide win32 server side applications". Why .NET servers aren’t supported? Win32 DataSnap servers and .NET clients does not make sense to me. In the other hand, .NET DataSnap servers + any kind of clients including Win32, .NET, whatever, makes perfect sense to me in a multi-tier application.
Can this new DataSnap framework be extended in such a way?
Best regards
July 21st, 2008 at 9:56 am
Alexandre,
Connect .NET client in DataSnap server complete make sense, with the new DataSnap you will be able to connect other client techonologies in DataSnap Server.
Best,
Andreano
July 21st, 2008 at 10:26 am
[...] is one of the big features in Tiburon. I’m pretty sure you will be surprised. Steve Shaughnessy started to talk about the new DataSnap. The new DataSnap is completely independent from COM/DCOM [...]
July 21st, 2008 at 11:34 am
m. Th.:
1 We have some really basic samples in place, but could use some more. Dan Miser and Andreano are working on some..
2 Yes. TSQLServerMthod decends for TDataSet.
3 TGenerator is specific to this sample. Intentionally made simple for this example. However we do have a more featured data generator in the product that is uses in out own automated tests. There is a sample called QCReport that uses it.
4) All source for Tiburon DataSnap enhancements is included in the product. All of the new work is implemented in Object Pascal. Some of it has been translated from Java to Object Pascal. This approach will make it easier for us to create .net and Java servers in the future.
5) We support native servers. We support native and managed clients. On the server, we are currently use Indy 10 for the server transport. We will probably only support blocking thread per connection in Tiburon. Going forward we are looking at non blocking async models.
6) We did not make speed improvements or remove the requirement for midas.dll for TClientDataSet. This is something I’d like to see addressed too. However as mentioned in this blog, DataSnap server methods allow for more than just TClientDataSet to be used to send/receive table shaped data.
7) I don’t think the behavior will be what you describe, but I’d have to do a some research to properly answer this question. My initial reaction to this though is to steer applications towards the use of Unicode data types in their apps and databases. We have Unicode enabled several of the dbExpress drivers already.
July 21st, 2008 at 11:39 am
Ivan Revelli:
This is another thing about TClientDataSet that I’d like to see fixed. However note that for .net clients, DataSnap will allow .net DataTable, DataRow, DataView objects to be passed as parameters. When we support .net servers, we will do the same on the server side.
July 21st, 2008 at 11:44 am
Alexandre Machado:
I agree that .net servers are very important. The only reason they are not included in this release was time constraints. Its a top priority feature going forward though. Native servers are important for us too. We have a large native customer base. The native server can host existing RemoteDataModules so this allows developers to leverage existing code while taking advantage of newer DataSnap features.
July 21st, 2008 at 11:49 am
Good article. Thanks Steve. For future post, can you actually provide the source as html instead of pictures. There are quite a few code highlighters out there that produce inline css and you just need to copy and paste it in. Possibly your blog engine has a code filter installed, if not I think this would be a good time to talk to the responsible people
July 21st, 2008 at 1:43 pm
Hannes Danzl:
Thanks I agree. I’ll try to update the article to use html.
July 21st, 2008 at 2:07 pm
[...] é uma das grandes novidades no Tiburon. Tenho certeza que vocês ficaram surpresos. Steve Shaughnessy começou a escrever sobre o novo [...]
July 21st, 2008 at 11:24 pm
Great and cool! two questions:
1. You wrote "DataSnap components are implemented in Object Pascal". Does this mean that single exe deployment for clients is possible?"
2. Is it planed to have .NET Server side support (talking to a Win32 cliiet) in the next Delphi .NET iteration?
Thanks for listening
July 22nd, 2008 at 2:06 am
Thanks, Steve, for being so kind to answer. I will comment only some points:
1.a.) Dan Miser?? He’s back? Wheeee!!….
(ok, not very DataSnap related, but…)
1.b.) Yes, more examples will be very good. Mind you, this is very different from TDataSet realm with which we’re used. I remember now when I discussed this feature with John Kaster (and others) in ng, I received enough messages showing a relative lack of understanding. Well, to be sincere, my paradigm was based on the system which we implemented which is like a ‘chat’ program but not between humans with words but between programs with data. (See #56795 for more details, perhaps you’ll find something useful there).
3. DUnit extensions specific to database work or any testing/stressing tools in this area will be _very_ welcomed, imho. Also, it would make a selling point for you and something appealing and en vogue with ‘CodeGear+DatabaseGear’ marriage
6. I (fully) understand you about TClientDataSet. Now you are like a giant with a dwarf on top. Also be sure to see in QC:
Report No: 56796 Status: Reported
Multiple cursors per dataset
http://qc.codegear.com/wc/qcmain.aspx?d=56796
7. The sensible point here is that this must work _regardless_ of the communication layer. If you’ll fix it in DBX (eg. with a proper driver, as you said) will NOT solve CodeGear’s problem. Ok, it will solve /your/ problem (as DBX main architect) but I remember you that outside are a HUGE installed base of non-Unicode databases (on many SQL and non-SQL backends) with a plethora of communication layers (IBX, BDE, IBO, FIBPlus, CRLab, AnyDac etc. - some of them in maintenance mode, as you know - BDE anyone?). Having now a breaking change, and forcing all to migrate to DBX (which has so different architecture, not mentioning the TClientDataset with its problems as we discussed) and to convert their DBs to Unicode, this is completely out of question (many many reasons - more upon request). Also, please consider that since D6-D7 (IIRC) upgrading for CodeGear was always a problem because D7 and now RAD Studio 2007 fulfills in a more than satisfactory manner our needs. Ok, Tiburon has neat features but if the price to upgrade means to ‘unicodize’ our databases (which means a complete data pump with all its quirks, /if/ this is possible - more on request) and/or to wait for a complete rework of our CodeGear and/or 3rd party library which we use to access the DB back-end, the vast majority of us will /not/ upgrade. That’s why I told you to ‘fix’ it at TField/TDataSet level. To see if it works, it’s very simple (can I humbly ask you to do it and to tell us the results? :-)):
Use Interbase/Firebird for the test:
a.) Create a Test database with charset WIN1253
b.) Create a table like this:
CREATE TABLE T1 (F1 Varchar(100));
c.) Do a Tiburon application using BDE & IBX (or if you want do two apps) to connect to the above database. Put the corresponding components to display a ‘SELECT * FROM T1 ORDER BY F1′ (sorry I don’t remember now which are [sheepish grin]) and also add the corresponding INSERT & UPDATE statements. Connect a TDBGrid to each one of them. Let’s say that you call the components BDETable1 and IBXTable1. In an OnClick handler of a button, do this (you can copy & paste if you want):
with IBXTable1 do
begin
Append;
FindField(’F1′).AsString:=’ύστερα’;
Append;
FindField(’F1′).AsString:=’ώρα’;
Append;
FindField(’F1′).Value:=’Τιμπουρόν’;
Append;
FindField(’F1′).AsString:=’άλφα’;
Append;
FindField(’F1′).Value:=’γάμμα’;
Append;
FindField(’F1′).AsString:=’βήτα’;
Append;
FindField(’F1′).AsString:=’δέλτα’;
Append;
FindField(’F1′).AsString:=’CodeGear’;
Post;
Close;
Open; //to have the ‘ORDER BY’ triggered
end;
Same for BDETable1.(I used a ‘with’ in order to be easy for you to do it).
You should see in both cases the following order:
CodeGear
Τιμπουρόν
άλφα
βήτα
γάμμα
δέλτα
ύστερα
ώρα
Also, from the same program change the select statement to (you can copy & paste them):
SELECT * FROM T1 where f1 containing ‘ε’ ORDER BY F1;
It should show you only a record: ύστερα
again change the select to:
SELECT * FROM T1 where f1 containing ‘α’ ORDER BY F1;
It should show you all the Greek records (6 recs) in the above order.
next change the select to:
SELECT * FROM T1 where f1 containing ‘ά’ ORDER BY F1;
It should show you άλφα and γάμμα (2 records) in this order.
and, finally, change the select to:
SELECT * FROM T1 where f1 containing ‘xyzzy’ ORDER BY F1;
It should show you no records.
FTR, we did the above tests and everything is as above if the correct encoding is used. We tried to ‘unicodize’ our dbs and we encountered problems in some of the above tests. (But we don’t use the above comm layers).
Thanks a lot once again for your clarifications and waiting for your response,
m. Th.
July 22nd, 2008 at 3:20 am
You’ve got the wrong road again. Once DCOM, now dbExpress. Dreadful. Remote methods *are not* stored procedures. RMI via "SQL" components? Many lines of code just to pass parameters, without type checking? Methods called via string literals? No compile time checking? No code completion? Oh my god!
MS introduces LINQ to make database programming like application programming and you go the other way round, making calling a damned, simple, function like invoking a database sp?
"There is no need for a proprietary interface definition language or complicated tools to generate stubs and proxies."
Why the server can’t publish something alike a WSDL that the client can interrogate and use at "design" and compile time? Cantù showed dynamic method invocation on his blog - and it has no access to IDE and compiler code - better something alike that then string literals here and there.
Another InternetExpress/WebSnap/ CodeGear technology?
Also if it is so dbExpress-bound we are not going to use it - many adcanced database features are not available in dbExpress drivers, and we stopped using Delphi data access components since D7, when the BDE was deprecated and dbEpress was the most buggy library ever included.
Really, when Swindell talked first about it I hoped in something much better than this.
July 22nd, 2008 at 5:28 am
What if the server returns simples object lists from ORM/webservices and I need to present them to the user to change and callback the server?
The server returns a TPerson with code and name, I need to build a dataset from this and present to the user. Once save call some methon on the server to post it.
July 22nd, 2008 at 5:36 am
This all sounds good to me. Anything that gets out of the COM mire has to be excellent. Let others flounder around in it if they need to. Taut, apposite, sound.
July 22nd, 2008 at 6:06 am
12 Ralf Grenzing:
1: Yes, single exe deployment is possible for the new DataSnap components. If you are using TClientDataSet you will still need the midas dll. Some of our built-in server methods make use of TClientDataSet on win32. When we do a .net implementation these same methods will use DataTables instead.
2: .net servers are a top priority. We only had so much time. Native servers were deemed higher priority. What is mostly needed is a separate implementation for reflection and sever transport.
July 22nd, 2008 at 6:23 am
13 m. Th.
Are you part of the Tiburon field test? Have you posted your unicode concerns on newsgroup? I think it would be good to move this discussion there. To start off, I’ll ask QA to build test case on this so we can take a closer look.
July 22nd, 2008 at 6:36 am
14 Luigi D. Sandon Says:
I don’t disagree on some of the issues with a dynamic approach that you highlight. There are ways to address some of them and we may do that in the future. However this is a light weight solution that is easy to employ. It is also a solution that is relatively easy for us to get working across many client and server side platforms providing a significant interop story for Delphi.
July 22nd, 2008 at 6:53 am
15. Solerman Kaplon Says:
You could populate a TDataSet on the server with the contents of the TPerson and return this from a server method. You could have another server method for saving that would take a TDataSet as an input parameter. Note that the messaging layer actually uses TDBXReader to send/receive table shaped data. So if you don’t need a capability of a TDataSet, you might find it slightly more efficient to use a TDBXReader. We wrap TDataSet, TParams, .net DataTable objects, etc with TDBXReader implementations when sending/receiving them as parameters and return values. In fact it should be possible to create a TDBXReader implementation that can stream an arbitrary collection of objects using a reflection implementation. Something we want to look into going forward.
July 22nd, 2008 at 7:24 am
[...] Shaughnessy has some insight into how the new DataSnap framework [...]
July 22nd, 2008 at 7:34 am
Frankly, to allow interop I’d use a standard technology (SOAP, XML RPC, REST, whatever you like) and wouldn’t develop a new proprietary one.
What I was looking for was a sound, performant RMI framework for Delphi applications that didn’t rely on COM/DCOM - but this one makes DCOM an appealing choice.
And if "ease of use" is typing ten lines to call a method, without any check it could work at runtime, we have very different views on that.
July 22nd, 2008 at 9:50 am
22. Luigi D. Sandon Says
We provide a JSON/RPC based messaging layer. This will make it easier to directly support JavaScript clients in the future. Other messaging layers are possible. There are other aspects of interoperability that should be considered as well. Server methods can be called from dbExpress and ADO.NET. Microsoft has been pushing stored procedures in ASP.NET apps for years. If you like that approach, now you can now build business logic in Delphi and choose between 9 or so different database vendors instead of being locked into SQL Server. Its also possible to support other connectivity frameworks. John Kaster’s team has worked on supporting PHP adodb based solution with the new DataSnap server for example. This approach also provides interoperability to an existing dbExpress client development model.
As for development complexity, calling a server method has the same development overhead of calling a sql query or stored procedure. This is more steps than calling a regular method, but remote methods tend to be significant operations. On the server you don’t need to deal with an interface designer or idl and the tools that typically surround such solutions. If you know Delphi, you know how to create a server method.
July 22nd, 2008 at 11:56 am
As I wrote in response to Swindell:
SERVERS WEB SERVERS
I never wrote a single line of Javascript, nor of PHP or ASP.NET, or any other "scripting" language. We do not code websites, thanks to heaven.
And never used SQL Server or ADO or any other MS tool in the past five years. But I got rid of dbExpress totally when it couldn’t handle Oracle TIMESTAMP fields, object types and much more.
I am working on cluster applications where nodes need to talk to each other and the whole clusters to their clients, clients that are far too complex to be coded in any damned web technology - and that’s why we are still using Delphi, otherwise we would have migrated years ago.
But I see I have to get rid of Datasnap too, and use another RMI framework - the better, no more Enterprise licenses, the Pro version will be enough, given that beyond the compiler, the editor and TForm we are almost no longer using anything "made in Codegear" - build a monument to your 3rd party developers - they kept you afloat in the past years.
"On the server you don’t need to deal with an interface designer or idl and the tools that typically surround such solutions. If you know Delphi"
I like early binding because it makes writing code faster, and code more robust.
I can’t understand why a server using the same RTTI it uses to call the method at run time can’t use it to "publish" automatically a methods/parameters descriptor when compiled to be used by calling clients at "design time" too, without requiring any tool to create it - maybe just for methods with a new "export" directive added to avoid to export any public method, which in turn could be a security risk if that method should be invoked within the server only and not outside.
That would be a great RAD solution, not yours. Forget VS and C# for a while, get back to the value Delphi delivered to developers years ago with powerful design tools and language - you need that mindset again. You lost it.
Once again you are making the mistake to chase the wrong target, targeting Delphi to markets where it is weaker instead of where it is stronger.
I see that companies and names changes, bad decisions are always the same. If it wasn’t for Unicode support that we badly need because our apps are used worldwide, I guess we could ignore Tiburon totally.
July 22nd, 2008 at 12:27 pm
My previous post stated
SERVER != WEB SERVERS
But this blog engine looks to disallow Delphi "not equal" syntax…
July 22nd, 2008 at 1:50 pm
24. Luigi D. Sandon Says:
As I mentioned earlier, I don’t think we can improve on some of the areas you have concerns. In particular, if early binding is necessary I don’t see why this cannot be introduced in the future. I would like to have some support for attributes in the Object Pascal language first. I think there is signifcant value though in the solution we are starting with, but realize there is more that can be done. I prefer that you stay engaged and continue providing feedback in field tests etc.
July 23rd, 2008 at 1:08 am
"I would like to have some support for attributes in the Object Pascal language first"
You are still thinking in terms of ".Netifying" and "C-sharping" Delphi. As long as you try to sail in the wake of the MS transatlantic liner, you will be just tossed here and there and go nowhere.
Delphi already has a form of "attributes", or "directives" are not that? They can’t be user defined, ok, but you can talk to the compiler team, can’t you? Why Object Pascal or whatever you call it now can’t evolve on its own as it always did, instead of apeing C#? Why do you have to wait for MS to implement something to copy it years later and lag behind as much as you can?
"I prefer that you stay engaged and continue providing feedback in field tests etc."
Thank you, but it’s difficult to provide feedback about something you don’t use daily, and the new Datasnap it’s really ueless for our applications.
July 23rd, 2008 at 8:29 am
Totally disagree w Luigi. I’ve used MIDAS since Delphi 3 for middle tier connections and some logic. This looks like a more flexible extension of MIDAS that we already use. I’d like to see more about using other clients.
July 23rd, 2008 at 9:43 am
Luigi –
The cool thing about this is that you don’t have to use it if you don’t want to.
I’m not sure it’s worth getting all worked up about things, especially since the only thing you’ve really seen of it is a blog post. Perhaps waiting and seeing for yourself would be a better thing?
Nick
July 23rd, 2008 at 9:56 am
Hi Steve,
First a lot thank you very much (again) for taking your time with our problems. I have some news for you WRT Unicode which (imho) would help you, Chris (Pattinson) & guys there. Also I need some clarifications from you (or from another one in the team) regarding this. Because I’m not in the field test (even if, perhaps, I would be interested to enter) what do you want: - contact me privately to send you by eMail the ‘findings’ and/or post them here? (Sorry for my somewhat convoluted message but I wrote it in a hurry in order to not let the QA guys to spend their time ‘blindly’. Also, if you want to give you some test cases and/or test something, I’m at your disposition.
HTH,
m. Th.
PS: Attributes in Delphi? - sure (there is also a QC entry on it). (Just) copying C#/Java. No. Study their approach, see what’s good & bad, put it in a blog or (better) in a ng for community shaping & acceptance and after this, implement it.
July 23rd, 2008 at 1:43 pm
30. m. Th.
I’ll send you a mail in regards to this inquiry.
July 24th, 2008 at 4:37 am
"The cool thing about this is that you don’t have to use it if you don’t want to."
The problem is we are using less and less what comes in the Delphi box and we pay for - and have to rely heavily in third party components. While in the D1-D5 times we relied heavily on Delphi itself, but some advanced GUI control (grids…) starting with D6-D7 we:
1) Moved most of the GUI to third party controls because issues with theming, modern interface needs, and long standing bugs (ActionBands anyone?). I know VCL got some attentions at last, but it’s not an edit with a button that will change my life…
2) Moved the database access to third party components because the BDE was deprecated but dbExpress took years to become working - and anyway I needed a deeper Oracle database support.
3) Moved localization to third party components because it got broken after D7 and it wasn’t fixed yet. Maybe in Tiburon will work… or all the Unicode will be pretty useless…
4) Moved the memory manager to a 3rd party one that was eventually somewhat integrated - but the separate one still offers more than what is available from the IDE.
5) Needed to use libraries outside Delphi for containers (beyond lists…), regular expression, encryption, zlib support, web browser integration, Win32 API support, exception logging and much more that almost anybody else has now in his standard framework - and not all the good ones are freeware.
6) Needed to use Indy snapshots because Delphi came with a buggy one - and didn’t use another library only because I didn’t find a suitable one yet.
Now it looks I have to move our servers to a third-party RMI framework too - or develop our one -, if this is all the "new" DataSnap has to offer. Hope I will see something more interesting.
Delphi is already expensive, compared to other tools - and I am asking myself how much we are paying for features we have no use for, and why. The price/features used ratio is getting fairly high.
Once we bought the Enterprise SKU because we needed both the Oracle SQL Link and Datasnap, then because we needed Datasnap - and now? Remote debugging only, because you leave it out the Pro? I’d really wish we could buy a "componentized" Delphi, leaving out all we don’t need.
July 24th, 2008 at 1:04 pm
We have developed SMART Clients (3-Tier) using Delphi since 2004 using TSOAPConnection on Client side. ISAPI soap servers on IIS 6 are very fast and reliable attending a lot of simultaneous connections. How can I do to use DataSnap using SOAP and MS IIS or APACHE on delphi 2009 (Tiburon)?
July 24th, 2008 at 5:28 pm
The existing SOAP implementation is still included in the product. However, the new DataSnap server currently only provides a JSON/RPC based messaging layer. I think the new DataSnap approach could work with a SOAP based messaging layer as well. On the server side, its possible that some existing soap DataModules could already be hosted by the new DataSnap server. Both systems rely on RTTI for dynamic method invocation, support IAppServer and have a lot of overlap in the type system for parameters and return values. I have been talking with Bruneau Babet who works on our SOAP based solution quite a bit on this. We’ll see what we can do after Tiburon.
July 27th, 2008 at 9:11 pm
I have added the following additional content to the second paragraph:
Some developers have expressed interest in statically typed client side proxies for server classes. These are not difficult to generate and support for this will be provided. In the initial Tiburon release we may have to provide this capability as a separate example utility due to time constraints. Tiburon already provides support to collect all the class/method/parameter metadata required to generate proxies allowing us to at least create an example utility to generate proxies. Note that the more “dynamic” execution of a command by name described in more detail below and a “static” proxy approach are both viable. There are trade offs to both approaches. The dynamic approach blends in well with applications that work more directly with SQL queries and stored procedures. The static approach provides better support for things like code insight and client side type safety when server side methods are refactored.
July 29th, 2008 at 4:24 am
Thank you.
August 4th, 2008 at 2:39 am
"Tiburon already provides support to collect all the class/method/parameter metadata required to generate proxies allowing us to at least create an example utility to generate proxies."
Can you please tell us more on this?
August 5th, 2008 at 6:35 am
I realized you are talking about "static" proxies, not dynamic proxies.
Can you remove my duplicate comments? Thanks.
August 12th, 2008 at 10:14 am
[...] Tiburon DataSnap enhancements http://blogs.codegear.com/steveshaughnessy/2008/07/20/38912/ [...]
September 4th, 2008 at 10:00 am
[...] DataSnap architecture, based on DBX, for lightweight, client/server [...]
November 3rd, 2008 at 12:04 pm
[...] Steve Shaughnessy blog post: Tiburon DataSnap enhancements [...]
December 1st, 2008 at 4:21 pm
In Delphi we have to do {$MethodInfo ON} in class defination …
I’m c++ builder programmer , I want how to make this directive work in c++ decleration or difination ?
what should i do ? ( __classmethod ) not work for me
{$MethodInfo ON} in delphi
???????????????? in c++ builder
Thank you .