yoy.be "Why-o-Why"

Done! Feeder now on PostgreSQL

2018-08-31 20:25  feederpq  coding delphi  [permalink]

Tadaa! Thanks to DataLank, I was able to switch feeder from SQLite to PostgreSQL in roughly one evening. You just need to change ifnull to coalesce, (and SQLiteData.pas for LibPQData.pas ofcourse) and storing Delphi's TDateTime values now map to genuine timestamp values, so it's a little different to do arithmetic with...

But I was getting some 'database locked' errors, so I guess it was the best thing to do, since SQLite works really well but isn't quite designed for use by a website, let alone multiple processes. Sorting by timestamps appears to take a little more processing power than I was expecting, so I may try to change back to storing them as double (since that's also the value TDateTime uses internally), or I may just be missing an index that could speed up the query. It takes some investigating to find out, but that's for another day...

Update 2018-10-12: With some regret, I need to inform you that I have switched back to SQLite, perhaps for the time being. Though operation was acceptable, I noticed quite some performance was lost on queries that depend on the values in the fields of type timestamp without timezone. I've had a look if I could fix this by adding indexes, but wasn't successful. I was also thinking about changing the date-time fields to float, as that's what's actually used in the Delphi code that handles the values, and would probably get better performance in comparing and sorting in PostgreSQL, but the time it would take to switch that around would exceed the time it takes to switch back to SQLite. And switching back to SQLite offers a situation that has shown to perform well in all cases. With this switch I can find some time (somewhere?) to read up on PostgreSQL and timestamp fields, and what I'm missing that was causing some queries to take abnormally long. The query that averages times between posts, prior to checking feeds for new posts would regularly take over 100 seconds, a query that SQLite would only take seconds over.

twitter reddit linkedin facebook google+

RSS is far from dead!

2018-08-08 00:55  feeder  delphi internet freeware  [permalink]

I've been using RSS/Atom feeds on and off since I've learned about them. A long time ago, Google had a nice feed reader, but decided to discontinue it. Users were left to search something new, and I settled on The Old Reader, combined with gReader since I had a smartphone, and all was well. For a while. After some time you notice you still get disturbed by some tiny issues you can't seem to get to go away, either with tweaking the configuration or with Stylus. So what does a developer do? Start to think about developing their own solution. Then plan to develop their own solution. Then develop their own solution. So I'm somewhat proud to present this little thing I've been tinkering on in off-hours the last month:

github.com/stijnsanders/feeder

I have a live version to try out here: http://yoy.be/home/feeder/ but it uses the neighbouring instance of tx for authentication. I should enable Google/Facebook/Github OAuth things instead, but finding out how that works is a few items lower on my wish-list (of things I wished I had the time to put into).

I wanted a feed reader without the extra's. I wanted to mark items as read that move out of view by scrolling down, and plays somewhat nice with the surrounding HTML and the browser. For now I like how it works. There's an issue with emoji's that apparently get eaten by UTF8Decode, but that could be a bug that got solved since good old Delphi 7. But now that Delphi has a community edition, I think I should bring most if not all of my other projects to this version instead of sticking to Delphi 7... But that's another story. (One you might notice some time in the future on my Delphi RSS feed...)

twitter reddit linkedin facebook google+

How to do timezones?

2018-07-17 23:53  timezones  coding delphi weblog  [permalink]

I feel stupid. I guess I've been put out of my comfort zone. I was thinking about doing something, and then I did, and I thought I'll just primarily make it work for me, but then I noticed "new" items were still two hours behind. Ofcourse that's due to the bias against UTC we have here in my current local timezone. So there you have it. All my programming career, I would just parse dates, and stop before this "+0200" or "-0900" thing that's there at the end. So the right thing to do is go all the way, and with a little effort make it work correctly for not just me, but for any user of this little thing.

But how? I remember these drop-downs with timezones where I typically look-up Brussels or Paris. But it's summer daylight-saving-time for the moment. So it switches to and from only one hour between UTC twice a year (until policy changes), how do I keep track of that? Is there something that I can have the system do this automatically for me? It feels like these are things I should have known all along, but dont. I guess I'll have to start reading somewhere...

twitter reddit linkedin facebook google+

StackOverFlow/Delphi: new blood?

2018-07-12 21:30  sodnewbies  delphi internet weblog  [permalink]

Stack Overflow — Newest 'delphi' questions

Am I seeing this correctly? I've been following this page somewhat less closely lately, but the majority of new questions is by people with not too high of a reputation score. And that's actually a good thing. Let me explain:

A while ago it looked like we were 'past StackOverflow peak'. StackOverflow started as an alternative to outperform all other question-and-answer sites for techies, by having a really well developed reputation system that allows a community to self-regulate. And it did. Both the reputation-system created a really fine repository of good questions and good answers, and all other question-and-answer websites were oblitherated (at least from the google top results on typical search-queries).

A few years later, StackOverflow appeared to struggle with having lost it's reason d'être: people with actual questions would easily mistake StackOverflow as a forum and saw most questions rapidly closed and reprimanded for not attaining an expected level of quality the community would hold to. This is a bad deal for newcomers and in general a source of bad feelings. They know about this over at StackOverflow. And have committed to take action. I haven't kept up to speed about what they're exactly plannig to do, but it could already be working.

Specifically for the questions tagged 'delphi', it's not only good that this way more people that just started collecting a reputation saldo, are posting valid questions and are getting helpful responses; I also think you can derive from it that more people are getting into Delphi. It's not up to me to tell whether that's in part because the most recent Delphi versions also successfully target mobile platforms, but if it's true I'm glad to know more people are about to experience the solidness of the Delphi eco-system, both in tooling and available components, and in solidity and reliability of your final result you're offering your users.

twitter reddit linkedin facebook google+

Another store from the trenches: GDI vs PDF

2018-07-06 18:31  gdipdf  coding dagboek delphi werk  [permalink]

One of the great things about Delphi is this object-oriented way of manipulating things called the Visual Component Library. There's much more to it than that, but in a sense you could describe it as a wrapper around Windows API's. One of these is GDI, it's old but it's still there and used for graphics and text-layout. It was born, way back when, from the idea of What You See Is What You Get (WYSIWYG) taken even further and that the same program logic could work for display on screen and writing to a printer. Delphi's VCL declares the TCanvas class you can use to design the output, and wether you pass a Printer.Canvas, Screen.Canvas, Form.Canvas or Bitmap.Canvas shouldn't really matter. This works great for forms and reports.

But in the printing business, there's much more than forms and reports to put on paper. Over there the PDF reigns supreme. It performs a similar task, but originated out of a very different set of parameters. A big module we created of the production software at work generates them according to specifications. How this works is very different as the components that make up a PDF document are constructed one by one and this doesn't look like the work you normally would do with a TCanvas instance.

But now and again these two worlds need to bridge over to eachother. One day I had to keep automatic PDF archive copies of some reports, and discovered the great SynPDF project from the same people that created the mORMot framework. It offers a TCanvas instance (TPdfDocumentGDI.VCLCanvas) that you can pass to code that knows what to with a TCanvas reference.  It works great.

I haven't needed to think about crossing the two the other way round, but recently I noticed a collegue did, and he also found a simple straight-forward solution for the problem: You can just ask the PDF reader on the system to render a PDF to a printer. It's as simple as starting a process with:

"C:\Program Files (x86)\Adobe\Acrobat Reader DC\Reader\AcroRd32.exe" /n /t "<document to print>" "<printer name>"

or

"C:\Program Files (x86)\SumatraPDF\SumatraPDF.exe" -print-to "<printer name>" "<document to print>" 

This works just great and even doesn't add too much code to your program. You may need to include the PDF viewer installer with your installer, but that's a minor issue and perhaps not even required as most people have a PDF reader installed by default.

twitter reddit linkedin facebook google+

Delphi project and subversion: set the build number to revision number

2018-06-09 09:37  dsvnbuild  coding delphi werk  [permalink]

At work, we have a number of interrelated projects in a single subversion repository. We've agreed to change the build number in the project's version properties to the current revision number of your work folder. And specifically not the revision number you might predict your changes will land in when you commit them.

We're currently a rather small team so it may be tempting to assume you'll just get the next revision number, but we stress newcomers to ignore that urge. As long as we're a small team, this works manually, but if we grow we should move over to scripted version resource entries and commit hooks that update the build number automatically.

So with a major and minor version of 2.0 for example, and a release number of 8, the full version could look like "2.0.8.51873".

This way of working has a number of advantages.

The binaries that get used in production, but exert a bug in their behaviour, show with their full version number (and the previous version number from before the bug occurred) the vicinity of revision numbers that introduced the code change that may lay at the cause of the bug. To get the exact revision number, you need to look up the SVN log (or blame) of the .dof or .dproj file, but it's quite sure it's a number closely above the number in the version. We've stressed second line support personnel to list this number when reporting the bug, which helps when researching reproducibility.

But even long before that, when it happens two (or) more of us inadvertently make changes to the same project, we either started from the same revision number in the work folder and see when comitting that something has to get merged and it does so automatically or we get a merge conflict; or we started from different revision numbers and get a conflict right away or a message that we're over due for an update to or work folder.

As I said we're a small team for the moment and it rarely happens so it saves us an update task with risk of conflict before we commit, and in general we can split the work that needs to be done between us so we shouldn't make changes to the same projects.

Then again, if the team would ever grow to something like really big, we would probably have to switch to something else than subversion, or even repositories per project, who knows...

twitter reddit linkedin facebook google+

AES v1.0.1

2018-05-21 20:18  aes101  coding delphi freeware  [permalink]

md5

I noticed something was wrong with the key generation schedule in my AES implementation. I had a close look with the FIPS 197 example vectors at hand and fixed it. I still should make some time to run it under pressure and see how it relates to reference implementations and popular implementations.

twitter reddit linkedin facebook google+

xxm v1.2.5.450

2018-05-03 22:43  xxm450  delphi freeware  [permalink]

For a while I thought there's not enough there to have a full release, but with the switch from XML to JSON, and NTLM and WebSocket support in xxmHttp and xxmHSys2, plus all the minor changes and tweaks that improve performance and security, it turns out to be quite a formidable release. Regretfully, by finally deciding to have all string arguments const anyway, any project's xxmp.pas will have to change. A long time I was hesitant about this because I feared binary incompatibilities, but I've tested all permutations between caller/called with/without const, and it all works. Except when building old code with the new xxm.pas will serve you with this minor one-time inconvenience of having to add const yourself to the arguments... I've thought about trying to create something automatic — like the XML to JSON converters — but since you're quite free to modify xxmp.pas to your liking, it may be more complex than it seems. (If you would like to see this anyway, let me know and I'll put some time into that.)

It's advised to do an extensive test-run with this new version before you update the live environment. If anything unexpected turns up, let me know!

xxm v1.2.5.450 (→gh) (→sf)


twitter reddit linkedin facebook google+

What do you think I should do?

2018-04-20 23:26  sendfile404  delphi freeware  [permalink]

Dilemma, dilemma! What should I do? Sometimes, you need to have a certain URL in a web-project that just serves op the contents of a local file. You could try to have the file in a static section of the website, or even a sub-domain for static content (if your budget allows it), but then it's out there for anyone. Best is to have a request be processed dynamically, so you can do some server-side logic first, for example to do authentication control, or generate the file first (perhaps with a graphics library?). That's where the SendFile method of the IXxmContext comes in. The code that implements it is pretty straight-forward: 

procedure TXxmGeneralContext.SendFile(const FilePath: WideString);
begin
inherited;
if State=ctHeaderNotSent then FSingleFileSent:=FilePath;
SendStream(TStreamAdapter.Create(TFileStream.Create(FilePath,fmOpenRead or fmShareDenyNone),soOwned));
end;

If the request's header has not been sent already, the private value FSingleFileSent is set to the file path, assuming the request is meant to have this file's data as response data. This is used for error handling and logging.

Then SendStream is called, which takes an IStream pointer, so a TStreamAdapter is used to wrap around a TFileStream, with ownership so the last IStream._Release will call the file stream's destructor.

One thing that's missing here perhaps is trying to figure out if the HTTP response header Content-Type is set to some suitable MIME-type for the file, but this is so untrivial that it's best left to the developer. So give it a second of thought when you're coding a call to SendFile.

So about this dilemma I'm having. Just imagine for a second you have this code in a project. Attention: this is very bad practice! It should be glaringly obvious to everybody that doing this opens the back door wide open and people with malicious intent can access any file they want on the machine, include system files, so please don't ever really do this:

[[Context.SendFile(Context['f'].Value);]]

Yikes. Very very bad! I feel dirty just for typing that, but just as an example, this code has a high probability of trying to open a file that doesn't exist, or otherwise have the TFileStream.Create throw an EFOpenError exception.

In that case, would it be better if xxm answers with a proper HTTP 404 (page not found) response? Now default exception handling kicks in, and left unhandled (hint) xxm will fashion a HTTP 500 response for you with the exception data. There's also a bit that will see if your local fragment registry can load a fragment for '404.xxm' that lets you design a nicer 'page not found' page than the default, but I'm having a hard time to guess if that would be something unexpected for someone somewhat new to xxm, calling SendFile on an inexistant file for the first time...

I'd love to hear from anyone on this, but for now I'll just let it be like this and let the normal exception play out, if any.

But wait, there's more. Deep burried within the dark corners of the Windows API, there's a thing called TransferFile. It basically lets you tell the system to take a file handle and a network handle and stream all the data from the one to the other, as much as possible right from the kernel. The way the friendly people over at Microsoft worked it out, and tied it to the running system so it would only work on Windows Server versions, makes it kind of unsuitable for where I want to take the xxm project.

But wait, there's more. You may have noticed the web-sphere is gripped with a frenzy for all things asynchronous. There are a few good things there, but it is mainly the best way to serve a magnitude more of concurrent requests by the same server. The short story is you try to avoid waiting on the system while it waits on network or disk. The long story envolves completion ports or libuv, but is in essence unfit to combine with what xxm is doing: having a separate DLL with code you just call to have a response generated for a request, since to do it properly every request to the operating system needs to be re-routed over your job/thread/task/fiber/yarn-management.

This shouldn't hold me back to get as close as possible to the middle ground between the two, where you have the option to build a response, but can hand over a (file)stream and have the HTTP server spool that as it sees fit,  once the situation really is just that straightforward. But this may be what xxm 2.0 could be about, if there ever will be something like that.

twitter reddit linkedin facebook google+

HTTP+HTML+Delphi authentication (how xxm does it)

2018-04-13 14:26  xxmauth  coding delphi internet freeware  [permalink]

Daraja Framework: HTTP+HTML form-based authentication

Jikes! This is strange. Yes you could go ahead and have a page with a login-form, that posts onto a handler that checks your password, and throws a 401 when it fails. But is that really what you need? I thought 401 is there to elicit the user's HTML-client (a.k.a. browser) to show a modal form asking for a password before re-posting the request. Just like xxm's Basic Authentication demo does, and it does this right at the center of the project, before your request is routed to any page or resource, so that all requests to the project need authentication. Also this way you don't need to code a check IsAuthenticated on every page or resource.

But — again — is this really what you need? The public nowadays doesn't respond well to systematic authentication like that, and also makes it impossible to do anything on the website while not being authenticated (yet). It's better form to welcome new users with a nice 'create new account' button (More about that here.) and perhaps more information on what's on offer, next to the logon form for existing users (with extra options like 'stay logged on on this station' and a 'forgot my password' link). There's an example in xxm's Session demo: The opening page has a log-on form, and Login.xxm does the rest. It doesn't really check user-account and password here as it exceeds the purpose of the demo.

To show you a working demo, you should have a look at tx: It has a central redirect for any page request from a user that should authenticate first; the logon-form with extra options to show users as a normal web-page; checks the entered password agains a properly salted hash and then redirects you to the page you came in for originally...

And there's much more to tell about authenticating users. I've tried to make a list here (it's in Dutch though), and that doesn't even scratch OAuth(2) yet...

Before I forget, did I mention xxm comes under a permissive MIT license? So you don't need to buy a commercial license!

twitter reddit linkedin facebook google+

Do I also need a four-letter-acronym to be cool these days?

2018-03-30 22:47  xx4la  coding delphi internet  [permalink]

→ Reddit: Any drawback to using Wordpress in front of a MERN application?

MERN?! What's that?

MERNMongoDB Express React node.js + Redux WebPack

Oh, I get it! It's one of those four-letter-acronyms that describes your software stack. The first one, and as it happens also the one I started on was:

LAMP: Linux Apache MySQL PHP

But trying things out on my own, I didn't get a hang of that Linux bit. I still blame the folks that sneered me off with "start with typing man man at the prompt". So I got stuck being a

WIMP: Windows IIS MySQL PHP

but later regained my poise and sting with

WASP: Windows ASP SQL Server PHP

which worked great for a while, but I moved on. Not quite with the hot and happening new one:

MEAN: MongoDB Express Angular node.js

but closer related to other desktop application work I was doing in Delphi. Having done some raw networking, and some raw HTTP, but also the IIS APIand implemented Internet Explorer's IInternetProtocoland FireFox' nsIHttpChannel (before they chucked XPCOM somewhere after version 3.6 and starting the rapid release schedule), and something something HTTP.SYS, I decided to start something to model all the common bits into one single interface so you could easily switch between implementations and environments. And hot-swap a binary without taking down the webserver/webservice. And do that after an automatic compile when you changed a file and refreshed your browser. And have a mix of HTML and server-side logic into the same files like PHP and ASP (and Cold Fusion...) And still have full response streaming, and not a big hard templating thing churning on a request first before being able to spew out the response in one go... And have a few of the basic things in place to help you with security to prevent malicious requests.

So I created xxm. And websites with it. Such as tx. So I guess I should invent suitable fout-letter-acronyms as well, then:

XIMR: xxm IIS MongoDB (over TMongoWire!)  Redis

XXJP:  xxm xxmHttpAU jQueryUI PostgreSQL

XESVxxm nginX (over SGI) SQLite Vue.js

Hmm, doesn't really sound all that great... Never mind then. I'll just enjoy it if xxm could serve as a solution for anybody in the very small niche of people that do both high-level server-side stuff with Delphi, and high-level dynamic-web-stuff, and want the two closely knit together...

twitter reddit linkedin facebook google+

A thin wrapper around SSPI SChannel.

2017-12-30 22:57  schannel1  coding delphi  [permalink]

I thought, if you already have something that does work over a network socket, can you have it encrypted of a TLS connection? If you search, you van find a lot, mainly using OpenSSL. If you read on a bit, you learn about LibreSSL, but if I understood correctly, Indy can't use that since it needs specially patched DLL's, that are stuck on some old version sadly enough...

But, I always keep searching for the thinnest possible wrapper. If there's a way to carry less bloat, or use an even thinner library, then yes please.

So I thought, Windows by itself, or at least some Microsoft things, make calls to the outside world over a TLS-line from time to time. So there has to be a DLL that does all the work for those. It would be strange if it exists, but it's not opened up. Some more searching leads to the realisation it's this SSPI thing that keeps turning up. There's a thing called SChannel you apparently need, but it's not as easy as just replacing your connect/recv/send calls...

Once there was a time when Microsoft wasn't quite planning to keep the 'network subsystem' to themselves, leaving the option open to get some from a different supplier. (Once there was a time it wasn't the matter of course that networking plays over TCP/IP/Ethernet, but that's another story altogether.) You still see that in the SSPI story. You're supposed to call a central function first to see what's available (by which vendor). Once you've tracked down the DLL you need (secur32.dll), you see it just has all you need right there in the exports table (and even just patches them on through to sspicli.dll, at least since some recent Windows version). So in the hope to have some simplification, I think I have now a basic minimal wrapper around the required calls to make it work, added to the minimal things I had in there to talk to WinSock2:

github TMongoWire commit d2c99a...

I hope this performs as expected, as I still have to put it through some more testing in different conditions, but that'll be for another day. And as the current season would have it, perhaps for another year. If you find anything, feel free to launch github issues or pull requests. Happy New Year.

 

twitter reddit linkedin facebook google+

Why I choose Delphi

2017-08-15 22:49  whydelphi  coding dagboek delphi  [permalink]

Strange, all these Why I choose Delphi articles lately:

Keep them coming! It's good to see it stressed that it's really a myth that there's not enough Delphi talent out there. Rember, Delphi's debugger by itself is so strong, a decent developer should be able to learn both Delphi and an existing code-base that works just by stepping through the code with the debugger and see what it does. Yes, the language is a little verbose; yes, it's perhaps even older than C/C++; but remember, so is COBOL, and I would almost say that's not cross-platform or a systems language, but those just had other meanings back in the days. (Did you know there's Delphi for AS400?)

So, why do I stick with Delphi? The answer is pretty straight-forward (and perhaps a little sad): give me any kind of computing problem, and I'll find a way to tackle it with Delphi. I've done so much different things, and still found a way to get a great system I enjoy working on, and still have a Delphi project that its compiler happily churns into a binary executable that performs really well. Yes, you could do that also in C/C++/Rust/D/Java/(etc.) but I can't, and don't really want to. There are always up-sides and down-sides,  but it feels like with Delphi you don't meet much of any down-sides, and if you do some-one else knows something to do about it.

twitter reddit linkedin facebook google+

MurMurHash3

2017-05-19 20:09  murmur3  delphi freeware  [permalink]

@stijnsanders do you have MurMurHash3 code in pascal aswell ?
i truly liked your optimized code for md5 and all those

— _pusher_ (@_pusher_0x90) 12 mei 2017

Why, thank you. Eeuh, is it at all optimized? I took some decisions that may perform a little better than the reference implementation, but I haven't taken any time to compare to see if it actually performs any better or worse...

So MurMurHash3... Let's have a look, it's on wikipedia, so it's a thing. And the reference C implementation is in the public domain, great! Looks straight-forward enough, could boil down to a translation-job...

Rougly two hours later, got triple zero (0 errors, 0 warnings, 0 hints). Now for checking if I got it all right. Hmm, not much there, but those x32_64 match, so barring any typo's, assuming this pretty straight translate job will result in the expected behaviour for the other two functions, this should be it:

md5.zip (25KB)

twitter reddit linkedin facebook google+

Checking xxm for PHP's vulnerabilities

2017-05-12 07:56  xxmphp1  delphi internet freeware  [permalink]

If I read about a newly discovered vulnerability related to PHP, for example this one here, I try to find out if it would apply to xxm as well. 

In this case I guess there's nothing more than sending out the message, again and again, to sanitize your inputs, and poperly encode your output. Strings are never just strings. They are always an internal representation of a bit of textual data. So always think about that taking string values in, and preparing strings for output. A few weeks back I had to speak up to someone who wrote OutJSON:='{"field1":"'+value1+'"}';Little Bobby Tables comes to mind, though I'm not sure 'JSON injection' could be so devastating as SQL injection. (And OutJSON:=JSON(['field1',value1]); is shorter!)

The other time I found out it's a really good idea to strip nastiness like EOL's (CRLF) from headers added to a response, just in case a malicious script is up to no good. Come to think of it, that's also just another case of properly sanitizing your inputs...

twitter reddit linkedin facebook google+

Delphi, the shrinking island.

2017-05-09 21:48  delphitrend  coding delphi  [permalink]

Is there a language that has a single word for 'the feeling of being on a shrinking island'? Anyway, this is somewhat sad to see, especially that the few most recent Delphi versions gave the impression there was a new uptake with more people getting persuaded, but it doesn't show in the curve.

(via)

twitter reddit linkedin facebook google+

'Relax' scripting vs xxm

2017-04-22 13:12  relaxxxm  coding delphi internet  [permalink]

Delphi Relax Web Scripting (Marco Tech Blog)

I'm sorry but I feel I must react. In general I keep silent, in the hope people by themselves will know better, but as I'm getting no input what-so-ever that that is the case, I feel tempted to write something about this.

First about what's at hand. I see this bit of code:

<h2>Employees</h2>
<ul>
@foreach (var emp in employee) {
  <li>@emp.FirstName @emp.LastName (@emp.PhoneExt)</li>
}
</ul>

and it looks kind-of OK. To the untrained eye it looks good and may even look tempting to write more in this syntax. This is a straight-forward example of a template that works with a templating engine that no doubt has many more capabilities and features. And then I thought, learned from practice, what I typically would get asked is to not show " ()" when the PhoneExt field is empty. I would not know how to make that happen in that template syntax. That's mainly because I know nothing about the template syntax. If I look into the documentation, I might find an @if predicate to make it happen, but let's move on:

This is what it could look like in xxm:

[[!var emp:TEmployee;
<<h2>Employees</h2>
<ul>>
foreach emp in FDMemTable1 do
begin
<<li>>=[emp.FirstName,' ',emp.LastName,' (',emp.PhoneExt,')']<</li>>
end;
<</ul>

Looks roughly simlar. A little more like Delphi syntax. And in fact it is. If you know [[, ]], << and >> get translated into Context.SendHTML() and Context.Send() calls behind the scenes (full details are here),  you know this code will result in the same output. Without templating engine! Streamed to the user's client! Perhaps even while the data is streaming in from the database server, in case it's a longer list, and in case there is a database server, Marco uses a memory-table for his example.

What I find important is that there's less going on between the native compiled logic and getting the data to the user launching a request. Not only a templating engine looks superfluous, this entire ORM thing is something I don't get. If it's a gigantic database model with so much tables that you clearly benefit from code-completion, then I agree, but I haven't come across something remotely close to that in web projects.

Also the HTTP-server itself is something I think that values extra attention. I've seen platforms and frameworks that offer you a wealth of capabilities and features, but hastily slapped on something that listens on TCP port for basic HTTP requests, in some cases on port 80, but more often 8080 or something else in the thousands. In real web environments, the server(s) has/have a lot more going on: load-balancing, reverse proxies, firewalling, authentication. Since we're in a Post-Snowden-era nowadays, we're all responsible to think about protecting privacy and get that HTTPS in order with the proper certification and encryption... Not to mention HTTP version 2 that's heading full steam towards being generally accepted/expected.

I can image the web-admin responsible for all that, isn't happy with your request to add this newfangled separate thing that's doing its own handling of HTTP requests. ISAPI DLL's or Apache modules play much nicer with existing IIS or Apache installations. (FastCGI is on the table, but for now xxm has SCGI available for other servers.) Even if your 'Delphi HTTP framework' of choice is specifically designed to tap your ORM of choice and offer a REST-API for your data-layer needs, it will still be one more stop along the way between the user's browser, and the delivery-setup, and the front-end, and the page-template, and the data-layer, and the database, and what the user actually needs or wants. I think of this in the postal office when there's twelve people in the queue in front of me.

I don't expect to convince much people of this way of working, but it works great for me. I remember the days with early PHP and ASP and how simple and straight-forward everything was. Knowing these work on scripting engines, I kept worrying about lost performance. This was the core reason to start xxm: employ the speed and power of the Delphi compiler to have a native library serve my websites. And it turns out that Delphi code looks quite nice between HTML to handle server-side logic, if I may say so. It took me a few years to make this happen, but I couldn't do without it any more. And people kind-of appreciate that for using this new application, all they need is their trusted browser and a URL.

twitter reddit linkedin facebook google+

TMongoWire v1.1, now on jsonDoc!

2017-04-04 19:46  mw_1_1  delphi freeware  [permalink]

→ TMongoWire

This took more effort than I anticipated, but I'm glad I saw it through. A tiny bit of history: when I started creating a Delphi connector to access a MongoDB instance, I had to create the tools to manipulate BSON, and since I really (really!) hate long lists of overloaded methods, I created something based on Variant values. It works great for ADO, so why not. Some time later I had to manipulate JSON, unrelated to MongoDB, so I took what I got, stripped BSON-specifics, into jsonDoc. There it evolved on its own and gained some useful features.

So it was time to re-work TMongoWire, making it use jsonDoc just like any other project. There's still a bit of BSON-specific code, but it deserves a unit of its own.  The downside is there's an extra unit to include in the project, but there were multiple already (jsonDoc, bsonTools, mongoWire and if needed mongoID, mongoAuth3 and mongoStream); the upside is the improved performance and features of jsonDoc, both of the current version and those the future could bring. Enough change to bump that version number:

https://github.com/stijnsanders/TMongoWire v1.1.0

Please let me know if the migration effort this generates is surmountable, and if I can do anything to help. In theory any references to bsonDoc, the BSON function and the IBSONDocument interface should keep working once replaced with references to respectively jsonDoc, JSON and IJSONDocument. The document object no longer implements IPersistStream, but this may have been stretching abstraction just a bit too far. We're on Delphi so plain old TStream should do, and apparently IPersistStream's Seek signature changed a tiny bit over Delphi versions, making it harder to make the code cross-version-ready.

twitter reddit linkedin facebook google+

Boxer

2017-03-24 21:57  boxer  delphi werk freeware  [permalink]

Drats. I thought I'll try something like Clover does, start with SetParent on any top-level window you could find with GetAncestor(GetForegroundWindow,GA_ROOT) and see what it gives. There's this old default MDI child mechanic that starts working, so I thought this might just work. But it looks like over at Microsoft the Windows 10 team decided to cut some corners. The theming doesn't do it's modern stuff and reverts to Vista-style borders, and the newfangled calulator, for example, won't even take the SetParent nicely any more. Oh well. For the time I'm still on Windows 7 at work, I guess it'll come in handy there for the time being. Probably in about a week I'll know if this could work where other desktop-management tools failed or were too cumbersome or too invasive...

How does it work? Boxer starts with an empty window, and will show an icon over the top-right corner of other windows. Click that to 'box' the window. The top bar shows a tab for each boxed window. Right-click to open a menu with options to unbox or close, and drag to reorder. Multiple instances play nice with each-other and show 'boxing handles'  next to eachother, the second instance labeled "A", the third "B", etc. To disable the boxing handles, right-click on the top bar to the right of all tabs.

Anyway, if you feel like having a try if you can do better, have a look at the code here. Download a binary here: Boxer.zip (209KB)

twitter reddit linkedin facebook google+

A story about two task-keeping-applications.

2017-03-18 17:22  s2tka  coding delphi werk freeware  [permalink]

Let me tell you a story of two task-keeping-applications. Once there was a team that was struggling to keep track of the work it was doing, had done, and still had to do. It was frustrating to work on the team, hard to schedule things, almost impossible to estimate when things where going to get done. And since clients are what they are, changing requirements and additional requests would cause much more disruption than expected of a team that calmly and firmly is determined to deliver the best of market solution.

An internal brainstorming-session about what to do about it, ended in two opposing visions about what decent issue-tracking should be about, and how a system that (co-!)operates with the team members should be structured internally, should behave, and which duties it should perform either by itself, or by effect of using its rules and restrictions correctly as designed.

In an attempt to get to the best solution, two teams were formed to develop each a project based on one of the two views. An evaluation would follow determining a winner, or if would it be possible to synergize the best parts of both into a single system.

A side note: from a managerial point of view this is a very tough decision. Allocating a lot of resources to work on internal structure, takes resources away of the work that brings in revenue, and such undertakings typically risk getting frivolous or spinning out of control. But I guess it's normal that all stop rowing to help keep the boat from sinking. So, apart from being intrinsically aware of the exceptionality of this opportunity, coordinators were given instructions to keep to a strict schedule in these projects and a determined focus on delivering a workable proof-of-concept quick. Sounds like a good work-ethic to apply generally, if you ask me.

A few weeks later progress was made, and the prototypes were already being used to keep track of the issues of these new task-keeping-applications and other projects. The main difference between design visions became apparent soon enough.

One application centered around the list of work items. Care was taken the entry form was extensive enough to have fields for all of the details about a work item, its categorization, its relation to the project, an outline of the projected outcome. An overview would show the current list, possibly filtered for those items assigned to you.

The other application was centered around getting information out of the system, especially structure and relation between items. Users would enter small specific reports, and add them to the best suitable node in a tree-structure, optionally marking relations with other items over branches. The overview showed an expandable structure starting at the root items. It also could have a filter applied, but would potentially show you an entirely different structure of the same data, when using a different relation type.

Having two working prototypes, attention gradually reverted back to the serious work and some of the frustrations of before were abated. After a few months an evaluation was undertaken.

One application had rendered itself useless. The first weeks of usage, a lot of entry happened, but without consensus about categorization, the list was enormous without a clear way of grouping relevant items. Duplicate entries and ambiguous task-descriptions were unresolved, causing confusion.

The other application was doing better. It needed work, but offered a good view of what had to get done.


Enough about the story. Reality, of course, is much more bleak. As no sane manager would make such a shift in resources of a troubled organization, the 'two teams' actually stand for the existing team using an off-the-shelf something badly administered on one side, and on the other side, well, just me, toiling on something potentially better, in my off-time.

I started — like many — with a plain text-file, then a spreadsheet. Then I took the step to design a database for it, but wanted something to do entry and retrieval with roughly the same ease-of-use that a spreadsheet would offer. Working with tree-structures for some other projects, I wanted a single structure to serve as the basis to store data in. Specifically with projects and tasks — as projects tend to have sub-projects, and tasks split into sub-tasks — using branches of a tree would enable to keep this distinction conveniently vague. If you add representation for entities like users and clients into this tree-structure, and relation between nodes over branches, you've got a richness to model much more of the world, and keep track of its changes, past and future.

From there it grew slowly, over years, into what it is now: tx. First versions suffered a notoriously bad interface, but I hope that has improved. For long I was its only user, but some cooperation features have been added since. What's left for me is to keep improving tx where possible, and demonstrating it to people what it's about.

Though entry and structure is important, where a task-keeping-system can really shine is helping to keep an overview. A cleverly designed filter can limit your view to exactly these items that are relevant to you, but in tx you get the additional option of having these items display with exactly those sections of the tree-structure they have in common. Also, I personally feel the most important items on any list, as long as it may be, should be on top. So when tx displays a list of items, they get ordered using their weight, determined by a combination of factors such as task-type and current status. As a task progresses from an active state to a final state, it may move down the list or even fade from view into an archival state.

And there's much, much more I could talk about, but it's all created out of necessity and designed adhering closely to a central vision of what a task-keeping-system should be and what it should be doing for you in order to be able to depend on it.

In conclusion, and for those people that — like me — skip long stories to the last paragraph, it's so very important that information systems succeed at keeping an up-to-date model of reality, and offer you the freedom and easy to update it's view of the world. Especially so with task-keeping software you depend on to keep track of the progress you make with projects, and to help stay on top of what's important. All of that served as the basis for developing tx.

twitter reddit linkedin facebook google+

How to import MSXML2_TLB

2017-03-13 15:07  msxml2  delphi  [permalink]

O my god I can't believe this isn't somewhere on the net already! I come across this question and ofcourse there's the default discussion about Indy and OpenSSL/LibreSSL, and Arnaud is very (very!) right about proposing to use WinHTTP. I personally would see if I could do it with MSXML2's XMLHTTP object. Contrary to what it looks like, it doesn't really have to do much about XML, and you can do really almost all of what you need out of a HTTP request with open, setRequestHeader, send and responseText or responseStream, with a much more straight-forward syntax. (If you can live with COM.)

Plus there's another pretty sound reason: back in the days, when using WinHTTP and/or VBScript (for the RegExp component...) you would make it very clear it would only work if you got Internet Explorer version 5.5 or newer installed. "How much back was this?" I hear you ask? It depends, but it's so far back that pretty much all of XML things came after, and XML being to closely related to data-processing, Microsoft needed to put MSXML*.dll deeper into the system than its web-browser. If it's tied to the MDAC or the actual core system I'm, not sure, but years of use and the success of AJAX caused it to get very many security updates. I suspect a far bit more than the DLL that handles your WinHTTP calls...

So, without further ado, do as I do and import the "Microsoft XML" type library in the most recent version you can find on your system:

On old Delphi versions (e.g. the venerable version 7) open 'Projects' on the main menu, then 'Import Type Library...', then select "Microsoft XML, v?.0 (Version ?.0)" from the list. Your system may have multiple versions installed, pick the highest version number (in my case 6.0). Uncheck "Generate Component Wrapper" as usually you don't need this, and actually have the added benefit of automatic reference counting when you use interface references to keep track of COM object instances.

On newer Delphi versions, I've noticed the import functionality has centralised, and is now under the 'Components' menu, the 'Import Component...' option and you need to select "Import a Type Library" from the first list, and then search for "Microsoft XML".

This will result in a automatically generated MSXML2_TLB.pas that you otherwise don't need to worry about. Except perhaps include it in all of your projects you'd launch HTTP requests from.
Use the CoXMLHTTP.Create class function to get a new XMLHTTP instance, and it's open and send methods to make magic happen.

uses MSXML2_TLB;

procedure TForm1.Button1Click(Sender: TObject);
var
r:XMLHTTP;
begin
r:=CoXMLHTTP.Create;
r.open('GET','http://www.google.com/search?q=hello+world',false,EmptyParam,EmptyParam);
r.send(EmptyParam);
Memo1.Text:=r.responseText;
end;

There's more ofcourse. This is just the basics. Depending on what you need to do there's HTTP POST and ways to correctly encode values into the request body you pass to send, using setRequestHeader. And if you need to handle larger chuncks of data, you should check out responseBody and perhaps TOleStream declared in AxCtrls.pas.

twitter reddit linkedin facebook google+

jsonDoc, jsonV 1.1.0

2017-03-09 01:22  jsonDoc110  delphi freeware  [permalink]

jsonDoc, jsonV v1.1.0

While working on TRethinkDB, I noticed something was wrong with document re-use. My original idea was to re-use a single IJSONDocument instance to process a list of similar documents, but keep the set of keys allocated, only overwriting the values from the new document. (If you're interested about the code, search jsonDoc.pas for FLoadIndex.) Nothing an extra internal-use interface can't fix. Then I noticed that IJSONDocument.ToString could do with a revision because it wasn't using the new IJSONEnumerator, though it should.

Then I noticed the feature I once added to function JSON, where you could declare embedded documents by listing a value of '[', and closing the document with a key of ']'.  The problem is that '[' could be a perfectly valid value! If it were to come via a variable, that would change the behaviour of the JSON call, and that's a big no-no. Also square brackets usually mean arrays in JSON-land,  so it's perhaps confusing that I chose square brackets because you already need them in the JSON([...]) call. So I changed it into a key suffix of '{', and a closing key of '}', counting on braces being really abnormal in key names. If you really need a key name that ends with '{', don't use the JSON function, but just d['x{']:='y';

A minor downside may be that JSON calls with braces in the string constants, break the comment block if you want to comment out a section of code with braces, but there's always (* *) or //. It by far doesn't outweigh how elegant this new solution is, but it's a breaking change (existing code using '[' values won't work correctly any more), so instead of version 1.0.6 I think it deserves a jump in minor version number to 1.1.0.

While I was at it I fixed some minor issues with jsonV, so remember to update that one as well. Hope you enjoy the changes. (Next up may be replacing TMongoWire's bsonDoc with jsonDoc...)

twitter reddit linkedin facebook google+

TRethinkDB

2017-03-06 23:03  TRethinkDB  delphi freeware  [permalink]

Delphi RethinkDB driver

I've given it another try and, in part thanks to a Windows executable that performs well,  and thanks to the gentle people over at the rethinkdb-dev group, I've got something working. It appears the basic functions you would expect from a driver work, but more advanced queries need some more extensive testing. (I've read about a driver testing harnass, but to replicate that around my driver, sounds like a separate project in its own.)

There are some strange sections in the design of ReSQL that show it was written for a weakly-typed language, but using variants and interfaces, I hope I've struck a nice balance between versatility and still having the best suitable methods showing up on auto-completion. Something that should be there may be hidden from view this way, but will probably be because of inaccuracy or error of my part.

Which leaves my in roughly the exact same position as I was closely after completing TMongoWire: having success learning about a new(ish) NoSQL DB the hard way by trying to connect to it directly with Delphi, but nothing in the way of an effective application that would use it for something vaguely useful. I've learned some people have effectively created things with TMongoWire (yey!) but I myself don't really have a good idea (or the motivation) to build something on its own that would use the features of these new database services to their best intent.

If you've got a good idea, please let me know. If you want to know more about TRethinkDB, let me know.

twitter reddit linkedin facebook google+

Am I an "old hand"?

2017-01-18 09:23  oldhand  coding dagboek delphi  [permalink]

Am I becoming an 'old hand'? Every time I see someone that doesn't have done as many years of work in Pascal, ask something about generics, I'm troubled again by this generics thing. I don't use it (much). I don't need it. I've been perfectly fine all of this work without them. Yes there is some hype nowadays about functional programming, but there's a grand glacial movement at the base of it: stong type systems. It all started even on the big metalwhere some bright minds had the sense of taking writing logic into a domain of itself with a gentle nod to mathematics, statistics and other adjacent domains. And a lot of great work was done in this domain. A lot. Some great names that now stand on their own came forth.

A recurring theme throughout of it is type systems, or the lack thereof. Or having it bolted on at a later stage. So by the time K&R came along, they took what they needed, what they could make work for them, what they needed to summon the Unix spirit from the depths. Pascal existed at the time, but C, like JavaScript much later, was born out of necessity. Created by the people there and then that needed a new wrench to tighten the new bolts of a new machine. With personal selections here and there, and careful considerations that copy a bit of the zeitgeist of the time.

What's left for us to do is try not to repeat ourselves too much. One way is keep a look-out for patterns and in new projects, know when to apply which one. Or even better, when to switch to another one. This, I think, is where it's starting to get really tough for the younger generation. There's a lot of base-knowledge to cover, some of it is superceded or obsolete, but we can't replace it with studying patterns. I've seen newbies take it on as lawas if it's forbidden to deviate from the design even just a little. Attacking establishments on the base of their deviations from a pattern. There's something to say about attacking the establishment, but blind fury will get you nowhere.

So this generics thing...  It also is in danger of being percieved as a rule, getting over-applied. As if the one true way is using it everywhere or as often as possible. Which is strange from where I stand, since I still see it as a convenience, where you can save on writing bits of code that would otherwise be painfully similar anyway. In operation the work would get handled by this extension to the type system, checking things for you in the background, making sure everything fits. But in the end the things that need to happen, occur much in the same way as if you would have written all those specialisations out in full.

That's why, as it turns out, I don't use them all that much. Perhaps it's also because of my fading belief in the object-oriented way. That too was presented some years ago as the way to go. Everything was supposed to be objects living on the internet. But it too kind of degraded in my view to a convenience, only one of the ways to manage the things inside of your program. Luckily before any of the ORM craze could tinge me. It makes you look around the horizon, in the past where you did more with arrays and pointers, and sideways where others pass messages around all the time. For now I feel like I'm stuck here, waiting for the ground to crack open and offer a new middle ground between interfaces, pointers, traits, channels, monads, and all of the above.

But that's just my humble opinion. I can't tell for sure what the next 60 years of programming machines will bring us.

twitter reddit linkedin facebook google+

TIL: 'method modifiers' are not reserved words.

2017-01-05 21:17  TILdirrw  coding delphi  [permalink]

Thing I learned today: 'method modifiers' are not reserved words. Apparently the Object Pascal parser 'consumes' these as labels that just happen to be there or not. Which means in other contexts they're free to use as identifiers, and so they're not reserved words... Strange but OK I guess as this keeps the tokenizer that does its job right before the parser, a little smaller... (How I stumbled upon this is a different story altogether, but one I may be sharing sometime soon...) I searched the documentation a bit and there they're called directives: Fundamental Syntactic Elements: Directives which I thought were things like {$XXX+}, but those are compiler directives...

So these variable names are accepted by the parser/compiler (it doesn't mind the smell at all):

procedure test;
var
stdcall,safecall,cdecl,overload,dynamic,register,abstract,reintroduce:integer;
begin
stdcall:=1;
safecall:=11;
cdecl:=111;
//...
end;

twitter reddit linkedin facebook google+

 

Archive... Search...