yoy.be "Why-o-Why"

Open source is nice, but is the protocol also open (enough)?

2019-01-15 08:57  openproto  actueel beurs coding computers internet  [permalink]

Hacker Noon: Bitcoin’s Biggest Hack In History: 184.4 Billion Bitcoin from Thin Air; Satoshi Hard Forks, Saves Bitcoin

See, this is something I'm very very worried about: things like Bitcoin — big public successful open-source projects — have the appearance of being complete open and public, but the protocol isn't really.

When I was first looking into Bitcoin and learning what it is about, really, I'm quite sure this can only have originated out of a tightly connected bunch of people that were very serious about 'disconnecting' from anything vaguely institutional. Any structure set up by people to govern any kind of transactions between them, has the tendency to limit liberties of people, for the people taking part in the system and sometimes also for those that don't. So it's only natural that Bitcoin at its code is a peer-to-peer protocol.

But. How do people that value anonymity and independence from any system, even get to find each-other and communicate to build things together? Well, the internet of course. But perhaps more importantly — and also since long before the internet — cryptography. Encoding messages so that only the one with the (correct) key can decode and read the message, helps to reduce the cloak-and-dagger stuff to exchanging these keys, and enables to send messages in the open. To the uninitiated onlooker it looks like a meaningless block of code, and in a sense it's exactly that. Unless you what to do with it, and have the key — or would like to have it.

Another use of encoded messages is proving it's really you that originally encoded a message. It's what's behind the Merkle tree that the blockchain runs on. That way the entire trail of transactions is out there in the open, all signed with safely stored private keys. The reader can verify with the public keys, and in fact these verifications buzz around the network and are used to supervise the current state of the blockchain, building a consensus. Sometimes two groups disagree and the chain forks, but that's another story.

The protocol, or the agreement of how to put this into bits and bytes in network packets, can get quite complex. It needs to be really tight and dependable from the get-go, see the article I linked to above. You could write it all down and still have nothing that works, so what typically happens is you create a program that does it and test it to see how it behaves. In this case it's a peer-to-peer networking program so you distribute it among your peers.

But when things get serious, you really need the protocol written out at some point. If you try that and can't figure out any more what really happens, you're in trouble. The protocol could help other people to create programs that do the same, if they would want to. This was something the early internet was all about: people got together to talk about "How are we going to do things?" and then several people went out and did it. And could interoperate just fine. (Or worked out their differences. In the best case.) It typically resulted in clean and clear protocols with the essence up front and a clear path to some additional things.

The existence of the open-source software culture it another story altogether, but I'm very worried it is starting to erode the requirement for clean protocols more and more. If people think "if we can't find out how the protocol exactly works, we can just copy the source of the original client/server" nobody will take the time to guard how the protocol behaves in corner cases and inadvertently backdoors will get left open, ready for use by people with bad intent.

twitter reddit linkedin facebook google+

NoSQL agent with SQL back-end(s)

2018-12-29 15:41  nosqlidea  coding delphi freeware  [permalink]

It happened again. I get this great idea that slowly develops, and gives the feeling I'm on to something, but nowhere near any time in the forseeable future to put in to it and get a proof-of-concept of a first project that makes it work. So, for what it's worth, I write it down here fast in the hope sharing it with you may give more chance to this idea getting useful.

I've read that some NoSQL solutions are actually about eventual consistency, meaning in the best case of a query for data that was just inserted or updated could already return this new data if the server(farm) ad already fully processed it. Worst case is that it just for a few milliseconds totally disappears, but that's another story altogether.

I haven't done anything serious with NoSQL yet, and really a lot on good old SQL, and recently with SQLite which I've really grown to love in a short period of time. But still there's something there that's really suited for the new style of programming that is going on with all the new web projects and this 'Internet of Things' everything is on about... To find out, I've been trying interfacing with a number of them from Delphi in the most direct way I could possibly find and make work with reasonable effort. I like how TMongoWire worked out, to talk with PostgreSQL all you need is in the libPQ.dll, but a number of others just stick with a plain HTTP API where you PUT and GET things on their own URL. There's a beauty to that, really. The structure of your documents is nicely contained in JSON, and HTTP is such a stable platform you're sure to be able to access it from almost any platform.

So that's where the idea came from: what if I made my own service where you can just put or get JSON documents? But on the back-end jsonDoc would do the heavy lifting and the storage itself could be in a decent SQL service. And/or it could be in something intermediate like memcached. And/or the saving to storage could be asynchronous somewhere close after the actual PUT call (hence the eventual consistency).

For example, you fill a collection of items with things with a number of fields, for example one is "Price", but later you need the items above or below a certain price, you would do SQL "select * from Items where Price<@p". So in this service I'm imagining, there would be meta-description on the collection that you've provided a SQL database somewhere, but the service is responsible for having done the "create table Items (ID some primary key, DATA json, Price decimal(8,2))" and filling it with the data.

And this would be the beauty of it: if you need an extra column later, you just say so, and the connector would be responsible for the "alter table Items add ..." and filling that column with the data from the stored items. Perhaps even slowly, asynchronously together with the other work. Or even another connector alltogether, let's say PostgreSQL and MySQL side by side, perhaps even as a fail-over for eachother.

But I'm dreaming. It would be a load of work just to get something to work, and even more work to get enough connectors to work good enough to even demonstrate how it would work. And then there's the performance trails... And the evangelising to see wether it solves other people's problems anyway... It would be a really great opportinity to finally cut my teeth on this IOCP thing.

twitter reddit linkedin facebook google+

Update to SQLite 3.26 as soon as possible

2018-12-24 14:52  sqlite326  coding delphi freeware  [permalink]

It's very rare something turns up in the SQLite package, but when it does is best to give it some attention. So, very esteemed users of TSQLite — and anyone else, really — I strongly suggest you upgrade to version 3.26 of sqlite3.dll since recently some dangerous vulnerabilities have been uncovered. I checked, and nothing extensive changed to the API so chances are nothing should break if you only update sqlite3.dll and not SQLite.pas and SQLiteData.pas. Also, happy Christmas!

twitter reddit linkedin facebook google+

TOML? YAML? ini-files? JSON? Loosen up!

2018-12-10 11:26  jsonloose  coding delphi freeware  [permalink]

TOML? YAML? plain old INI-files? or shiny and new — but strict — JSON? Nàh. I've made myself something better. In jsonDoc.pas I've recently added an optional compiler define JSONDOC_JSON_LOOSE that makes the JSON parser a whole lot genter to work with. Some things are no longer required: the root document doesn't need to start and end with braces ("{}"), the quotes around the key names are no longer required (unless they contain special chars), the comma's between key-value pairs are no longer required, it accepts equal ("=") between keys and values instead of colon (":") and can even do entirely without.

And while I was at it there's also JSONDOC_JSON_PASCAL_STRINGS that allows you to write string literals the Delphi way: between single quotes, and double single quotes where you want a single quote in the string value. This way — really handy in Windows environments — it doesn't require you to escape backslashes.

The only downside maybe is that you need to add the defines to the compiler parameters, but for plain old configuration something like this:

x=5
y=10
output='C:\test\output\'

looks a lot nicer and like what we're used to, and gets parsed just the same as if it was written like this:

{
"x":5,
"y":10,
"output":"C:\\test\\output\\"
}

twitter reddit linkedin facebook google+

xxm 2.0? About the big plan

2018-11-28 22:50  xxm2plan  coding delphi weblog freeware  [permalink]

Should I begin on xxm v2.0? Delphi having a Community Edition is really really great news, not only for all of the Delphi community, but also for me as a hobbyist programmer with a number of freeware opensource projects out there. Except I was one of those that stuck with Delphi 7 up till now, so next to myself being fully convinced to finally take the step, I also need to have a look project-per-project how to lift them into these modern times and upgrade them to full Delphi 10 projects.

One of my darlings is xxm. You can compile xxm with modern Delphi just fine. I switched string to AnsiString or WideString explicitly nearly everywhere, so you shouldn't get as much as a hint. (Except mayby somwhere where I still have case x in ['A'..'Z'] of or something like that where x is a char and thus is a WideChar, and the compiler doesn't like that). But I'm worried about how I chose to set up the project. I thought I was best to just inherit from IUnknown and define a decent interface and build everything around that. But several xxm projects later, some that are even running some decent production load at work, I learned some things, most importantly these two:

It looks like xxm doesn't need to work with object instances. Requests are coming in, something needs to build a response, but having to have your objects set up first before you can get crackin' now appears as something I should have tried to avoid. Specifics and identities are handled by the underlying communication framework anyway, so you can get by with just a reference, and request more with that reference only just when you need it.

The other is that with the dynamics of IUnknown pointers, you're still dragging this COM thing with you. Yes it may be wafer-thin really and proven tech, but it's still something removing you from what really counts: pointers to where the code is to run. If you're really (really!) pushing to get the last drop of jouce from the orange performance-wise, it may be just that last thing that's in the way. Even when you avoid any locking in reference counting.

So this is the big plan for the moment: if I get to put some decent time into the project, over the next year(s), I'm considering starting a blank project xxm 2.0, start with a plain C style interface with just procedure and function pointers — a really really ancient looking interface — that has everything that IXxmContext and related interfaces have now. Then re-start with xxlProto and xxmHttp to make that work,  then a new xxmCGI, xxmSCGI, xxmISAPI, xxmAhttpd (and perhaps now really a xxmFastCGI, but no xxm IInternetProtocol any more, sorry) and then have a first new push to see how it performs, with IOCP and http pipelining, and who knows HTTP/2 or HTTP/3 that'll have materialized by then.

After that I was thinking I need a new implementation of IXxmContext that just patches through to the new xxm 2.0 calls, so you can very easily and without a worry add it to an existing xxm v1 project and run it with the xxm v2 handlers, enjoy the performance benifits and slowly change things over to xxm v2 where you need to — or not at all, really.

I hope it's a good plan, but for now it's all a dream. I haven't been able to put much time into my hobby projects lately, but it fluctuates and if it picks up some I'll see how far I'll get. Along the way I still hope to find more people that are searching for a good way to combine good old web-scripting with the power and speed of the Delphi compiler, and can count on xxm to offer good debugging, easy live-updating and a stable and secure platform for their web projects.

twitter reddit linkedin facebook google+

TMongoWire on jsonDoc v1.1.8

2018-11-01 17:24  mwjd118  coding delphi freeware  [permalink]

→ TMongoWire commit 78c7c62: jsonDoc v1.1.8

I may have decoupled the bit that works with structured data (using Variants), and converts to/from JSON, into a separate project jsonDoc, and have converted TMongoWire to run on jsonDoc, I also should take the time to update TMongoWide when jsonDoc gets improvements. The fact is I'm doing much more with JSON than on MongoDB recently, so I risk letting this slip out of sight.

What I also should do is set the TMongoWire repository up so it uses the jsonDoc repository as a git module, but for now it's just about one single file so I'll see when I get to it (...).

twitter reddit linkedin facebook google+

"Leren programmeren is even belangrijk als leren lezen schrijven"

2018-09-11 14:38  nwsprog  actueel coding politiek  [permalink]

→ VRTNWS: Leren programmeren is even belangrijk als leren lezen schrijven

O jee. Hoe gaat het lesmateriaal daarvoor er uit zien? Wie zou dat kunnen samenstellen? En wat gaan ze geven als programmeren, want het is intussen best een behoorlijk breed kennisdomain met veel onderverdelingen. Bepaalde dingen van het computerwereldje liggen bepaalde mensen zelfs beter dan andere dingen. Denk maar aan het verschil tussen taal-vakken en wetenschap-vakken.

Maar misschien nog belangrijker: zit er ook een stuk etiek in? Je kan alles programmeren wat je maar wil, en dus ook programma's die doelbewust misbruik maken van het vertrouwen van de achteloze gebruiker. Of erger nog, schade berokkenen. Om nog maar te zwijgen van de recente revelatie dat alle complexe software die werkt op maatschappelijke data blijkbaar ook latent racisme en discriminatie braafjes toepast zonder verpinken.

Toen ik het middelbaar doorworstelde hadden ze net een vak 'technologische opvoeding' ingevoerd. We hebben een bureaulamp gemaakt. En iets gedaan met blauwe bakjes met in het groot "OR" en "AND" op. Het zou uiteindelijk een verkeerslichtje moeten doen maar dat werkte niet (meer?). En er was ook iets met kleine baksteentjes en lijm in plaats van mortel, maar dat had het gelukkig niet gehaald van de onzin-filter die de professionals vooraan in de klas gelukkig aan de dag leggen.

Ik herinner me dat we enkele jaren later toch al wat puntjes konden verdienen met een mooie pivot-chart of een etiketten mail-merge. Dus ergens denk ik als ze nu eerst zien dat scholen iets gemakkelijker wat computers in de klassen krijgen, dan zal de rest wel volgen.

twitter reddit linkedin facebook google+

Done! Feeder now on PostgreSQL

2018-08-31 20:25  feederpq  coding delphi  [permalink]

Tadaa! Thanks to DataLank, I was able to switch feeder from SQLite to PostgreSQL in roughly one evening. You just need to change ifnull to coalesce, (and SQLiteData.pas for LibPQData.pas ofcourse) and storing Delphi's TDateTime values now map to genuine timestamp values, so it's a little different to do arithmetic with...

But I was getting some 'database locked' errors, so I guess it was the best thing to do, since SQLite works really well but isn't quite designed for use by a website, let alone multiple processes. Sorting by timestamps appears to take a little more processing power than I was expecting, so I may try to change back to storing them as double (since that's also the value TDateTime uses internally), or I may just be missing an index that could speed up the query. It takes some investigating to find out, but that's for another day...

Update 2018-10-12: With some regret, I need to inform you that I have switched back to SQLite, perhaps for the time being. Though operation was acceptable, I noticed quite some performance was lost on queries that depend on the values in the fields of type timestamp without timezone. I've had a look if I could fix this by adding indexes, but wasn't successful. I was also thinking about changing the date-time fields to float, as that's what's actually used in the Delphi code that handles the values, and would probably get better performance in comparing and sorting in PostgreSQL, but the time it would take to switch that around would exceed the time it takes to switch back to SQLite. And switching back to SQLite offers a situation that has shown to perform well in all cases. With this switch I can find some time (somewhere?) to read up on PostgreSQL and timestamp fields, and what I'm missing that was causing some queries to take abnormally long. The query that averages times between posts, prior to checking feeds for new posts would regularly take over 100 seconds, a query that SQLite would only take seconds over.

twitter reddit linkedin facebook google+

"How can i do C++ for free?"

2018-08-30 10:54  freecpp  coding weblog  [permalink]

Someone at work asked me "Is there a way to do some C++ for free?", and told me to mail some links. I did and thought it's a kind of nice overview, so I post it here as well.

If you're looking for a free C++ development environment, the first thing I can think of is:

http://www.codeblocks.org/ 

but one of the main trendsetters has ofcourse always been Microsoft Visual C++, you can obtain here:
https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&rel=15 

or ex-Borland's version with language extensions to make its UI designer work:

https://www.embarcadero.com/products/cbuilder/starter

if you're willing to get to work with just a compiler, then this one is getting good remarks recently, not only because it's giving usefull error messages: 
https://clang.llvm.org/ 

but there are plenty more:
https://gcc.gnu.org/ 
https://digitalmars.com/download/freecompiler.html 
http://openwatcom.org/download.php 
https://software.intel.com/en-us/c-compilers 
https://www.embarcadero.com/free-tools/ccompiler (or perhaps this one https://edn.embarcadero.com/article/20633 )

(or the list from the creator of C++ itself: http://www.stroustrup.com/compilers.html  )

but C++ is getting some negative remarks lately:
https://duckduckgo.com/?q=why+C%2B%2B+sucks 

so what is recommendable to learn? currently good candidates are:

https://dlang.org/ 
https://golang.org/ 
https://www.rust-lang.org/en-US/ 
https://www.lua.org/ 
https://docs.microsoft.com/en-us/dotnet/csharp/ 

and ofcourse:
https://www.embarcadero.com/products/delphi/starter/free-download 

twitter reddit linkedin facebook google+

How to do timezones?

2018-07-17 23:53  timezones  coding delphi weblog  [permalink]

I feel stupid. I guess I've been put out of my comfort zone. I was thinking about doing something, and then I did, and I thought I'll just primarily make it work for me, but then I noticed "new" items were still two hours behind. Ofcourse that's due to the bias against UTC we have here in my current local timezone. So there you have it. All my programming career, I would just parse dates, and stop before this "+0200" or "-0900" thing that's there at the end. So the right thing to do is go all the way, and with a little effort make it work correctly for not just me, but for any user of this little thing.

But how? I remember these drop-downs with timezones where I typically look-up Brussels or Paris. But it's summer daylight-saving-time for the moment. So it switches to and from only one hour between UTC twice a year (until policy changes), how do I keep track of that? Is there something that I can have the system do this automatically for me? It feels like these are things I should have known all along, but dont. I guess I'll have to start reading somewhere...

twitter reddit linkedin facebook google+

Another store from the trenches: GDI vs PDF

2018-07-06 18:31  gdipdf  coding dagboek delphi werk  [permalink]

One of the great things about Delphi is this object-oriented way of manipulating things called the Visual Component Library. There's much more to it than that, but in a sense you could describe it as a wrapper around Windows API's. One of these is GDI, it's old but it's still there and used for graphics and text-layout. It was born, way back when, from the idea of What You See Is What You Get (WYSIWYG) taken even further and that the same program logic could work for display on screen and writing to a printer. Delphi's VCL declares the TCanvas class you can use to design the output, and wether you pass a Printer.Canvas, Screen.Canvas, Form.Canvas or Bitmap.Canvas shouldn't really matter. This works great for forms and reports.

But in the printing business, there's much more than forms and reports to put on paper. Over there the PDF reigns supreme. It performs a similar task, but originated out of a very different set of parameters. A big module we created of the production software at work generates them according to specifications. How this works is very different as the components that make up a PDF document are constructed one by one and this doesn't look like the work you normally would do with a TCanvas instance.

But now and again these two worlds need to bridge over to eachother. One day I had to keep automatic PDF archive copies of some reports, and discovered the great SynPDF project from the same people that created the mORMot framework. It offers a TCanvas instance (TPdfDocumentGDI.VCLCanvas) that you can pass to code that knows what to with a TCanvas reference.  It works great.

I haven't needed to think about crossing the two the other way round, but recently I noticed a collegue did, and he also found a simple straight-forward solution for the problem: You can just ask the PDF reader on the system to render a PDF to a printer. It's as simple as starting a process with:

"C:\Program Files (x86)\Adobe\Acrobat Reader DC\Reader\AcroRd32.exe" /n /t "<document to print>" "<printer name>"

or

"C:\Program Files (x86)\SumatraPDF\SumatraPDF.exe" -print-to "<printer name>" "<document to print>" 

This works just great and even doesn't add too much code to your program. You may need to include the PDF viewer installer with your installer, but that's a minor issue and perhaps not even required as most people have a PDF reader installed by default.

twitter reddit linkedin facebook google+

Delphi project and subversion: set the build number to revision number

2018-06-09 09:37  dsvnbuild  coding delphi werk  [permalink]

At work, we have a number of interrelated projects in a single subversion repository. We've agreed to change the build number in the project's version properties to the current revision number of your work folder. And specifically not the revision number you might predict your changes will land in when you commit them.

We're currently a rather small team so it may be tempting to assume you'll just get the next revision number, but we stress newcomers to ignore that urge. As long as we're a small team, this works manually, but if we grow we should move over to scripted version resource entries and commit hooks that update the build number automatically.

So with a major and minor version of 2.0 for example, and a release number of 8, the full version could look like "2.0.8.51873".

This way of working has a number of advantages.

The binaries that get used in production, but exert a bug in their behaviour, show with their full version number (and the previous version number from before the bug occurred) the vicinity of revision numbers that introduced the code change that may lay at the cause of the bug. To get the exact revision number, you need to look up the SVN log (or blame) of the .dof or .dproj file, but it's quite sure it's a number closely above the number in the version. We've stressed second line support personnel to list this number when reporting the bug, which helps when researching reproducibility.

But even long before that, when it happens two (or) more of us inadvertently make changes to the same project, we either started from the same revision number in the work folder and see when comitting that something has to get merged and it does so automatically or we get a merge conflict; or we started from different revision numbers and get a conflict right away or a message that we're over due for an update to or work folder.

As I said we're a small team for the moment and it rarely happens so it saves us an update task with risk of conflict before we commit, and in general we can split the work that needs to be done between us so we shouldn't make changes to the same projects.

Then again, if the team would ever grow to something like really big, we would probably have to switch to something else than subversion, or even repositories per project, who knows...

twitter reddit linkedin facebook google+

GMail: make the labels list compacter

2018-06-08 10:32  GMailLabelsCompacter  coding internet werk  [permalink]

If you remember from before, I have so much labels in GMail that I didn't like that the box to change the labels on a message with, is so small. Stylus to the rescue.

Now there's this new GMail design, and even in compact display, the list of labels on the left doesn' fit my screen. Also I don't like the font the subject line is rendered in. So a bit of inspection later, I add these lines to my overrides:

.z0 {
margin: 0px;
height: 32px;
padding: 0px 0px 0px 64px;
}
.z0>.L3{
height:24px;
}
.ha>.hP {
font-family: "PT Sans", sans-serif;
}
.aim {
height: 18px;
}
.J-N {
padding: 0px 12px 0px 32px;
}
.J-LC
{
padding: 0px 12px 0px 48px;
}

twitter reddit linkedin facebook google+

AES v1.0.1

2018-05-21 20:18  aes101  coding delphi freeware  [permalink]

md5

I noticed something was wrong with the key generation schedule in my AES implementation. I had a close look with the FIPS 197 example vectors at hand and fixed it. I still should make some time to run it under pressure and see how it relates to reference implementations and popular implementations.

twitter reddit linkedin facebook google+

HTTP+HTML+Delphi authentication (how xxm does it)

2018-04-13 14:26  xxmauth  coding delphi internet freeware  [permalink]

Daraja Framework: HTTP+HTML form-based authentication

Jikes! This is strange. Yes you could go ahead and have a page with a login-form, that posts onto a handler that checks your password, and throws a 401 when it fails. But is that really what you need? I thought 401 is there to elicit the user's HTML-client (a.k.a. browser) to show a modal form asking for a password before re-posting the request. Just like xxm's Basic Authentication demo does, and it does this right at the center of the project, before your request is routed to any page or resource, so that all requests to the project need authentication. Also this way you don't need to code a check IsAuthenticated on every page or resource.

But — again — is this really what you need? The public nowadays doesn't respond well to systematic authentication like that, and also makes it impossible to do anything on the website while not being authenticated (yet). It's better form to welcome new users with a nice 'create new account' button (More about that here.) and perhaps more information on what's on offer, next to the logon form for existing users (with extra options like 'stay logged on on this station' and a 'forgot my password' link). There's an example in xxm's Session demo: The opening page has a log-on form, and Login.xxm does the rest. It doesn't really check user-account and password here as it exceeds the purpose of the demo.

To show you a working demo, you should have a look at tx: It has a central redirect for any page request from a user that should authenticate first; the logon-form with extra options to show users as a normal web-page; checks the entered password agains a properly salted hash and then redirects you to the page you came in for originally...

And there's much more to tell about authenticating users. I've tried to make a list here (it's in Dutch though), and that doesn't even scratch OAuth(2) yet...

Before I forget, did I mention xxm comes under a permissive MIT license? So you don't need to buy a commercial license!

twitter reddit linkedin facebook google+

Do I also need a four-letter-acronym to be cool these days?

2018-03-30 22:47  xx4la  coding delphi internet  [permalink]

→ Reddit: Any drawback to using Wordpress in front of a MERN application?

MERN?! What's that?

MERNMongoDB Express React node.js + Redux WebPack

Oh, I get it! It's one of those four-letter-acronyms that describes your software stack. The first one, and as it happens also the one I started on was:

LAMP: Linux Apache MySQL PHP

But trying things out on my own, I didn't get a hang of that Linux bit. I still blame the folks that sneered me off with "start with typing man man at the prompt". So I got stuck being a

WIMP: Windows IIS MySQL PHP

but later regained my poise and sting with

WASP: Windows ASP SQL Server PHP

which worked great for a while, but I moved on. Not quite with the hot and happening new one:

MEAN: MongoDB Express Angular node.js

but closer related to other desktop application work I was doing in Delphi. Having done some raw networking, and some raw HTTP, but also the IIS APIand implemented Internet Explorer's IInternetProtocoland FireFox' nsIHttpChannel (before they chucked XPCOM somewhere after version 3.6 and starting the rapid release schedule), and something something HTTP.SYS, I decided to start something to model all the common bits into one single interface so you could easily switch between implementations and environments. And hot-swap a binary without taking down the webserver/webservice. And do that after an automatic compile when you changed a file and refreshed your browser. And have a mix of HTML and server-side logic into the same files like PHP and ASP (and Cold Fusion...) And still have full response streaming, and not a big hard templating thing churning on a request first before being able to spew out the response in one go... And have a few of the basic things in place to help you with security to prevent malicious requests.

So I created xxm. And websites with it. Such as tx. So I guess I should invent suitable fout-letter-acronyms as well, then:

XIMR: xxm IIS MongoDB (over TMongoWire!)  Redis

XXJP:  xxm xxmHttpAU jQueryUI PostgreSQL

XESVxxm nginX (over SGI) SQLite Vue.js

Hmm, doesn't really sound all that great... Never mind then. I'll just enjoy it if xxm could serve as a solution for anybody in the very small niche of people that do both high-level server-side stuff with Delphi, and high-level dynamic-web-stuff, and want the two closely knit together...

twitter reddit linkedin facebook google+

Best practices for user account management

2018-02-27 11:17  i3036bis  coding internet  [permalink]

Google Cloud Platform Blog: 12 best practices for user account, authorization and password management

Bon, ik moet dringend de lijst die ik hier opgesteld had nog eens bijwerken met de hedendaagse methodieken...

twitter reddit linkedin facebook google+

A thin wrapper around SSPI SChannel.

2017-12-30 22:57  schannel1  coding delphi  [permalink]

I thought, if you already have something that does work over a network socket, can you have it encrypted of a TLS connection? If you search, you van find a lot, mainly using OpenSSL. If you read on a bit, you learn about LibreSSL, but if I understood correctly, Indy can't use that since it needs specially patched DLL's, that are stuck on some old version sadly enough...

But, I always keep searching for the thinnest possible wrapper. If there's a way to carry less bloat, or use an even thinner library, then yes please.

So I thought, Windows by itself, or at least some Microsoft things, make calls to the outside world over a TLS-line from time to time. So there has to be a DLL that does all the work for those. It would be strange if it exists, but it's not opened up. Some more searching leads to the realisation it's this SSPI thing that keeps turning up. There's a thing called SChannel you apparently need, but it's not as easy as just replacing your connect/recv/send calls...

Once there was a time when Microsoft wasn't quite planning to keep the 'network subsystem' to themselves, leaving the option open to get some from a different supplier. (Once there was a time it wasn't the matter of course that networking plays over TCP/IP/Ethernet, but that's another story altogether.) You still see that in the SSPI story. You're supposed to call a central function first to see what's available (by which vendor). Once you've tracked down the DLL you need (secur32.dll), you see it just has all you need right there in the exports table (and even just patches them on through to sspicli.dll, at least since some recent Windows version). So in the hope to have some simplification, I think I have now a basic minimal wrapper around the required calls to make it work, added to the minimal things I had in there to talk to WinSock2:

github TMongoWire commit d2c99a...

I hope this performs as expected, as I still have to put it through some more testing in different conditions, but that'll be for another day. And as the current season would have it, perhaps for another year. If you find anything, feel free to launch github issues or pull requests. Happy New Year.

 

twitter reddit linkedin facebook google+

DirDiff v2.0.3.512

2017-10-27 00:19  DirDiff512  coding freeware  [permalink]

DirDiff v2.0.3.512

Fixed issue with UTF-8 sensitive characters in ANSI file.
Fixed issue with Ctrl+Shift+Up/Down past start/end of files list.
Enable switching checkboxes on tree view with space key press.

twitter reddit linkedin facebook google+

Momoa

2017-09-26 10:39  momoa  coding  [permalink]

We've had XML. We've had JSON. There's a thing called YAML. And then there's Protocol Buffers and Thrift and a number of others.

And still, with each there's something is not quite right. So here is yet another proposition, humbly offered for adoption:

Binary. Why binary? Parsing speed matters. There's a belief that binary is not human readable, but:

ASCII control codes. Why ASCII control codes? They're out of use. Except 0x0A (and 0x0D) for new lines and 0x09 for tabs. I've come across a 0x0C and 0x1B when talking to printers, but that's it. And all modern editors know what to do with them. Best case may even be they show them as something foreign, but still they're visibliy right there with the other text.

A list of keys and values. It's tempting to provide structure and clearly indicate which is what, but it's unneccessary. A parser is smart enough to know these come two by two, and to pair them up when handing over to something for processing.

Types of values. A value has a preceding byte denoting what it is, and what rule to follow for the succeeding bytes.

0x02 string: read the string up to the next type byte. If a type byte needs to be actually part of the string, escape it with 0x07

0x03 number:  read a string up to the next type byte and convert it from text notation to something numeric. Depending on the context it may be something specific or variadic. By using the text notation we retain some human readability, and also get an acceptable storage to information ratio (smaller numbers take less bytes)

0x05 boolean true: with nothing more

0x06 boolean false: same as above but with opposite value

0x01 embedded key-value list: treat the following sequence of key-value pairs, delimited by a 0x04 closing type byte, as an embedded list

0x08 array: treat the following as a sequence of values only, delimited by a 0x04 closing type byte

There's no specific type byte assigned to null or undefined, but can be encoded as a single 0x03 without data following it.

Keys are themselves values, typically of type string (0x02). A possible permissible exception in specific contexts may be to encode sparse arrays as an embedded list (0x01) where all keys are of type number (0x03).

And now for a name for it... I know, let's type Jason into IMDB... Sounds nice, and serves as a tribute to the artist. So let the file extension be ".momo" and the MIME type be "application/momoa"

twitter reddit linkedin facebook google+

Why I choose Delphi

2017-08-15 22:49  whydelphi  coding dagboek delphi  [permalink]

Strange, all these Why I choose Delphi articles lately:

Keep them coming! It's good to see it stressed that it's really a myth that there's not enough Delphi talent out there. Rember, Delphi's debugger by itself is so strong, a decent developer should be able to learn both Delphi and an existing code-base that works just by stepping through the code with the debugger and see what it does. Yes, the language is a little verbose; yes, it's perhaps even older than C/C++; but remember, so is COBOL, and I would almost say that's not cross-platform or a systems language, but those just had other meanings back in the days. (Did you know there's Delphi for AS400?)

So, why do I stick with Delphi? The answer is pretty straight-forward (and perhaps a little sad): give me any kind of computing problem, and I'll find a way to tackle it with Delphi. I've done so much different things, and still found a way to get a great system I enjoy working on, and still have a Delphi project that its compiler happily churns into a binary executable that performs really well. Yes, you could do that also in C/C++/Rust/D/Java/(etc.) but I can't, and don't really want to. There are always up-sides and down-sides,  but it feels like with Delphi you don't meet much of any down-sides, and if you do some-one else knows something to do about it.

twitter reddit linkedin facebook google+

Idea: assembly that flags when to release virtual registers

2017-05-31 23:52  asmvrr  coding computers  [permalink]

I just had a fragment of an idea. I want to write it down, just to let it go for now as I've got other things to do, and to be sure I can pick it up later exactly where I left off.

First situating what it's about: I have been reading up on WebAssembly, and to my surprise the intermediate representation is stack based (just like Java's JVM and .Net's CIL). I'm not sure why because it feels to me this makes registry assigning when constructing the effective platform-dependent instructions harder, but I may be wrong. Finding out objectively is a project on it's own, but sits on the pile 'lots of work, little gain'.

I also went through the great set of MIT 6.004 lectures by Chris Terman which really gives you a good view of 'the other side' of real assembly since it's actually born out of designing these processing units built out of silicon circuits. It prompted me to make this play thing, but again pushing that through with a real binary encoding of the instructions made it a 'lots of work, little gain' project, and I really don't have access to any kind of community that routinely handles circuit design, so it stalled there.

Before that, I read something about hyper-threading, and what it's really about. It turns out modern CPU cores actually handle two streams of incoming instructions, have a set of instruction decoding logic for each stream (and perhaps branch prediction), but share a lot of the other stuff, like the L1 cache,  and especially a set of virtual registers that the logical registers the instruction stream thinks it's using is mapped on to. Mapping used registers freely over physical slots makes sense when you're making two (or more?) streams of instructions work, but it's important to know when the value in the register is no longer needed. Also if the register is only needed for just a few instructions, pipelining comes in to play and could speed up processing a great deal. But for now the CPU has to guess about all this.

When playing around with a virtual machine of my own, I instinctively made the stack grow up, since you request just another block of memory, plenty of those, and start filling it from index 0. It shows I haven't really done much effective assembler myself, as most systems have the stack grow down. What's everybody seems to have forgotten is that this is an ugly trick from old days, where you would have (very!) limited memory and use (end of) the same block for the stack, and with more work going on the stack could potentially grow into your data, or even worse your code, producing garbled output or even crashing the system. (Pac-man kill screen comes to mind, although that's technically a range overflow.) Modern systems still have stack growing down, but virtually allocate a bit of the address-space at the start of that stack-data-block to invalid memory, so stack-overflows cause a hardware exception and have the system intervene. It's a great trick for operating system (and compilers alike) to have checks and balances happen at zero cost to performance.

The consensus nowadays is that nobody writes assembler any more. It's important to know about it, it's important to have access to it, but there is so much of it, it's best left to compilers to write it for you. In the best case it may find optimizations for you you didn't even think about yourself. But this works both ways. Someone writes the compiler(s), and need to teach it about all the possible optimizations. I can imagine the CPU's instruction set manual comes in handy, but that's written by someone also, right? I hope these people talk to eachother. Somewhere. Someday. But I guess they do as with x86-64 they've kind of agreed on a single ABI... and they've also added some registers. Knowing about the virtual registers allocation going on behind the scenes, it could be that that was just raising an arbitrarily imposed limit.

So this is where I noticed a gap. When performing all kinds of optimizations and static analysis on the code when compiling, and especially with register allocation, it's already known when a register's value is no longer relevant to future instructions. What if the compiler could encode this into the instruction bits? If I were ever to pick up where I left, and have a try at a binary encoding for a hypothetical processing core, the instruction set would have bits flagging when the data in registers becomes obsolete. Since this would be a new instruction set, and I guess it's more common to need the value in a register only once, I might make it the default that a value in a register becomes obsolete by default, and you'd use a suffix in assembler to denote you want to use the value for something extra later as well.

twitter reddit linkedin facebook google+

Delphi, the shrinking island.

2017-05-09 21:48  delphitrend  coding delphi  [permalink]

Is there a language that has a single word for 'the feeling of being on a shrinking island'? Anyway, this is somewhat sad to see, especially that the few most recent Delphi versions gave the impression there was a new uptake with more people getting persuaded, but it doesn't show in the curve.

(via)

twitter reddit linkedin facebook google+

'Relax' scripting vs xxm

2017-04-22 13:12  relaxxxm  coding delphi internet  [permalink]

Delphi Relax Web Scripting (Marco Tech Blog)

I'm sorry but I feel I must react. In general I keep silent, in the hope people by themselves will know better, but as I'm getting no input what-so-ever that that is the case, I feel tempted to write something about this.

First about what's at hand. I see this bit of code:

<h2>Employees</h2>
<ul>
@foreach (var emp in employee) {
  <li>@emp.FirstName @emp.LastName (@emp.PhoneExt)</li>
}
</ul>

and it looks kind-of OK. To the untrained eye it looks good and may even look tempting to write more in this syntax. This is a straight-forward example of a template that works with a templating engine that no doubt has many more capabilities and features. And then I thought, learned from practice, what I typically would get asked is to not show " ()" when the PhoneExt field is empty. I would not know how to make that happen in that template syntax. That's mainly because I know nothing about the template syntax. If I look into the documentation, I might find an @if predicate to make it happen, but let's move on:

This is what it could look like in xxm:

[[!var emp:TEmployee;
<<h2>Employees</h2>
<ul>>
foreach emp in FDMemTable1 do
begin
<<li>>=[emp.FirstName,' ',emp.LastName,' (',emp.PhoneExt,')']<</li>>
end;
<</ul>

Looks roughly simlar. A little more like Delphi syntax. And in fact it is. If you know [[, ]], << and >> get translated into Context.SendHTML() and Context.Send() calls behind the scenes (full details are here),  you know this code will result in the same output. Without templating engine! Streamed to the user's client! Perhaps even while the data is streaming in from the database server, in case it's a longer list, and in case there is a database server, Marco uses a memory-table for his example.

What I find important is that there's less going on between the native compiled logic and getting the data to the user launching a request. Not only a templating engine looks superfluous, this entire ORM thing is something I don't get. If it's a gigantic database model with so much tables that you clearly benefit from code-completion, then I agree, but I haven't come across something remotely close to that in web projects.

Also the HTTP-server itself is something I think that values extra attention. I've seen platforms and frameworks that offer you a wealth of capabilities and features, but hastily slapped on something that listens on TCP port for basic HTTP requests, in some cases on port 80, but more often 8080 or something else in the thousands. In real web environments, the server(s) has/have a lot more going on: load-balancing, reverse proxies, firewalling, authentication. Since we're in a Post-Snowden-era nowadays, we're all responsible to think about protecting privacy and get that HTTPS in order with the proper certification and encryption... Not to mention HTTP version 2 that's heading full steam towards being generally accepted/expected.

I can image the web-admin responsible for all that, isn't happy with your request to add this newfangled separate thing that's doing its own handling of HTTP requests. ISAPI DLL's or Apache modules play much nicer with existing IIS or Apache installations. (FastCGI is on the table, but for now xxm has SCGI available for other servers.) Even if your 'Delphi HTTP framework' of choice is specifically designed to tap your ORM of choice and offer a REST-API for your data-layer needs, it will still be one more stop along the way between the user's browser, and the delivery-setup, and the front-end, and the page-template, and the data-layer, and the database, and what the user actually needs or wants. I think of this in the postal office when there's twelve people in the queue in front of me.

I don't expect to convince much people of this way of working, but it works great for me. I remember the days with early PHP and ASP and how simple and straight-forward everything was. Knowing these work on scripting engines, I kept worrying about lost performance. This was the core reason to start xxm: employ the speed and power of the Delphi compiler to have a native library serve my websites. And it turns out that Delphi code looks quite nice between HTML to handle server-side logic, if I may say so. It took me a few years to make this happen, but I couldn't do without it any more. And people kind-of appreciate that for using this new application, all they need is their trusted browser and a URL.

twitter reddit linkedin facebook google+

HTML: label, no more "for" for me!

2017-04-13 22:44  htmllabelfor  coding internet werk  [permalink]

If only I had known sooner! I forgot where I picked this up, but apparently if you put <label> around an <input>, typically of type checkbox or radio, browsers automatically know the label is for that control. Before, I would write my <input id="x"> first, then a <label for="x"> after. To keep code neat, I would put it on a separate line, but the EOL inbetween would not be clickable to actuate the control. This is a really minor issue, but still. Now that I know you can just write this:

<label><input type="checkbox" name="Toggle1" value="1" checked="1" /> Toggle1: clicking text after a checkbox should toggle the checkbox!</label>

Because, there are two kinds of people: those that click the box to switch a checkbox, and those that click the text right of the checbox. You might not even know that you do, but you do don't you. If you're of the latter type, it's just one of those minor frustrations, that a click on the text-label sometimes doesn't do what you expect, and you have to:

  1. first pick up that's this that's going on, possible because you've selected a bit of the text
  2. align your eyes to the checkbox
  3. align the mouse-cursor over the checkbox
  4. click the checkbox, confirming the previous one or more click are actually wasted

But there you have it. Heaven has great UX. Here we need to make do with what we get. (And need to make sure it's the way we like it for those bits that we have control over.)

twitter reddit linkedin facebook google+

 

Archive... Search...