2018 ...

januari februari (2) maart (2) april (2) mei (2) juni (3) juli (4) augustus (3) september (3) oktober (2) november (3) december (3)

TOML? YAML? ini-files? JSON? Loosen up!

2018-12-10 11:26  jsonloose      [permalink]

TOML? YAML? plain old INI-files? or shiny and new — but strict — JSON? Nàh. I've made myself something better. In jsonDoc.pas I've recently added an optional compiler define JSONDOC_JSON_LOOSE that makes the JSON parser a whole lot genter to work with. Some things are no longer required: the root document doesn't need to start and end with braces ("{}"), the quotes around the key names are no longer required (unless they contain special chars), the comma's between key-value pairs are no longer required, it accepts equal ("=") between keys and values instead of colon (":") and can even do entirely without.

And while I was at it there's also JSONDOC_JSON_PASCAL_STRINGS that allows you to write string literals the Delphi way: between single quotes, and double single quotes where you want a single quote in the string value. This way — really handy in Windows environments — it doesn't require you to escape backslashes.

The only downside maybe is that you need to add the defines to the compiler parameters, but for plain old configuration something like this:

x=5
y=10
output='C:\test\output\'

looks a lot nicer and like what we're used to, and gets parsed just the same as if it was written like this:

{
  "x":5,
  "y":10,
  "output":"C:\\test\\output\\"
}

→

Update to SQLite 3.26 as soon as possible

2018-12-24 14:52  sqlite326      [permalink]

It's very rare something turns up in the SQLite package, but when it does is best to give it some attention. So, very esteemed users of TSQLite — and anyone else, really — I strongly suggest you upgrade to version 3.26 of sqlite3.dll since recently some dangerous vulnerabilities have been uncovered. I checked, and nothing extensive changed to the API so chances are nothing should break if you only update sqlite3.dll and not SQLite.pas and SQLiteData.pas. Also, happy Christmas!

→

NoSQL agent with SQL back-end(s)

2018-12-29 15:41  nosqlidea      [permalink]

It happened again. I get this great idea that slowly develops, and gives the feeling I'm on to something, but nowhere near any time in the forseeable future to put in to it and get a proof-of-concept of a first project that makes it work. So, for what it's worth, I write it down here fast in the hope sharing it with you may give more chance to this idea getting useful.

I've read that some NoSQL solutions are actually about eventual consistency, meaning in the best case of a query for data that was just inserted or updated could already return this new data if the server(farm) ad already fully processed it. Worst case is that it just for a few milliseconds totally disappears, but that's another story altogether.

I haven't done anything serious with NoSQL yet, and really a lot on good old SQL, and recently with SQLite which I've really grown to love in a short period of time. But still there's something there that's really suited for the new style of programming that is going on with all the new web projects and this 'Internet of Things' everything is on about... To find out, I've been trying interfacing with a number of them from Delphi in the most direct way I could possibly find and make work with reasonable effort. I like how TMongoWire worked out, to talk with PostgreSQL all you need is in the libPQ.dll, but a number of others just stick with a plain HTTP API where you PUT and GET things on their own URL. There's a beauty to that, really. The structure of your documents is nicely contained in JSON, and HTTP is such a stable platform you're sure to be able to access it from almost any platform.

So that's where the idea came from: what if I made my own service where you can just put or get JSON documents? But on the back-end jsonDoc would do the heavy lifting and the storage itself could be in a decent SQL service. And/or it could be in something intermediate like memcached. And/or the saving to storage could be asynchronous somewhere close after the actual PUT call (hence the eventual consistency).

For example, you fill a collection of items with things with a number of fields, for example one is "Price", but later you need the items above or below a certain price, you would do SQL "select * from Items where Price<@p". So in this service I'm imagining, there would be meta-description on the collection that you've provided a SQL database somewhere, but the service is responsible for having done the "create table Items (ID some primary key, DATA json, Price decimal(8,2))" and filling it with the data.

And this would be the beauty of it: if you need an extra column later, you just say so, and the connector would be responsible for the "alter table Items add ..." and filling that column with the data from the stored items. Perhaps even slowly, asynchronously together with the other work. Or even another connector alltogether, let's say PostgreSQL and MySQL side by side, perhaps even as a fail-over for eachother.

But I'm dreaming. It would be a load of work just to get something to work, and even more work to get enough connectors to work good enough to even demonstrate how it would work. And then there's the performance trails... And the evangelising to see wether it solves other people's problems anyway... It would be a really great opportinity to finally cut my teeth on this IOCP thing.

→