Freeware:

                |

AES v1.0.1

2018-05-21 20:18  aes101      [permalink]

→ md5

I noticed something was wrong with the key generation schedule in my AES implementation. I had a close look with the FIPS 197 example vectors at hand and fixed it. I still should make some time to run it under pressure and see how it relates to reference implementations and popular implementations.

xxm v1.2.5.450

2018-05-03 22:43  xxm450     [permalink]

For a while I thought there's not enough there to have a full release, but with the switch from XML to JSON, and NTLM and WebSocket support in xxmHttp and xxmHSys2, plus all the minor changes and tweaks that improve performance and security, it turns out to be quite a formidable release. Regretfully, by finally deciding to have all string arguments const anyway, any project's xxmp.pas will have to change. A long time I was hesitant about this because I feared binary incompatibilities, but I've tested all permutations between caller/called with/without const, and it all works. Except when building old code with the new xxm.pas will serve you with this minor one-time inconvenience of having to add const yourself to the arguments... I've thought about trying to create something automatic — like the XML to JSON converters — but since you're quite free to modify xxmp.pas to your liking, it may be more complex than it seems. (If you would like to see this anyway, let me know and I'll put some time into that.)

It's advised to do an extensive test-run with this new version before you update the live environment. If anything unexpected turns up, let me know!

→ xxm v1.2.5.450 (→gh) (→sf)

• project registry "xxm.json" (auto-conversion of "xxm.xml")
• project file "Web.xxmp" JSON (auto-conversion from XML)
  
  ### ATTENTION ### ATTENTION ### ATTENTION ###
  xxmProject and xxm handlers are fitted with a transitional
  automatic conversion procedure that will convert XML project files (Web.xxmp)
  and the project registry (xxm.xml) into JSON files. (Web.xxmp in place,
  storing a Web_xxmp.bak copy; xxm.json)
  This may cause incompatibilities if different xxm versions are required to
  coexist on the same installation(s).
  This procedure (and all code using XML) is scheduled to be removed at a
  later, currently undetermined, version.
• fixed issue with custom parserValues spilling into next file processed
• IXxm* interfaces: all string arguments now with 'const'
  
  ### ATTENTION ### ATTENTION ### ATTENTION ###
  This may cause errors on generated xxmp.pas:
  xxmp.pas() Error: Declaration of 'LoadPage' differs from previous declaration
xxmp.pas() Error: Declaration of 'LoadFragment' differs from previous declaration
  Please update both methods by adding 'const' to the arguments with a string type.
• xxmIsapi*: Redirect: set 'Cache-control' to 'nocache' unless it was set already
• xxmHttpAU: added manifest to avoid default UAC elevation request
• xxmHttp,xxmHSys: Transfer-Encoding chunked (set Context.BufferSize to something >0 !!!)
• xxmHttp,xxmHSys: NTLM authentication (using SSPI, add "ntlm":true to the project in xxm.json)
• xxmHSys2: WebSocket support (see demo's: 13 WebSockets)
• DispositionAttach: replace unacceptable characters (\/:*?"<>|) with '_'
• SendFile, SendStream before header sent, will by default have ContentType
  application/octet-stream instead of text/html, unless already set before
  (with Context.ContentType!)

What do you think I should do?

2018-04-20 23:26  sendfile404     [permalink]

Dilemma, dilemma! What should I do? Sometimes, you need to have a certain URL in a web-project that just serves op the contents of a local file. You could try to have the file in a static section of the website, or even a sub-domain for static content (if your budget allows it), but then it's out there for anyone. Best is to have a request be processed dynamically, so you can do some server-side logic first, for example to do authentication control, or generate the file first (perhaps with a graphics library?). That's where the SendFile method of the IXxmContext comes in. The code that implements it is pretty straight-forward: 

procedure TXxmGeneralContext.SendFile(const FilePath: WideString);
begin
  inherited;
  if State=ctHeaderNotSent then FSingleFileSent:=FilePath;
  SendStream(TStreamAdapter.Create(TFileStream.Create(FilePath,fmOpenRead or fmShareDenyNone),soOwned));
end;

If the request's header has not been sent already, the private value FSingleFileSent is set to the file path, assuming the request is meant to have this file's data as response data. This is used for error handling and logging.

Then SendStream is called, which takes an IStream pointer, so a TStreamAdapter is used to wrap around a TFileStream, with ownership so the last IStream._Release will call the file stream's destructor.

One thing that's missing here perhaps is trying to figure out if the HTTP response header Content-Type is set to some suitable MIME-type for the file, but this is so untrivial that it's best left to the developer. So give it a second of thought when you're coding a call to SendFile.

So about this dilemma I'm having. Just imagine for a second you have this code in a project. Attention: this is very bad practice! It should be glaringly obvious to everybody that doing this opens the back door wide open and people with malicious intent can access any file they want on the machine, include system files, so please don't ever really do this:

[[Context.SendFile(Context['f'].Value);]]

Yikes. Very very bad! I feel dirty just for typing that, but just as an example, this code has a high probability of trying to open a file that doesn't exist, or otherwise have the TFileStream.Create throw an EFOpenError exception.

In that case, would it be better if xxm answers with a proper HTTP 404 (page not found) response? Now default exception handling kicks in, and left unhandled (hint) xxm will fashion a HTTP 500 response for you with the exception data. There's also a bit that will see if your local fragment registry can load a fragment for '404.xxm' that lets you design a nicer 'page not found' page than the default, but I'm having a hard time to guess if that would be something unexpected for someone somewhat new to xxm, calling SendFile on an inexistant file for the first time...

I'd love to hear from anyone on this, but for now I'll just let it be like this and let the normal exception play out, if any.

But wait, there's more. Deep burried within the dark corners of the Windows API, there's a thing called TransferFile. It basically lets you tell the system to take a file handle and a network handle and stream all the data from the one to the other, as much as possible right from the kernel. The way the friendly people over at Microsoft worked it out, and tied it to the running system so it would only work on Windows Server versions, makes it kind of unsuitable for where I want to take the xxm project.

But wait, there's more. You may have noticed the web-sphere is gripped with a frenzy for all things asynchronous. There are a few good things there, but it is mainly the best way to serve a magnitude more of concurrent requests by the same server. The short story is you try to avoid waiting on the system while it waits on network or disk. The long story envolves completion ports or libuv, but is in essence unfit to combine with what xxm is doing: having a separate DLL with code you just call to have a response generated for a request, since to do it properly every request to the operating system needs to be re-routed over your job/thread/task/fiber/yarn-management.

This shouldn't hold me back to get as close as possible to the middle ground between the two, where you have the option to build a response, but can hand over a (file)stream and have the HTTP server spool that as it sees fit,  once the situation really is just that straightforward. But this may be what xxm 2.0 could be about, if there ever will be something like that.

HTTP+HTML+Delphi authentication (how xxm does it)

2018-04-13 14:26  xxmauth       [permalink]

→ Daraja Framework: HTTP+HTML form-based authentication

Jikes! This is strange. Yes you could go ahead and have a page with a login-form, that posts onto a handler that checks your password, and throws a 401 when it fails. But is that really what you need? I thought 401 is there to elicit the user's HTML-client (a.k.a. browser) to show a modal form asking for a password before re-posting the request. Just like xxm's Basic Authentication demo does, and it does this right at the center of the project, before your request is routed to any page or resource, so that all requests to the project need authentication. Also this way you don't need to code a check IsAuthenticated on every page or resource.

But — again — is this really what you need? The public nowadays doesn't respond well to systematic authentication like that, and also makes it impossible to do anything on the website while not being authenticated (yet). It's better form to welcome new users with a nice 'create new account' button (More about that here.) and perhaps more information on what's on offer, next to the logon form for existing users (with extra options like 'stay logged on on this station' and a 'forgot my password' link). There's an example in xxm's Session demo: The opening page has a log-on form, and Login.xxm does the rest. It doesn't really check user-account and password here as it exceeds the purpose of the demo.

To show you a working demo, you should have a look at tx: It has a central redirect for any page request from a user that should authenticate first; the logon-form with extra options to show users as a normal web-page; checks the entered password agains a properly salted hash and then redirects you to the page you came in for originally...

And there's much more to tell about authenticating users. I've tried to make a list here (it's in Dutch though), and that doesn't even scratch OAuth(2) yet...

Before I forget, did I mention xxm comes under a permissive MIT license? So you don't need to buy a commercial license!

Do I also need a four-letter-acronym to be cool these days?

2018-03-30 22:47  xx4la      [permalink]

→ Reddit: Any drawback to using Wordpress in front of a MERN application?

MERN?! What's that?

MERN: MongoDB Express React node.js + Redux WebPack

Oh, I get it! It's one of those four-letter-acronyms that describes your software stack. The first one, and as it happens also the one I started on was:

LAMP: Linux Apache MySQL PHP

But trying things out on my own, I didn't get a hang of that Linux bit. I still blame the folks that sneered me off with "start with typing man man at the prompt". So I got stuck being a

WIMP: Windows IIS MySQL PHP

but later regained my poise and sting with

WASP: Windows ASP SQL Server PHP

which worked great for a while, but I moved on. Not quite with the hot and happening new one:

MEAN: MongoDB Express Angular node.js

but closer related to other desktop application work I was doing in Delphi. Having done some raw networking, and some raw HTTP, but also the IIS API, and implemented Internet Explorer's IInternetProtocol, and FireFox' nsIHttpChannel (before they chucked XPCOM somewhere after version 3.6 and starting the rapid release schedule), and something something HTTP.SYS, I decided to start something to model all the common bits into one single interface so you could easily switch between implementations and environments. And hot-swap a binary without taking down the webserver/webservice. And do that after an automatic compile when you changed a file and refreshed your browser. And have a mix of HTML and server-side logic into the same files like PHP and ASP (and Cold Fusion...) And still have full response streaming, and not a big hard templating thing churning on a request first before being able to spew out the response in one go... And have a few of the basic things in place to help you with security to prevent malicious requests.

So I created xxm. And websites with it. Such as tx. So I guess I should invent suitable fout-letter-acronyms as well, then:

XIMR: xxm IIS MongoDB (over TMongoWire!)  Redis

XXJP:  xxm xxmHttpAU jQueryUI PostgreSQL

XESV: xxm nginX (over SGI) SQLite Vue.js

Hmm, doesn't really sound all that great... Never mind then. I'll just enjoy it if xxm could serve as a solution for anybody in the very small niche of people that do both high-level server-side stuff with Delphi, and high-level dynamic-web-stuff, and want the two closely knit together...

Gedaan met diesels?

2018-03-01 22:07  nmdiesels       [permalink]

→ VRT NWS: Gaan we onze afgedankte diesels naar Afrika sturen?

Zie, ik moet echt precies op het juiste moment om een benzine-wagen gevraagd hebben... Eind vorig jaar was het tijd voor een nieuwe lease-contract op het werk, dus dacht ik vragen staat vrij. Ik doe op vier jaar ocharme iets van een enkele omtrek om de aarde, dus geeft me asjeblief toch een wagen op benzine. Al was het omdat het inderdaad te voorspellen was dat dit jaar de dieselprijs (eindelijk!) de benzine-prijs zou inhalen. Daarnaast is er nog het kleine netelige feitje dat de wasem uit de pijp bij een dieselwagen blijkbaar behoorlijk wat smeerlapperij bevat. Ik zou niet zover gaan om dieselrijders te veroordelen voor deelname aan moord, maar jammer genoeg is er wel een relatie vast te stellen tussen luchtvervuiling en de sterftestatistieken.

Tot nu toe beklaag ik het me niet. Wel weet ik nu iets wat ik graag iets vroeger had geweten: Met een gemiddelde van 5,5 liter per honderd kilometer voor een diesel, en — voorlopig toch nog, dit nieuwe rijtuig moet zijn eerste 30 megameter nog volmaken — 7,5 bij een benzine-motor, is het omkeren van het verschil tussen diesel- en benzineprijs op slag verworden tot een psychologische grens. Wat niet wil zeggen dat heel misschien ik nog aan mijn gemiddelde rijstijl kan sleutelen om meer dan een grote 600 kilometer per tankbeurt er uit te krijgen.

Een heel klein iets waar ik mee zit, en ik niet direct weet aan wie ik het zou kunnen vragen is het volgende: Ik merk een klein verschilletje in gedrag van de aansturing bij het koppelen. Bij het versnellen en schakelen in de eerste drie versnellingen, lijkt de motor net even naar adem te happen voor het koppel zich aandient. Ik heb al geprobeerd of ik misschien het aankoppelpunt verkeerd inschat, of ik iets meer of net minder het gaspedaal moet geven net voor het koppelen, maar ik vond nog niet wat best werkt. Ik weet ook niet of het vanzelf zou verdwijnen als de motor wat is ingewerkt, of ik het spontaan onbewust leer vermijden, maar voorlopig is dat zo wat het enige wat me opvalt aan de wagen. Dat en dat ik goed het hoofd er bij hou om van die groene tuiten te nemen bij het tanken, niet meer die dikke zwarte. Zelfs die vind ik beter ruiken dan vroeger...

Best practices for user account management

2018-02-27 11:17  i3036bis     [permalink]

→ Google Cloud Platform Blog: 12 best practices for user account, authorization and password management

Bon, ik moet dringend de lijst die ik hier opgesteld had nog eens bijwerken met de hedendaagse methodieken...

Alles op krediet in Amerika!

2018-02-08 20:55  amcred      [permalink]

→ Talks at Google: Frank Abagnale: "Catch Me If You Can" @43:15

Dat het zo erg is wist ik niet. Ten eerste, heel de video is zeker de moeite. De mens van de film "Catch Me If You Can" vertelt zelf wat er eigenlijk is gebeurd. Mensen vinden zo in de doorsnee geniaal wat hij gedaan heeft, maar hij zag het vooral als een vorm van overleven. De elementen brachten hem door samenloop van omstandigheden ertoe om te doen wat hij deed, en een deel was onethisch en zelfs crimineel, maar hij heeft zijn straf uitgezeten. Zo laat hij het toch uitschijnen, want wat zeker is is dat hij slim is. Wat vast staat is dat hij echt geslaagd is aan de balie. En hij met recht en rede intussen jarenlang ervaring heeft uitgebouwd in financiële fraude.

Dus dat precies hij uitlegt dat hij alles op krediet aankoopt — en zelfs kredietkaarten voor zijn zoons voorzag vanaf het kon — lijkt wel heel erg vreemd voor deze gemiddelde Europeaan. Blijkbaar zitten de Amerikanen met iets als de "credit-score" en kom je in problemen als die te laag is. Wat ben ik blij dat wij dat niet hebben. Als ik het juist begrijp is het in Europa minder de gewoonte om veel op krediet te kopen, toch in België, maar Amerika zit blijkbaar vast helemaal aan de overkant van het spectrum. Daar is het gebruikelijk om je diep in de schulden te gooien als je beslist verder te studeren. Heel erg vreemd.

Waar zit het verschil dan? Met de recente grootschalige privacy-inbreuken die ze daar hebben (hoewel we er intussen ook bij ons enkele kennen!) zijn nogal wat bank- en rijbewijs-gegevens van modale mensen in slechte handen gevallen. En blijkbaar heb je daarmee genoeg om kredietlijnen te openen in naam van andere mensen en met het geld te gaan lopen! Ik neem aan dat dit in Europa toch iets beter is geregeld. Ik denk dat ik nu iets beter snap dat je daar beter af bent om meer op krediet te doen, maar als er zoveel spelers in zijn dat het een rommeltje wordt, dat het bij wijze van spreken langs de straat begint rond te slingeren, dan hoop ik van harte dat er daar toch een paar zijn die er in slagen om het uiteindelijk bij te sturen.

Update: Quartz: Guess which foolish strategy people most often use to pay back credit card debt Blijkbaar zijn de credit-cards daar zo ingeburgerd dat ze anders werken dan hier. Je krijgt een kredietlijn en kan kiezen wanneer je hoeveel terugbetaalt. Ze heffen er in bepaalde gevallen natuurlijk interesten op, wat je best probeert te vermijden, maar het zijn dus extra dingen om in de gaten te houden.

A thin wrapper around SSPI SChannel.

2017-12-30 22:57  schannel1     [permalink]

I thought, if you already have something that does work over a network socket, can you have it encrypted of a TLS connection? If you search, you van find a lot, mainly using OpenSSL. If you read on a bit, you learn about LibreSSL, but if I understood correctly, Indy can't use that since it needs specially patched DLL's, that are stuck on some old version sadly enough...

But, I always keep searching for the thinnest possible wrapper. If there's a way to carry less bloat, or use an even thinner library, then yes please.

So I thought, Windows by itself, or at least some Microsoft things, make calls to the outside world over a TLS-line from time to time. So there has to be a DLL that does all the work for those. It would be strange if it exists, but it's not opened up. Some more searching leads to the realisation it's this SSPI thing that keeps turning up. There's a thing called SChannel you apparently need, but it's not as easy as just replacing your connect/recv/send calls...

Once there was a time when Microsoft wasn't quite planning to keep the 'network subsystem' to themselves, leaving the option open to get some from a different supplier. (Once there was a time it wasn't the matter of course that networking plays over TCP/IP/Ethernet, but that's another story altogether.) You still see that in the SSPI story. You're supposed to call a central function first to see what's available (by which vendor). Once you've tracked down the DLL you need (secur32.dll), you see it just has all you need right there in the exports table (and even just patches them on through to sspicli.dll, at least since some recent Windows version). So in the hope to have some simplification, I think I have now a basic minimal wrapper around the required calls to make it work, added to the minimal things I had in there to talk to WinSock2:

github TMongoWire commit d2c99a...

I hope this performs as expected, as I still have to put it through some more testing in different conditions, but that'll be for another day. And as the current season would have it, perhaps for another year. If you find anything, feel free to launch github issues or pull requests. Happy New Year.

DirDiff v2.0.3.512

2017-10-27 00:19  DirDiff512     [permalink]

→ DirDiff v2.0.3.512

Fixed issue with UTF-8 sensitive characters in ANSI file.
Fixed issue with Ctrl+Shift+Up/Down past start/end of files list.
Enable switching checkboxes on tree view with space key press.

Spanje, ojé ojé

2017-10-19 19:33  ES2017      [permalink]

O jee, wat zit dat scheef in Spanje! Het lijkt alsof ik nog niet zo lang uitvoerig het nieuws volg, en eigenlijk weet ik nog helemaal niet genoeg over geopolitiek, maar ik begrijp er genoeg van om te snappen dat wat nu gebeurt niet direct voor de beste sfeer zal zorgen in Catalonië. Naar mijn bescheiden mening is er momenteel maar één iets dat Madrid zou moeten doen: een nationaal referendum uitschrijven voor de gehele Spaanse bevolking om nauwkeurig te peilen wat iedereen er van denkt als Catalonië alleen op weg zou gaan. (Eventueel vanuit een uiteindelijke koninklijke opdracht.) Niet alleen weet je zo wat de rest van het land precies erover denkt, maar ook in Catalonië zelf heb je zo misschien een iets accuratere peiling dan de half illegale, half onderdrukte volksbevraging die er nu al geweest was. Daarenboven kan je er mee aan de opstandigen echt laten merken dat je het meent met de zoektocht naar een geweldloze oplossing die voor alle betrokkenen de best mogelijke resultaten belichaamt.

Maar néé, ze lijken precies de tegenovergestelde richting uit te gaan, jammer genoeg. Ik begrijp dat een natie zijn territoriale integriteit wil beschermen, maar als er een constitutioneel bezwaar onder de gemiddelde bevolking leeft, moet die daar toch ook rekening mee houden? Soit de vrees voor gewapend conflict begint er zo wat in te zitten, toch hier in de rest van Europa. Plus dat Europa zelf liever niet heeft dat nu de ene na de andere regio afscheidingsdrang gaat uitwerken. Territoriale drang is zo nu wel bijna een eeuw uit de mode, maar als ik het goed begrijp, en zoals we in Oost-Oekraïne hebben gezien, vanaf voedselprijzen lokaal fel gaan stijgen en andere levensmiddelen ook schaars beginnen te worden, kan het snel nog veel slechter worden.

Update 27/10: volgende fase: er blijkt nu echt onafhankelijkheid uitgeroepen te zijn.

RIP AIM

2017-10-13 13:08  ripaim  [permalink]

→ Verge: AIM shutting down after 20 years instant messenger

Oh my. Things come and go. But they're inspired by some things, and get replaced by others. I'm not sure this has been written down anywhere, so I feel compulsed to do it here: what's the relation to microblogging and instant messaging?

Ever since we've been hooking up computers to eachother, we've been sending messages. (Just like we did before that, without computers.) At first there were bulletin board systems and then later the internet with e-mail and things, but those slow out-of-sync messages had to have an 'instant' synchronized counterpart. So there was IRC and ICQ and others, each with their specifics that made them interesting.

But when people grow up, there's less to talk about. To remind us why we're not talking, most instant message platforms had something you could write why you're not to be disturbed, that gets to be displayed next to your name in the contacts-list of the people you're listed with. Some people were really creative with this. It would be interesting and enjoyable to look up the history of these messages of these people, just for a laugh or because you could trace where they've been. It was fashionable to change your status several times a day.

Then came Twitter. It hit after the sweet spot right there in the middle. It completely dropped the instant message thing and was a list of status-updates, complete with the limit on the length of the message. Ofcourse people would still (ab)use it to react or reply, so '@' mentions and re-tweets were born and soon after the '#' prefix that would show as a link to the search-list with that lemma. #hastag!  From there it grew into a phenomenon of its own, and gradually lost it implication of an indication of what you're up to.

Er was eens... (Over politiek en statistiek)

2017-10-09 18:03  Polistiek      [permalink]

Er was eens een televisieprogramma. Eerst was een politicus aan het woord, legt zijn standpunt uit en haalt cijfers aan van een onderzoek om dat te staven. Daarna krijgt een andere politicus het woord, die een tegenovergesteld gedachtengoed genegen is, legt uit hoe zijn standpunt afwijkt, en haalt ander cijferwerk aan als ondersteuning.

Poef. Beide heren zijn hun geloofwaardigheid kwijt. Dat is toch bij mij het geval. In de grote grijze massa van het legislatief moeras waar we allen de verantwoordelijkheid dragen een scheiding aan te brengen tussen goed en kwaad, botsen we keer op keer op het vrij mogen vormen van een mening. Om toch een akkoord te bekomen over hoe het verder moet, lijkt het soms normaal dat je de ander dan overtuigt van jouw mening. Ik laat het als oefening voor de lezer om te onderzoeken hoe het anders kan, maar als er dan toch moet overtuigd worden, moet er boven al geargumenteerd worden.

Argument per argument sluip je dichter bij het ongelijk van je tegenstander. Een argument staat nergens zonder het cijferwerk. Onderzoeken, statistieken, berekeningen. Die vind je niet zomaar en zijn soms duur om vakkundig op te (laten) stellen. Wil het toeval nu dat allerhande organisaties en stuurgroepen veroordeeld zijn tot een bestaan in de marge, en ze hun budget nuttig besteed doen lijken net door zo'n dingen waar smeerlapperij voor nodig is zoals contact met de burger. Dat de resultaten kant en klaar 'gekaderd' zijn om je argumentatie te staven neem je er schoorvoetend dan maar bij.

Misschien hebben ze daarom het dragen van een das verplicht, als die iedereen een basis betrouwbaarheid geeft, is er aan het begin een gelijk spelersveld. Ik beeld me er dan bij in dat ze zich afvragen hoe ze het doen aan de andere kant van het glazen plafond, maar dat zal aan mij liggen.

Statistiek is saai. Dus de journalistiek is die ook liever kwijt dan rijk, je krijgt er geen extra lezers door. Of confronteert net te veel hoe het zit met die lezers. Of het gebrek daaraan. Dus moet je zelf op zoek. Soms is het een publiek geheim dat bepaalde cijfers waardeloos zijn, soms slagen ze er net heel slecht in om te verbloemen dat de cijfers zomaar uit een donker gat zijn ontsproten.

Soit, we moeten vooruit. Beste politicus, je krijgt mijn stem als je alle cijfers kan kaderen in uw uitgestippelde beleid, en onderwijl armoede, werkloosheid, milieu-problemen en begrotingstekorten kan wegwerken. Wacht, dat komt me bekend voor. Ah, natuurlijk. Mijn fout, dat moet zijn: je krijgt mijn stem als onafhankelijk statistisch onderzoek aantoont dat jouw uitgestippelde beleid armoede, werkloosheid, milieu-problemen en het begrotingstekort zal wegwerken. Succes.

Wat als Limburg onafhankelijk zou zijn?

2017-10-05 09:48  limburg     [permalink]

→ VRT NWS: Wat als Limburg onafhankelijk zou zijn?

O jee, zo braaf! Wil het toeval nu dat ik ook ooit al eens een denkoefening als deze had gemaakt, en die zag er behoorlijk anders uit. Ten eerste dacht ik meer aan Baskenland in plaats van Catalonië, en dus aan Noord- en Zuid-Limburg samen, een echte grens-kwestie. En twee nationale afscheuringen dus. Maar ook een echte politieke verschuiving. Er ligt namelijk een stukje goed uitgewerkte maar veel te weinig gebruikte spoorweg ten noorden van Maastricht, waar nieuw Limburg plots de volledige controle over krijgt. Als een poort tussen wereld-havens Antwerpen en Gent-Zeebrugge-Terneuzen-Sas-Van-Gent-En-De-Rest-Allemaal en het Ruhrgebied, kunnen ze de prijs zetten die ze willen per goederentrein. Oh, en de Limburgse luchtmacht kan gewoon wel lekker Raffale's kopen, waarschijnlijk.

Momoa

2017-09-26 10:39  momoa    [permalink]

We've had XML. We've had JSON. There's a thing called YAML. And then there's Protocol Buffers and Thrift and a number of others.

And still, with each there's something is not quite right. So here is yet another proposition, humbly offered for adoption:

Binary. Why binary? Parsing speed matters. There's a belief that binary is not human readable, but:

ASCII control codes. Why ASCII control codes? They're out of use. Except 0x0A (and 0x0D) for new lines and 0x09 for tabs. I've come across a 0x0C and 0x1B when talking to printers, but that's it. And all modern editors know what to do with them. Best case may even be they show them as something foreign, but still they're visibliy right there with the other text.

A list of keys and values. It's tempting to provide structure and clearly indicate which is what, but it's unneccessary. A parser is smart enough to know these come two by two, and to pair them up when handing over to something for processing.

Types of values. A value has a preceding byte denoting what it is, and what rule to follow for the succeeding bytes.

0x02 string: read the string up to the next type byte. If a type byte needs to be actually part of the string, escape it with 0x07

0x03 number:  read a string up to the next type byte and convert it from text notation to something numeric. Depending on the context it may be something specific or variadic. By using the text notation we retain some human readability, and also get an acceptable storage to information ratio (smaller numbers take less bytes)

0x05 boolean true: with nothing more

0x06 boolean false: same as above but with opposite value

0x01 embedded key-value list: treat the following sequence of key-value pairs, delimited by a 0x04 closing type byte, as an embedded list

0x08 array: treat the following as a sequence of values only, delimited by a 0x04 closing type byte

There's no specific type byte assigned to null or undefined, but can be encoded as a single 0x03 without data following it.

Keys are themselves values, typically of type string (0x02). A possible permissible exception in specific contexts may be to encode sparse arrays as an embedded list (0x01) where all keys are of type number (0x03).

And now for a name for it... I know, let's type Jason into IMDB... Sounds nice, and serves as a tribute to the artist. So let the file extension be ".momo" and the MIME type be "application/momoa"

About vehicles of the (near!) future: electric or autonomous (not and).

2017-09-18 13:48  autoelec     [permalink]

Little old humble me will try to give a naunced depiction of their complex and layered opinion about something that's recently been getting some news.

I regret that most news-items on these subjects tend to handle both the new models that have a fully electric drive-train, and the fact that onboard systems of sensors and advanced signals processing may be used to autonomously let these new models operate on the streets.

Not too long ago it was made painfully clear that the seat belt should be mandatory in every automobile, and also the fact of strapping yourself in with them. In a similar light, the design of systems that could handle the performance of said automobiles has indeed proceeded by so much as to have a far better safety record with operating vehicles than us humans. So there's no doubt that the distance between a future where self-driving cars are mentioned in the text of law to some extent, is measured in years, not decades. Perhaps even months since the time is now to begin thinking about authorization and certification.

The matter of a novel power train, on the other side, is a different thing. Electrical vehicles are not new. What is new, is that battery-technology has steadily improved, and that a certain captain of industry has thrown their weight behind an undertaking to launch mass-production of purely electric vehicles(*1), trailblazing into a new domain the settled automobile-constructors weren't committed to explore (yet).  Even more important is that he is succeeding at it. And building a giant battery-producing plant just for it.

What is also painfully clear is that we may just be ahead of the required battery-technology-improvement we actually need. Getting us to drive electric vehicles is hard, not only because we cling to what we know, but also because there's no way to create all the required batteries with the methods for creating them we master now. Also recharging a battery has a vastly different dynamic than refilling a gas tank; this doesn't mean that I think it should. Quite to the contrary, but it should in itself offer an improvement over how we did things before.

Where the automobile-constructors of old are set out to explore, is at the cutting edge if these combustion engines most of us are using. There's an apparent room for improvement, but apparently it only very slowly gets filled. It may have been a very good move to switch 'back' the Formula 1 specifications to a 6-cylinder design with a fuel content more in line with most mainstream vehicles. Any advancements engineered there should find their way in construction of engines a few years down the road. But that apparently takes years.

So that's why the coverage of autonomous electric vehicles kind of rubs me the wrong way. It would be the best thing for us that autonomous cars are here soon, and find mainstream acceptance quick; and that electric vehicles take over the majority when they're ready. With advances made in bio-fuel and high-milage-engines, to be expected over one or more decades, I dare even predict that autonomous control will get even more kilometers out of what you put in, be it liters or Ampère-hours.

(*1): and space-rockets, and solar panels with an in-house battery, and an attempt to improve tunnel-boring.

This one day at work

2017-09-07 11:00  fromthetrenches      [permalink]

Here's another nice story 'from the trenches'. Packaging stations use a barcode scanner to scan the barcode on the items that need packaging. We were able to buy a batch of really good barcode scanners, second hand, but newer and better than those that we had. A notice came in from an operator: "with this new scanner, we get the wrong packaging material proposal." The software we wrote for the packaging stations, would check the database for the item which is the best suitable packaging material to package it in. It's a fairly complicated bit of logic that used the order and product details, linked to the warehouse stockkeeping and knows about the several exceptions required by postal services of the different destination countries.

So I checked the configuration of this station first. Always try to reproduce first: the problem could go away by itself, or exist 'between the user and the keyboard', or worse only happen intermittent depending on something yet unknown... Sure enough, product '30cm wide' would get a packaging proposal of the '40cm box' which is incorrect since it fits the '30cm box'. Strange. The station had the 'require operator packaging material choice confirmation' flag set to 1, so I checked with 0 and sure enough, it proposed the '30cm box' (with on-screen display, without operator confirm, just as the flag says)...

So into the code. Hauling the order-data and product-data from the live DB into the dev DB (I still thank the day I thought of this tool to reliably transport a single order between DB's). Opening the source-code for the packaging software, starting the debugger while processing the order, and... nothing. Nicely proposing the '30cm box' every time, with any permutation of the different flags (and there are a few, so a lot of combinations...)

Strange. Very strange. Going over things again and again, checking with other orders and other products, nothing. I declared the issue 'non-reproducable' an flagged it 'need more feedback', not really knowing of any would come from anywhere.

A short while later, a new notice from the same station: "since your last intervention, scanned codes concatenate". What, huh? I probably forgot to switch the 'require operator packaging material choice confirmation' back to 1, but how could that cause codes to concatenate? I went to have a look, and indeed, when a barcode is scanned (the device emulates keyboard signals for the digits and a press of 'Enter') the input-box would select-all, so the number is displayed, and would get overwritten by the next input. This station didn't. The caret was behind the numbers, and the next scan would indeed concatenate the next code into the input field.

Strange. Very strange. Into to the code first: there's a SelectAll call, but what could be wrong with that? And how to reproduce? What I did was write a small tool that displayed the exact incoming data from the keyboard, since it's apparently all about this scanner. Sure enough, the input was: a series of digits (those from the barcode), an 'Enter', and 'Arrow Down'. A-ha! These were second hand scanners, remember? God knows what these scanners were used for before, but if having the scanner send an extra 'arrow down' after each code, is the kludge it takes to solve some mystery problem in software out of your control, than that is what a fellow support engineer has to do... Got to have some sympathy for that. And the '40cm box' was indeed just below the '30cm box' in the list, so the arrow down would land in the packaging material selection dialog, causing the initial issue.

Download the manual for the scanners, scan the 'reset all suffixes to "CR"' configuration code, done.

(Update: got some nice comments on reddit)

Why I choose Delphi

2017-08-15 22:49  whydelphi      [permalink]

Strange, all these Why I choose Delphi articles lately:

• Code Velocity: Why I Choose Delphi
• The Poscast at Delphi.org: Why I Keep Choosing Delphi
• Fixed by Code: Why I keep preferring Delphi
• Jens Fudge: Whi I choose Delphi
• Hit Compile and Run!: Why I Choose Delphi
• Marco Cantu: October 2017 Why I Love Delphi Summary

Keep them coming! It's good to see it stressed that it's really a myth that there's not enough Delphi talent out there. Rember, Delphi's debugger by itself is so strong, a decent developer should be able to learn both Delphi and an existing code-base that works just by stepping through the code with the debugger and see what it does. Yes, the language is a little verbose; yes, it's perhaps even older than C/C++; but remember, so is COBOL, and I would almost say that's not cross-platform or a systems language, but those just had other meanings back in the days. (Did you know there's Delphi for AS400?)

So, why do I stick with Delphi? The answer is pretty straight-forward (and perhaps a little sad): give me any kind of computing problem, and I'll find a way to tackle it with Delphi. I've done so much different things, and still found a way to get a great system I enjoy working on, and still have a Delphi project that its compiler happily churns into a binary executable that performs really well. Yes, you could do that also in C/C++/Rust/D/Java/(etc.) but I can't, and don't really want to. There are always up-sides and down-sides,  but it feels like with Delphi you don't meet much of any down-sides, and if you do some-one else knows something to do about it.

Odo v0.4.1.511

2017-06-21 23:15  odo511    [permalink]

→ Odo

delen in de winst? waarom niet sparen

2017-06-12 10:59  liqres2     [permalink]

→ Open VLD wil werknemers laten delen in de winst van bedrijven

Ah, zijn de liberalen daar mee bezig? Ik heb een ander idee. Wat als middelgrote bedrijven en grote bedrijven zouden worden verplicht een 'liquidatiereserve' aan te leggen? Een bedrag dat pas vrij komt als een bedrijf er mee wil stoppen of een fractie van zijn personeel aan de deur wenst te zetten.

Ik loop blijkbaar al een tijdje met het idee (2011!) maar het speelt toch maar in je hoofd. Als je weet wat bedrijven kunnen/moeten doen met sommen geld die om verschillende redenen in het bedrijf aanwezig zijn, kan je misschien zelfs verplichten om zo'n fonds lokaal te investeren. Hawel kijk, dat zou zowel de linker- als de rechter-flank dienen. En de lokale economie. (En de schatkist.) En uiteindelijk ook de mensen die jaren de onderneming dienen maar dan moeten afvloeien...

Idea: assembly that flags when to release virtual registers

2017-05-31 23:52  asmvrr     [permalink]

I just had a fragment of an idea. I want to write it down, just to let it go for now as I've got other things to do, and to be sure I can pick it up later exactly where I left off.

First situating what it's about: I have been reading up on WebAssembly, and to my surprise the intermediate representation is stack based (just like Java's JVM and .Net's CIL). I'm not sure why because it feels to me this makes registry assigning when constructing the effective platform-dependent instructions harder, but I may be wrong. Finding out objectively is a project on it's own, but sits on the pile 'lots of work, little gain'.

I also went through the great set of MIT 6.004 lectures by Chris Terman which really gives you a good view of 'the other side' of real assembly since it's actually born out of designing these processing units built out of silicon circuits. It prompted me to make this play thing, but again pushing that through with a real binary encoding of the instructions made it a 'lots of work, little gain' project, and I really don't have access to any kind of community that routinely handles circuit design, so it stalled there.

Before that, I read something about hyper-threading, and what it's really about. It turns out modern CPU cores actually handle two streams of incoming instructions, have a set of instruction decoding logic for each stream (and perhaps branch prediction), but share a lot of the other stuff, like the L1 cache,  and especially a set of virtual registers that the logical registers the instruction stream thinks it's using is mapped on to. Mapping used registers freely over physical slots makes sense when you're making two (or more?) streams of instructions work, but it's important to know when the value in the register is no longer needed. Also if the register is only needed for just a few instructions, pipelining comes in to play and could speed up processing a great deal. But for now the CPU has to guess about all this.

When playing around with a virtual machine of my own, I instinctively made the stack grow up, since you request just another block of memory, plenty of those, and start filling it from index 0. It shows I haven't really done much effective assembler myself, as most systems have the stack grow down. What's everybody seems to have forgotten is that this is an ugly trick from old days, where you would have (very!) limited memory and use (end of) the same block for the stack, and with more work going on the stack could potentially grow into your data, or even worse your code, producing garbled output or even crashing the system. (Pac-man kill screen comes to mind, although that's technically a range overflow.) Modern systems still have stack growing down, but virtually allocate a bit of the address-space at the start of that stack-data-block to invalid memory, so stack-overflows cause a hardware exception and have the system intervene. It's a great trick for operating system (and compilers alike) to have checks and balances happen at zero cost to performance.

The consensus nowadays is that nobody writes assembler any more. It's important to know about it, it's important to have access to it, but there is so much of it, it's best left to compilers to write it for you. In the best case it may find optimizations for you you didn't even think about yourself. But this works both ways. Someone writes the compiler(s), and need to teach it about all the possible optimizations. I can imagine the CPU's instruction set manual comes in handy, but that's written by someone also, right? I hope these people talk to eachother. Somewhere. Someday. But I guess they do as with x86-64 they've kind of agreed on a single ABI... and they've also added some registers. Knowing about the virtual registers allocation going on behind the scenes, it could be that that was just raising an arbitrarily imposed limit.

So this is where I noticed a gap. When performing all kinds of optimizations and static analysis on the code when compiling, and especially with register allocation, it's already known when a register's value is no longer relevant to future instructions. What if the compiler could encode this into the instruction bits? If I were ever to pick up where I left, and have a try at a binary encoding for a hypothetical processing core, the instruction set would have bits flagging when the data in registers becomes obsolete. Since this would be a new instruction set, and I guess it's more common to need the value in a register only once, I might make it the default that a value in a register becomes obsolete by default, and you'd use a suffix in assembler to denote you want to use the value for something extra later as well.

MurMurHash3

2017-05-19 20:09  murmur3     [permalink]

@stijnsanders do you have MurMurHash3 code in pascal aswell ?
i truly liked your optimized code for md5 and all those

— _pusher_ (@_pusher_0x90) 12 mei 2017

Why, thank you. Eeuh, is it at all optimized? I took some decisions that may perform a little better than the reference implementation, but I haven't taken any time to compare to see if it actually performs any better or worse...

So MurMurHash3... Let's have a look, it's on wikipedia, so it's a thing. And the reference C implementation is in the public domain, great! Looks straight-forward enough, could boil down to a translation-job...

Rougly two hours later, got triple zero (0 errors, 0 warnings, 0 hints). Now for checking if I got it all right. Hmm, not much there, but those x32_64 match, so barring any typo's, assuming this pretty straight translate job will result in the expected behaviour for the other two functions, this should be it:

md5.zip (25KB)

Checking xxm for PHP's vulnerabilities

2017-05-12 07:56  xxmphp1      [permalink]

If I read about a newly discovered vulnerability related to PHP, for example this one here, I try to find out if it would apply to xxm as well. 

In this case I guess there's nothing more than sending out the message, again and again, to sanitize your inputs, and poperly encode your output. Strings are never just strings. They are always an internal representation of a bit of textual data. So always think about that taking string values in, and preparing strings for output. A few weeks back I had to speak up to someone who wrote OutJSON:='{"field1":"'+value1+'"}';. Little Bobby Tables comes to mind, though I'm not sure 'JSON injection' could be so devastating as SQL injection. (And OutJSON:=JSON(['field1',value1]); is shorter!)

The other time I found out it's a really good idea to strip nastiness like EOL's (CRLF) from headers added to a response, just in case a malicious script is up to no good. Come to think of it, that's also just another case of properly sanitizing your inputs...

Delphi, the shrinking island.

2017-05-09 21:48  delphitrend     [permalink]

Is there a language that has a single word for 'the feeling of being on a shrinking island'? Anyway, this is somewhat sad to see, especially that the few most recent Delphi versions gave the impression there was a new uptake with more people getting persuaded, but it doesn't show in the curve.

(via)

'Relax' scripting vs xxm

2017-04-22 13:12  relaxxxm      [permalink]

→ Delphi Relax Web Scripting (Marco Tech Blog)

I'm sorry but I feel I must react. In general I keep silent, in the hope people by themselves will know better, but as I'm getting no input what-so-ever that that is the case, I feel tempted to write something about this.

First about what's at hand. I see this bit of code:

<h2>Employees</h2>
<ul>
@foreach (var emp in employee) {
  <li>@emp.FirstName @emp.LastName (@emp.PhoneExt)</li>
}
</ul>

and it looks kind-of OK. To the untrained eye it looks good and may even look tempting to write more in this syntax. This is a straight-forward example of a template that works with a templating engine that no doubt has many more capabilities and features. And then I thought, learned from practice, what I typically would get asked is to not show " ()" when the PhoneExt field is empty. I would not know how to make that happen in that template syntax. That's mainly because I know nothing about the template syntax. If I look into the documentation, I might find an @if predicate to make it happen, but let's move on:

This is what it could look like in xxm:

[[!var emp:TEmployee;
<<h2>Employees</h2>
<ul>>
foreach emp in FDMemTable1 do
begin
 <<li>>=[emp.FirstName,' ',emp.LastName,' (',emp.PhoneExt,')']<</li>>
end;
<</ul>

Looks roughly simlar. A little more like Delphi syntax. And in fact it is. If you know [[, ]], << and >> get translated into Context.SendHTML() and Context.Send() calls behind the scenes (full details are here),  you know this code will result in the same output. Without templating engine! Streamed to the user's client! Perhaps even while the data is streaming in from the database server, in case it's a longer list, and in case there is a database server, Marco uses a memory-table for his example.

What I find important is that there's less going on between the native compiled logic and getting the data to the user launching a request. Not only a templating engine looks superfluous, this entire ORM thing is something I don't get. If it's a gigantic database model with so much tables that you clearly benefit from code-completion, then I agree, but I haven't come across something remotely close to that in web projects.

Also the HTTP-server itself is something I think that values extra attention. I've seen platforms and frameworks that offer you a wealth of capabilities and features, but hastily slapped on something that listens on TCP port for basic HTTP requests, in some cases on port 80, but more often 8080 or something else in the thousands. In real web environments, the server(s) has/have a lot more going on: load-balancing, reverse proxies, firewalling, authentication. Since we're in a Post-Snowden-era nowadays, we're all responsible to think about protecting privacy and get that HTTPS in order with the proper certification and encryption... Not to mention HTTP version 2 that's heading full steam towards being generally accepted/expected.

I can image the web-admin responsible for all that, isn't happy with your request to add this newfangled separate thing that's doing its own handling of HTTP requests. ISAPI DLL's or Apache modules play much nicer with existing IIS or Apache installations. (FastCGI is on the table, but for now xxm has SCGI available for other servers.) Even if your 'Delphi HTTP framework' of choice is specifically designed to tap your ORM of choice and offer a REST-API for your data-layer needs, it will still be one more stop along the way between the user's browser, and the delivery-setup, and the front-end, and the page-template, and the data-layer, and the database, and what the user actually needs or wants. I think of this in the postal office when there's twelve people in the queue in front of me.

I don't expect to convince much people of this way of working, but it works great for me. I remember the days with early PHP and ASP and how simple and straight-forward everything was. Knowing these work on scripting engines, I kept worrying about lost performance. This was the core reason to start xxm: employ the speed and power of the Delphi compiler to have a native library serve my websites. And it turns out that Delphi code looks quite nice between HTML to handle server-side logic, if I may say so. It took me a few years to make this happen, but I couldn't do without it any more. And people kind-of appreciate that for using this new application, all they need is their trusted browser and a URL.

Archive... Search...