yoy.be "Why-o-Why"

freeware

2005-06-10 20:46  freeware  freeware  [permalink]

---Here are a few applications I wrote, a few of them I use myself, a few were just for fun...

They're put public here as FreeWare, some are OpenSource. I haven't figured out yet if I need to put them here under a license or even which one. So for now they're all rights reserved and use em at your risk, though there's really nothing to fear because they really don't do more then they're supposed to do.

Anyway, if you really want to have a look at the SourceCode, let me know -- StijnSanders

(Click on these icons:)

[TreeBrowse] [DirDiff] [DirFind] [odo] [RE] [jsonDoc] [Connect 4] [CursorTime] [MetaClick] [MetaKeys][BarCode] [MailCount] [Ro] [Fa] [WebTop] [SideSwitch] [Drink]

AllSorts (shell extension)
Growthris
IconSuite
WikiEngine
TortoiseSVN icons
xxm
github.com/stijnsanders

twitter reddit linkedin facebook google+

jsonDoc v1.2

2019-01-18 08:53  jsondoc12  coding delphi freeware  [permalink]

I don't really know where my open-source/freeware projects get used by other people (and I don't really care all that much, really), but ofcouse where I use my private libraries in the projects of my day job, I've got a first-row seat to see how they perform. No, even better, I'm the stagehand. I know it's important to keep the two separated though. At work I can only use and call libraries, never work on them. I can make changes, but I would have to do so like anybody else: clone a local repository, store the changes there, and when they're ready have the good manners of sending out a pull request. If 'at-home’-‘hobbyist-programmer’-me decides to merge it, it makes my employer a co-creator of the project, creating a new legal situation that my employer would need to know about.

What I can also do, ofcourse, is let private-me know that ‘at-work’-me would really like this or that extra feature or interface, and then when I can put some evening time into programming, I can see if I get round to it. Github issues are nice for that. Or Post-It's on my car key...

But recently I've got a real decent suggestion from a collegue. It didn't come in over a pull request, but it made sense, so why not put it in at the base:

https://github.com/stijnsanders/jsonDoc/commit/fb01e0865fa888c7441234726dc4a90b0c802582 

The IJSONDocument interface has Parse to load data from a string holding JSON data, and a function ToString to convert the contents back to a string. For jsonDoc (and bsonDoc before that) I want concise syntax, so I though I'd make Parse a function that returns Self so you can chain calls. But my collegue wondered why IJSONDocument doesn't have a property AsString:WideString;. In theory it would have read ToString write Parse, except Parse needs to be a procedure, not a function. So I had a close look and changed it around. Another option would be to add an extra procedure SetString, but that would mean the virtual method pointer table for the IJSONDocument interface would have an extra item, to a method with exactly the same behaviour, which makes little sense.

So I changed it around. If you really like/need chaining, there's still the function JSON overload that takes a Variant. If you pass it a string, it'll call plain function JSON to get a new instance, and call Parse for you on the string.

twitter reddit linkedin facebook google+

NoSQL agent with SQL back-end(s)

2018-12-29 15:41  nosqlidea  coding delphi freeware  [permalink]

It happened again. I get this great idea that slowly develops, and gives the feeling I'm on to something, but nowhere near any time in the forseeable future to put in to it and get a proof-of-concept of a first project that makes it work. So, for what it's worth, I write it down here fast in the hope sharing it with you may give more chance to this idea getting useful.

I've read that some NoSQL solutions are actually about eventual consistency, meaning in the best case of a query for data that was just inserted or updated could already return this new data if the server(farm) ad already fully processed it. Worst case is that it just for a few milliseconds totally disappears, but that's another story altogether.

I haven't done anything serious with NoSQL yet, and really a lot on good old SQL, and recently with SQLite which I've really grown to love in a short period of time. But still there's something there that's really suited for the new style of programming that is going on with all the new web projects and this 'Internet of Things' everything is on about... To find out, I've been trying interfacing with a number of them from Delphi in the most direct way I could possibly find and make work with reasonable effort. I like how TMongoWire worked out, to talk with PostgreSQL all you need is in the libPQ.dll, but a number of others just stick with a plain HTTP API where you PUT and GET things on their own URL. There's a beauty to that, really. The structure of your documents is nicely contained in JSON, and HTTP is such a stable platform you're sure to be able to access it from almost any platform.

So that's where the idea came from: what if I made my own service where you can just put or get JSON documents? But on the back-end jsonDoc would do the heavy lifting and the storage itself could be in a decent SQL service. And/or it could be in something intermediate like memcached. And/or the saving to storage could be asynchronous somewhere close after the actual PUT call (hence the eventual consistency).

For example, you fill a collection of items with things with a number of fields, for example one is "Price", but later you need the items above or below a certain price, you would do SQL "select * from Items where Price<@p". So in this service I'm imagining, there would be meta-description on the collection that you've provided a SQL database somewhere, but the service is responsible for having done the "create table Items (ID some primary key, DATA json, Price decimal(8,2))" and filling it with the data.

And this would be the beauty of it: if you need an extra column later, you just say so, and the connector would be responsible for the "alter table Items add ..." and filling that column with the data from the stored items. Perhaps even slowly, asynchronously together with the other work. Or even another connector alltogether, let's say PostgreSQL and MySQL side by side, perhaps even as a fail-over for eachother.

But I'm dreaming. It would be a load of work just to get something to work, and even more work to get enough connectors to work good enough to even demonstrate how it would work. And then there's the performance trails... And the evangelising to see wether it solves other people's problems anyway... It would be a really great opportinity to finally cut my teeth on this IOCP thing.

twitter reddit linkedin facebook google+

Update to SQLite 3.26 as soon as possible

2018-12-24 14:52  sqlite326  coding delphi freeware  [permalink]

It's very rare something turns up in the SQLite package, but when it does is best to give it some attention. So, very esteemed users of TSQLite — and anyone else, really — I strongly suggest you upgrade to version 3.26 of sqlite3.dll since recently some dangerous vulnerabilities have been uncovered. I checked, and nothing extensive changed to the API so chances are nothing should break if you only update sqlite3.dll and not SQLite.pas and SQLiteData.pas. Also, happy Christmas!

twitter reddit linkedin facebook google+

TOML? YAML? ini-files? JSON? Loosen up!

2018-12-10 11:26  jsonloose  coding delphi freeware  [permalink]

TOML? YAML? plain old INI-files? or shiny and new — but strict — JSON? Nàh. I've made myself something better. In jsonDoc.pas I've recently added an optional compiler define JSONDOC_JSON_LOOSE that makes the JSON parser a whole lot genter to work with. Some things are no longer required: the root document doesn't need to start and end with braces ("{}"), the quotes around the key names are no longer required (unless they contain special chars), the comma's between key-value pairs are no longer required, it accepts equal ("=") between keys and values instead of colon (":") and can even do entirely without.

And while I was at it there's also JSONDOC_JSON_PASCAL_STRINGS that allows you to write string literals the Delphi way: between single quotes, and double single quotes where you want a single quote in the string value. This way — really handy in Windows environments — it doesn't require you to escape backslashes.

The only downside maybe is that you need to add the defines to the compiler parameters, but for plain old configuration something like this:

x=5
y=10
output='C:\test\output\'

looks a lot nicer and like what we're used to, and gets parsed just the same as if it was written like this:

{
"x":5,
"y":10,
"output":"C:\\test\\output\\"
}

twitter reddit linkedin facebook google+

xxm 2.0? About the big plan

2018-11-28 22:50  xxm2plan  coding delphi weblog freeware  [permalink]

Should I begin on xxm v2.0? Delphi having a Community Edition is really really great news, not only for all of the Delphi community, but also for me as a hobbyist programmer with a number of freeware opensource projects out there. Except I was one of those that stuck with Delphi 7 up till now, so next to myself being fully convinced to finally take the step, I also need to have a look project-per-project how to lift them into these modern times and upgrade them to full Delphi 10 projects.

One of my darlings is xxm. You can compile xxm with modern Delphi just fine. I switched string to AnsiString or WideString explicitly nearly everywhere, so you shouldn't get as much as a hint. (Except mayby somwhere where I still have case x in ['A'..'Z'] of or something like that where x is a char and thus is a WideChar, and the compiler doesn't like that). But I'm worried about how I chose to set up the project. I thought I was best to just inherit from IUnknown and define a decent interface and build everything around that. But several xxm projects later, some that are even running some decent production load at work, I learned some things, most importantly these two:

It looks like xxm doesn't need to work with object instances. Requests are coming in, something needs to build a response, but having to have your objects set up first before you can get crackin' now appears as something I should have tried to avoid. Specifics and identities are handled by the underlying communication framework anyway, so you can get by with just a reference, and request more with that reference only just when you need it.

The other is that with the dynamics of IUnknown pointers, you're still dragging this COM thing with you. Yes it may be wafer-thin really and proven tech, but it's still something removing you from what really counts: pointers to where the code is to run. If you're really (really!) pushing to get the last drop of jouce from the orange performance-wise, it may be just that last thing that's in the way. Even when you avoid any locking in reference counting.

So this is the big plan for the moment: if I get to put some decent time into the project, over the next year(s), I'm considering starting a blank project xxm 2.0, start with a plain C style interface with just procedure and function pointers — a really really ancient looking interface — that has everything that IXxmContext and related interfaces have now. Then re-start with xxlProto and xxmHttp to make that work,  then a new xxmCGI, xxmSCGI, xxmISAPI, xxmAhttpd (and perhaps now really a xxmFastCGI, but no xxm IInternetProtocol any more, sorry) and then have a first new push to see how it performs, with IOCP and http pipelining, and who knows HTTP/2 or HTTP/3 that'll have materialized by then.

After that I was thinking I need a new implementation of IXxmContext that just patches through to the new xxm 2.0 calls, so you can very easily and without a worry add it to an existing xxm v1 project and run it with the xxm v2 handlers, enjoy the performance benifits and slowly change things over to xxm v2 where you need to — or not at all, really.

I hope it's a good plan, but for now it's all a dream. I haven't been able to put much time into my hobby projects lately, but it fluctuates and if it picks up some I'll see how far I'll get. Along the way I still hope to find more people that are searching for a good way to combine good old web-scripting with the power and speed of the Delphi compiler, and can count on xxm to offer good debugging, easy live-updating and a stable and secure platform for their web projects.

twitter reddit linkedin facebook google+

TMongoWire on jsonDoc v1.1.8

2018-11-01 17:24  mwjd118  coding delphi freeware  [permalink]

→ TMongoWire commit 78c7c62: jsonDoc v1.1.8

I may have decoupled the bit that works with structured data (using Variants), and converts to/from JSON, into a separate project jsonDoc, and have converted TMongoWire to run on jsonDoc, I also should take the time to update TMongoWide when jsonDoc gets improvements. The fact is I'm doing much more with JSON than on MongoDB recently, so I risk letting this slip out of sight.

What I also should do is set the TMongoWire repository up so it uses the jsonDoc repository as a git module, but for now it's just about one single file so I'll see when I get to it (...).

twitter reddit linkedin facebook google+

RSS is far from dead!

2018-08-08 00:55  feeder  delphi internet freeware  [permalink]

I've been using RSS/Atom feeds on and off since I've learned about them. A long time ago, Google had a nice feed reader, but decided to discontinue it. Users were left to search something new, and I settled on The Old Reader, combined with gReader since I had a smartphone, and all was well. For a while. After some time you notice you still get disturbed by some tiny issues you can't seem to get to go away, either with tweaking the configuration or with Stylus. So what does a developer do? Start to think about developing their own solution. Then plan to develop their own solution. Then develop their own solution. So I'm somewhat proud to present this little thing I've been tinkering on in off-hours the last month:

github.com/stijnsanders/feeder

I have a live version to try out here: http://yoy.be/home/feeder/ but it uses the neighbouring instance of tx for authentication. I should enable Google/Facebook/Github OAuth things instead, but finding out how that works is a few items lower on my wish-list (of things I wished I had the time to put into).

I wanted a feed reader without the extra's. I wanted to mark items as read that move out of view by scrolling down, and plays somewhat nice with the surrounding HTML and the browser. For now I like how it works. There's an issue with emoji's that apparently get eaten by UTF8Decode, but that could be a bug that got solved since good old Delphi 7. But now that Delphi has a community edition, I think I should bring most if not all of my other projects to this version instead of sticking to Delphi 7... But that's another story. (One you might notice some time in the future on my Delphi RSS feed...)

twitter reddit linkedin facebook google+

DirDiff v2.0.4.535

2018-06-23 00:31  dirdiff535  freeware  [permalink]

DirDiff v2.0.4.535

There was a bug when diff-ing more than 2 files or folders, where lines matching between other files than the first/left one would show as matched.

twitter reddit linkedin facebook google+

AES v1.0.1

2018-05-21 20:18  aes101  coding delphi freeware  [permalink]

md5

I noticed something was wrong with the key generation schedule in my AES implementation. I had a close look with the FIPS 197 example vectors at hand and fixed it. I still should make some time to run it under pressure and see how it relates to reference implementations and popular implementations.

twitter reddit linkedin facebook google+

xxm v1.2.5.450

2018-05-03 22:43  xxm450  delphi freeware  [permalink]

For a while I thought there's not enough there to have a full release, but with the switch from XML to JSON, and NTLM and WebSocket support in xxmHttp and xxmHSys2, plus all the minor changes and tweaks that improve performance and security, it turns out to be quite a formidable release. Regretfully, by finally deciding to have all string arguments const anyway, any project's xxmp.pas will have to change. A long time I was hesitant about this because I feared binary incompatibilities, but I've tested all permutations between caller/called with/without const, and it all works. Except when building old code with the new xxm.pas will serve you with this minor one-time inconvenience of having to add const yourself to the arguments... I've thought about trying to create something automatic — like the XML to JSON converters — but since you're quite free to modify xxmp.pas to your liking, it may be more complex than it seems. (If you would like to see this anyway, let me know and I'll put some time into that.)

It's advised to do an extensive test-run with this new version before you update the live environment. If anything unexpected turns up, let me know!

xxm v1.2.5.450 (→gh) (→sf)


twitter reddit linkedin facebook google+

What do you think I should do?

2018-04-20 23:26  sendfile404  delphi freeware  [permalink]

Dilemma, dilemma! What should I do? Sometimes, you need to have a certain URL in a web-project that just serves op the contents of a local file. You could try to have the file in a static section of the website, or even a sub-domain for static content (if your budget allows it), but then it's out there for anyone. Best is to have a request be processed dynamically, so you can do some server-side logic first, for example to do authentication control, or generate the file first (perhaps with a graphics library?). That's where the SendFile method of the IXxmContext comes in. The code that implements it is pretty straight-forward: 

procedure TXxmGeneralContext.SendFile(const FilePath: WideString);
begin
inherited;
if State=ctHeaderNotSent then FSingleFileSent:=FilePath;
SendStream(TStreamAdapter.Create(TFileStream.Create(FilePath,fmOpenRead or fmShareDenyNone),soOwned));
end;

If the request's header has not been sent already, the private value FSingleFileSent is set to the file path, assuming the request is meant to have this file's data as response data. This is used for error handling and logging.

Then SendStream is called, which takes an IStream pointer, so a TStreamAdapter is used to wrap around a TFileStream, with ownership so the last IStream._Release will call the file stream's destructor.

One thing that's missing here perhaps is trying to figure out if the HTTP response header Content-Type is set to some suitable MIME-type for the file, but this is so untrivial that it's best left to the developer. So give it a second of thought when you're coding a call to SendFile.

So about this dilemma I'm having. Just imagine for a second you have this code in a project. Attention: this is very bad practice! It should be glaringly obvious to everybody that doing this opens the back door wide open and people with malicious intent can access any file they want on the machine, include system files, so please don't ever really do this:

[[Context.SendFile(Context['f'].Value);]]

Yikes. Very very bad! I feel dirty just for typing that, but just as an example, this code has a high probability of trying to open a file that doesn't exist, or otherwise have the TFileStream.Create throw an EFOpenError exception.

In that case, would it be better if xxm answers with a proper HTTP 404 (page not found) response? Now default exception handling kicks in, and left unhandled (hint) xxm will fashion a HTTP 500 response for you with the exception data. There's also a bit that will see if your local fragment registry can load a fragment for '404.xxm' that lets you design a nicer 'page not found' page than the default, but I'm having a hard time to guess if that would be something unexpected for someone somewhat new to xxm, calling SendFile on an inexistant file for the first time...

I'd love to hear from anyone on this, but for now I'll just let it be like this and let the normal exception play out, if any.

But wait, there's more. Deep burried within the dark corners of the Windows API, there's a thing called TransferFile. It basically lets you tell the system to take a file handle and a network handle and stream all the data from the one to the other, as much as possible right from the kernel. The way the friendly people over at Microsoft worked it out, and tied it to the running system so it would only work on Windows Server versions, makes it kind of unsuitable for where I want to take the xxm project.

But wait, there's more. You may have noticed the web-sphere is gripped with a frenzy for all things asynchronous. There are a few good things there, but it is mainly the best way to serve a magnitude more of concurrent requests by the same server. The short story is you try to avoid waiting on the system while it waits on network or disk. The long story envolves completion ports or libuv, but is in essence unfit to combine with what xxm is doing: having a separate DLL with code you just call to have a response generated for a request, since to do it properly every request to the operating system needs to be re-routed over your job/thread/task/fiber/yarn-management.

This shouldn't hold me back to get as close as possible to the middle ground between the two, where you have the option to build a response, but can hand over a (file)stream and have the HTTP server spool that as it sees fit,  once the situation really is just that straightforward. But this may be what xxm 2.0 could be about, if there ever will be something like that.

twitter reddit linkedin facebook google+

HTTP+HTML+Delphi authentication (how xxm does it)

2018-04-13 14:26  xxmauth  coding delphi internet freeware  [permalink]

Daraja Framework: HTTP+HTML form-based authentication

Jikes! This is strange. Yes you could go ahead and have a page with a login-form, that posts onto a handler that checks your password, and throws a 401 when it fails. But is that really what you need? I thought 401 is there to elicit the user's HTML-client (a.k.a. browser) to show a modal form asking for a password before re-posting the request. Just like xxm's Basic Authentication demo does, and it does this right at the center of the project, before your request is routed to any page or resource, so that all requests to the project need authentication. Also this way you don't need to code a check IsAuthenticated on every page or resource.

But — again — is this really what you need? The public nowadays doesn't respond well to systematic authentication like that, and also makes it impossible to do anything on the website while not being authenticated (yet). It's better form to welcome new users with a nice 'create new account' button (More about that here.) and perhaps more information on what's on offer, next to the logon form for existing users (with extra options like 'stay logged on on this station' and a 'forgot my password' link). There's an example in xxm's Session demo: The opening page has a log-on form, and Login.xxm does the rest. It doesn't really check user-account and password here as it exceeds the purpose of the demo.

To show you a working demo, you should have a look at tx: It has a central redirect for any page request from a user that should authenticate first; the logon-form with extra options to show users as a normal web-page; checks the entered password agains a properly salted hash and then redirects you to the page you came in for originally...

And there's much more to tell about authenticating users. I've tried to make a list here (it's in Dutch though), and that doesn't even scratch OAuth(2) yet...

Before I forget, did I mention xxm comes under a permissive MIT license? So you don't need to buy a commercial license!

twitter reddit linkedin facebook google+

DirDiff v2.0.3.512

2017-10-27 00:19  DirDiff512  coding freeware  [permalink]

DirDiff v2.0.3.512

Fixed issue with UTF-8 sensitive characters in ANSI file.
Fixed issue with Ctrl+Shift+Up/Down past start/end of files list.
Enable switching checkboxes on tree view with space key press.

twitter reddit linkedin facebook google+

Odo v0.4.1.511

2017-06-21 23:15  odo511  freeware  [permalink]

Odo

v0.4.1.511
Fixed a bug that caused mouse clicks not getting counted and nobody told me :,(


twitter reddit linkedin facebook google+

MurMurHash3

2017-05-19 20:09  murmur3  delphi freeware  [permalink]

@stijnsanders do you have MurMurHash3 code in pascal aswell ?
i truly liked your optimized code for md5 and all those

— _pusher_ (@_pusher_0x90) 12 mei 2017

Why, thank you. Eeuh, is it at all optimized? I took some decisions that may perform a little better than the reference implementation, but I haven't taken any time to compare to see if it actually performs any better or worse...

So MurMurHash3... Let's have a look, it's on wikipedia, so it's a thing. And the reference C implementation is in the public domain, great! Looks straight-forward enough, could boil down to a translation-job...

Rougly two hours later, got triple zero (0 errors, 0 warnings, 0 hints). Now for checking if I got it all right. Hmm, not much there, but those x32_64 match, so barring any typo's, assuming this pretty straight translate job will result in the expected behaviour for the other two functions, this should be it:

md5.zip (25KB)

twitter reddit linkedin facebook google+

Checking xxm for PHP's vulnerabilities

2017-05-12 07:56  xxmphp1  delphi internet freeware  [permalink]

If I read about a newly discovered vulnerability related to PHP, for example this one here, I try to find out if it would apply to xxm as well. 

In this case I guess there's nothing more than sending out the message, again and again, to sanitize your inputs, and poperly encode your output. Strings are never just strings. They are always an internal representation of a bit of textual data. So always think about that taking string values in, and preparing strings for output. A few weeks back I had to speak up to someone who wrote OutJSON:='{"field1":"'+value1+'"}';Little Bobby Tables comes to mind, though I'm not sure 'JSON injection' could be so devastating as SQL injection. (And OutJSON:=JSON(['field1',value1]); is shorter!)

The other time I found out it's a really good idea to strip nastiness like EOL's (CRLF) from headers added to a response, just in case a malicious script is up to no good. Come to think of it, that's also just another case of properly sanitizing your inputs...

twitter reddit linkedin facebook google+

TMongoWire v1.1, now on jsonDoc!

2017-04-04 19:46  mw_1_1  delphi freeware  [permalink]

→ TMongoWire

This took more effort than I anticipated, but I'm glad I saw it through. A tiny bit of history: when I started creating a Delphi connector to access a MongoDB instance, I had to create the tools to manipulate BSON, and since I really (really!) hate long lists of overloaded methods, I created something based on Variant values. It works great for ADO, so why not. Some time later I had to manipulate JSON, unrelated to MongoDB, so I took what I got, stripped BSON-specifics, into jsonDoc. There it evolved on its own and gained some useful features.

So it was time to re-work TMongoWire, making it use jsonDoc just like any other project. There's still a bit of BSON-specific code, but it deserves a unit of its own.  The downside is there's an extra unit to include in the project, but there were multiple already (jsonDoc, bsonTools, mongoWire and if needed mongoID, mongoAuth3 and mongoStream); the upside is the improved performance and features of jsonDoc, both of the current version and those the future could bring. Enough change to bump that version number:

https://github.com/stijnsanders/TMongoWire v1.1.0

Please let me know if the migration effort this generates is surmountable, and if I can do anything to help. In theory any references to bsonDoc, the BSON function and the IBSONDocument interface should keep working once replaced with references to respectively jsonDoc, JSON and IJSONDocument. The document object no longer implements IPersistStream, but this may have been stretching abstraction just a bit too far. We're on Delphi so plain old TStream should do, and apparently IPersistStream's Seek signature changed a tiny bit over Delphi versions, making it harder to make the code cross-version-ready.

twitter reddit linkedin facebook google+

Boxer

2017-03-24 21:57  boxer  delphi werk freeware  [permalink]

Drats. I thought I'll try something like Clover does, start with SetParent on any top-level window you could find with GetAncestor(GetForegroundWindow,GA_ROOT) and see what it gives. There's this old default MDI child mechanic that starts working, so I thought this might just work. But it looks like over at Microsoft the Windows 10 team decided to cut some corners. The theming doesn't do it's modern stuff and reverts to Vista-style borders, and the newfangled calulator, for example, won't even take the SetParent nicely any more. Oh well. For the time I'm still on Windows 7 at work, I guess it'll come in handy there for the time being. Probably in about a week I'll know if this could work where other desktop-management tools failed or were too cumbersome or too invasive...

How does it work? Boxer starts with an empty window, and will show an icon over the top-right corner of other windows. Click that to 'box' the window. The top bar shows a tab for each boxed window. Right-click to open a menu with options to unbox or close, and drag to reorder. Multiple instances play nice with each-other and show 'boxing handles'  next to eachother, the second instance labeled "A", the third "B", etc. To disable the boxing handles, right-click on the top bar to the right of all tabs.

Anyway, if you feel like having a try if you can do better, have a look at the code here. Download a binary here: Boxer.zip (209KB)

twitter reddit linkedin facebook google+

A story about two task-keeping-applications.

2017-03-18 17:22  s2tka  coding delphi werk freeware  [permalink]

Let me tell you a story of two task-keeping-applications. Once there was a team that was struggling to keep track of the work it was doing, had done, and still had to do. It was frustrating to work on the team, hard to schedule things, almost impossible to estimate when things where going to get done. And since clients are what they are, changing requirements and additional requests would cause much more disruption than expected of a team that calmly and firmly is determined to deliver the best of market solution.

An internal brainstorming-session about what to do about it, ended in two opposing visions about what decent issue-tracking should be about, and how a system that (co-!)operates with the team members should be structured internally, should behave, and which duties it should perform either by itself, or by effect of using its rules and restrictions correctly as designed.

In an attempt to get to the best solution, two teams were formed to develop each a project based on one of the two views. An evaluation would follow determining a winner, or if would it be possible to synergize the best parts of both into a single system.

A side note: from a managerial point of view this is a very tough decision. Allocating a lot of resources to work on internal structure, takes resources away of the work that brings in revenue, and such undertakings typically risk getting frivolous or spinning out of control. But I guess it's normal that all stop rowing to help keep the boat from sinking. So, apart from being intrinsically aware of the exceptionality of this opportunity, coordinators were given instructions to keep to a strict schedule in these projects and a determined focus on delivering a workable proof-of-concept quick. Sounds like a good work-ethic to apply generally, if you ask me.

A few weeks later progress was made, and the prototypes were already being used to keep track of the issues of these new task-keeping-applications and other projects. The main difference between design visions became apparent soon enough.

One application centered around the list of work items. Care was taken the entry form was extensive enough to have fields for all of the details about a work item, its categorization, its relation to the project, an outline of the projected outcome. An overview would show the current list, possibly filtered for those items assigned to you.

The other application was centered around getting information out of the system, especially structure and relation between items. Users would enter small specific reports, and add them to the best suitable node in a tree-structure, optionally marking relations with other items over branches. The overview showed an expandable structure starting at the root items. It also could have a filter applied, but would potentially show you an entirely different structure of the same data, when using a different relation type.

Having two working prototypes, attention gradually reverted back to the serious work and some of the frustrations of before were abated. After a few months an evaluation was undertaken.

One application had rendered itself useless. The first weeks of usage, a lot of entry happened, but without consensus about categorization, the list was enormous without a clear way of grouping relevant items. Duplicate entries and ambiguous task-descriptions were unresolved, causing confusion.

The other application was doing better. It needed work, but offered a good view of what had to get done.


Enough about the story. Reality, of course, is much more bleak. As no sane manager would make such a shift in resources of a troubled organization, the 'two teams' actually stand for the existing team using an off-the-shelf something badly administered on one side, and on the other side, well, just me, toiling on something potentially better, in my off-time.

I started — like many — with a plain text-file, then a spreadsheet. Then I took the step to design a database for it, but wanted something to do entry and retrieval with roughly the same ease-of-use that a spreadsheet would offer. Working with tree-structures for some other projects, I wanted a single structure to serve as the basis to store data in. Specifically with projects and tasks — as projects tend to have sub-projects, and tasks split into sub-tasks — using branches of a tree would enable to keep this distinction conveniently vague. If you add representation for entities like users and clients into this tree-structure, and relation between nodes over branches, you've got a richness to model much more of the world, and keep track of its changes, past and future.

From there it grew slowly, over years, into what it is now: tx. First versions suffered a notoriously bad interface, but I hope that has improved. For long I was its only user, but some cooperation features have been added since. What's left for me is to keep improving tx where possible, and demonstrating it to people what it's about.

Though entry and structure is important, where a task-keeping-system can really shine is helping to keep an overview. A cleverly designed filter can limit your view to exactly these items that are relevant to you, but in tx you get the additional option of having these items display with exactly those sections of the tree-structure they have in common. Also, I personally feel the most important items on any list, as long as it may be, should be on top. So when tx displays a list of items, they get ordered using their weight, determined by a combination of factors such as task-type and current status. As a task progresses from an active state to a final state, it may move down the list or even fade from view into an archival state.

And there's much, much more I could talk about, but it's all created out of necessity and designed adhering closely to a central vision of what a task-keeping-system should be and what it should be doing for you in order to be able to depend on it.

In conclusion, and for those people that — like me — skip long stories to the last paragraph, it's so very important that information systems succeed at keeping an up-to-date model of reality, and offer you the freedom and easy to update it's view of the world. Especially so with task-keeping software you depend on to keep track of the progress you make with projects, and to help stay on top of what's important. All of that served as the basis for developing tx.

twitter reddit linkedin facebook google+

jsonDoc, jsonV 1.1.0

2017-03-09 01:22  jsonDoc110  delphi freeware  [permalink]

jsonDoc, jsonV v1.1.0

While working on TRethinkDB, I noticed something was wrong with document re-use. My original idea was to re-use a single IJSONDocument instance to process a list of similar documents, but keep the set of keys allocated, only overwriting the values from the new document. (If you're interested about the code, search jsonDoc.pas for FLoadIndex.) Nothing an extra internal-use interface can't fix. Then I noticed that IJSONDocument.ToString could do with a revision because it wasn't using the new IJSONEnumerator, though it should.

Then I noticed the feature I once added to function JSON, where you could declare embedded documents by listing a value of '[', and closing the document with a key of ']'.  The problem is that '[' could be a perfectly valid value! If it were to come via a variable, that would change the behaviour of the JSON call, and that's a big no-no. Also square brackets usually mean arrays in JSON-land,  so it's perhaps confusing that I chose square brackets because you already need them in the JSON([...]) call. So I changed it into a key suffix of '{', and a closing key of '}', counting on braces being really abnormal in key names. If you really need a key name that ends with '{', don't use the JSON function, but just d['x{']:='y';

A minor downside may be that JSON calls with braces in the string constants, break the comment block if you want to comment out a section of code with braces, but there's always (* *) or //. It by far doesn't outweigh how elegant this new solution is, but it's a breaking change (existing code using '[' values won't work correctly any more), so instead of version 1.0.6 I think it deserves a jump in minor version number to 1.1.0.

While I was at it I fixed some minor issues with jsonV, so remember to update that one as well. Hope you enjoy the changes. (Next up may be replacing TMongoWire's bsonDoc with jsonDoc...)

twitter reddit linkedin facebook google+

TRethinkDB

2017-03-06 23:03  TRethinkDB  delphi freeware  [permalink]

Delphi RethinkDB driver

I've given it another try and, in part thanks to a Windows executable that performs well,  and thanks to the gentle people over at the rethinkdb-dev group, I've got something working. It appears the basic functions you would expect from a driver work, but more advanced queries need some more extensive testing. (I've read about a driver testing harnass, but to replicate that around my driver, sounds like a separate project in its own.)

There are some strange sections in the design of ReSQL that show it was written for a weakly-typed language, but using variants and interfaces, I hope I've struck a nice balance between versatility and still having the best suitable methods showing up on auto-completion. Something that should be there may be hidden from view this way, but will probably be because of inaccuracy or error of my part.

Which leaves my in roughly the exact same position as I was closely after completing TMongoWire: having success learning about a new(ish) NoSQL DB the hard way by trying to connect to it directly with Delphi, but nothing in the way of an effective application that would use it for something vaguely useful. I've learned some people have effectively created things with TMongoWire (yey!) but I myself don't really have a good idea (or the motivation) to build something on its own that would use the features of these new database services to their best intent.

If you've got a good idea, please let me know. If you want to know more about TRethinkDB, let me know.

twitter reddit linkedin facebook google+

MetaClick v1.3.0.500

2017-02-03 22:54  MetaClick500  freeware  [permalink]

MetaClick v1.3.0.500


twitter reddit linkedin facebook google+

MetaKeys v1.1.0.500

2017-02-03 22:52  MetaKeys500  freeware  [permalink]

MetaKeys v1.1.0.500


twitter reddit linkedin facebook google+

DirFind v2.0.4.494

2016-12-16 22:30  DirFind494  delphi freeware  [permalink]

DirFind

version 2.0.4.494


twitter reddit linkedin facebook google+

MetaClick v1.2.3.497

2016-12-10 23:57  metaclick497  freeware  [permalink]

MetaClick

v1.2.3.497
- fixed issue about division by zero at start-up
- now open source on github

twitter reddit linkedin facebook google+

 

Archive... Search...