The Titchmarsh Effect November 18, 2018

Alan Titchmarsh is the presenter of a number of similarly themed programmes in which instant transformations are done on people's gardens without their knowledge. A team of experts creates beautiful designs, lots of problems are overcome along the way, usually to a very tight timescale, and the owner - or should I say victim? - is always delighted, or at least pretends to be. It’s all great TV.

But consider: If the owner were a keen gardener the makeover wouldn’t have been needed in the first place. And if they wanted a beautiful garden and could afford to pay for it they’d have done so. Who is going to maintain Alan’s loving creations?

Something these programmes rarely seem to do is return a few years later and see how well it stood the test of time. I fear that were they to do so it would make for some rather more depressing TV, with the inspiration long gone, original concepts overgrown and weeds rampant. Entropy rules.

It's rather like that with software projects. Everything is hunky-dory on the day of delivery but the system has to be maintained, usually by people with a lower level of expertise than the original team. The result is a gradual deterioration in the quality and reliability of the product. There's also an ever-escalating cost of repairs because each time work needs to be done a new team has to be brought up to speed and before they can make progress they have to first unpick the mistakes of the previous team.

For both gardens and software, the root cause is usually excessive complexity and an inappropriate choice of technology. The principle too often ignored is

"If you can't maintain it, don't build it."

If your client can afford a full-time gardener, then go ahead with the elaborate design. If not, give him a lawn and some shrubs. Similarly with software; only use tools and techniques that are readily understood by the people who will be tasked, years from now, with the maintenance of the system.

The problem with complexity

Every new technology is touted by its evangelists as the answer to all problems, but it ain't necessarily so. Just because it works well in a highly professional environment with long-term support guaranteed, you shouldn't assume it will automatically meet your needs now and into the future. The languages and frameworks that are favored today by the software industry all have a lifetime, after which it will be hard to find anyone experienced in their use. That translates to high costs and uncertain results.

The response of the software industry to complexity is to add more complexity. That appears bizarre but actually it’s not as daft as it sounds if we compartmentalize the complexity properly to keep it well hidden from all but those who really need and are able to handle it. The trouble is we tend to proudly display it as a sign of machismo, placing it in full view where it often becomes a barrier to understanding.

Languages like JavaScript and the frameworks like Angular and React are raw, naked software; built to work well in the hands of experts. You should never expose this nakedness to your customers, most of whom will find it upsetting. They would prefer to see it decently clothed by the domain in which they operate.

Story-Driven Design

Customer needs and wishes start life expressed as stories, written in English by domain experts, and if you want to fully benefit from the experience of those experts you should depart as little as possible from that format. SQL is the leading example of this principle - nobody writes database code in C or JavaScript. SQL is probably the best advert there is for languages that match the domain and keep the complexity hidden. And it’s not alone; although it may be stretching the definition of a language just a little, Excel macros can also be described as a DSL.

Where are the DSLs?

You may argue that for other domains, such languages don't readily exist and will have to be written before any useful progress can be made on the project itself. You might also believe that special skills will have to be brought in and that the resulting effort cannot possibly be cost-effective.

Well it may be true that the DSL you need probably doesn’t exist and will have to be written, but let me assure you that a DSL is no harder to write than the complex nakedness it is required to cover. Yes, there’s an overhead, but once the initial preparation has been done, development and maintenance of the end product get easier, so in the longer term the savings are significant. It’s also a fact that once you’ve written the core of a DSL, adding new features to it becomes a simple task.

A well-designed DSL is highly modular, with each language feature handled by its own private code section. Much of the work is done by calls to core functions, which get massive use that flushes out bugs, leaving them little room to hide in. As the language develops the core gets less and less attention, so earlier features are rarely disrupted by new additions. Functions outside the core are rarely called from anywhere outside their own modules, which minimizes the tendency for spaghetti code to develop.

Writing a good DSL requires a positive attitude towards encapsulation. The starting point for any new functionality is the syntax you require it to provide to its users, so it works best if the functions being implemented have a tight, well-defined API. Google Maps is a good example of a highly complex product with an elegant, lean API that allows it to be readily encapsulated in DSL code. With experience you find that almost anything can be encapsulated; in most cases the hardest part is in finding a source syntax that reads well and is unambiguous, that is, unable to be confused with other functionality using the same language keywords. An example would be the need for care when implementing words like set and create, both of which are likely to be heavily overloaded in a typical DSL.

For WordPress

If your project is based on WordPress then much if not all of the work has already been done. The EasyCoder plugin is a DSL that implements a set of language features relevant to the needs of web page builders. Its comprehensive feature set includes support for managing DOM elements and also things like JSON and vector graphics. The package comes with a REST server that can quickly be leveraged to provide support for load-on-demand code, HTML and CSS, facilitating the construction of large single-page websites.

EasyCoder is built using its own plugin architecture that lets new syntax to be added without any need to change existing scripts. New functionality can be provided by anyone; not just theEasyCoder team itself. No conventional compiler techniques are used; script commands are handled by simple text processors that analyze the incoming stream and create a description of what they find, in the form of JavaScript objects. These are handled at runtime by code that is essentially just a thin wrapper around the underlying functionality that is being exposed as script. Any competent JavaScript programmer will take only a few hours to get to grips with the structure of both the compiler and the runtime.

For the rest

If the functionality offered by the standard product is inadequate, even after adding your own plugins, it should be noted that EasyCoder is the name for both the browser language provided to WordPress users and for the core engine itself. What this means is that the engine is largely independent of the language features so it can be the basis of an entirely new DSL where the only basic requirement is that it should all be written in JavaScript. The entire source code is available on GitHub - see the About page - and anyone is welcome to adapt it to their own needs.

Categories: Uncategorized