Story Driven Development with DSL and REST

Every new website project starts with a "story"; a narrative that describes the appearance of the site and the way it responds to user interactions and other events. Stories can be visual (picture-based), they can be expressed as text, or they may simply exist as conceptual entities in the shared awareness of the development team. The last of these leaves no permanent record other than a series of emails and Slack messages, but in most cases the website itself displays its purposes clearly enough.

Story Driven Development (SDD) may or may not be a recognized term, but I use it here to mean a formal, documented process with 3 steps:

1 Write the stories
2 Do the development
3 Test the result against the stories

We start with the expressed requirements of the client. These are formalized as design documents and used by the client to verify the final product. Along the way there may be many steps all bundled together as "Do the development", and for all but the most basic websites a program of some kind will run in the browser to control aspects of the design. JavaScript is now the only game in town as far as the core programming language is concerned, but it is increasingly layered with tools and frameworks like Angular, React and Vue. These are all tools for professional programmers that leave little space for the client to be involved in the development process and little opportunity to verify progress until the final product is delivered. SDD doesn't care how you do the development, only that it's following the stories.

For over 50 years the software industry has had a poor record of delivering what is wanted. Projects overrun on both time and budget and frequently fail to work as requested. Sometimes they collapse in unrecoverable heaps, but the response of the industry is usually to throw more complexity at the problem in an often futile attempt to squash bugs before they occur. The consequence, even for initially successful projects, is an ever-growing dependency on a coterie of highly-paid, highly trained software engineers who will not be around years later to maintain the products they have built. The result is a downward spiral of reliability and a loss of fitness for purpose, as many customers are unwilling or unable to afford the equally high cost of maintenance.

But it doesn't have to be like this. With more emphasis on the stories themselves we can build code that is understandable by product owners as well as software engineers. It's a fundamental principle of Open Source that "with enough eyeballs, all bugs are shallow", and it goes without saying that the easier your code is to understand the easier it will be years down the road to find people who can be safely trusted to maintain it.

Let's look at an example.

We have a web page that displays some paragraphs of information. If there is too little content the message may not be properly conveyed, but if too much then users may be overwhelmed by it. So we decide to compromise by using a 'concertina' design pattern, putting a "more..." link at the end of each topic, which when clicked causes further text and/or images to be added to the topic. The "more..." link disappears and a "...less" link appears at the end of the new text. When this is clicked things are restored to their previous state.

The basic story for this feature looks something like this:

"When the user clicks the More button...
1 Close any popout blocks that are open
2 Hide the More button
3 Make the popout block visible"

Let's now rewrite step 3 in terms that relate to the content of the web page:

Set the display style of the div whose id is "popout-1" to "block"

This is pretty clear and unambiguous, but you may need to do more than one thing to the popout div, such as setting a background color or giving it a border. Also, the syntax is a little clumsy and could be made clearer. To avoid repeating the same text each time, let's create a named entity that we can use whenever we want to refer to the div. We can call it Popout1 so it's easy to see which id it relates to. In software terms this entity is usually called a variable or a property and the action we want to perform is to attach this entity to the div element in the DOM:

attach Popout1 to the div whose id is "popout-1"

Now when we want to show the text, all we need to say is

set the display style of Popout1 to "block".

When a story like this is translated into JavaScript it leaves the normal, everyday world and enters the domain of the programmer. From here on, every addition, every change, however simple, needs a programmer to cast the necessary magic spells.

Although the story is expressed in fairly ordinary English it's also a formal, unambiguous description of some actions. This is really what defines a computer language, so in an ideal world there would be a compiler able to handle instructions like these. Languages of this kind do exist - they're called DSLs - Domain-Specific Languages. Rather than catering for general-purpose programming concepts, a DSL uses a syntax and vocabulary that has particular meaning in the domain for which the DSL is written. Here in the domain of the web page it's all things relating to browsers and what they do.

DSLs can be used by programmers and non-programmers alike, though the former can be expected to produce better code more quickly and with fewer bugs. The code may not be simple - a DSL script may be asked to do complex things and if complexity is really there it's not possible to hide it just by writing it differently. The difference is that no matter how complex a DSL script becomes it can still be read and understood by anyone having sufficient domain knowledge, without also having to be a JavaScript expert. Every command in a DSL script has a real-world meaning and it takes only minutes to learn the essential features of the scripting language.

A good DSL has other advantages. It can offer a built-in debugger that allows you to step through the script, examining variables as you go. Most bugs are quickly revealed by this technique. And in a WordPress context, much of the coding can be done by people with ordinary Editor rights on the website. This is because DSL scripts live in the page with the HTML, so it makes a DSL very convenient to use for simple web pages that only require a small amount of coding to handle their features. No need to find a place for custom JS files; everything is done using the WordPress editor.

You may have realized that the DSL I've been talking about all this time is EasyCoder. Others may exist of course, and I would encourage you to look out for them, but I'll continue to focus on what EasyCoder offers to website builders.

You may also have formed the impression that EasyCoder is only for simple websites. Most of the examples are of necessity simple because that's the nature of examples. In fact, EasyCoder has some industrial-strength features, that I'll describe in part 2 of this tutorial series.

Placing a script in the page with the HTML is fine for simple websites, but the obvious objection will be raised that it makes the source code visible for the world to see and to copy. Also, the tendency now is to go for single-page designs, which result in much larger scripts that combine the features of what would otherwise have been several pages. Whether JavaScript or EasyCoder, the larger the script the longer it takes to download, and in the case of EasyCoder, to compile prior to running.

Both of these objections are dealt with by using REST. As supplied, EasyCoder comes with its own REST server in the form of a PHP script in the plugin folder. This manages content in any number of MySQL tables added to those already used by WordPress. The tables all share the same structure; an id, a name and a value, and one of these tables can be used to hold your scripts. Another can hold chunks of HTML for use by your pages, and a third your  CSS definitions. Other tables hold data specific to the needs of your website.

An EasyCoder web page can comprise as little as 3 lines of code, such as

variable Script
rest get Script from `https://mysite.com/wp-content/plugins/

cat easycoder/rest.php/

cat ec_scripts/name/main`
run Script

In this script the word cat performs string concatenation, keeping the lines short enough to fit on a mobile screen.

The script declares a variable, performs a GET request on the REST server (asking for the item named 'main' in the 'ec-scripts' table), fills the variable with what returns then runs the script. Everything from this point on is handled by the script. This is the ultimate in hiding your data from curious eyes, yet the script and the HTML are still there, easily accessible by anyone with the right accreditation.