Cognitive Consonance

There is, in my view, an inconsistency with the clojure core API with regards to type checking.

Consider the two functions contains? and even?

contains? returns true if a certain collection has a certain key:

=> (contains? {'a 1} 'a)
true
=> (contains? {'a 1} 'b)
false

If you pass an object which is not a collection, contains? silently returns false.

=> (contains? 1 'a)
false

I.e., a type error is not distinguishable from a collection which does not contain a certain element.

even? , the function to check if a certain number is, well, even, behaves in a completely different fashion:

=> (even? 'a)
java.lang.ClassCastException: clojure.lang.Symbol cannot be cast to java.lang.Number (NO_SOURCE_FILE:0)

A type error on the parameter raises an exception with even?.

From a design philosophy I really do not like this inconsistent behavior: For the same type of error (a typical error pattern) the API behaves in a clearly different way.

The 2 points that are important (but, alas, are not the fundamental issue of this post) are:

• Core functions should have a coherent and consistent way of dealing with type errors. This is the most important point.
• If they have a consistent way of dealing with type errors, my preference would be for a behavior like even? (ie, throw) and not like contains?.

Yes, I do understand that other reasons might have taken precedence (like performance). I still don’t like it.

But the beauty of clojure is that one can redefine these “core” functions. For instance, I prefer to have

(defn contains? [coll key]
  (if (coll? coll)
    (clojure.core/contains? coll key)
    (throw (new ClassCastException
                     (str (type coll) " cannot be cast as collection")))
  )
)

[Newbie alert: there might be better ways to design a function like this (suggestions welcome).]

Now, one as to be careful not to import the original contains? into a namespace. Easy done in clojure:

(ns myuser (:refer-clojure :exclude [contains?]))

This has to be done before the definition of the new contains? (note that, when calling the old contains? it includes the full namespace).

Of course, redefining core functions (even if inside namespaces not seen outside) is a bit like laying a mine field and probably has to be done with care. Irrespective of that, it is good to be able to have a language which allows one to express itself with the syntax and semantics that one desires, and not to be constrained to the whims of the original developer.

So far, no big problems found with Clojure (at least until now).

Having read Tim Bray’s tips for Clojure newbies from a newbie, I decided to write a version of my own.

First, I generally agree with Tim’s observations, so this post is more like minor extension. In fact I recommend reading Tim’s version first.

My background so that you know my point of view: Added to Java, C and Python experience I have lots of exposure to Prolog (which is, like Clojure, homoiconic) and some experience with OCaml (a functional language). I have had some practical contact with other “modern languages over JVM”: Scala and Groovy. No Lisp experience (other than basic elisp).

Reading suggestions: Other than Tim’s comments I would add: The clojure site information is really not very useful as a starting point, the reference part assumes that you already know quite a bit and it is though on newbies, the only thing that I tend to use is the API section. Like Tim, I also use Mark Volkmann’s introduction (It is my main documentation, and I would be a bit more positive than Tim about it. For an intro article it is great. I strongly recommend it as a starting point and as the reading anchor during the first weeks). Next week I plan to order Stuart Halloway’s book, so I still cannot comment on that.

On a more general note, while learning Clojure, I’ve found Paul Graham’s On Lisp (available for free), a gem and I would strongly recommend it. It is not an easy read, it probably takes months to digest the content. But it is really a great book.

Clojure is a fast moving target. Documentation of many modules and functions might not be up to date with the current code. I have noticed that sometimes the best “documentation” ends up being reading the code (clojure is hosted on github and so are many satellite projects – In fact getting familiar with github is probably another recommendation).

Some of the contrib stuff is a bit too green, and I would recommend inspection of some of the modules before using them. Just because it is accepted on clojure.contrib it does not mean it is production quality, has even bare functionality or the exposed API is stable. As an example the graph API is minimal and I question if the graph structure directed-graph is enough to represent a general directed graph (future changes to it will probably break existing code built on top of it). This is not a criticism, a new product is bound to be fast changing, and agile methods of development will entail lots of instability at the beginning. But a caveat should be added to the stability and completeness of parts of clojure-contrib.

Regarding editors and IDEs, I would probably recommend for you to stick with what you feel more comfortable with (vi, emacs, netbeans, eclipse, …). Note that Clojure, being a Lisp derivative has a big share of its user base on emacs. I mainly use Netbeans using the wonderful enclojure plug-in. I can only say that encloure is stable enough for usage and I recommend it if you are a Netbeans type of person.

I also second Tim’s comments on namespace hell. Uses/imports/requires can be quite confusing. Hell is too harsh of a word, but purgatory seems an accurate description . Also, as Tim says, the clojure mailing list and IRC channels are very, very helpful.

One of the most annoying kinds of bugs come from typing problems, things like this:

(defn hiddenBug [a b]
  (println a) ;lets do a println for debug purposes
  (println b) ;lets do a println for debug purposes
  (if-not (= a b) (println "they are different!"))
)

Now lets call this:

(hiddenBug 'x "x")
x
x
they are different!

Notice that, when you are debugging a and b will seem equal on a println, but they are not (one is a symbol, another a string)!

The biggest gotcha that I have been getting is the expectation (Java based) that stupid things raise exceptions, but sometimes they don’t. Here is an example:

user=> (contains? '(a b) 'a)
false
user=> (contains? 'blab 'a)
false
user=> (contains? (list 'a 'b) 'a)
false
user=> (contains? ['a 'b] 'a)
false
user=> (contains? '(a b) :a)
false
user=> (contains? '(:a b) :a)
false

In all these cases, the Java-expecting gnome inside me was hoping something of a throw (as the first argument is not of the type required by contains? ) . It is not clear to me if this is a design issue with contains? only, or it is something that is standard along the whole API. But I notice this from the API reference:

(even? n)
Returns true if n is even, throws an exception if n is not an integer

So it seems that the design is not homogeneous throughout the API as some functions throw exceptions. I would probably prefer a throw when the type of arguments is wrong, but people with lots of Lisp experience might have a different view on the issue. Anyway I would like to understand if the lack of homogeneity is a feature or a bug.

More than 10 years ago I participated in the development of an University IT system (the front- and backend to maintain grades and that sort of stuff). The system was based on a DB/2 backend (a very nice database system) with the business code stored on a Prolog interpreter (Prolog interpreter which was in-house developed) and the web backend being a Java servlet engine (the old JServ, the thingy pre-Tomcat from Apache). Prolog is famed to be slow, and Java (at that point in time) was very slow. Surprise, surprise… the bottleneck was on the DB/2 server. Eventually, as the system grow (and the database hardware was beefed up) the bottleneck come forward to the business and web tiers, but the problem was sorted by just adding more machines: The contention was on a bunch of parallel independent process, they could be run on separate machines.

The example above illustrates why the concurrency problem posed by multiple core CPUs and GPUs, might not be that much important:

1. Many problems are not CPU bound anyway, and even if they are, the bottleneck might be elsewhere. Another example: I am the proud owner of 3 cheap, slow laptops (one being a netbook). For my use case I really don’t need faster applications, I wonder how many users really need more than they already have?
2. Even if more CPU/GPU power is needed, a loosely coupled model (without much interprocess communication and contention issues) might be enough. This is typically the case of many web apps, which can scale by just adding more computers which run independent processes.

Concurrency, even with modern abstractions, is hard. It should be avoided if possible and it can be avoided in many applications. If it cannot be avoided, maybe a loosely coupled model is enough… Guido van Rossum has a nice take on this issue.

This is important as concurrency is being touted as an important criteria to evaluate languages. Modern functional languages (think Scala and Clojure) are being touted as a better option precisely because they are better to do concurrency (both because of functional – “no changing state” – programming and the availability of libraries implementing nice concurrency paradigms like actors).

When addressing this importance of this issue, I would propose, that people would ask themselves this: “Am I developing computationally intensive software?” and “If I am developing computationally intensive software, can I live with loosely coupled models of computation, preferably processes with no shared memory?”

This is not to say that there are not some cases where tightly coupled computing is a good idea. It is just that, this complex solution might be an overkill for many problems.

I would just like to add that I am not defending my cause, in fact it is quite the opposite. There is actually some content produced here, in the past, on how to tackle concurrent programming:

1. LOSITAN – A multicore-aware Jython-based (Python for the JVM) Web Start application to do selection detection.
2. An introductory tutorial on concurrent computing targeting computational biologists – Part 1, 2 and 3

When you read about programming language comparisons, the main narrative for comparison is normally about the paradigm(s) supported. Lisp, Haskell, Scala, Clojure fall mainly in the functional realm. Prolog is logic. Smalltalk OO. C and Fortran, imperative. Most of them are not “pure” paradigm (e.g. you can make nice OO designed programs in C – just check GTK’s GLib library if you disagree, imperative coding in Prolog, and so on…), but that is besides the point.

The point is that, when comparing programming languages, the main issue of discussion is the bloody paradigm thing.

Paradigm is not really that important! In fact, as said above, you normally can tweak a language to write in your favorite paradigm. Sure the ability to do that varies from case to case, but in most cases that I can think of, it is really not difficult to cross paradigm boundaries. In fact, I would go as far as to argue that it is easier to do proper OO design with C using GLib then with the highly complex and convoluted C++.

Before going into the fundamental point that I want to make, I would also note that ecology matters: Are there good libraries? Good documentation? Does it run on a virtual machine? Portability? Nice community? User base? That is, when comparing programming languages all that is around the language is more important than the language itself. Just ask all the poor of us poor Prolog/Lisp/Haskell fans why are we doing Java/C++ during most of our day? It puts bread on the table, and, for the most of us, that is the most important criteria (I prefer not to starve!).

But, going to the main point here, I would like to propose that one of the fundamental points in comparing programming languages from a technical standpoint is homoiconicity.

Just to remember, an homoiconic language is a language where the program is represented as the core language data-type. Code is a data type.

If you classify languages according to homoiconicity, then they split in completely different ways:

1. The homoiconic bunch: Lisp, Prolog, Ioke, Clojure, …
2. The non-homoiconic bunch: Cobol, Fortran, C, Java, Goovy, Scala, Haskell, OCaml, [A very long list follows]…

From this point of view, the comparison of say, Clojure to Scala as sister-languages makes little sense, as they fall in different groups.

Homoiconic languages lend themselves to – by construction – metaprogramming and extensibility (think very easy embedded DSLs). And some of these features are difficult (with varying levels of difficulty) to implement in non-homoiconic languages. At best (as “best” I am thinking of some scripting languages like Python), they are awkward to do in a non homoiconic language.

As a side jab, last time a checked, Scala was very very poor on metaprogramming (has that changed?), making it the only “modern” language which seems to be scorning metaprogramming. Scala can still be DSL-extensible (I offer my own example both in Scala and Grovy: Ronald: A Domain-Specific Language to study the interactions between malaria infections and drug treatments.

One could argue of the value of doing programs that reason about themselves (and that idea has very bad karma coming from assembler – an idea so old and so disconnected from current reality that I am not even going to discuss it). I am surely on the side that proper metaprogramming is one of the core features of any elegant, productive and declarative solution.

Also, a very nice side effect of having code as data, is that the syntax of homoiconic languages is normally very, very simple (as in trivial to learn). This is just a side effect, but compare this with the learning curve of, say, C++ syntax. There is also a philosophical issue here: you get a simple, highly flexible environment, where complexity is tacked not by having a complex mammoth that tries to address all possible cases, but by a set of plastic, bendable building blocks.

Homoiconicity is not a black-and-white feature. For instance, Lisp macros are not first-class objects (I am a Clojure newbie, so feel free to correct me) so you cannot metaprogram with them. Prolog seems to come close. In fact, to a Prolog programmer, Lisp macros seem especially inelegant as the are “out of the system”.

As of this moment (this will change in the future), I would suggest some care in installing the new Ubuntu release, especially on newbie computers that depend on Mobile Internet. The shipped kernel has a bug with some USB dongles: It mounts the small SSD drive on the dongle, instead of making the modem available. This means that, when you connect the dongle there will be no Internet. If you are a geek, the problem is easy to correct, but for normal users (wasn’t one of the points of Ubuntu to make free software easily available to normal users?) it will be a bad experience.

Furthermore, after the problem above is corrected, many times is fails to get the DNS info on connection.

These are both old problems, that have been solved in the past and have re-emerged.

How serious is this? Most Vodafone dongles (Vodafone is a very big pan-European mobile operator) that I know off show this problem.

How Ubuntu Q&A let this one pass, let alone mark this bug as only of “medium” importance baffles me. The number of normal users that this has the potential to impact is substantially high.

I am doing some development in Clojure (a Lisp type language for the JVM). Lisp as in a clone tailored for the JVM, not Lisp as only “functional programming”. I note, by the way, that more than functional programming, Lisp is an homoiconic language.

I developed a simple system to specify Swing menus in clojure, here is an example:

Simple Menu

The following “micro-language” was developed to specify this:

 (getMenuBar actionManager '(
    (menu {
      :text "Project" :key "P"
      :content (
        (item {:text "New" :key "N"})
        (item {:text "Open" :key "O"  })
        (item {:text "Close" :key "O" :id "Close" :enabled false})
        (item {:text "Recent" :key "R"})
        (separator)
        (item {:text "Exit" :key "E"})
      )
    })
    (menu {
      :text "Options" :key "O"
      :content (
        (item {:text "Rendering" :key "R"})
      )
    })
))

The code is very easy to read, I hope: two menu items, with a few menu entries with text, ability to enable/disable and accelerator keys, plus a separator.

Notice the actionManager on top, is it the (very simple) event processing function which receives only a text as parameter (to identify the selection). The text is simply the menu text, or, if specified an id. Not the most general solution, but enough for simple menu structures.

The code? Below is the _complete_ implementation.

 
(ns org.tiago.swing
  ;(:require clojure.contrib.def)
  (:use
    [clojure.contrib.seq-utils :only (flatten)]
    [clojure.contrib.def :only (defnk)]
  )
  (:import
    (java.awt.event ActionListener KeyEvent)
    (javax.swing JFrame JMenu JMenuBar JMenuItem)
  )
)
 
(defnk createFrame [title :menuBar nil]
  (def frame (new JFrame title))
  (. frame setDefaultCloseOperation (. JFrame EXIT_ON_CLOSE))
  (if menuBar (. frame setJMenuBar menuBar))
  (. frame pack)
  (. frame setVisible true)
  frame
)
 
(defmulti addMItem (fn [manager x & rst] (first x)))
(defmethod addMItem 'item [manager content menu]
  (let [params (second content)]
    (def mItem (new JMenuItem (:text params)))
    (if (contains? params :id) (. mItem putClientProperty "id" (:id params)))
    (if (contains? params :key) (. mItem setMnemonic (. (:key params) charAt 0)))
    (. menu add mItem)
    (. mItem addActionListener manager)
 
  )
)
(defmethod addMItem 'separator [manager sep menu]
  (. menu addSeparator)
)
 
(defmulti getMBItem first)
(defmethod getMBItem 'menu [desc]
  (let [params (second desc) manager (last desc)]
    (def menu (new JMenu (:text params)))
    ;Assuming mnemonic is ASCII CODE.
    ;java7 has . KeyEvent getExtendedKeyCodeForChar
    (if (contains? params :key) (. menu setMnemonic (. (:key params) charAt 0)))
    (if (contains? params :id) (. menu putClientProperty "id" (:id params)))
    (dorun (map #(addMItem manager % menu) (:content params)))
    menu
  )
)
(defmethod getMBItem :default [arg] (new JMenu "UNK"))
 
(defn getMenuBar [actionManager menuItems]
  (let [manager (
      proxy [ActionListener]
      []
      (actionPerformed [e] (let [obj (.getSource e)
                                 id (.getClientProperty obj "id")]
        (actionManager (if (nil? id) (. obj getText) id))
      ))
   )]
   (def menuBar (new JMenuBar))
   (dorun (map #(. menuBar add %)
            (map #(getMBItem (concat % (cons manager ()))) menuItems)))
    menuBar
  )
)

OK, comments have to be added .
From a declarative point of view, not bad at all.

My first Lisp program. It completely baffles me that, 25 years of programming with all the languages imaginable (including some functional like Caml or highly declarative like Prolog), I never tried Lisp.

I have been toying with Processing. Processing is..

…an open source programming language and environment for people who want to program images, animation, and interactions. It is used by students, artists, designers, researchers, and hobbyists for learning, prototyping, and production.

Processing is actually a full blown IDE with a language based on Java (it is JVM based).

I am fascinated by its community and how most things seems to cleverly done.

Here is my first example (applet alert!). After it loads, move the mouse over the applet to see it in action (you might have to mouse click for it to start on some browsers, like Opera):

Please install Java!

The code for the above is just:

void setup() {
    size(400, 200);
}
 
color elColor = color(255,255,0);
 
void draw() {
  background(255);
  float delta = (1.0*mouseY)/height;
  if (random(10)<1) {
      elColor = color(random(255), random(255), random(255));
  }
  fill(elColor);
  ellipse(width/2, height/2, 80.0*mouseY/height, 50.0*mouseX/width);
  fill(255.0*mouseY/width,255.0*mouseX/width, 0, 255*mouseX/width);
  quad(0, 0,
      delta*width/2, (1-delta)*height/2,
      delta*width/2, height - (1-delta)*height/2,
      0, height);
  quad(width, 0,
      width - delta*width/2, (1-delta)*height/2,
      width - delta*width/2, height - (1-delta)*height/2,
      width, height);
}

The vibrant community behind seems to produce quite a lot of really neat examples. Go and check for yourself…

It is interesting to see how different people tackle the ongoing multicore (and GPU) software “revolution”. There are strong philosophical differences on how to develop for these new concurrent architectures. Lets start with the extremes.

The most interesting extreme comes from Guido van Rossum (aka Python benevolent dictator for life): He suggests that if you want to use the available processing power of multiple cores you should have separated processes, let me quote:

[...] doesn’t mean that multiple processes (with judicious use of IPC) aren’t a much better approach to writing apps for multi-CPU boxes than threads.

Just Say No to the combined evils of locking, deadlocks, lock granularity, livelocks, nondeterminism and race conditions.

Some similar arguments are made by the message passing crowd, which seems to be quite happy with a model based on explicit message passing between separated processes.

The fundamental idea here is that shared memory between parallel computing threads can lead to a lot of grief and sorrow, thus is is better if all the data memory space is the sole propriety of a single thread. Communication occurs in a explicit form (e.g., message passing among executing code) between threads that do not share anything (other than messages).

The opposite idea can be found on the typical C/C++/Fortran, lower-level crowd: One single process, many threads, a single memory space shared among threads with concurrent access controlled through a low level mechanism like semaphores. This seems also to be the underlying idea of the OpenMP system. These folks believe that programmers can tackle parallel complexity easily (well, at least it is not an impossible, daunting task according to this philosophy).

The point of contention comes from the fact that multiple execution flows introduce a completely new class of bugs coming from the need to coordinate a lot of things going on in parallel. The worst problem introduced is non-determinism: You can execute the same program twice, WITH THE SAME INPUT and get different results. Why? Because the different threads/processes will be scheduled in unpredicted ways by the operating system (or virtual machine) which can yield different results. This severely increases the difficulty to test and debug software. The shared memory crowd (the shared memory model is more efficient and flexible as, well, memory is directly shared) will say that we can deal with this. The message passing crowd suggests that having some restrictions and explicit communication will make life easier (or, less complicated).

The Java crowd is where you can find the most variety of opinions, but the core JVM and Java language itself seems to follow the C/C++ philosophy (though with some candy thrown in, like the Fork/Join framework). But on top of that you can find everything with a vocal support community: Tuple spaces, Map/Reduce, Message passing, etc. This is not to say that the Python and C/C++ communities are monolithic (they are not! Just check the C implementations of MPI and PVM), but you really can find a lot alternatives with vibrant communities on top of the JVM.

A sort of middle of the ground approach was introduced de facto with the programming language Erlang: Erlang allows for multiple threads, but the communication is shared-nothing and based on message passing. I.e. while there is one single process with multiple threads, there is no shared-memory per se and all inter-thread communication is based on message passing. This Actor model based language has influenced some recent language libraries in Scala, Groovy and Clojure, among others where the actor model is the main concurrent programming model.

Many functional languages (like Erlang, Scala and Clojure) proponents also suggest that mutability (ie, the concept of variable stemming from imperative languages like C, Java, C#, Basic, C++, 99% of used languages) is not easily amenable to parallel programming and suggest that immutable data structures make life much easier: If what is shared cannot be changed then much less bugs can be introduced.

To sum it up: Some people suggest concurrent programming is difficult and it is better to minimize communication to tackle that difficulty. Others suggest that concurrent programming is workable and tightly-coupled memory-sharing systems are OK. Some also suggest (functional crowd) that immutable data structures help.

Further reading:
Concurrent computing (Wikipedia)
Scala actors – My preferred introduction to Actors (which happens to be based on Scala)
Erlang Concurrency Message passing (Wikipedia)

My opinion: Shared memory models are for real men! I am just a regular bloke, so I stick with message passing models. The complexity of bugs introduced by concurrent programming is much much worse compared to the existing sequential paradigm. In most of the cases that I have encountered, the restrictions imposed by message passing are acceptable compared to the benefits. Even with message passing and immutable data structures, concurrent programming is still very hard and bug prone (non-determinism is still quite possible with message passing). I expect (hope) that new R&D will allow us to tame this complexity. Avoid shared memory/tightly coupled systems like the plague!

If you search the web you can find some discussions on whether IDEs for dynamic languages can be as helpful as IDEs for static languages. The issue is that static languages like Java have compile-time (thus easy to get at IDE-time) information in order to provide that fundamental code-completion functionality (among many others). If the IDE knows that a certain parameter is a String, than it is simple: it will present to you all the String methods when you type in the dot. For dynamic languages things get more complex are there is formally no (by definition) compile-time information. Some people would argue that there are ways around it (which you can already find in existing IDEs, I remember having some sort of code completion, years ago, on SPE – for Python). I will not add anything to that discussion here, this preamble was mainly for putting the reader in context. I am more interested in discussing good IDEs for DSLs.

With DSLs you get, most of the times, added syntax. Worse than that, you might fall into situations where you have changed (not only added) the initial language syntax; furthermore those syntax changes might even become valid only in runtime (imagine that a method is added to a class that is supplying DSL methods).

One example comes from Ioke and Prolog operator precedence and associativity rules which are changeable (see the previous post). It is not trivial to know if something like 1+2 is even syntactically valid (*). Even if it is syntactically valid things like association rules might change. In languages like Groovy you can add (e.g., through categories) methods to code blocs (from classes that can be dynamically changed). Then there is dynamic dispatching and macros. What is valid in a certain piece of code can be different from what is valid a few lines below. In fact, complete information of what is valid in a certain code block might require code execution. Or, to put in another way, it might be very difficult to have a completely helpful IDE! In this scenario there are 3 considerations that I think are worth being done:

1. One should not be discouraged for not having perfect solutions. Maybe it is not possible to determine all that can be expressed in a certain code block, but sometimes good approximations are enough.
2. On this issue, one good example comes from Prolog: In Prolog, syntax can be changed mainly through the use of the p directive (and through asserts and retracts). The p directive changes operators but is very easy to analyze pre-compilation/interpretation. So, the way DSLs are normally be constructed lend themselves very easily to code analysis which can be used by IDEs. This unfortunately not the case in most real-world languages.
3. It would be cool to have a language where DSL specifications could be automatically used to construct IDEs. The current real-world DSL-able languages (Ruby, Groovy, …) are DSL-enabled through indirect techniques which can be used to build DSLs (Dynamic reception, operator overload, whatever), in fact many of these techniques exist with other objectives than creating DSLs. If there was a declarative and explicit way to create DSLs, that information could be used to inform IDEs on parsing and other issues. An embedded, core way, to explicitly specify DSLs.

(*) I suppose some will see this as an argument for the fact that you can do pretty stupid (or at least unintuitive) things with DSLs. Well, you can do stupid things with everything. The question is not if you can or not, but the extent of bad use cases and how bad uses can creep in easily. Another (interesting) discussion, but not for now.

I was reading Ola Bini’s post about operators in Ioke (Ioke being the new language that Ola is developing).

It is a common saying around LISPers that everything that is being done in “modern” languages is a return to LISP. And the argument holds some ground. The truth is, among the 4 most conceptually influential programming languages that I can think of (Lisp/Functional, Fortran/Imperative, Smalltalk/OO, Prolog/Logic), the bad option (Fortran) won as it is the major philosophical contributor to current programming languages (much more than Smalltalk).

Take the reinvention of operators on Ioke as per the post above. This concept is available in Prolog for decades. It is all there: precedence (i.e. 2*3+4 means (2*3)+4 and not 2*(3+4)). Associativity (left or right – ie. 3-2-1 is 0 (3-2)-1 and not 2 3-(2-1) ). And even more as new operators can be defined and can be made of alphanumeric characters (want to create a new operator called say, “in”? go ahead). In fact people were doing DSLs a long time ago (in the small Prolog community at least) using techniques such as these.

The next thing that you will need (and we are getting there with macros and AST access) is no default interpretation. This is especially important with arithmetic, let me give an example:

Imagine the expression 1+x. Most languages will evaluate this expression and will return the sum of 1 + x. If x is defined and say is 4, then 1+x is 5. If x is not defined then an error (compile or run)-time will be raised. This is an absolute disgrace for DSLs with are essentially declarative (i.e., detached from semantics). “1+x” might be something that you want to evaluate now (and get the result) or might be something that you want to specify in order to evaluate later (say, I want to do a chart of all values of x between 1 and 5, or I want to differentiate), look at this pseudo-code

1
2
3
4
5
6
7

Var x
Exp expression = 1 + x**2
 
chart(expression, [[x,[1, 5]]]) //do a chart, x between 1 and 5
evaluate(expression, [[x,3]]) //Evaluate expression where x is 3 (i.e.  10)
diffe = differentiate(expression, x) //returns the expression 2*x
prettyprint(expression) //Pretty prints the expression.

Most people automatically associate the operation evaluate to 1+x**2. That might be so in an imperative world (can I call it shitty world?). But in an declarative/DSL world 1+x**2 is just that, an expression, it has no meaning attached per se. What you do with it depends on the context. Pretty print it, differentiate it, integrate it, or even evaluate it by instantiating x to 3 and getting the “precious” 10.

Update: I was rereading the post and noticed that it might be read as seeing Ola’s work as less interesting. Not at all: I actually think the way forward is precisely improving the current “imperative” setting in the way Ola is doing.