PERMALINK

The rmap library was one of my first Clojure libraries. It lets you create recursive maps, both programmatically and in literal form. They are recursive in the sense that entries in the map can refer to other entries.

The library has seen its first major update. It is far simpler now, and more powerful because of it. Version 2.0 of this library has been released, and this blog post will tell you what has changed and why. Since most of you won’t know the library already, let’s begin with some history.

A little history

The recursive property of maps in rmap work by lazily evaluating the values, using a macro. To give you an idea how it works, this is what it used to look like:

(def my-map
  (rmap X {:what "awesome!"
           :clj  (str "Clojure is " (:what X))})

(type my-map)
;=> rmap.internals.RMap

my-map
;=> {:what ??, :clj ??}

(:clj my-map)
;=> "Clojure is awesome!"

my-map
;=> {:what "awesome!"
;=>  :clj  "Clojure is awesome!"}

You give the rmap macro a symbol and a map. The map values can use the symbol to refer to the map itself, and thus other entries. The macro returns a special kind of map, one that evaluates the entries on request and caches them.

While I haven’t touched the library in a while, I had a use for it again in my current project. When going through the source code I saw that it was showing its age. Both the API and the internals were complex and over-engineered. Again, it was one of my first libraries, so cut me some slack :)

Most of the complexity came from allowing recursive values to fetch other entries in any map-like manner. In other words, the value expressions had access to the map datastructure itself. But because it was a map with special evaluation rules, a completely new map type was created for this purpose. The type needed to implement all kinds of interfaces, such as ILookup, IFn, Seqable, IPersistentCollection and IPersistentMap to name a few.

Because of the special semantics of the map, difficult decisions had to be made on what it means for an entry to be “evaluated”. Is the value cached? Where is it cached? Does this influence “ancestor” maps sharing the same entry? Does it influence already created “derivative” maps with the same entry? Or those created after the evaluation? Are the entries realized on equality checks?

In the end, the library was setup in such a way that the users could make all these choices themselves, only adding to the complexity.

Six years later

When revisiting the rmap library, I thought about how to simplify it. Going back and forth between several designs, I found a solution that I have already partially hinted at. It is somewhat inspired by the Integrant library. What if the library does not allow the recursive values to use a map to access the other entries, but a single, simple and clearly defined way instead? A plain function for example, called ref for example?

This is exactly how rmap 2.0 is setup. It changes the library in the following ways:

• There is no special map implementation anymore, just plain Clojure maps
• Recursive values access other entries through a plain function
• Recursive values are now first class objects
• There is no caching, except during evaluation of the recursive values
• It also works on vectors, as those are also associative

With the new version the beginning of the former example now looks like this:

(def my-map
  (rmap {:what "awesome!"
         :clj  (str "Clojure is " (ref :what))}))

(type my-map)
;=> clojure.lang.PersistentArrayMap

my-map
;=> {:what ?? :clj ??}

Within the rmap macro the values now refer to other values using the (ref [key] [key not-found]) function and the result is a plain Clojure map. This seemingly small change to the user has a profound impact on the implementation. From 691 to 75 lines of code, actually. It also makes all those caching choices go away and by that offer far simpler semantics, as you will now see.

Valuating

You can see in the 2.0 example that the recursive values in the map are still lazy, i.e. not instantly evaluated. Let’s see what happens if we continue our 2.0 example like we did with the old rmap:

(:clj my-map)
;=> ??

my-map
;=> {:what ??, :clj ??}

Instead of getting the “Clojure is awesome” string, you get back an unevaluated recursive value. This is because we now use a plain Clojure map and recursive values are now first-class objects. To get the desired result, a map with recursive values needs to be “valuated” first. For this the library has the valuate! function. All it does is “updating” a Clojure map by evaluating all recursive values.

To evaluate a recursive value a ref function is needed. The valuate! function it creates such a ref function and is passed to the recursive values. Moreover, the ref function passes itself along recursively while evaluating referred entries. It is exactly at this ref function where caching takes place, and nowhere else. This ensures that a recursive value is evaluated only once, even if it referred to multiple times.

Let’s use this valuate! function on our basic map:

(valuate! my-map)
;=> {:what "awesome!"
;=>  :clj  "Clojure is awesome!"}

Now we have our expected result, again in a standard Clojure map. This simplicity makes the semantics very clear. The original map is also not affected. Descendent maps act like normal maps, since recursive values are first class. If you want the values to evaluate in the context of another map, you simply valuate an updated map, like so:

(valuate! (assoc my-map :what "recursive!"))
;=> {:what "recursive!"
;=>  :clj  "Clojure is recursive!"}

A companion function of valuate! is valuate-keys!. It behaves similar to valuate!, but returns a map where only the entries of specified keys and its dependencies are evaluated.

Wrapping up

One last macro is provided by rmap 2.0, which is rmap!. It is the same as rmap (without a bang), but is instantly valuated. For example:

(rmap! {:what "simple!"
        :clj  (str "Clojure is " (ref :what))})
;=> {:what "simple!"
;=>  :clj  "Clojure is simple!"}

This rmap! macro may be all you need for your purposes. The other macros and functions are provided to give you all the building blocks you might need. This way the library aims to be both simple and easy.

And that wraps up an overview of rmap 2.0. You will read more about it soon on this blog though, as there are two more libraries to be announced, and one of them complements nicely with the new rmap!

You can find more detailed documentation here.

As always, have fun!

Oh, one more thing

If you do like to have a special map implementation that does caching, automatic evaluation and such, it is certainly possible to create one based on the rmap 2.0 core API. The example below gives you a map-like read-only datastructure that allows access patterns like (get my-map :key), (my-map :key) and (:key my-map):

(deftype RMap [cache]
  clojure.lang.IFn
  (invoke [this key]
    (.valAt this key nil))
  (invoke [this key not-found]
    (.valAt this key not-found))

  clojure.lang.ILookup
  (valAt [this key]
    (.valAt this key nil))
  (valAt [this key not-found]
    (swap! cache valuate-keys! key)
    (get @cache key not-found)))

(defmethod print-method RMap [rmap ^java.io.Writer writer]
  (print-method @(.cache rmap) writer))

(defn ->my-rmap [m]
  (RMap. (atom m)))

(defmacro my-rmap [m]
  `(->my-rmap (rmap ~m)))

(def my-map
  (my-rmap {:foo 1
            :bar (inc (ref :foo))})))

my-map
;=> {:foo ??, :bar ??}

(:bar my-map)
;=> 2

my-map
;=> {:foo 1, :bar 2}

However, this approach does reintroduce a “hidden” cache again, along with its mutable properties; something rmap 2.0 tries to avoid, at least in its core. The library does have a similar wrapper, but this is experimental for now.