When I set myself the task of learning how to use EDN, or Extensible Data Notation, in Clojure, I couldn’t find a simple tutorial on how it worked, so I figured I would write one for anyone else having the same troubles I did. I was keen to learn EDN, as I am working on a Clojure based web application in my spare time, and wanted a serialisation format I could send data in that could be easily understood by both my Clojure and ClojureScript programs.
Disclaimer: This is my first blog post on Clojure, and I’m still learning the language. So if you see anything that can be improved / is incorrect, please let me know.
If you’ve never looked at EDN, it looks a lot like Clojure, which is not surprising, as it is actually a subset of Clojure notation. For example, this is what a vector looks like in EDN:
[1 2 3 4 "five" true]
Which should be fairly self explanatory. If you want to know more about the EDN specification, have a read. You should be able to read through in around ten minutes, as it’s nice and lightweight.
The first place I started with EDN, was with the clojure.edn namespace, which has a very short API documentation and this was my first point of confusion. I could see a read and read-string method… but couldn’t see how I would actually write EDN? Coming from a background that was used to JSON, I expected there to be some sort of equivalent Clojure to-edn function lying around, which I could not seem to find. The concept connection I was missing, was that the since EDN was a subset of Clojure, and the Clojure Reader supports, EDN, you only had to look at the IO Clojure functions to find that there are functions pr and prn whose job it is to take an object and “By default, pr and prn print in a way that objects can be read by the reader.” Now pr and prn output to the current output stream. However we can use either pr-str or prn-str to give us a string output, which is far easier to use for our examples. Let’s have a quick look at an example of that, with a normal Clojure Map:
(ns edn-example.core
(require [clojure.edn :as edn]))
(def sample-map {:foo "bar" :bar "foo"})
(defn convert-sample-map-to-edn
"Converting a Map to EDN"
[]
;; yep, converting a map to EDN is that simple"
(prn-str sample-map))
(println "Let's convert a map to EDN: " (convert-sample-map-to-edn))
;=> Let's convert a map to EDN: {:foo "bar", :bar "foo"}
(println "Now let's covert the map back: " (edn/read-string (convert-sample-map-to-edn)))
;=> Now let's covert the map back: {:foo bar, :bar foo}Obviously this could have been written simpler, but I wanted to break it down into individual chunks to make learning a little bit easier. As you can see, to convert our sample-map into EDN, all we had to do was call prn-str on it to return the EDN string. From there, to convert it back to EDN, it’s as simple as passing that string into the edn/read-string function, and we get back a new map with the same values as before.
So far, so good, but the next tricky bit (I found) comes when we want to actually extend EDN for our own usage. The immediate example that came to mind for me, was for use with defrecords. Out of the box, prn-str will convert a defrecord into EDN without any external intervention. For example:
;;lets get more fancy, and convert a defrecord to and from EDN
(defrecord Goat [stuff things])
(def sample-goat (->Goat "I love Goats", "Goats are awesome"))
(defn convert-sample-goat-to-edn
"Converting a Goat to EDN"
[]
(prn-str sample-goat))
(println "Let's convert our defrecord Goat into EDN: " (convert-sample-goat-to-edn))
;=> Let's convert our defrecord Goat into EDN: #edn_example.core.Goat{:stuff "I love Goats", :things "Goats are awesome"}The nice thing here, is that prn-str provides us with a EDN tag for our defrecord out of the box: “#edn_example.core.Goat”. This let’s the EDN reader know that this is not a standard Clojure type, and that it will need to be handled differently from normal. The EDN reader makes it very easy to tell it how to handle this new EDN tag:
;; This is a map of reader functions that match up to the #tags we have.
;; we can use map->Goat, as we get back a map from the EDN block after the tag
;; deserialises as a map, and we can just pass that through.
(def edn-readers {'edn_example.core.Goat map->Goat})
(defn convert-edn-to-goat
"Convert EDN back into a Goat. We will use the :readers option to pass through a map
of tags -> readers, so EDN knows how to handle our custom EDN tag."
[]
(edn/read-string {:readers edn-readers} (convert-sample-goat-to-edn)))
(println "Let's try converting EDN back to a Goat: " (convert-edn-to-goat))
;=> Let's try converting EDN back to a Goat: #edn_example.core.Goat{:stuff I love Goats, :things Goats are awesome}You can see that when we call edn/read-string we pass through an option of :readers with a map of our custom EDN tag as the key ( 'edn_example.core.Goat ), to functions that return what we finally want from our EDN deserialisation as the values ( map->Goat ). This is the magic glue that tells the EDN reader what to do with your custom EDN tag, and we can tell it whatever we want it to do from that point forward. Since we have used custom EDN tags, if we didn’t do this, the EDN reader would throw an exception saying “No reader function for tag edn_example.core.Goat”, when we attempted to deserialise the EDN.
Therefore, as the EDN reader parses:
#edn_example.core.Goat{:stuff I love Goats, :things Goats are awesome}
It first looks at #edn_example.core.Goat, and matches that tag to the map->Goat function. This function is then passed the deserialised value of {:stuff “I love Goats”, :things “Goats are awesome”} to it. map->Goat takes that map, and converts it into our Goat defrecord, and presto we are able to serialise and deserialise our new Goat defrecord.
This isn’t just limited to derecords, it could be any custom EDN we want to use. For example, if we wanted to write our own crazy EDN string for our Goat defrecord, rather than use the default, we could flatten out the map into a sequence, like so:
(defn alternative-edn-for-goat
"Creates a different edn format for the goat. Flattens the goat map into a sequence of keys and values"
[^Goat goat]
(str "#edn-example/Alt.Goat" (prn-str (mapcat identity goat))))
(println "Lets convert our Goat to our custom EDN format: " (alternative-edn-for-goat sample-goat))
;=> Lets convert our Goat to our custom EDN format: #edn-example/Alt.Goat(:stuff "I love Goats" :things "Goats are awesome")We can then apply the same strategy as we did before to convert it back:
(defn convert-alt-goat-edn
"Takes the altenative EDN and converts it into a Goat"
[elems]
(map->Goat (apply hash-map elems)))
(defn convert-alt-edn-to-goat
"Convert the alternative edn format for a goat back to a Goat"
[]
(edn/read-string {:readers {'edn-example/Alt.Goat convert-alt-goat-edn}} (alternative-edn-for-goat sample-goat)))
(println "Lets our custom EDN back into a Goat: " (convert-alt-edn-to-goat))
;=> Lets convert our custom EDN back into a Goat: #edn_example.core.Goat{:stuff I love Goats, :things Goats are awesome}Finally, there is the question of what do you do when you don’t know all the EDN tags that you will be required to parse? Thankfully the EDN reader handles this elegantly as well. You are able to provide the reader with a :default option, that gets called if a custom namespace can’t be found for the given EDN string, and gets passed the tag and the deserialised value as well.
For example, here is a function that simply converts the incoming EDN string into a map with :tag and :value values for the incoming EDN:
(defn default-reader
"A default reader, for when we don't know what's coming in."
[t v]
{:tag t :value v})
(defn convert-unknown-edn
"We don't know what this EDN is, so let's give it to the default reader"
[]
(edn/read-string {:default default-reader} (alternative-edn-for-goat sample-goat)))
(println "Let's handle some unknown EDN: " (convert-unknown-edn))
;=> Let's handle some unknown EDN: {:tag edn-example/Alt.Goat, :value (:stuff I love Goats :things Goats are awesome)}That about wraps up EDN. Hopefully that makes things easier for people who want to start using EDN, and encountered some of the stumbling blocks I did when trying to fit all the pieces together. EDN is a very nice data format for usage with Clojure communication.
The full source code for this example can be found on Github as well, if you want to have a look. Simply clone it and execute lein run to watch it go.