Mumblings and Grumblings on Computers

My Clojure Workflow, Reloaded, Reloaded

Several months ago Stuart Sierra wrote a post on the Relevance blog which I found very interesting. In it, he described a way of doing Clojure development that I wanted to try. At first glance, I was interested in it only because it gave me a good workflow for testing within the Clojure repl. I love Test-Driven development and I have always had problems practicing it well in Clojure. I had previously tried two approaches. To begin with, I tried to make do with simply running lein test in the shell when I needed to run tests. The context switch coupled with the slow JVM start up time made this approach unfeasible.

Moving on from this, I attempted to run tests in the REPL without the use of Stuart's tools.namespace library. I learned after a few hours of attempting that I was simply too dumb. I have no idea how code loading works in Clojure. I do not know how to reliably reload code and all its dependencies. I never really knew what the state of the system was at any given time. Once I started making use of the workflow suggested by Sierra I was actually able to reliably and quickly run tests in the repl. This drastically improved my feelings about Clojure development.

A lot of my problems are likely related to the fact that I don't really know what I'm doing. I understand some of the fundamentals around functional programming and LISP. But I do not understand the specific Java-ness of Clojure. I have never written Java more complex than is necessary to maintain the Braintree Java Client Library. I'm certain that for someone with more familiarity with Java, it would be very easy to start up an instance of Jetty when initializing the system and then setting up the ring handler within. I don't know how to do that. For this reason, I was unable to full commit to this technique. I had to make a compromise. Here is how I init the system and start the app.

(defn system []
  (let [db-url (or (System/getenv "DATOMIC_URL") "datomic:mem://sayc-app")]
    {:db-url db-url
     :handler (sayc-app.web/create-handler db-url)
     :web-port (or (try (Integer/parseInt (System/getenv "PORT"))
                        (catch NumberFormatException _ nil))

(defn start [system]
  (d/create-database (:db-url system))
  (migrate/migrate (d/connect (:db-url system)))
  (assoc system
         :server (ring/run-jetty (:handler system) {:port (or (:web-port system) 8080) :join? false})
         :db-connection (d/connect (:db-url system))))

Since I don't really understand jetty I have to rely on ring/run-jetty to initialize the server. This means there will be some hidden elements of global state. I wish I could get rid of it and I don't think this is exactly what Stuart Sierra had in mind, but it works for me.

Other than the benefit of being able to reliably and quickly run tests, there was a different huge benefit. Constructing an application this way leads you to really consider where all of your global state lies. When you initialize the database connection at such a high level, it is much harder to rely on it deep within the application. This is what led me to the approach I outlined in my previous post. When you initialize this complex system at such a high level, you have to think much harder about what needs access to what.

It is however not all sunshine and roses. This way of working is much harder than just hacking out some Clojure code. It is a quite heavyweight solution. There are parts of it that I found useful that I would like to separate out. I love the good code reloading and I love the ease with which I could run tests. However, designing an application in this way is definitely overkill for the simple toy app I've been working on. Granted, I don't think anyone would have ever thought otherwise. I'm certain that when one is waist-deep in an extermely complicated application this workflow is invaluable.

Why Are We Settling for this in Clojure?

I did a quick Github search to find instances of people passing around global database connections. It didn't take long for me to find a bunch of examples of this.

(def ^:dynamic ^java.sql.Connection *db* nil)

(def *db*
  {:subprotocol "postgresql"
  :subname "//localhost:5432/blogjr"
  :username "blogjr"

(def ^{:doc "The binding contains the database connection settings."}
  *db* {:classname "com.mysql.jdbc.Driver"})

Why are we okay with this? This isn't the post global state future we were promised. If you are using this technique, there are a limited number of options to swap out your database. You can either redefine a dynamic variable or using with-bindings. I had a conversation on Twitter recently with one of the few people I know who has actually been paid to write Clojure about this topic:

I'm quick to believe that there is an issue with what I'm about to propose. It's very likely in fact. I have never done serious work in Clojure. The largest app I've written in Clojure is maybe 300 LoC. So take this with the world's largest grain of salt (#OXYMORON #ALERT).

We're going to take advantage of one of the primitive functional programming techniques -- currying. Currying is a process whereby a function with multiple arguments is transformed into a function which takes one argument returning a function takes a single argument and so on... For example,

(defn foo [a b c] (+ a b c))

(defn foo-curried [a] (fn [b] (fn [c] (+ a b c))))

(= (foo 1 2 3) (((foo-curried 1) 2) 3))

So we write our actions like this.

(defn create-bid [db]
  (fn [request]
    (transact db tx-data)

In starting the web server, I create the database connection and initialize the web handler like so.

(defn build-routes [db]
  (defroutes routes
    (POST "/bids" [] (actions/create-bid db))
    (resources "/")))

(defn create-handler [db]
  (build-routes db)
  (-> (var routes)

I like this. I much prefer it to dealing with the global database connection. Am I missing something? Is there a reason that this isn't preferable? Is it overkill?

I likely feel uncomfortable with the status quo because there are a lot of things left in Clojure I don't understand. I don't really get what it means to reassign these global variables. I don't know what the best way is to do this. I do understand how to pass arguments to functions. I've been doing that since I first touched a Lisp in my first year of college 7 years ago. If I understood things better, I would probably feel more comfortable with this than I do, but I don't know much.

An Action's Tale

I have been working recently on a web app written in Clojure. I wanted to learn both Clojure and Bridge better and this app has helped me in both regards. I want to tell you the story of an action in this app and how I implemented it. It is a story that demonstrates one of my great struggles while I write Clojure code. As I work in Clojure, I often feel myself writing Rails code with more parentheses. I had this very problem with the action that is the subject of our story. This action parses a request, builds up a new object representing the result of a hand of Bridge and associates it with an in-progress Chicago Bridge game. As, you can see the code is very procedural. First, get the data we need, check if everything is present, then update the storage, then respond with a redirect.

(defn add-hand-to-chicago-game [request]
  (let [id (get-in request [:params :id])
        game (get @chicago-games id)
        hand (get-in request [:params :hand])]
    (if (not game) (not-found "NOT FOUND"))
    (swap! chicago-games assoc id (conj game (process-hand-params hand)))
    (redirect (str "/chicago_games/" id))))

This really bothered me. This is not the beauty of functions operating on data that I was longing to see. It got even worse once I added some validations. This version of the action doesn't even exist in the Git repo anymore so I can't show you. I hated it so much.

Finally, I bit the bullet. I broke this action apart into functions upon functions upon functions. Ahhhh, here was the beauty I was looking for...sort of. Granted, it didn't really look that beautiful. First, it's giant. I took a seven line function and ballooned it out into almost 40 lines.

(defn extract-params [request]
  {:id (get-in request [:params :id])
   :hand (get-in request [:params :hand])})

(defn enrich-with-prereqs [request]
   (assoc request :game (get @chicago-games (:id request))))

(defn validate-prereqs [prereqs]
  (assoc prereqs :errors (validate/hand (:hand prereqs))))

(defn derive-status [prereqs]
  (assoc prereqs :status
           (nil? (:game prereqs)) 404
           (not (empty? (:errors prereqs))) 422
           :else 201)))

(defn persist [request]
  (let [new-game (conj (:game request) (process-hand-params (:hand request)))]
    (swap! chicago-games assoc (:id request) new-game)
    (assoc request :game new-game)))

(defn conditionally-persist [request]
  (if (= (:status request) 201)
    (persist request)

(defn render [request]
    (= (:status request) 404) (not-found "NOT FOUND")
    :else (views/display-chicago-game (:id request) (:game request) (:errors request))))

(defn add-hand-to-chicago-game [request]
    (-> request extract-params enrich-with-prereqs validate-prereqs
        derive-status conditionally-persist render))

I suspect anyone who wanted to get any real work done in this language would object to this style. And I would totally agree with them. However, for this toy app, it works fine.

In thinking about this, my mind is drawn towards thinking about Ring. Ring is a Clojure middleware stack and spec. It takes a similar approach to Ruby's Rack framework. It works by applying a series of functions to the data that is the HTTP request and returning data that will be the HTTP response. In order to get the aesthetic I want, perhaps I should be pushing more of the contents of this action into the Ring stack. Rather than extracting params in the action, perhaps a param-extractor middleware which is built up with a map of pairs of verbs and actions pointing to a map representing the parameters to be extracted. At this point, the contents of the action could be changed to only validation, persistence, and rendering. I may attempt spiking this out to see what it would look like. Until then, that's the story of this action and I'm sticking to it.