A Clojurescript API Server in Docker using ExpressJS
I have been writing code in Clojure for some time, most recently running it in Docker containers on Kubernetes. This approach has been primarily for ease of testing and deployment, rather than for scaling. However, now I am looking at applications where I want to start scaling out dynamically, and I started to get a bit concerned about the startup time of my containers — the JVM that Clojure runs on, can take a bit of time to get out of bed. Since ClojureScript allows us to compile Clojure into Javascript, I took a look at running ClojureScript in NodeJS.
So you know what to expect, I’m going to talk about the ClojureScript project file, then the Javascript package file (needed for native Javascript packages), then the code itself, and finally the Dockerfile to build the Docker image.
Clojure and ClojureScript
If you haven’t encountered it before, Clojure is a Lisp dialect which runs on a JVM and interoperates with Java. As a Lisp dialect, Clojure is a functional programming language, dynamic in nature, has a code as data philosophy, and encourages immutability.
ClojureScript is a compiler for Clojure, which generates Javascript. This gives two complementary directions on its use. Firstly, it gives another language which can be used in the browser (addressing concerns some people have around Javascript). Secondly, if you are already writing code in Clojure on your server, this gives a way of running some of that code in the browser (e.g. to do validation).
The Challenge
What I wanted to do was to generate server-side Javascript from ClojureScript, allowing me to continue to use Clojure as a language, but to overcome some of the challenges around Java-based microservices. Before I looked at ClojureScript, I assumed this would be a trivial matter — just write Clojure as I had been, but compile it in a different way. The fact that I am writing this blog, hints at the fact that it was not so simple.
The primary challenge is that although Clojure itself can be pretty much consumed by the ClojureScript compiler, some features rely on Java, some libraries are not ClojureScript-compatible, and sometimes I use Java libraries or other JVM-based DSLs (such as Drools). This meant that I could not just use my standard stack, I needed to look for some replacements.
The General Approach
I immediately found that huge chunks of my stack weren’t compatible — this included the Ring libraries I use for actually creating a web server, and the Liberator library I use for routing and validating API calls. I decided to use ExpressJS as the obvious choice to replace these (although I may look into Loopback in the future.
Fortunately, my logging library of choice, Timbre, is ClojureScript-ready, so I could continue to use that.
Since this is a simple proof of concept (I’m just going to serve some canned responses), that’s all I need.
The project.clj File
I use Leiningen to build my Clojure projects, so I need to create a project.clj file to handle everything. This begins with some standard items (including the project name which is map-server, because of where I want to eventually go with this):
(defproject map-server "0.1.0-SNAPSHOT"
:description "ExpressJS in ClojureScript proof of concept"
:url "http://example.com/tbc"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
Next we have the dependencies. I tend to group these into areas, and comment the area:
:dependencies [; Core Clojure
[org.clojure/clojure "1.8.0"]
[org.clojure/clojurescript "1.9.946"]
; Logging
[com.taoensso/timbre "4.10.0"]]
Next we have any plugins. The important one for us is the plugin to actually build ClojureScript (usually abbreviated to cljs):
:plugins [[lein-cljsbuild "1.1.7"]]
Finally, we have the information needed to build the Javascript from the ClojureScript. This includes items such as where to find the source files, where to write the output files, and where to find the overall main function:
:cljsbuild {
:builds [{:source-paths ["src/cljs"]
:compiler {:output-to "resources/public/core.js"
:optimizations :advanced
:externs ["node_modules/body-parser/index.js"
"node_modules/express/lib/middleware/init.js"
"node_modules/express/lib/middleware/query.js"
"node_modules/express/lib/router/index.js"
"node_modules/express/lib/router/layer.js"
"node_modules/express/lib/router/route.js"
"node_modules/express/lib/application.js"
"node_modules/express/lib/express.js"
"node_modules/express/lib/request.js"
"node_modules/express/lib/response.js"
"node_modules/express/lib/utils.js"
"node_modules/express/lib/view.js"
"externs/externs.js"]
:target :nodejs
:main "map-server.core"}}]})
Two key items to note are the optimizations and the externs. I’ve opted to turn on full optimisation here, which uses Google’s Closure. This can sometimes cause problems if you are using pure Javascript libraries which are not Closure-ready (in which case you can set it to none if you wish).
The second item is the externs list. Because I will be using ExpressJS as a standard Javascript library, and because it is not Closure-ready, I need to specify where to find all the various entrypoints to functions in it.
The package.json File
I will be using ExpressJS, which is a pure Javascript library. ClojureScript will therefore not know how to import it, so I will use npm for that. I will therefore create a package.json file to make that easier to do. This is a very simple file, basically just listing ExpressJS as a dependency:
{
"name": "map-server",
"version": "0.0.1",
"description": "ExpressJS / ClojureScript proof of concept",
"main": "app.js",
"directories": {
"doc": "doc",
"test": "test"
},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Ian Finch",
"license": "EPL",
"dependencies": {
"express": "^4.16.1"
}
}
Now I just need to run npm install and I get ExpressJS and its dependencies in the location I already specified in my project.clj.
The Source Code
Clojure can be very specific about where files go, so I tend to use the same directory structure that Leiningen’s scaffolding sets up. I therefore put my core ClojureScript for my map-server namespace at src/cljs/map-server/core.cljs.
I begin with a namespace declaration, together with require statements for any libraries I will be using:
(ns map-server.core
(:require [cljs.nodejs :as node]
[taoensso.timbre :as log]))
Next, I do the equivalent of NodeJS’s require, to make the ExpressJS module available:
(def express (node/require "express"))
Now I write a simple request handler for any API calls. Because I am using ExpressJS, these will be in the format expected by ExpressJS — a function which accepts a request and a response object, and which returns a modified response object:
(defn handler
"Function to handle a simple request"
[req res]
(log/info "Request:" (.-method req) (.-url req))
(-> res
(.status 200)
(.json (clj->js {:message "Hello Clojurescript"}))))
Just like in standard Clojure, a function name beginning with a dot invokes a method call on the object given as the first parameter. Obviously, the difference here is that this is a Javascript object rather than a Java object. ClojureScript extends this notation, so that if the character after the dot is a dash, it retrieves the value with that name from the object. So, in the above function, the expression (.-url req) gets the contents of the variable url from the object req (in Javascript this would be req.url). The thread function makes the function calls less obvious, but if I unthread it, the expression (.status res 200) invokes the status method on the res object with the value 200 (or res.status(200) in Javascript).
Finally, because we want to return a JSON structure, I use the handy clj->js function to convert my ClojureScript map into a Javascript object, then call the response’s json function to set that as the body of the response.
You can therefore see that the above function simply logs a message describing the request, sets a success status, and supplies a hard-coded message as the body of the response.
It’s also convenient to know when our web server has started up, so I also wrote a brief callback which just logs a message:
(defn server-started-callback
"Callback triggered when ExpressJS server has started"
[]
(log/info "App started at http://localhost:3000"))
We can now pull all this together in our main function, which creates a new Express instance, adds our handler for when someone requests /, and starts listening on port 3000:
(defn -main
"Our main handler"
[& args]
(doto (new express)
(.get "/" handler)
(.listen 3000 server-started-callback)))
So, now I can fire up my server and try a couple of requests. Here is the output:
[docker@minikube:~]$ curl http://localhost:3000
{"message":"Hello ClojureScript"}
[docker@minikube:~]$ curl http://localhost:3000/foo
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /foo</pre>
</body>
</html>
And here is what I see in the console:
INFO [map-server.core:11] - App started at http://localhost:3000
INFO [map-server.core:32] - Request: GET /
Note that the /foo request doesn’t show in the logs, since it doesn’t reach my handler.
We Need More Idioms
So, that works, but it feels like Javascript written as Clojure. Additionally, if we add a second URI, we will need a new handler for that, which will have to do its own logging, its own JSON conversions, etc.
So, let’s convert our handler function into a generic one which handles the grunt work, then our URI-specific handlers only need to add their specific functionality:
(defn handler
"Function to handle a generic route"
[handler-fn]
(fn [req res]
(log/info "Request:" (.-method req) (.-url req))
(-> res
(.status 200)
(.json (clj->js (handler-fn req res))))))
This is almost identical to the earlier handler, but it now returns a function which does the actual handling of the request, rather than doing it itself. To accomplish this, it takes as a parameter a function which does the request-specific part of handling the request. You can see that passed in function being called in the last line of the above code.
We can now modify our main function to use the above function in its routing calls.
(defn -main
"Our main handler"
[& args]
(doto (new express)
(.get "/" (handler (fn [request response] {:message "Hello ClojureScript"})))
(.get "/xyzzy" (handler (fn [request response] {:result "Nothing happens"})))
(.listen 3000 server-started-callback)))
We can run this, to check it behaves as expected:
[docker@minikube:~]$ curl http://localhost:3000
{"message":"Hello ClojureScript"}
[docker@minikube:~]$ curl http://localhost:3000/xyzzy
{"result":"Nothing happens"}
[docker@minikube:~]$
To make it easier to configure, we can make the routes more flexible by accepting multiple types of argument (strings, objects and functions) and converting them all to functions. This would allow us to use a more concise syntax:
(defn -main
"Our main handler"
[& args]
(doto (new express)
(.get "/" (handler "Hello ClojureScript"))
(.get "/xyzzy" (handler {:result "Nothing happens"}))
(.get "/echo" (handler (fn [req _] {:method (.-method req) :url (.-url req)})))
(.listen 3000 server-started-callback)))
We can do this by adding a function to test the type and convert it to a canonical function format:
(defn canonicalise-fn
"Create a function, depending on input type"
[item]
(cond (fn? item) item
(string? item) (fn [_ _] {:message item})
:else (fn [_ _] item)))
We glue this all together by calling the canonical function from our handler:
(defn handler
"Function to handle a generic route"
[handler-fn]
(let [canonical-fn (canonicalise-fn handler-fn)]
(fn [req res]
(log/info "Request:" (.-method req) (.-url req))
(log/debug "Request:" (obj->clj req))
(-> res
(.status 200)
(.json (clj->js (canonical-fn req res)))))))
So, now we have our server, we just need to stick it in a Docker container.
The Dockerfile
The Dockerfile is really straightforward:
- Our ClojureScript has been compiled into a Javascript file at resources/public/core.js (as defined in our project.clj).
- Our supporting modules (ExpressJS and dependencies) are in node-modules.
- To run our Javascript, we just use the node command followed by the name of our Javascript file.
So, we need a base node image (I’ll use the alpine version for a smaller container), we need to copy our files to a suitable directory, and then issue the node command.
So our Dockerfile is this:
FROM node:alpine
COPY map-server/resources/public/core.js /usr/src/node/map-server.js
COPY map-server/node_modules /usr/src/node/node_modules
WORKDIR /usr/src/node
CMD node map-server.js
This gives me an 80MB Docker image, which starts up quickly and is available to serve web pages immediately.
Summary
As a Proof of Concept, this has demonstrated that it is possible to build a containerised API server in ClojureScript, leveraging Javascript modules.
My next step will be to look at some more API-specific server modules (such as loopback.io) or some of the ClojureScript libraries which are starting to appear (such as macchiato-framework.github.io).
The code used in this article is available on GitHub at ianfinch/cljs-expressjs