Pre-Conj Interview: Zach Tellman

October 23, 2014

Introduction

Zach Tellman is the next interview participant. He is giving a talk at Clojure/conj about composition. The background to his talk is available, if you like.

Interview

LispCast: Briefly, how did you get into Clojure?

Zach Tellman: At my first job, the main language was C#. However, I worked with Tom Faulhaber, who is a Common Lisp guy from way back, and at the time was just working on the initial implementation of clojure.pprint. I started playing around with Clojure, and started building Penumbra in mid 2009. I haven't looked back since.

LC: Composition is a very important concept in the Clojure world, yet you claim that many libraries can't compose cleanly. Why is it such a problem?

ZT: Clojure is a young language, and is fairly unique in the way it tries to couple functional idioms with less-than-functional host environments. It also lacks any real organizational principle: a namespace can encompass an enormous amount of functionality (see clojure.core), or a very small, focused piece of functionality (see most of Ring). Both of these allow us enormous flexibility, but it also means that Clojure programmers are without either precedent or obvious affordances when starting out with Clojure. It's understandable that we'd make some missteps.

If we look at the first two or three SQL libraries written for Clojure, it's notable that they all used macros. It became clear after not too long, though, that this was a really limiting approach - the only way to compose with a macro was with more macros. This doesn't mean that macros shouldn't be used, of course, just that we should always be aware of how they constrain the surrounding code. Abstractions that are incredibly useful in application-level code might be irritating and arbitrary in a library.

I think a lot of people begin to innately understand these design considerations after a few years of using Clojure, but I haven't heard anyone try to articulate them in a general form. I guess we'll see how I do at that.

LC: You mention Ring. I think it's a great example of a system with very good composability. Middleware written by different programmers can be combined without problems. Ring achieves this by being a standard protocol. So there is coordination between the programmers in that they both agree to the centralized standard. Are there other patterns besides standards that can allow for composition?

ZT: Ring is useful because it's focused. Modeling HTTP requests as a pure function is a lossy abstraction, but it works for the vast majority of requests. If it wasn't so simple, I don't think it would have had nearly as much adoption (see the n-many mostly unused async extensions to Ring that try to accurately model the entire problem space). But by using Ring, we commit ourselves to its simplified view of the world, as well as its thread-per-connection execution model. This is almost always an acceptable simplification, but it transitively limits everything downstream of it. Now we find ourselves in a situation where the entire ecosystem is predicated on this simplified view of things, and anyone who tries to do something more general has to start over from scratch.

So to actually answer your question, composition can't happen without conventions. But conventions, like macros (which really are just conventions of code structure), constrain and shape the code around them. The tradeoffs and resulting design space are what I want to explore.

LC: You've mentioned that Clojure does not constrain the code. One of the most common questions I answer from Clojure beginners is how to structure their code given the freedom allowed in Clojure. Do you think conventions that support composition could help answer that question?

ZT: I use a lot of Java-land libraries, so I spend a lot of time reading Javadocs. The nice thing about Javadocs is that even if the actual docstrings are minimal, it gives you a full, unambiguous dependency graph between all the different pieces of code. There's no analogue for that in Clojure, which makes reuse a lot harder than it otherwise would be.

I'm not sure what the solution for this is. I think standardizing on a narrower sort of code structure, like the "class" convention in Javascript, isn't that useful or likely to be popular. Standardized (and better) documentation, though, along with tooling to effectively browse it, would be enormously helpful. Formal specifications of data shapes or types, like Schema or core.typed, might be helpful but I don't think they're a necessary precondition.

LC: You mention dependency graph. Have you had a chance to look at CrossClj? Would that kind of thing be helpful? What would it need to be more helpful?

ZT: I've looked at it, it's a good place to start, but raw dependency data isn't useful without the tooling to integrate it into our development process. Light Table and others are playing around in this space, but it's a genuinely hard problem, because it's not a one-size-fits-all visualization problem. I don't have any solutions to offer, but I'm certainly paying attention to how things develop.

LC: So there are going to be a lot of beginners at the conj. Are there any resources that could prepare them to make the most of your talk? Any blog posts or videos that would get them ready to participate actively?

ZT: I'll talk about macros, transducers, and core.async, among other things. Being conversant in each will help, but isn't necessary.

LC: Where can people follow you and your adventures?

ZT: I twit about software sometimes at https://twitter.com/ztellman. I write longer stuff less often at http://ideolalia.com/.

LC: Ok, last question: What is the average airspeed velocity of an unladed Clojure REPL?

ZT: JVM or V8?

LC: I don't know that!

Thanks for the interview. It was very informative.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.