April 04, 2015
Leon Barrett's talk at Clojure/West is about parallelism in Clojure.
Background
Clojure is well known for its parallel programming super powers. Immutable data structures, concurrency primitives, and a few convenient constructs like future and pmap have been there since the beginning. But what's even cooler is how people have been able to build on the strong foundation Clojure established to create new parallel abstractions. Leon Barrett will talk about some of these. The description mentions reducers, tesser, and claypoole.
Rich Hickey gave a talk about reducers back in 2012, focusing on the ideas and abstractions they are based on. A more practical talk was given by Renzo Borgatti at Strange Loop 2013. Kyle Kingsbury gave a talk about tesser, a library which extends Clojure's parallel abstractions to execute in a distributed manner. And Leon Barrett himself wrote a recent blog post about Claypoole.
Homepage - GitHub - Google+
This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.
That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.
Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.
You might also like
April 03, 2015
Jeanine Adkisson's talk at Clojure/West is about designing and prototyping a language using Clojure.
Background
Lisps have been used for a long time to prototype other languages. Scheme was designed and used for just that purpose. The most popular books on compiler design often start with Lex and Yacc, which are just really old tools built when memory was so restricted you had to do your parsing in stages. Instaparse is way better for quickly exploring syntax. And since lambda calculus (which Clojure is based on) is computationally complete, you're in good hands exploring the entire range of computational semantics.
However, these are just generalities. Jeanine Adkisson will likely bring wit and depth to the talk, as she did in her Clojure/conj talk. She's a language designer, so she's got plenty of experience to draw from.
As an interesting background to language prototyping in Clojure, check out Bodil Stokke talking about her language Bodol.
Homepage - GitHub - Twitter
This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.
That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.
Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.
You might also like
April 03, 2015
Gary Fredericks' talk at Clojure/West is about a functional approach to generating random numbers.
Background
Random number generation (RNG) is typically achieved by using mutable state. And of course, that means they're not great for parallel processing, let alone being generally harder to reason about.
An immutable random number generator returns the next random bytes and a new generator for the next number. A splittable random number generator adds a new function called split, which takes a generator and returns two new generators. It lets you guarantee that the same numbers are generated every time regardless of interleaving.
Haskell's Quickcheck uses splittable random number generators. The talk description states that Clojure's test.check, which is based on Quickcheck, would benefit from splittable random number generators, presumably by allowing for concurrency.
I found an interesting paper about an efficient implementation of Splittable RNGs. Also, the dunaj-project has some interesting work in this area, using java.util.SplittableRandom (Java 8) and Transducers.
Homepage - GitHub - Twitter
This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.
That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.
Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.
You might also like
April 03, 2015
Introduction
Alan Dipert and Micha Niskin generously agreed to do an interview about their talk at Clojure/West. The background to their talk is available, if you like.
Interview
LispCast: How did you get into Clojure?
Micha Niskin: I watched the SICP lectures from MIT and was blown away by the simplicity of Lisp. At the time I was programming in Perl, C, and Java, mostly (I was actually on a programming hiatus building boats in Maine, but you know, you never just stop programming). So I started doing some things in Scheme (Racket) and I really enjoyed it. I was learning awesome new things at a rate I'd never experienced before. Then Alan recommended I try Clojure. I was initially skeptical of the syntax, but persistent collections were a real revelation and I quickly joined the cult.
Alan Dipert: When I learned about arrays in Visual Basic 4, someone told me that some language "LISP" was somehow made out of arrays, which I giggled at because it sounded so silly. Later, in college, my coolest professor challenged my Java class to complete assignments in alternate JVM languages. I took the opportunity to learn some Scala, and for the final assignment, Clojure. Clojure blew my mind - what didn't immediately click was somehow very obviously something I would need to learn and would unlock more awesome things. The Clojure, and the learning, haven't stopped since then.
LC: Last year you all released Hoplon. Everyone else was zigging toward React, while you zagged to a completely different paradigm. I can't help but see a parallel with Boot. Nearly all Clojure projects use Leiningen, and here you are zagging again. I think you've explained why already. What I want to know is the thinking that leads to two very zaggy approaches. Were the existing libraries not scratching your itches? Or do you have a deep desire to develop your own tools from scratch?
AD: Hoplon is a packaging of a vision we have for how to write and maintain Single Page Applications (SPAs). We had been developing this vision for about two years before Hoplon's release, and were driven to see the vision through because we needed practical tools for professional use and found nothing that existed to be suitable.
In the first year (2012) of pre-Hoplon exploration, it became clear we needed to either find or develop a model for state that could work in a browser. Of the technologies we evaluated, we explored FRP most thoroughly 1. We built a series of applications in ClojureScript using Flapjax 2.
Our experience with Flapjax informed our opinion that of the units of composition in FRP - behaviors and event streams - only behaviors are necessary for driving browser UIs. This is because UIs are views into memory/data, not into processes. For anything to appear in a UI, it must first appear in memory somewhere. We think behaviors, which are conceptually similar to cells in a spreadsheet, are the ultimate tool for coordinating change in data (NOT process) over time. We think core.async is probably the ultimate tool for coordinating process (NOT data) over time. FRP in its various formulations tries to do both, and not in a way that we find especially helpful.
We codified these opinions in early 2013 into Javelin 3, a ClojureScript library for modeling data in SPA UIs. What Javelin didn't solve -- or even imply -- was how to actually display the data being modeled.
If a UI is really just a stylized view into the heap, then any data structure is a kind of UI. If Lisp is the ultimate way to compose and manipulate data structures, then it must also be the ultimate way to compose and manipulate UIs. This is the thinking behind the second big piece of Hoplon, "the HLisp semantics", our answer to the question of display that Javelin leaves totally open. We worked on and around this concept in the 2nd year of pre-Hoplon (2013).
Concretely, HLisp is a set of semantics 5 that map HTML markup into the Lisp evaluation model. That is, the HTML expression <ul><li>one</li></ul> is identical to the ClojureScript expression (ul (li "one")). Both evaluate to a live Node object in heap that can be passed to and returned from functions. Under HLisp, a list might be constructed like this: (apply ul (map li ["one" "two" "three"])). Because these expressions evaluate to native browser objects, they can compose with a vast array of plugins and libraries available in the frontend ecosystem.
A big positive of HLisp is that HTML becomes a syntax for a subset of ClojureScript. We find this especially beneficial because it means any designer or UX expert familiar with HTML can easily contribute. We've never worked on a team that didn't include at least one designer/UX expert, and we believe this demographic is totally underserved by existing tools. Part of our vision is to empower designers to experience the same level of "flow" that we Lispers know comes with simple, clear models that we can infinitely extend.
Realizing fully HLisp pushed the limits of every build tool we tried to make it work in, because it was different than what anyone had ever wanted to do before. It was clear to us that the kind of transformation HLisp required needed to be accessible, and we began to imagine how a build system capable of these kinds of things could work. We dreamed of compiling Markdown, into HAML, into HTML, into ClojureScript, into JavaScript, reliably, simply, and accessibly.
We developed Boot1 to serve these needs, and shipped Hoplon atop it last year. We haven't had much reason to change Javelin, HLisp, or Hoplon over the past couple years, but we have rolled the set of ideas for improvement accumulated over a year of Boot1 use into the nascent Boot2. Today, Boot2 is capable of the Markdown -> JavaScript transformation and much more. We are really happy and excited that it is drawing enthusiasm from Clojure and ClojureScript users outside of Hoplon-world.
To finally answer your question: It's not that we think being different or building our own tools are virtues; we're just lucky to have had a series of opportunities -- starting with our friendship -- to experience, study deeply, and attempt to solve as definitively as possible the big programming challenges we've faced.
LC: In terms of Boot, what were those challenges that Leiningen did not solve? And how does Boot solve them?
AD:
- Leiningen does not supply a general file system watcher; every plugin must provide its own
- Leiningen does not coordinate filesystem access between plugins; the user must be mindful of every plugin's file use and weary of filesystem contention
- Leiningen plugins cannot easily be composed or invoked from the REPL, which requires the user to start a new JVM for every experiment
For these reasons I would say that Leiningen plugins must all duplicate essential functionality, don't compose well, and are cumbersome to develop and experiment with.
Hoplon existed for a short time as a Leiningen plugin, but like all ambitious Leiningen plugins it became its own build tool and didn't compose especially well with other plugins. Once your Leiningen plugin becomes essentially its own build tool, it's not hard to imagine taking the last step.
This is what we did and Boot was the result; Hoplon dissolved into a half-dozen Boot "tasks". Boot solves the problems we encountered in Leiningen:
- Boot tasks are "stateful middlewares" composed in a pipeline, similar to Ring middleware. Tasks can decide whether the tasks after them should run, the same way a Ring middleware can decide whether or not to invoke the next handler. With a reference to the next task, a task can invoke the next task once, multiple times, or not at all. With this ability, "watch" is no longer special, and it becomes a regular task. Library authors are empowered to write other watch-like tasks that initiate or prohibit builds depending on any condition, not just filesystem state.
- The value passed through the pipeline of tasks represents the file system, and is called a FileSet. It is an immutable snapshot of a filesystem. If a task needs to effect a change, addition, or deletion to the filesystem, it performs an operation on the FileSet -- returning a new FileSet -- NOT directly on the filesystem as a mutation operation. After the last task returns the final FileSet, Boot synchronizes the FileSet to the target directory. Access to the value of a FileSet doesn't need to be synchronized because it is immutable and the actual filesystem is not in contention. FileSet operations are efficient because Boot uses a content-addressed, hard-link backed structural sharing scheme to minimize actual file operations.
- A Boot task is a function, nothing more. Boot tasks can be created, tested, and composed in the REPL or in any Clojure program just like any Clojure function can. Facilitating task composition is a "unit of classpath isolation", the "pod", which is a Clojure runtime that can have dependencies independent of the rest of the build. So, these task functions can have dependencies independent of the project they might operate on. For instance, a task might supply typechecking using core.typed in a pod, but this doesn't imply the user know or care to add core.typed to their project's dependencies. Because tasks are functions, and because we have a unit of classpath, most builds can be orchestrated in a single JVM. Builds are easier to develop, and more efficient to run.
LC: This sounds great! Is there an efficient path you would recommend for switching a project from Leiningen to Boot?
AD: The best place to go after installing is definitely the wiki.
On the wiki there are many community-contributed tasks and projects to look at for ideas and examples. There is an active and helpful community that you can connect with on IRC (in #hoplon on freenode) and at http://hoplon.discoursehosting.net/
If you have an idea for a script or small program, this is a very nice introductory tutorial that demonstrates making a standalone, boot-powered script that can be called from the command line.
LC: Does Boot have dependency management? Does it work with Clojars?
MN: Sure, Boot uses Pomegranate, the same Aether library that Leiningen uses. Boot can also use Leiningen style wagons (like the S3 private wagon, for instance). The built-in tasks include tasks to install jars in your local Maven repo and to push to remote repos like Clojars. Dependencies can be loaded dynamically at runtime, and Boot has "pods" in which you can load dependencies that conflict with something in the current runtime.
Incidentally, I've been looking at Apache Ivy a bit, it looks really interesting. Maybe it's something we can use in the Clojure world.
LC: Where can people follow you and Boot online? How can they help?
MN: The Boot website is the main entry point into the weird world. We also hang out in the #hoplon channel on Freenode IRC where we provide support for all things tailrecursion but mostly just enjoy the interesting conversations that go on in there :) Help is always welcomed, people can check the open issues or create an issue here or on the associated kanban board here. We try to schedule RC releases weekly every Monday.
LC: Where can people follow you online?
AD: We can be followed at https://twitter.com/alandipert and https://twitter.com/michaniskin.
LC: If Clojure were a food, what food would it be?
AD: We both agree that Clojure is like anchovies: it's weird, relatively unpopular, but an essential ingredient in almost anything that's good. You should only ever need a small amount to make the dreariest meals magnificent. But never tell -- nobody needs to know exactly what they're so enjoying.
This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.
That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.
Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.
You might also like
March 30, 2015
Fumiko Hanreich's talk at Clojure/West is about HoneySQL, a Clojure DSL for representing SQL queries.
Background
People in the Clojure community talk a lot about representing things as data. This is an approach inherited from its Lisp forebears. HoneySQL is a data-oriented DSL that represents SQL queries. You build up SQL statements in a structured form, not string concatenation.
HoneySQL is a good example of a case where the Clojure philosophy allows for a solution to an existing problem. The problem is that SQL is designed as a human-facing language. That's fine by itself, but it is not a good language for composing pieces of a query into a larger query--the kind of things a program would do. HoneySQL provides an intermediate representation that is way more amenable to manipulation. In a way, it does the same thing for SQL as Lisp s-expressions did for code.
Twitter
This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.
That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.
Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.
You might also like
March 30, 2015
Summary: Where to put your tests is a common question. You could put them anywhere, but you want to pick a place that makes it easy to find, easy to exclude from production, and work well with your tools. My recommendation is to follow what most projects do, which takes care of all of these requirements.
You want to write some tests in Clojure. Maybe they're unit tests. Maybe they're integration tests. The first question you must answer is where do you put your tests?
And you don't want them just anywhere. You actually have some important requirements dealing with where they are:
- They should not be compiled/run on production.
- They need to be easily findable.
- They should be runnable from the command line.
- They should be runnable from the REPL.
Here's my recommendation, which is the de facto standard of organizing your tests. It works with Leiningen, CIDER, and vim-fireplace.
First, you make a new namespace structure in a test/ directory. It should mirror the src/ directory.
If you have:
src/
lispcast/
core.clj
init.clj
util.clj
Then your test directory should look like:
test/
lispcast/
core_test.clj
init_test.clj
util_test.clj
But also notice the second point: that the structure is the same, but the names are slightly different, but it a systematic way. It's really easy: you just add -test to the namespace name, which becomes _test in the file name. Then, you put all the tests that test lispcast.core into lispcast.core-test. Now they're easy to find!
If you need to write a test that crosses two different namespaces (like an integration test might), then you can just make a new test namespace that doesn't correspond to one or the other.
Leiningen will load the test/ directory selectively, depending on if you're deploying to production (it won't load test/) or running the tests (it will load test/).
So, it's that easy. You are free, of course, to put the tests wherever you like. But this is my recommendation!
If you're getting into testing in Clojure, you should check out LispCast Intro to clojure.test. It's an interactive course. It has animations, screencasts, exercises, code samples, and text.
You might also like
March 29, 2015
This summary was graciously written by Nola Stowe. She's a programmer, the co-founder of DevChix, and a prolific teacher. She recently ran ClojureBridge Austin. Please shout out to her and say thanks!
John Hume's talk at Clojure/West is about Clojure internals.
Background
You can get a lot done in Clojure without understanding much about its internals. But when your code doesn't do what you expect, whether in terms of behavior or performance, it can be extremely useful to dig into the underlying Java code.
John's presentation will walk the audience through some of that code by following a simple map-literal expression through Clojure's reader and analyzer, particularly focusing on code generation in clojure.lang.Compiler, and then onto runtime evaluation. The hope is that you'll come out of the talk better able to analyze what's really happening in your own Clojure code.
Github - Twitter - Website
This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.
That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.
Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.
You might also like
