Symbols and Namespaces in Clojure and Common Lisp

This is the first in what I hope will be a series of articles documenting my experiences while learning Clojure. I've been playing with Clojure for a couple of years, but I've struggled to find the time to do more than just scratch the surface. At last I have some time to get into Clojure more deeply.

I have a background in Common Lisp, so I expect to spend a fair amount of my time grappling with the differences between Clojure and Common Lisp. As a start on that, this article discusses an area in which Clojure and Common Lisp differ at a fundamental level — symbols and namespaces.

An Overview

Coming from Common Lisp, I've found it a little hard to get my head around Clojure symbols and namespaces — it took time to stop taking certain things for granted.

I feel pretty comfortable with my understanding of the differences now, and I'm summarising my understanding here for several reasons. Firstly, the act of writing things down will force me to think things through in detail and make sure I really do understand; secondly, I hope I can make things easier for anyone else coming to Clojure from Common Lisp; and finally I hope this will be of interest to anyone interested in how Clojure relates to other Lisps.

The following statements are true for Clojure but not for Common Lisp:

• Symbols are simply names.
• Symbols do not belong to a namespace.
• Namespaces are mappings from symbols to vars, nothing more.
• Symbols are not storage locations.
• There is a notion of symbol resolution at compile time.
• It is not possible to use a symbol to refer to a private var from outside its namespace.
• Two symbols with the same name are distinct.
• It's not possible to read a form in one namespace and print it in another.

These statements are discussed in detail below, but I'll begin by giving an overview of symbols in Clojure without talking about Common Lisp. I'll mention aspects of symbols in Common Lisp as we need them. For now, I'll just mention that Clojure namespaces are analogous to Common Lisp packages.

An Overview of Symbols in Clojure

The following REPL interaction provides an overview of symbols in Clojure.

(name 'foo)
;; => "foo"

(name 'my-namespace/foo)
;; => "foo"

(namespace 'my-namespace/foo)
;; => "my-namespace"

(namespace 'foo)
;; => nil

Here we see that:

• A symbol may or may not have a slash within it.
• All symbols have a name, which is the whole thing if it doesn't have a slash and the part after the slash otherwise.
• A symbol with a slash (which is said to be namespace-qualified) has a namespace part.
• A symbol without a slash (which is said to be unqualified or simple) has no namespace part.

Symbols Do Not Belong to a Namespace

In Clojure a symbol is simply a name, nothing more. In particular, symbols do not belong to namespaces.

The following example illuminates this:

(ns a)
(defn f1 [] 'foo)

(ns b)
(defn f2 [] 'foo)

(ns c)
(= 'foo (a/f1) (b/f2))
;; => true

The symbol foo is the same no matter which namespace it is mentioned in.

Even namespace-qualified symbols do not belong to a namespace. What's more, we can have namespace-qualified symbols whose namespace part does not name a namespace that exists:

(find-ns 'no-such-ns)
;; => nil

'no-such-ns/foo
;; => no-such-ns/foo

This is in contrast to Common Lisp (note that : is used to access symbols within another package):

(in-package a)
(export 'f1)
(defun f1 () 'foo)

(in-package b)
(export 'f2)
(defun f2 () 'foo)

(in-package c)
(eql (a:f1) (b:f2))
;; => NIL

and

(find-package 'no-such-package)
;; => NIL

'no-such-package:foo
;; Error: Reader cannot find package NO-SUCH-PACKAGE.

Namespaces are Mappings from Symbols to Vars, Nothing More

In Clojure namespaces are mappings from unqualified symbols to vars. The mappings for public vars are distinct from the mappings for private vars. If one namespace refers or uses another namespace, the public mappings of the referred namespace are copied to the referring namespace as private mappings.

Note also that it's vars that are public or private in Clojure, not symbols.

Common Lisp packages contain their symbols, and a use relationship between two packages is maintained (effectively) as a pointer from the using package to the used package.

A consequence of this is that referring/using in Clojure is less dynamic than using in Common Lisp. The following example shows REPL interactions in two Clojure namespaces, a and b, interleaved in time. If we define a new public var in a after referring a from b, we cannot access the new public var from b until we refer a from b again.

a> (def foo 42)
                      b> (refer 'a)
                      b> foo
                      42
a> (def bar 99)
                      b> bar
                      ;; Evaluation aborted.
                      ;; Unable to resolve symbol: bar

                      ;; -- If we refer again all is ok...

                      b> (refer 'a)
                      b> bar
                      99

Here's the same thing in Common Lisp, showing the dynamic nature of the use relationship:

A > (defvar foo 42)
A > (export 'foo)
                      B > (use-package 'a)
                      B > foo
                      ;; => 42
A > (defvar bar 99)
A > (export 'bar)
                      B > bar
                      ;; => 99

Another consequence is that Clojure has no equivalent of the following in Common Lisp: if a package a exports a symbol foo, and package b uses package a, it is possible for b to export foo.

Symbols Are Not Storage Locations

In Clojure a symbol is simply a name. In Common Lisp a symbol is a storage location with a name.

In Clojure storage locations are provided by vars, which are named within a namespace by an unqualified symbol. (def foo ...) creates a var and adds a mapping in the current namespace from the symbol foo to the var.

There is a Notion of Symbol Resolution at Compile Time

In Clojure the process of taking a symbol and finding the var that it names is called resolution. Resolution happens at compile time. In Common Lisp there is no equivalent of resolution; the determination of a symbol's package happens at read time and nothing remains to be done later.

We can see how resolution works using the resolve function:

user> '+
;; => +

user> (resolve '+)
;; => #'clojure.core/+

It is not Possible to use a Symbol to Refer to a Private Var from outside its namespace

In Common Lisp we can refer to an unexported symbol using the double-colon notation, for example:

my-package::my-symbol

In Clojure it is not possible to use a symbol to refer to a private var from outside its namespace.

Two Symbols With the Same Name are Distinct

In Clojure, two symbols with the same name are distinct:

(identical? 'foo 'foo)
;; => false

but they are considered equal:

(= 'foo 'foo)
;; => true

In Common Lisp, within a package there is only one symbol with a given name:

(eq 'foo 'foo)
;; => T

and of course they are also considered equal by the default equality test:

(eql 'foo 'foo)
;; => T

It's Not Possible to Read a Form in One Namespace and Print it in Another

In Common Lisp, because symbols are tied to packages at read time and because it's possible to refer to unexported symbols, it's possible to read a form in one package and print it in another.

Here's a Common Lisp function that does the translation:

(defun translate-text-between-packages (text
                                        from-package
                                        to-package)
  (let* ((*package* from-package)
         (form (read-from-string text))
         (*package* to-package))
    (with-output-to-string (*standard-output*)
      (pprint form))))

And a sample use of the function:

(make-package 'editor-package)
(make-package 'repl-package)

(defvar repl-package::a)

(translate-text-between-packages "(+ repl-package::a b)"
                                 (find-package
                                  'editor-package)
                                 (find-package
                                  'repl-package))
;; => "(+ A EDITOR-PACKAGE::B)"

The printed form differs from the read form in the places where symbols need to be package-qualified in different ways. (Use of upper and lower case is a little unusual in Common Lisp. By default symbols are converted to upper case on input.)

This is simply not possible in Clojure.

For more detail, see a question I posed on Stack Overflow.

Closing Thoughts

A few closing thoughts...

This article is a bit of a rag-bag. I suspect it's possible to take a small number of the differences I've discussed and derive the remaining differences as logical consequences, but I haven't made that leap yet.

There are good reasons for at least some of the differences. One thing that is easier in Clojure is the writing of macros that avoid unintended name capture — at macroexpansion time symbols are turned into their namespace-qualified equivalents.

I haven't yet understood the reasons for all the design choices made by Clojure in this area and the trade-offs that need to be considered. I've spent a fair bit of time being confused and trying to put aside my knowledge of Common Lisp, and it will take a while yet before the Clojure Way fully sinks in. Once that's happened, I'll be in a better position to judge whether I like the design choices made by Clojure. For now I'll just say that there are some things that I miss from Common Lisp.