Our next potentially unfamiliar Lisp feature is locally.  Before proceeding, I would recommend that you review the articles on declare and declaim.

The locally special operator gives us an intermediate control between declare and declaim.  We had declaim, which causes a change in the compiler behaviour that persists until explicitly reset, but possibly resetting at the end of the compilation unit, at the discretion of the implementers.  We had declare, which affects the behaviour of the compiler within the current form.  Now, with locally, we have a way to change compiler behaviour over several forms, but with an explicit end to this change.  Here’s an example of how this might be used.  In this file, we have three functions.  Two of them have been debugged, and we want to optimize them because they’re expensive functions that slow down our testing.  The third function is buggy, and we want to turn on debugging settings to try to help solve the issues.  So, we declaim the compiler settings for optimization in the file as a whole, but use locally to switch over to debug settings for the one function we want to compile that way:
 

(declaim (optimize (debug 0) (safety 0) (speed 3)))

(eval-when (:compile-toplevel)
  (format t "~%~%~%About to compile normal-1~%")
  (sb-ext:describe-compiler-policy))

(defun normal-1 (x)
  (declare (sb-ext:muffle-conditions sb-ext:compiler-note))
  (+ 2 x))

(locally
    (declare (optimize (debug 3) (safety 3) (speed 0)))

  (eval-when (:compile-toplevel)
    (format t "~%~%~%About to compile debugging-1~%")
    (sb-ext:describe-compiler-policy))

  (defun debugging-1 (x)
    (declare (sb-ext:muffle-conditions sb-ext:compiler-note))
    (+ 3 x)))

(eval-when (:compile-toplevel)
  (format t "~%~%~%About to compile normal-2~%")
  (sb-ext:describe-compiler-policy))

(defun normal-2 (x)
  (declare (sb-ext:muffle-conditions sb-ext:compiler-note))
  (+ 4 x))

CL-USER> (compile-file "locally")
; compiling file "locally.lisp" (written 26 FEB 2014 09:06:22 PM):
; compiling (DECLAIM (OPTIMIZE # ...))

About to compile normal-1
  Basic qualities:
COMPILATION-SPEED = 1
DEBUG = 0
SAFETY = 0
SPACE = 1
SPEED = 3
INHIBIT-WARNINGS = 1
  Dependent qualities:
SB-C::CHECK-CONSTANT-MODIFICATION = 1 -> 0 (no)
SB-C::TYPE-CHECK = 1 -> 0 (no)
SB-C::CHECK-TAG-EXISTENCE = 1 -> 0 (no)
SB-C::LET-CONVERSION = 1 -> 3 (on)
SB-C:ALIEN-FUNCALL-SAVES-FP-AND-PC = 1 -> 0 (no)
SB-C:VERIFY-ARG-COUNT = 1 -> 0 (no)
SB-C::INSERT-DEBUG-CATCH = 1 -> 0 (no)
SB-C::RECOGNIZE-SELF-CALLS = 1 -> 3 (yes)
SB-C::FLOAT-ACCURACY = 1 -> 3 (full)
SB-C:INSERT-STEP-CONDITIONS = 1 -> 0 (no)
SB-C::COMPUTE-DEBUG-FUN = 1 -> 0 (no)
SB-C::PRESERVE-SINGLE-USE-DEBUG-VARIABLES = 1 -> 0 (no)
SB-C::INSERT-ARRAY-BOUNDS-CHECKS = 1 -> 0 (no)
SB-C::STORE-XREF-DATA = 1 -> 3 (yes)
SB-C:STORE-COVERAGE-DATA = 1 -> 0 (no)
; compiling (DEFUN NORMAL-1 ...)

About to compile debugging-1
  Basic qualities:
COMPILATION-SPEED = 1
DEBUG = 3
SAFETY = 3
SPACE = 1
SPEED = 0
INHIBIT-WARNINGS = 1
  Dependent qualities:
SB-C::CHECK-CONSTANT-MODIFICATION = 1 -> 3 (yes)
SB-C::TYPE-CHECK = 1 -> 3 (full)
SB-C::CHECK-TAG-EXISTENCE = 1 -> 3 (yes)
SB-C::LET-CONVERSION = 1 -> 0 (off)
SB-C:ALIEN-FUNCALL-SAVES-FP-AND-PC = 1 -> 3 (yes)
SB-C:VERIFY-ARG-COUNT = 1 -> 3 (yes)
SB-C::INSERT-DEBUG-CATCH = 1 -> 3 (yes)
SB-C::RECOGNIZE-SELF-CALLS = 1 -> 0 (no)
SB-C::FLOAT-ACCURACY = 1 -> 3 (full)
SB-C:INSERT-STEP-CONDITIONS = 1 -> 3 (full)
SB-C::COMPUTE-DEBUG-FUN = 1 -> 3 (yes)
SB-C::PRESERVE-SINGLE-USE-DEBUG-VARIABLES = 1 -> 3 (yes)
SB-C::INSERT-ARRAY-BOUNDS-CHECKS = 1 -> 3 (yes)
SB-C::STORE-XREF-DATA = 1 -> 3 (yes)
SB-C:STORE-COVERAGE-DATA = 1 -> 0 (no)
; compiling (DEFUN DEBUGGING-1 ...)

About to compile normal-2
  Basic qualities:
COMPILATION-SPEED = 1
DEBUG = 0
SAFETY = 0
SPACE = 1
SPEED = 3
INHIBIT-WARNINGS = 1
  Dependent qualities:
SB-C::CHECK-CONSTANT-MODIFICATION = 1 -> 0 (no)
SB-C::TYPE-CHECK = 1 -> 0 (no)
SB-C::CHECK-TAG-EXISTENCE = 1 -> 0 (no)
SB-C::LET-CONVERSION = 1 -> 3 (on)
SB-C:ALIEN-FUNCALL-SAVES-FP-AND-PC = 1 -> 0 (no)
SB-C:VERIFY-ARG-COUNT = 1 -> 0 (no)
SB-C::INSERT-DEBUG-CATCH = 1 -> 0 (no)
SB-C::RECOGNIZE-SELF-CALLS = 1 -> 3 (yes)
SB-C::FLOAT-ACCURACY = 1 -> 3 (full)
SB-C:INSERT-STEP-CONDITIONS = 1 -> 0 (no)
SB-C::COMPUTE-DEBUG-FUN = 1 -> 0 (no)
SB-C::PRESERVE-SINGLE-USE-DEBUG-VARIABLES = 1 -> 0 (no)
SB-C::INSERT-ARRAY-BOUNDS-CHECKS = 1 -> 0 (no)
SB-C::STORE-XREF-DATA = 1 -> 3 (yes)
SB-C:STORE-COVERAGE-DATA = 1 -> 0 (no)
; compiling (DEFUN NORMAL-2 ...)

; locally.fasl written
; compilation finished in 0:00:00.004
#P"locally.fasl"
NIL
NIL

I’ve used some SBCL-specific features to report the compilation settings, and also to suppress compilation notes that otherwise would complicate this output.  You can see that the normal-1 and normal-2 functions are compiled with optimizations, while the debugging-1 function is compiled with debug settings.

Next on our tour of less commonly used Lisp features is load-time-value.  This special operator allows the programmer to write a form that is executed when the file is loaded, and to take the primary value returned by that form and insert it into the code as a literal object.

So, why might one typically use this?  The obvious use case is to perform expensive operations to initialize a local variable inside a function.  While the programmer could choose to do this work in a top-level form assigned to the value field of an interned symbol, that violates encapsulation principles by making the variable visible outside the scope of its intended user.

To demonstrate, I’ll start with the alternative solution.  For purposes of this demonstration imagine that expensive-function does something difficult that takes a long time to run.  Adding one to a number is not commonly a difficult operation, but it’s my stand-in.  So, here’s the solution without load-time-value:
 

(defparameter *lookup-table*
  (labels
      ((expensive-function (x)
         (format t "WORKING-BAD~%")
         (+ 1 x)))
    (mapcar #'expensive-function '(1 2 3 4 5 6 7))))

(defun demonstrate-bad ()
  (format t "lookup-table is ~A~%" *lookup-table*))

Loading and executing is as follows:
 

CL-USER> (load "load-time-value")
WORKING-BAD
WORKING-BAD
WORKING-BAD
WORKING-BAD
WORKING-BAD
WORKING-BAD
WORKING-BAD
T
CL-USER> (demonstrate-bad)
lookup-table is (2 3 4 5 6 7 8)
NIL

So, when the file is compiled, our lookup table is assembled, and the expensive function is executed.  When the demonstrate-bad function is called, the lookup table is not recomputed, so it doesn’t have to make that expensive calculation every time it’s invoked.  The disadvantage here is that I’ve had to intern a symbol called *lookup-table*.  Other functions can locate that symbol, and can modify its contents, and if I have to do this many times I’ll have to be careful that my lookup table names don’t collide.  Also, by separating the definition of the table from the function that uses it, it becomes less obvious to the maintainer what *lookup-table* is for, and who might be using or modifying it.

Now, here’s the solution with load-time-value:
 

(defun demonstrate ()
  (let ((lookup-table
         (load-time-value
          (labels 
              ((expensive-function (x)
                 (format t "WORKING~%")
                 (+ 1 x)))
            (mapcar #'expensive-function '(1 2 3 4 5 6 7))))))
    (format t "lookup-table is: ~A~%" lookup-table)))

CL-USER> (load "load-time-value")
WORKING
WORKING
WORKING
WORKING
WORKING
WORKING
WORKING
T
CL-USER> (demonstrate)
lookup-table is: (2 3 4 5 6 7 8)
NIL

The behaviour is the same as in the other example, but now the lookup table is a local variable within demonstrate.  It’s not stored in an interned symbol, so other functions can’t locate or modify it.  It can’t collide with variable names in other functions, and by putting it in this function it makes it very clear to the maintainer that this table is used exclusively within the demonstrate function.

It is important to note that the load-time-value form may be executed before other top-level forms in the same file have loaded.  This means that if the form invokes functions defined in the same file, those functions may or may not be known at the time the load-time-value form executes.  The relative order is implementation-defined, so the programmer cannot assume any particular order in portable code.  To avoid confusion, such load-time-value forms should only refer to symbols that are known to have been loaded earlier, in other files, or to symbols that are part of the Common Lisp image.  This is why I have used labels here, rather than a separate defun-ed function.

Next, the ldiff and tailp functions.  These are fairly simple in their descriptions, but it is important to note that they look for sublists that are eq, not equal.  That is, their special behaviour is only apparent on sublists that are part of the list they’re being compared against.  An example might help:
 

CL-USER> (let ((list-1 '(1 2 3 4 5 6))
               (list-2 '(4 5 6)))
           (format t "ldiff on non-eq lists: ~A~%"
                   (ldiff list-1 list-2)))
ldiff on non-eq lists: (1 2 3 4 5 6)
NIL
CL-USER> (let* ((list-1 '(1 2 3 4 5 6))
                (list-2 (cdddr list-1)))
           (format t "ldiff on true sublists: ~A~%"
                   (ldiff list-1 list-2)))
ldiff on true sublists: (1 2 3)
NIL
CL-USER> (let ((list-1 '(1 2 3 4 5 6))
               (list-2 '(4 5 6)))
           (format t "tailp on non-eq lists: ~A~%"
                   (tailp list-2 list-1)))
tailp on non-eq lists: NIL
NIL
CL-USER> (let* ((list-1 '(1 2 3 4 5 6))
                (list-2 (cdddr list-1)))
           (format t "tailp on true sublists: ~A~%"
                   (tailp list-2 list-1)))
tailp on true sublists: T
NIL

So, as long as you remember this constraint, and also remember that the order of the arguments is reversed between tailp and ldiff, you will have no troubles with these functions.

For a description of ldb, please see the earlier article on byte.

Now, the newcomer to Lisp, arriving from C++, has certainly encountered lambda before, as part of a construct to build functions that can be passed to functions.  Just about any introduction to Lisp will show lambda forms being passed to mapcar or sort.  There is, however, a short digression we can go into in order to clear up some confusion.

While reading over Lisp code, you’ve almost certainly seen lambda used in two visually distinct ways:
 

(let ((my-list (list 1 2 3 4 5)))
  (mapcar #'(lambda (x) (* x 3)) my-list))

(let ((my-list (list 1 2 3 4 5)))
  (mapcar (lambda (x) (* x 3)) my-list))

What is the difference between these two?  Why do you sometimes see one form, and sometimes another?

You’ll recall we talked about read-macros, and that the form #’<STUFF> is rewritten (function <STUFF>).  The standard Lisp implementation also defines a macro for lambda, which expands to #’, as follows:
 

CL-USER> (macroexpand '(lambda (x) (* x 3)))
#'(LAMBDA (X) (* X 3))
T

So, in fact, there is no difference at all between the two forms, they are interpreted the same way by the Lisp instance.  It comes down to a matter of preference.  Personally, I use the prefixed form of lambda, because I find that it sticks out a bit more that way, a lambda is built a bit like a function invocation, and the syntactic prefix helps to separate that in my mind when quickly skimming through code.