Monthly Archives: March 2014

The less-familiar parts of Lisp for beginners — locally

Our next potentially unfamiliar Lisp feature is locally.  Before proceeding, I would recommend that you review the articles on declare and declaim.

The locally special operator gives us an intermediate control between declare and declaim.  We had declaim, which causes a change in the compiler behaviour that persists until explicitly reset, but possibly resetting at the end of the compilation unit, at the discretion of the implementers.  We had declare, which affects the behaviour of the compiler within the current form.  Now, with locally, we have a way to change compiler behaviour over several forms, but with an explicit end to this change.  Here’s an example of how this might be used.  In this file, we have three functions.  Two of them have been debugged, and we want to optimize them because they’re expensive functions that slow down our testing.  The third function is buggy, and we want to turn on debugging settings to try to help solve the issues.  So, we declaim the compiler settings for optimization in the file as a whole, but use locally to switch over to debug settings for the one function we want to compile that way:
 

(declaim (optimize (debug 0) (safety 0) (speed 3)))

(eval-when (:compile-toplevel)
  (format t "~%~%~%About to compile normal-1~%")
  (sb-ext:describe-compiler-policy))

(defun normal-1 (x)
  (declare (sb-ext:muffle-conditions sb-ext:compiler-note))
  (+ 2 x))

(locally
    (declare (optimize (debug 3) (safety 3) (speed 0)))

  (eval-when (:compile-toplevel)
    (format t "~%~%~%About to compile debugging-1~%")
    (sb-ext:describe-compiler-policy))

  (defun debugging-1 (x)
    (declare (sb-ext:muffle-conditions sb-ext:compiler-note))
    (+ 3 x)))

(eval-when (:compile-toplevel)
  (format t "~%~%~%About to compile normal-2~%")
  (sb-ext:describe-compiler-policy))

(defun normal-2 (x)
  (declare (sb-ext:muffle-conditions sb-ext:compiler-note))
  (+ 4 x))

The output of compilation is as follows:
 
CL-USER> (compile-file "locally")
; compiling file "locally.lisp" (written 26 FEB 2014 09:06:22 PM):
; compiling (DECLAIM (OPTIMIZE # ...))

About to compile normal-1
  Basic qualities:
COMPILATION-SPEED = 1
DEBUG = 0
SAFETY = 0
SPACE = 1
SPEED = 3
INHIBIT-WARNINGS = 1
  Dependent qualities:
SB-C::CHECK-CONSTANT-MODIFICATION = 1 -> 0 (no)
SB-C::TYPE-CHECK = 1 -> 0 (no)
SB-C::CHECK-TAG-EXISTENCE = 1 -> 0 (no)
SB-C::LET-CONVERSION = 1 -> 3 (on)
SB-C:ALIEN-FUNCALL-SAVES-FP-AND-PC = 1 -> 0 (no)
SB-C:VERIFY-ARG-COUNT = 1 -> 0 (no)
SB-C::INSERT-DEBUG-CATCH = 1 -> 0 (no)
SB-C::RECOGNIZE-SELF-CALLS = 1 -> 3 (yes)
SB-C::FLOAT-ACCURACY = 1 -> 3 (full)
SB-C:INSERT-STEP-CONDITIONS = 1 -> 0 (no)
SB-C::COMPUTE-DEBUG-FUN = 1 -> 0 (no)
SB-C::PRESERVE-SINGLE-USE-DEBUG-VARIABLES = 1 -> 0 (no)
SB-C::INSERT-ARRAY-BOUNDS-CHECKS = 1 -> 0 (no)
SB-C::STORE-XREF-DATA = 1 -> 3 (yes)
SB-C:STORE-COVERAGE-DATA = 1 -> 0 (no)
; compiling (DEFUN NORMAL-1 ...)

About to compile debugging-1
  Basic qualities:
COMPILATION-SPEED = 1
DEBUG = 3
SAFETY = 3
SPACE = 1
SPEED = 0
INHIBIT-WARNINGS = 1
  Dependent qualities:
SB-C::CHECK-CONSTANT-MODIFICATION = 1 -> 3 (yes)
SB-C::TYPE-CHECK = 1 -> 3 (full)
SB-C::CHECK-TAG-EXISTENCE = 1 -> 3 (yes)
SB-C::LET-CONVERSION = 1 -> 0 (off)
SB-C:ALIEN-FUNCALL-SAVES-FP-AND-PC = 1 -> 3 (yes)
SB-C:VERIFY-ARG-COUNT = 1 -> 3 (yes)
SB-C::INSERT-DEBUG-CATCH = 1 -> 3 (yes)
SB-C::RECOGNIZE-SELF-CALLS = 1 -> 0 (no)
SB-C::FLOAT-ACCURACY = 1 -> 3 (full)
SB-C:INSERT-STEP-CONDITIONS = 1 -> 3 (full)
SB-C::COMPUTE-DEBUG-FUN = 1 -> 3 (yes)
SB-C::PRESERVE-SINGLE-USE-DEBUG-VARIABLES = 1 -> 3 (yes)
SB-C::INSERT-ARRAY-BOUNDS-CHECKS = 1 -> 3 (yes)
SB-C::STORE-XREF-DATA = 1 -> 3 (yes)
SB-C:STORE-COVERAGE-DATA = 1 -> 0 (no)
; compiling (DEFUN DEBUGGING-1 ...)

About to compile normal-2
  Basic qualities:
COMPILATION-SPEED = 1
DEBUG = 0
SAFETY = 0
SPACE = 1
SPEED = 3
INHIBIT-WARNINGS = 1
  Dependent qualities:
SB-C::CHECK-CONSTANT-MODIFICATION = 1 -> 0 (no)
SB-C::TYPE-CHECK = 1 -> 0 (no)
SB-C::CHECK-TAG-EXISTENCE = 1 -> 0 (no)
SB-C::LET-CONVERSION = 1 -> 3 (on)
SB-C:ALIEN-FUNCALL-SAVES-FP-AND-PC = 1 -> 0 (no)
SB-C:VERIFY-ARG-COUNT = 1 -> 0 (no)
SB-C::INSERT-DEBUG-CATCH = 1 -> 0 (no)
SB-C::RECOGNIZE-SELF-CALLS = 1 -> 3 (yes)
SB-C::FLOAT-ACCURACY = 1 -> 3 (full)
SB-C:INSERT-STEP-CONDITIONS = 1 -> 0 (no)
SB-C::COMPUTE-DEBUG-FUN = 1 -> 0 (no)
SB-C::PRESERVE-SINGLE-USE-DEBUG-VARIABLES = 1 -> 0 (no)
SB-C::INSERT-ARRAY-BOUNDS-CHECKS = 1 -> 0 (no)
SB-C::STORE-XREF-DATA = 1 -> 3 (yes)
SB-C:STORE-COVERAGE-DATA = 1 -> 0 (no)
; compiling (DEFUN NORMAL-2 ...)

; locally.fasl written
; compilation finished in 0:00:00.004
#P"locally.fasl"
NIL
NIL

I’ve used some SBCL-specific features to report the compilation settings, and also to suppress compilation notes that otherwise would complicate this output.  You can see that the normal-1 and normal-2 functions are compiled with optimizations, while the debugging-1 function is compiled with debug settings.

The less-familiar parts of Lisp for beginners — load-time-value

Next on our tour of less commonly used Lisp features is load-time-value.  This special operator allows the programmer to write a form that is executed when the file is loaded, and to take the primary value returned by that form and insert it into the code as a literal object.

So, why might one typically use this?  The obvious use case is to perform expensive operations to initialize a local variable inside a function.  While the programmer could choose to do this work in a top-level form assigned to the value field of an interned symbol, that violates encapsulation principles by making the variable visible outside the scope of its intended user.

To demonstrate, I’ll start with the alternative solution.  For purposes of this demonstration imagine that expensive-function does something difficult that takes a long time to run.  Adding one to a number is not commonly a difficult operation, but it’s my stand-in.  So, here’s the solution without load-time-value:
 

(defparameter *lookup-table*
  (labels
      ((expensive-function (x)
         (format t "WORKING-BAD~%")
         (+ 1 x)))
    (mapcar #'expensive-function '(1 2 3 4 5 6 7))))

(defun demonstrate-bad ()
  (format t "lookup-table is ~A~%" *lookup-table*))

Loading and executing is as follows:
 

CL-USER> (load "load-time-value")
WORKING-BAD
WORKING-BAD
WORKING-BAD
WORKING-BAD
WORKING-BAD
WORKING-BAD
WORKING-BAD
T
CL-USER> (demonstrate-bad)
lookup-table is (2 3 4 5 6 7 8)
NIL

So, when the file is compiled, our lookup table is assembled, and the expensive function is executed.  When the demonstrate-bad function is called, the lookup table is not recomputed, so it doesn’t have to make that expensive calculation every time it’s invoked.  The disadvantage here is that I’ve had to intern a symbol called *lookup-table*.  Other functions can locate that symbol, and can modify its contents, and if I have to do this many times I’ll have to be careful that my lookup table names don’t collide.  Also, by separating the definition of the table from the function that uses it, it becomes less obvious to the maintainer what *lookup-table* is for, and who might be using or modifying it.

Now, here’s the solution with load-time-value:
 

(defun demonstrate ()
  (let ((lookup-table
         (load-time-value
          (labels 
              ((expensive-function (x)
                 (format t "WORKING~%")
                 (+ 1 x)))
            (mapcar #'expensive-function '(1 2 3 4 5 6 7))))))
    (format t "lookup-table is: ~A~%" lookup-table)))

The output of load and invoke is:
 
CL-USER> (load "load-time-value")
WORKING
WORKING
WORKING
WORKING
WORKING
WORKING
WORKING
T
CL-USER> (demonstrate)
lookup-table is: (2 3 4 5 6 7 8)
NIL

The behaviour is the same as in the other example, but now the lookup table is a local variable within demonstrate.  It’s not stored in an interned symbol, so other functions can’t locate or modify it.  It can’t collide with variable names in other functions, and by putting it in this function it makes it very clear to the maintainer that this table is used exclusively within the demonstrate function.

It is important to note that the load-time-value form may be executed before other top-level forms in the same file have loaded.  This means that if the form invokes functions defined in the same file, those functions may or may not be known at the time the load-time-value form executes.  The relative order is implementation-defined, so the programmer cannot assume any particular order in portable code.  To avoid confusion, such load-time-value forms should only refer to symbols that are known to have been loaded earlier, in other files, or to symbols that are part of the Common Lisp image.  This is why I have used labels here, rather than a separate defun-ed function.