Monthly Archives: November 2013

List-printing code for upcoming posts

I’m about to start what may be a long series of posts about the Lisp language for newcomers arriving from a C++ background.  Before I get to that, I’m going to write a bit of handy code to produce drawings of lists similar to those we often see in Lisp textbooks.

To review, lists in Lisp are a bit different from std::slist classes in the C++ STL.  In C++, a list has a single entry point, and, while you can have multiple references to the list, they all have the same view on the list.  In Lisp, you could think of a reference to a list as a data pair.  The reference has a pointer to the list object itself, but it also has a position marker, like an iterator, which is its entry point into the list.  Because lists in Lisp are singly-linked, you can have multiple references to the same underlying object, but they’ll have different views of the list.  To demonstrate this, here’s some printing code.
 

;; Some code to pretty-print a list with multiple entry points
;;
(defparameter *rec-width* 10)

(defun plot-list-worker (&rest entries)
  "Given inputs of the form (list1 \"list1\") (list2 \"list2\")..., pretty-prints the list with multiple entry points.  That is, the first entry in 'lists' is the symbol-name proper superlist of the second, which is a proper superlist of the third, etc."
  (let* ((big-list (first (first entries)))
         (total-list-len (length big-list)))

    ;; print the entry points
    (let ((prev-indents '()))
      (dolist (entry entries)
        (let* ((e-handle (first entry))
               (e-name (second entry))
               (indent (1+ (* *rec-width* 
                              (- total-list-len 
                                 (length e-handle))))))
          (dolist (p-i (reverse prev-indents))
            (format t "~VT." (+ p-i 2)))
          (format t "~VT~A~%" indent e-name)
          (push indent prev-indents)))
      (dotimes (i 2)
        (dolist (p-i (reverse prev-indents))
          (format t "~VT." (+ p-i 2)))
        (format t "~%")))

    ;; print the boxes
    (dotimes (i total-list-len)
      (format t "~VT-----" (1+ (* i *rec-width*))))
    (format t "~%")
    (dotimes (i total-list-len)
      (format t "~VT|A|D| -->" (1+ (* i *rec-width*))))
    (format t "NIL~%")
    (dotimes (i total-list-len)
      (format t "~VT-----" (1+ (* i *rec-width*))))
    (format t "~%")

    ;; print the drops to the values
    (dotimes (i 2)
      (dotimes (j total-list-len)
        (format t "~VT |" (1+ (* j *rec-width*))))
      (format t "~%"))

    (dotimes (i total-list-len)
      (let ((j (- total-list-len i 1)))
        (dotimes (k j)
          (format t "~VT |" (1+ (* k *rec-width*))))
        (format t "~VT~A~%" (1+ (* j *rec-width*)) (nth j big-list))))
))

(defmacro plot-lists (&rest lists)
  "Set up and invoke plot-lists-worker"
  (let ((arglist))
    (dolist (lname lists)
      (push (list 'list lname (symbol-name lname)) arglist))
    `(plot-list-worker ,@(reverse arglist))))

Now, I can create a single list with multiple entry points, and plot them.  Here’s the invocation and output, from the SLIME development environment.
 

CL-USER> (let* ((clist (list 5)) 
                (blist (append (list 3 4) clist))
                (alist (append (list 1 2) blist)))
           (plot-lists alist blist clist))
 ALIST
   .                 BLIST
   .                   .                 CLIST
   .                   .                   .
   .                   .                   .
 -----     -----     -----     -----     -----
 |A|D| --> |A|D| --> |A|D| --> |A|D| --> |A|D| -->NIL
 -----     -----     -----     -----     -----
  |         |         |         |         |
  |         |         |         |         |
  |         |         |         |        5
  |         |         |        4
  |         |        3
  |        2
 1
NIL

The ‘alist’ view on the list sees a list of 5 objects, while ‘blist’ cannot see the first two objects, and sees only a list of 3.

There is, however, another way in which Lisp lists differ from C++ std::slist objects.  Lists in Lisp can be spliced together.  It is possible to have two different lists which join partway down their lengths, so that they share all elements beyond the join.  I’m not going to try to draw that.  There is no parallel to this in C++ std::slist objects.

OK, with that taken care of, further posts will follow.

Getting Lisp programming help

I’m not really going to say much here, there are places you can go to get help with Lisp problems.  One can go looking at the usual suspects, like stackoverflow or the Lisp reddit page.  You can also look for help on the #lisp IRC channel on freenode.net.  There may even be a Lisp users group in your local area.

As always, though, don’t perform drive-by questioning where you pop in, brusquely ask a question, then run off.  By asking questions on any of these, you’re talking with real people who have to take the time to explain to you, they’re doing you a favour.  Spend a few hours reading the text of other questions, or watching the interactions of the people on the IRC channel.  Get to understand the culture of the group you’re going to be asking questions of, even if just a little bit.  Also, spend some time trying to figure out the solution to your problem before going for help, and, if appropriate, describe what you tried to do and what went wrong.  The replies you receive are likely to be more helpful then, as people explain not only how you should approach the problem, but also why your unsuccessful attempts did not work out.

Later, when you feel your understanding is sufficient, consider spending time watching those resources for an opportunity to help newcomers.

Online references for learning Lisp

If you start to get serious about using Lisp, you’re likely to want to buy some books to have on the shelf.  I’ve got three:  Practical Common Lisp by Peter Siebel, ANSI Common Lisp by Paul Graham, and The Art of the Metaobject Protocol by Gregot Kiczales, Jim des Rivieres, and Daniel G. Bobrow.

For people who want to try out the language before committing money to the attempt, though, there are online sources.  Notably, Practical Common Lisp has been made available online by the author.  It can be found here.

Another very useful online resource is the Common Lisp HyperSpec, which I mentioned earlier in this series.  It provides good syntax descriptions and examples of all of the standard-defined features of Common Lisp.

You may also want to check out the advice outlined on this page.

A well-appointed public library will usually have a few books on Lisp programming on the shelf.

Building with dependencies in Lisp

A simple project, whether in C++ or in Lisp, is easy enough to implement.  In C++, you’d write one .h file and one .cpp file, and when you wanted to test changes, you would compile with something like this

g++ -ggdb example.cpp -o example

However, as the project becomes more complicated, it typically gets broken up into separate files.  In C++, you might reserve a pair of .h/.cpp files for one class or one hierarchy of descended classes.  Then, as changes continue, it becomes important to figure out what files need to be recompiled when a header file has changed.  This is where GNU Make, and its relatives like cmake and qmake, come in.

What about Lisp?  Well, in a similar fashion, you tend to break up your project into separate files.  It isn’t common to break out your file into declarations and definitions, the way you try to do in C++, but it is possible to do a bit of that, if desired.  However, you still have cases where one .lisp file depends on definitions created in another .lisp file.  In that case, it’s important to load the dependent file ahead of the depending one, to avoid runtime errors at load time.  How is this managed?

In Lisp, the commonly used dependence manager is called ASDF. A particular project is built with an .asd file declaring what files are needed for the project, and what files depend on what other files.  Here’s an example of an .asd file from one of my old projects, a sudoku puzzle generator:
 

(defpackage puzzle-system
  (:use :common-lisp :asdf
#+clisp :screen
#-clisp :cl-ncurses
))

(in-package :puzzle-system)

(defsystem "puzzle"
    :description "puzzle:  An interactive Sudoku game."
    :version "0.2"
    :author "Christopher Neufeld <EMAIL-REDACTED>"
    :licence "GPL"
    :components ((:file "generate")
                 (:file "printed-puzzles" 
                        :depends-on ("generate" "eval-difficulty"))
                 (:file "interactive" 
                        :depends-on ("generate" "eval-difficulty"))
                 (:file "eval-difficulty" 
                        :depends-on ("generate")))
    :depends-on (
                 #-clisp :cl-ncurses
                         ))

This declares that we’re creating a system called ‘puzzle-system’.  If compiled under clisp, it depends on the :screen feature, otherwise it depends on :cl-ncurses (the cl-ncurses package is now deprecated, using the older UFFI where now we would use CFFI and cl-charms).  There are four input files, some of which depend on others.  By creating this .asd file, the programmer can avoid having to figure out what files need to be recompiled and loaded after a change has been made.  One simply invokes this command:

(asdf:oos ‘asdf:load-op ‘<SYSTEM-NAME>)

The asdf system then compiles the appropriate files and loads them.  You should ensure that asdf is loaded when your Lisp starts up.  For SBCL, I use this ~/.sbclrc file:
 

(require :asdf)
(setf asdf:*central-registry*
      (list* '*default-pathname-defaults*
             #p"/home/neufeld/.sbcl/systems/"
             asdf:*central-registry*))

This tells SBCL that it should load the :asdf system at startup.  It also tells it where to look for additional systems.  In the directory /home/neufeld/.sbcl/systems, I have placed symbolic links to the .asd files of the different Lisp packages I have downloaded.  For instance, there are links to cl-ncurses.asd, cl-ppcre.asd, cffi.asd, and so on.  This way, I can build a project that depends on one of those packages, and the ASDF make system will locate the appropriate system wherever it might be, and include it as appropriate.

ASDF is capable of much more than this, I’ve presented little more than the basic mode of operation.  As your projects become more sophisticated, you may find yourself learning the more advanced features of ASDF for proper compiler and loader control.  This, though, is a beginner’s guide to dependency tracking and compiling using ASDF in Lisp.

Lisp code efficiency

For the C++ programmer wondering whether to try his hand at Lisp, one thing that make come up is the question of code efficiency.  The Gnu Compiler Collection produces good, fast code.  Can a program compiled in Lisp do as well?

Well, there are tricks to improve Lisp code.  SBCL will even produce output showing what expressions it could not optimize, generally because the compiler could not infer the type of some object.  Using this output, the programmer can speed up the hot spots in the code by providing compiler hints, basically promises that a certain object has a certain type.

Here’s an example of how one might do this in SBCL.  We’ll write a simple program that computes the average and standard deviation of a list of numbers.
 

(declaim (optimize (debug 0) (safety 0) (speed 3)))

(defun get-stats (list-of-numbers)
  (let ((sum-x 0.0d0)
        (sum-x2 0.0d0)
        (num-pts 0)
        avg stddev)
    (dolist (num list-of-numbers)
      (incf sum-x num)
      (incf sum-x2 (* num num))
      (incf num-pts))

    (when (> num-pts 0)
      (setf avg (/ sum-x num-pts))
      (when (> num-pts 1)
        (setf stddev (sqrt (/ (- (* num-pts sum-x2) (* sum-x sum-x))
                              (* num-pts (1- num-pts)))))))

    (values avg stddev)))

Now, when this is compiled in SBCL, we get the following optimizer output:
 

; compiling (DECLAIM (OPTIMIZE # ...))
; compiling (DEFUN GET-STATS ...)

; file: /home/neufeld/programming/lisp/blogging/optimizing.lisp
; in: DEFUN GET-STATS
;     (/ SUM-X NUM-PTS)
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a FLOAT.
; 
; note: unable to
;   convert x/2^k to shift
; due to type uncertainty:
;   The first argument is a NUMBER, not a INTEGER.

;     (* NUM-PTS SUM-X2)
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The second argument is a NUMBER, not a FLOAT.
; 
; note: unable to
;   convert x*2^k to shift
; due to type uncertainty:
;   The second argument is a NUMBER, not a INTEGER.

;     (* SUM-X SUM-X)
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a RATIONAL.
;   The second argument is a NUMBER, not a FLOAT.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a FLOAT.
;   The second argument is a NUMBER, not a RATIONAL.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a SINGLE-FLOAT.
;   The second argument is a NUMBER, not a DOUBLE-FLOAT.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a DOUBLE-FLOAT.
;   The second argument is a NUMBER, not a SINGLE-FLOAT.
; 
; note: unable to
;   convert x*2^k to shift
; due to type uncertainty:
;   The first argument is a NUMBER, not a INTEGER.
;   The second argument is a NUMBER, not a INTEGER.

;     (- (* NUM-PTS SUM-X2) (* SUM-X SUM-X))
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a RATIONAL.
;   The second argument is a NUMBER, not a FLOAT.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a FLOAT.
;   The second argument is a NUMBER, not a RATIONAL.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a SINGLE-FLOAT.
;   The second argument is a NUMBER, not a DOUBLE-FLOAT.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a DOUBLE-FLOAT.
;   The second argument is a NUMBER, not a SINGLE-FLOAT.

;     (/ (- (* NUM-PTS SUM-X2) (* SUM-X SUM-X)) (* NUM-PTS (1- NUM-PTS)))
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a FLOAT.
; 
; note: unable to
;   convert x/2^k to shift
; due to type uncertainty:
;   The first argument is a NUMBER, not a INTEGER.

;     (SQRT (/ (- (* NUM-PTS SUM-X2) (* SUM-X SUM-X)) (* NUM-PTS (1- NUM-PTS))))
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a SINGLE-FLOAT.
;   The result is a (VALUES
;                    (OR (MEMBER 0.0 0.0d0) (DOUBLE-FLOAT (0.0d0))
;                        (SINGLE-FLOAT (0.0)) (COMPLEX SINGLE-FLOAT)
;                        (COMPLEX DOUBLE-FLOAT))
;                    &OPTIONAL), not a (VALUES FLOAT &REST T).
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a DOUBLE-FLOAT.
;   The result is a (VALUES
;                    (OR (MEMBER 0.0 0.0d0) (DOUBLE-FLOAT (0.0d0))
;                        (SINGLE-FLOAT (0.0)) (COMPLEX SINGLE-FLOAT)
;                        (COMPLEX DOUBLE-FLOAT))
;                    &OPTIONAL), not a (VALUES FLOAT &REST T).

;     (INCF SUM-X NUM)
; --> LET* 
; ==>
;   (+ SUM-X #:G3)
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a RATIONAL.
;   The second argument is a NUMBER, not a FLOAT.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a FLOAT.
;   The second argument is a NUMBER, not a RATIONAL.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a SINGLE-FLOAT.
;   The second argument is a NUMBER, not a DOUBLE-FLOAT.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a DOUBLE-FLOAT.
;   The second argument is a NUMBER, not a SINGLE-FLOAT.

;     (* NUM NUM)
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a RATIONAL.
;   The second argument is a NUMBER, not a FLOAT.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a FLOAT.
;   The second argument is a NUMBER, not a RATIONAL.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a SINGLE-FLOAT.
;   The second argument is a NUMBER, not a DOUBLE-FLOAT.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a DOUBLE-FLOAT.
;   The second argument is a NUMBER, not a SINGLE-FLOAT.
; 
; note: unable to
;   convert x*2^k to shift
; due to type uncertainty:
;   The first argument is a NUMBER, not a INTEGER.
;   The second argument is a NUMBER, not a INTEGER.

;     (INCF SUM-X2 (* NUM NUM))
; --> LET* 
; ==>
;   (+ SUM-X2 #:G5)
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a RATIONAL.
;   The second argument is a NUMBER, not a FLOAT.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a FLOAT.
;   The second argument is a NUMBER, not a RATIONAL.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a SINGLE-FLOAT.
;   The second argument is a NUMBER, not a DOUBLE-FLOAT.
; 
; note: unable to
;   optimize
; due to type uncertainty:
;   The first argument is a NUMBER, not a DOUBLE-FLOAT.
;   The second argument is a NUMBER, not a SINGLE-FLOAT.

;     (INCF SUM-X NUM)
; --> LET* 
; ==>
;   (+ SUM-X #:G3)
; 
; note: forced to do GENERIC-+ (cost 10)
;       unable to do inline float arithmetic (cost 2) because:
;       The first argument is a NUMBER, not a DOUBLE-FLOAT.
;       The second argument is a NUMBER, not a DOUBLE-FLOAT.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES DOUBLE-FLOAT
;                                                                &REST T).
;       unable to do inline float arithmetic (cost 2) because:
;       The first argument is a NUMBER, not a SINGLE-FLOAT.
;       The second argument is a NUMBER, not a SINGLE-FLOAT.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES SINGLE-FLOAT
;                                                                &REST T).
;       etc.

;     (* NUM NUM)
; 
; note: forced to do GENERIC-* (cost 30)
;       unable to do inline float arithmetic (cost 4) because:
;       The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT).
;       The second argument is a NUMBER, not a SINGLE-FLOAT.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES
;                                                         (COMPLEX SINGLE-FLOAT)
;                                                         &REST T).
;       unable to do inline float arithmetic (cost 4) because:
;       The first argument is a NUMBER, not a SINGLE-FLOAT.
;       The second argument is a NUMBER, not a SINGLE-FLOAT.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES SINGLE-FLOAT
;                                                                &REST T).
;       etc.

;     (INCF SUM-X2 (* NUM NUM))
; --> LET* 
; ==>
;   (+ SUM-X2 #:G5)
; 
; note: forced to do GENERIC-+ (cost 10)
;       unable to do inline float arithmetic (cost 2) because:
;       The first argument is a NUMBER, not a DOUBLE-FLOAT.
;       The second argument is a NUMBER, not a DOUBLE-FLOAT.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES DOUBLE-FLOAT
;                                                                &REST T).
;       unable to do inline float arithmetic (cost 2) because:
;       The first argument is a NUMBER, not a SINGLE-FLOAT.
;       The second argument is a NUMBER, not a SINGLE-FLOAT.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES SINGLE-FLOAT
;                                                                &REST T).
;       etc.

;     (INCF NUM-PTS)
; --> LET* 
; ==>
;   (+ NUM-PTS #:G7)
; 
; note: forced to do GENERIC-+ (cost 10)
;       unable to do inline fixnum arithmetic (cost 1) because:
;       The first argument is a UNSIGNED-BYTE, not a FIXNUM.
;       The result is a (VALUES (INTEGER 1) &OPTIONAL), not a (VALUES FIXNUM
;                                                                     &REST T).
;       unable to do inline fixnum arithmetic (cost 2) because:
;       The first argument is a UNSIGNED-BYTE, not a FIXNUM.
;       The result is a (VALUES (INTEGER 1) &OPTIONAL), not a (VALUES FIXNUM
;                                                                     &REST T).
;       etc.

;     (> NUM-PTS 0)
; 
; note: forced to do GENERIC-> (cost 10)
;       unable to do inline fixnum comparison (cost 3) because:
;       The first argument is a UNSIGNED-BYTE, not a FIXNUM.
;       unable to do inline fixnum comparison (cost 4) because:
;       The first argument is a UNSIGNED-BYTE, not a FIXNUM.
;       etc.

;     (> NUM-PTS 1)
; 
; note: forced to do GENERIC-> (cost 10)
;       unable to do inline fixnum comparison (cost 3) because:
;       The first argument is a (INTEGER 1), not a FIXNUM.
;       unable to do inline fixnum comparison (cost 4) because:
;       The first argument is a (INTEGER 1), not a FIXNUM.
;       etc.

;     (* NUM-PTS SUM-X2)
; 
; note: forced to do GENERIC-* (cost 30)
;       unable to do inline fixnum arithmetic (cost 4) because:
;       The first argument is a (INTEGER 2), not a FIXNUM.
;       The second argument is a NUMBER, not a FIXNUM.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES FIXNUM &REST T).
;       unable to do inline (signed-byte 64) arithmetic (cost 5) because:
;       The first argument is a (INTEGER 2), not a (SIGNED-BYTE 64).
;       The second argument is a NUMBER, not a (SIGNED-BYTE 64).
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES (SIGNED-BYTE 64)
;                                                                &REST T).
;       etc.

;     (* SUM-X SUM-X)
; 
; note: forced to do GENERIC-* (cost 30)
;       unable to do inline float arithmetic (cost 4) because:
;       The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT).
;       The second argument is a NUMBER, not a SINGLE-FLOAT.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES
;                                                         (COMPLEX SINGLE-FLOAT)
;                                                         &REST T).
;       unable to do inline float arithmetic (cost 4) because:
;       The first argument is a NUMBER, not a SINGLE-FLOAT.
;       The second argument is a NUMBER, not a SINGLE-FLOAT.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES SINGLE-FLOAT
;                                                                &REST T).
;       etc.

;     (- (* NUM-PTS SUM-X2) (* SUM-X SUM-X))
; 
; note: forced to do GENERIC-- (cost 10)
;       unable to do inline float arithmetic (cost 2) because:
;       The first argument is a NUMBER, not a DOUBLE-FLOAT.
;       The second argument is a NUMBER, not a DOUBLE-FLOAT.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES DOUBLE-FLOAT
;                                                                &REST T).
;       unable to do inline float arithmetic (cost 2) because:
;       The first argument is a NUMBER, not a SINGLE-FLOAT.
;       The second argument is a NUMBER, not a SINGLE-FLOAT.
;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES SINGLE-FLOAT
;                                                                &REST T).
;       etc.

;     (1- NUM-PTS)
; ==>
;   (- NUM-PTS 1)
; 
; note: forced to do GENERIC-- (cost 10)
;       unable to do inline fixnum arithmetic (cost 1) because:
;       The first argument is a (INTEGER 2), not a FIXNUM.
;       The result is a (VALUES (INTEGER 1) &OPTIONAL), not a (VALUES FIXNUM
;                                                                     &REST T).
;       unable to do inline fixnum arithmetic (cost 2) because:
;       The first argument is a (INTEGER 2), not a FIXNUM.
;       The result is a (VALUES (INTEGER 1) &OPTIONAL), not a (VALUES FIXNUM
;                                                                     &REST T).
;       etc.

;     (* NUM-PTS (1- NUM-PTS))
; 
; note: forced to do GENERIC-* (cost 30)
;       unable to do inline fixnum arithmetic (cost 4) because:
;       The first argument is a (INTEGER 2), not a FIXNUM.
;       The second argument is a (INTEGER 1), not a FIXNUM.
;       The result is a (VALUES (INTEGER 2) &OPTIONAL), not a (VALUES FIXNUM
;                                                                     &REST T).
;       unable to do inline (signed-byte 64) arithmetic (cost 5) because:
;       The first argument is a (INTEGER 2), not a (SIGNED-BYTE 64).
;       The second argument is a (INTEGER 1), not a (SIGNED-BYTE 64).
;       The result is a (VALUES (INTEGER 2) &OPTIONAL), not a (VALUES
;                                                              (SIGNED-BYTE 64)
;                                                              &REST T).
;       etc.
; 
; compilation unit finished
;   printed 41 notes

So, let’s look at the output of the optimizer.  The first complaint is about the division to obtain the average.  The optimizer knows that sum-x is a number, but doesn’t know whether it’s a float, integer, double.  It also notes that, if it knows the first number is an integer and the second is a power of two, that it can perform the division by bit-shifting.  This will not be the case, so we’ll modify the code to inform it of the true restrictions.

The second complaint is about a multiplication.  Again, it has to produce general multiplication code, instead of producing code optimized for a particular numeric type.  You see more warnings of this type through all of the output.  So, it’s time to modify the code to help the optimizer.

We point out certain types to the compiler, and we ensure that, in one spot where it needs to use a number several times, that the number has been pre-converted to a double-precision float.  The new code looks like this:
 

(declaim (optimize (debug 0) (safety 0) (speed 3)))

(defun get-stats2 (list-of-numbers)
  (let ((sum-x 0.0d0)
        (sum-x2 0.0d0)
        (num-pts 0)
        (avg 0.0d0)
        (stddev 0.0d0))
    (declare (double-float sum-x))
    (declare (double-float sum-x2))
    (declare (double-float avg))
    (declare (double-float stddev))
    (declare (fixnum num-pts))

    (dolist (num list-of-numbers)
      (let ((num-double (coerce num 'double-float)))
        (incf sum-x num-double)
        (incf sum-x2 (* num-double num-double))
        (incf num-pts)))

    (when (> num-pts 0)
      (setf avg (/ sum-x num-pts))
      (when (> num-pts 1)
        (setf stddev (sqrt (/ (- (* num-pts sum-x2) (* sum-x sum-x))
                              (* num-pts (1- num-pts)))))))

    (values avg stddev)))

What does this do to performance?  Testing the unoptimized and optimized versions:
 

CL-USER> (progn
           (time 
            (let ((mylist (list 2. 2. 1. 3. 2.)))
              (dotimes (i 10000000)
                (get-stats mylist))))
           (time 
            (let ((mylist (list 2. 2. 1. 3. 2.)))
              (dotimes (i 10000000)
                (get-stats2 mylist)))))
Evaluation took:
  2.514 seconds of real time
  2.516157 seconds of total run time (2.500156 user, 0.016001 system)
  [ Run times consist of 0.044 seconds GC time, and 2.473 seconds non-GC time. ]
  100.08% CPU
  8,575,512,723 processor cycles
  3,520,004,080 bytes consed

Evaluation took:
  0.598 seconds of real time
  0.596037 seconds of total run time (0.596037 user, 0.000000 system)
  [ Run times consist of 0.016 seconds GC time, and 0.581 seconds non-GC time. ]
  99.67% CPU
  2,040,223,555 processor cycles
  1,280,003,632 bytes consed

We see that the optimizations resulted in a better than four-fold improvement in runtime.

The interested programmer can even ask to see the generated code, by issuing the command (disassemble ‘get-stats).

So, after all this, how fast is Lisp-generated code?  Well, a good resource to get answers to that is to look in the Computer Language Benchmarks Game.  This is a page where people can submit their solutions to some specific computational problems in the programming language of their choice, and the solutions are built, executed, and compared on a reference system to produce statistics on the relative efficiency of the different solutions to the same problem.  A benchmark comparison of some SBCL vs. g++ code can be found here.