Moving onwards, the casual Lisp programmer arriving from C++ might not have encountered deftype in his or her reading.  There’s a fine description and example of the macro in the CLHS.  This macro allows the programmer to define a new type specifier.

So, why might the programmer want to create new type specifiers?  Most commonly, programmers either want to use them along with declare and the to provide compiler hints, helping the optimizer to know what underlying types are being used.  The programmer-defined type specifier is somewhat similar to a C++ typedef, by defining your own type specifier you can easily alter code when you have to change the underlying type, without having to search through your code for occurrences of the type.

New type specifiers also might be useful in typecase statements:
 

(defun equidimensional (a)
  (or (< (array-rank a) 2)
      (apply #'= (array-dimensions a))))

(defun all-ints (a)
  (let ((len (array-total-size a)))
    (dotimes (i len)
      (unless (integerp (row-major-aref a i))
        (return-from all-ints nil)))
    t))

(deftype square-matrix (&optional type size)
  `(and (array ,type (,size ,size))
        (satisfies equidimensional)))

(deftype integer-square-matrix (&optional size)
  `(and (square-matrix integer ,size)
        (satisfies all-ints)))

(defun demonstrate ()
  (let ((objects (list :ABC
                       (make-array '(2 3))
                       (make-array '(2 2) :initial-element 1)
                       (make-array '(2 2) :initial-element 1.1))))
    (dolist (obj objects)
      (format t "~S~26T" obj)
      (typecase obj
        (integer-square-matrix
         (format t "is an integer square matrix"))
        (square-matrix
         (format t "is a square matrix"))
        (array
         (format t "is a non-square matrix"))
        (t
         (format t "is a non-matrix")))
      (format t "~%"))))

CL-USER> (demonstrate)
:ABC                      is a non-matrix
#2A((0 0 0) (0 0 0))      is a non-square matrix
#2A((1 1) (1 1))          is an integer square matrix
#2A((1.1 1.1) (1.1 1.1))  is a square matrix
NIL

We talked about define-setf-expander.  That’s a fairly powerful macro that allows setf to operate on things that aren’t what the C++ programmer would think of as an “lvalue”.  There is also defsetf.  Less powerful, but simpler to use, so it’s generally preferred when its use is possible.

There are two forms of defsetf, a short form for very simple cases, and a long form for slightly longer cases.

If you have an access function that takes a certain number of arguments, and a setting form that takes exactly one more argument, the new value to substitute, and if that second function returns this new value, then you can use the short form.  Here’s an example:
 

(defparameter *num-counters* 10)
(defparameter *counter-array* (make-array *num-counters*
                                          :initial-element 0))

(defun get-next-counter (index)
  (assert (<= 0 index (1- *num-counters*)))
  (incf (aref *counter-array* index)))

(defun set-counter-value (index val)
  (assert (<= 0 index (1- *num-counters*)))
  (assert (integerp val))
  (setf (aref *counter-array* index) (1- val))
  val)

(defsetf get-next-counter set-counter-value)

(defun demonstrate ()
  (format t "Next counter 0: ~D~%" (get-next-counter 0))
  (format t "Next counter 0: ~D~%" (get-next-counter 0))
  (format t "Next counter 0: ~D~%" (get-next-counter 0))
  (format t "Next counter 1: ~D~%" (get-next-counter 1))
  (format t "Setting next counter 0 value to 100~%")
  (setf (get-next-counter 0) 100)
  (format t "Next counter 0: ~D~%" (get-next-counter 0))
  (format t "Next counter 1: ~D~%" (get-next-counter 1)))

CL-USER> (demonstrate)
Next counter 0: 1
Next counter 0: 2
Next counter 0: 3
Next counter 1: 1
Setting next counter 0 value to 100
Next counter 0: 100
Next counter 1: 2
NIL

When the arguments list to the function you’re working with is a bit more complicated, such as if it has optional arguments, the long form may still be suitable:
 

(defparameter *test-set* (list :A :B :C :C :D :C :A :B :A :C))

(defun member-in-test-set (sym &key (skip 0))
  (let ((rv (member sym *test-set*)))
    (do ()
        ((or (not rv) (= skip 0)) rv)
      (setf rv (member sym (cdr rv)))
      (decf skip))))

(defun set-member-in-test-set (sym new-sym &key (skip 0))
  (let ((rv (member sym *test-set*)))
    (do ()
        ((or (not rv) (= skip 0)))
      (setf rv (member sym (cdr rv)))
      (decf skip))
    (when rv
       (rplaca rv new-sym))
    new-sym))

(defsetf member-in-test-set (sym &key (skip 0)) (new-sym)
    `(set-member-in-test-set ,sym ,new-sym :skip ,skip))

(defun demonstrate ()
  (format t "Test set is: ~S~%" *test-set*)
  (format t "Setting the second occurence of :A to :Y~%")
  (setf (member-in-test-set :A :skip 1) :Y)
  (format t "Test set is: ~S~%" *test-set*))

CL-USER> (demonstrate)
Test set is: (:A :B :C :C :D :C :A :B :A :C)
Setting the second occurence of :A to :Y
Test set is: (:A :B :C :C :D :C :Y :B :A :C)
NIL

As I mentioned in the article about define-setf-expander, you might initially ask why bother writing a setf expander when the programmer can simply call the appropriate modification functions directly.  The most likely reason you’d use an expander instead is that you have existing macros that make use of setf for some work, and you’d like to use it to setf one of its passed arguments, but that argument doesn’t look like an lvalue.  Rather than writing runtime checks against type and changing the macro to account for all such cases, you can simplify your code by writing an setf expander and hiding the special-case code underneath it.

Another feature of Lisp that the newcomer might not have encountered is define-symbol-macro.  Think of this as something very close to a C/C++ macro.  It is a way to define a symbol that will be expanded at compile time.  Where the more familiar Lisp macros have a function-like syntax, this creates a macro with a symbol-like syntax, so it is called without surrounding parentheses.

Here’s a short bit of code to demonstrate this:
 

(define-symbol-macro now (get-time-of-day))
(define-symbol-macro first-fred (car fred))

(defun demonstrate ()
  (format t "The value of symbol 'now' is: ~A~%" now)
  (let ((fred (list 1 2 3 4 5)))
    (format t "The value of symbol 'first-fred' is ~A~%" 
            first-fred)
    (let ((fred (list :A :B :C :D :E)))
      (format t "The value of symbol 'first-fred' is ~A~%" 
              first-fred))))

CL-USER> (demonstrate)
The value of symbol 'now' is: 1388450944
The value of symbol 'first-fred' is 1
The value of symbol 'first-fred' is A
NIL
CL-USER> (macroexpand 'now)
(GET-TIME-OF-DAY)
T
CL-USER> (macroexpand 'first-fred)
(CAR FRED)
T

We continue our overview of Lisp features possibly missed by the beginner arriving from C++ with define-setf-expander.  In C, you had the concept of lvalues, assignable expressions such as a variable name or a pointer or array dereference.  In C++, you got to things like operator=, to make assignable operations to objects, or functions that returned a modifiable reference, which is usable as an lvalue.

In Lisp, a similar concept is the setf expander.  The setf macro is the common means by which an existing place has its value modified.  In that way, it fills the niche used by the assignment operator in C/C++.  Lisp also has things that you would think of as lvalues, setf cannot be used to assign a value to arbitrary expressions.

When the programmer wants to generalize setf to situations that are more complex than a simple setf-able place, define-setf-expander is one way to do it.  However, before you spend too much effort on this feature, check out defsetf, which I will cover in an upcoming post.  The defsetf macro, when the situation is simple enough to allow its use, is less complicated.

Here’s an example of define-setf-expander at work.  I define a function, max-value, that, when given an array of numbers, returns the largest number in the array, and a second return value holding the list of indices where those values can be found.  Imagine I also have a context where I want to replace all instances of the largest value in the array with a new value, using setf.  This is where a setf-expander is helpful.
 

(defun max-value (arr)
  "Returns 2 values.  The first is the maximum scalar value in the array of numbers.  The second is a list of lists of array indices."
  (labels
      ((convert-to-indices (row-major-index dimensions)
         (let (rv)
           (dolist (dim (reverse dimensions))
             (multiple-value-bind (div rem)
                 (floor row-major-index dim)
               (push rem rv)
               (setf row-major-index div)))
           rv)))

    (let* ((adim (array-dimensions arr))
           (num-elements (array-total-size arr))
           max-val indices)
      (dotimes (i num-elements)
        (let ((val (row-major-aref arr i)))
          (cond
            ((or (not max-val)
                 (> val max-val))
             (setf max-val val
                   indices (list (convert-to-indices i adim))))
            ((= val max-val)
             (push (convert-to-indices i adim) indices)))))
      (values max-val indices))))

(define-setf-expander max-value (arr &environment env)
  (multiple-value-bind (tmpspace old-values new-values
                                 setting-form getting-form)
      (get-setf-expansion arr env)
    (declare (ignore new-values setting-form))
    (let ((nval (gensym)))
      (values tmpspace
              old-values
              `(,nval)
              `(multiple-value-bind (val indices)
                   (max-value ,getting-form)
                 (declare (ignore val))
                 (dolist (one-ind indices)
                   (let ((linear-ind (apply 'array-row-major-index 
                                            ,getting-form one-ind)))
                     (setf (row-major-aref ,getting-form linear-ind)
                           ,nval)))
                 ,nval)
              `(max-value ,getting-form)))))

(defun demonstrate ()
  (let ((test-array (make-array '(4 3) 
                                :initial-contents '((0 1 2) 
                                                    (3 4 11) 
                                                    (11 7 8)
                                                    (9 10 11)))))
    (format t "Input array is: ~A~%" test-array)
    (multiple-value-bind (max-val indices)
        (max-value test-array)
      (format t "The maximum value is: ~A~%" max-val)
      (format t "Appearing at indices: ~A~%" indices))
    (format t "We setf the max-value to 50.~%")
    (setf (max-value test-array) 50)
    (format t "The array is now: ~A~%" test-array)))

CL-USER> (demonstrate)
Input array is: #2A((0 1 2) (3 4 11) (11 7 8) (9 10 11))
The maximum value is: 11
Appearing at indices: ((3 2) (2 0) (1 2))
We setf the max-value to 50.
The array is now: #2A((0 1 2) (3 4 50) (50 7 8) (9 10 50))
NIL

I will point out that the define-setf-expander macro is not something that I have frequently seen used, it’s a helpful tool for certain unusual cases, but is not commonly needed.  In most contexts, the programmer will write a special-purpose function to perform the effects of the setf, but when the setf is buried in an existing macro, this can be a useful way to re-use the macro without having to write special-case code all through it.

Continuing with the features of Lisp that a newcomer arriving from C++ might not have encountered, we look at define-modify-macro.  The rationale behind this is that certain operations that you might naively think of as modifying a data structure might actually create a new data structure.  This is generally not true of functions that act on objects, but may be true of functions that act on lists.

The C++ programmer is used to having explicit syntax to decide whether a variable is passed by value or by reference.  In Lisp, that is determined by the data type, rather than in the declaration of the function.  Numbers and symbols are passed by value.  If a parameter to a function evaluates as a number or a symbol, and the function modifies the parameter in its body, the effect of such modification is not propagated back to the caller.  On the other hand, if the parameter is a structure, object, or array, it is passed by reference, and modifications made to the contents of such entities will persist after the function returns.  Lists, though, are a bit complicated.  The thing that is passed is a pointer to a node in a singly-linked list.  If the list is modified, for instance by reversing its contents, the caller will still have a pointer to the same node, but now the forward links from that node won’t necessarily traverse the whole list.  You go from having a reference to the beginning of the list to having a reference to somewhere in the middle of the list, and the forward linkage means you can’t back up to see what has been moved ahead of your entry point.  Consequently, functions that modify lists will usually return a new list, rather than modifying the linkages of the passed list.

It’s a common mistake for a beginner to expect the sort function, when acting on a list as opposed to an array, to modify the list and make it sorted.  In fact, it returns a new list, and destroys the original list (the programmer cannot make any assumptions about what the input list looks like after sort is called).  The programmer is responsible for retaining the returned value and using it when the sorted list is desired.

The define-modify-macro macro allows you to construct the fairly common construct of calling a function on a list and assigning the result back to the place (think of this like a variable or pointer) that held the original list.  Here’s an example of how this might work with reverse.
 

(define-modify-macro reversef (&rest args)
  reverse)

(defun demonstrate ()
  (let* ((input-list '(9 1 3 2 4 8 7 6))
         (input-list-copy input-list))
    (format t "Input list: ~A~%" input-list)
    (reverse input-list)
    (format t "Input list after calling reverse: ~A~%" 
            input-list)
    (reversef input-list)
    (format t "Input list after calling reversef: ~A~%"
            input-list)
    (format t "Input list copy after calling reversef: ~A~%"
            input-list-copy)))

CL-USER> (demonstrate)
Input list: (9 1 3 2 4 8 7 6)
Input list after calling reverse: (9 1 3 2 4 8 7 6)
Input list after calling reversef: (6 7 8 4 2 3 1 9)
Input list copy after calling reversef: (9 1 3 2 4 8 7 6)
NIL

Notice that only the place passed to reversef sees the change.  The input-list-copy variable, while originally pointing at the same list as input-list, is a different place, and is not updated.  Where, originally, the two places pointed to the same list, they now point to different lists.