Tag Archives: programming

Finding libraries for Lisp programming

If you’re interested in trying out Lisp programming, you might wonder how you can find libraries for useful functions.  Maybe you want to try your hand at some Mandelbrot set generation, but don’t want to have to figure out how to write out the graphical files for viewing.  Perhaps you want to do something with networking.

The starting point I’d recommend is the common lisp wiki at www.cliki.net. There, you can find links to working libraries that perform many useful duties, and let you concentrate on the code you’re trying to write.  For instance, under graphics libraries, you will find about 70 entries ranging from PNG/JPG output to the Cairo API, to OpenGL linkages.  Chances are you’ll find what you need for your project there, and if not, there are other possibilities, as we’ll see later, when I talk about CFFI.

Lisp sounds interesting, but… Answering the Objections

I’ve written a few series of posts intended to introduce some of the features of Lisp that make it look and feel different from C++.  I’ve talked about macros, object-oriented programming, and exception handling, because the way these work in Lisp differs qualitatively from their C++ counterparts.

Differences in function names or minor differences in syntax are fairly trivial distinctions for the average C++ programmer, who might find him/herself exposed to such things frequently, writing Perl scripts, designing yacc/bison grammars, and so on.  But some of the differences between Lisp and C++ are quite fundamental, and experience with both might sometimes help the programmer to design their code, in whatever language they are using.  For some programmers, thinking in Lisp can clarify the problem, and some have argued that adding Lisp to one’s set of languages makes one a better programmer in all languages.  Whether or not that is true, Lisp can, at least, be entertaining for programmers who write code for the enjoyment of it.

As I’ve said before, I’m not trying to convince people to become Lisp programmers here.  This isn’t a “my language is better than your language” set of posts.  Even if it were, my most familiar programming language is still C++, Lisp for me is mostly for fun and for prototyping.  But perhaps you’ve been reading about Lisp, maybe here, maybe elsewhere, and have started to think you might want to play around with it a bit, just for fun.  But then some objections come to mind, and you’re not sure it’s worth the trouble to set up for Lisp programming.  I’m going to try to address some of these possible objections in another series of posts.

I’ll be addressing these objections over the next series of posts.

Collection of Lisp exception handling posts

Just putting all the links here in one place for easy access.  This was a 6-part series describing exception handling in Lisp, specifically to point out the similarities and differences to exception handling in C++.  I went over both the more familiar C++ try/throw/catch model and a restart technique that handles exceptions without unwinding the stack. Here is the series of posts:

Part 1

Part 2

Part 3

Part 4

Part 5

Part 6

 

Exception handling in Lisp, as seen from C++, Part 6

Now, we’ll streamline the code a bit.

Recall that if a handler function returns, rather than making a non-local exit through a restart or by signaling another condition, then the condition that caused the handler to be called in the first place is deemed not to have been handled, and we continue up the call stack looking for handlers to handle the condition.  This means that we can make our number-fixing function simply exit if it fails to find a valid fix, and invoke the restart if it succeeds.  In that case, we’ll continue up the call stack to the general-purpose “continue” handler at the top.

If we do this, then our unreadable number condition no longer needs a field to indicate whether or not it has been successfully corrected.  The only way code can proceed past the signal function call is if the restart is invoked, which will only happen if there is a corrected value available.

The result is that we have an analysis function that can correct bad numbers if necessary.  All that is required is that an enclosing context binds a handler function to the not-readable-number condition, and that that handler function, if it successfully supplies a corrected number, fills in the slot in the condition and calls the number-was-fixed restart.  There might be several different functions available for this, and the caller might choose how to do it based on some enclosing context.  For instance, it might be that another function uses a recognized words list from French, or Chinese.  The analyse-stream function just knows that somebody might fix the number, and indicate that by calling the number-was-fixed restart.

Finally, what about arithmetic errors?  The ratio of two numbers can give an arithmetic error in at least two ways under Lisp.  The denominator of the fraction could be zero, or the result of the division could lead to a floating-point overflow or underflow.  If this happens, we want our code to produce a message, but then continue running with the next pair of numbers, rather than unwinding the stack to the toplevel and exiting.  We do this by wrapping the arithmetic operation in its own handler-case, so that conditions of type arithmetic-error (or classes derived from it) are handled locally, and not passed back up the call stack.

Here’s how the code looks now:
 

;; Some examples of using conditions in Lisp code.
;;
(declaim (optimize (debug 3) (safety 3)))

(define-condition not-readable-number (error)
  ((object-seen         :initarg :object-seen
                        :initform nil
                        :reader get-object-seen)
   (corrected-value     :initform nil
                        :accessor get-corrected-value))
  (:documentation "The condition that will be used to communicate
exceptions when trying to read numbers.")
  (:report (lambda (c s)
             (format s "The value \"~A\" could not be interpreted as a number."
                     (get-object-seen c)))))

(defun condition-demo (pathname &key fix-bad-numbers)
  (format t "Starting to analyse file ~A~%" pathname)
  (handler-case
      (with-open-file (s pathname :direction :input)
        (if fix-bad-numbers
            (handler-bind ((not-readable-number 
                            #'fix-number-technique-1)) 
              (analyse-stream s))
            (analyse-stream s)))

    (file-error (c)
      (format t "A file error was encountered while trying to analyse ~A.~%The error returned was:~%~A~%" pathname c))
    (end-of-file ()
      (format t "Successfully reached the end of the file.~%"))
    (condition (c)
      (format t "A non-file error was encountered while trying to analyse ~A.~%The error returned was:~%~A~%" pathname c))))

;; Goes through the stream, dividing entries one by the next
(defun analyse-stream (stream)
  (do ()
      (nil)
    (let ((num1 (get-next-number stream))
          (num2 (get-next-number stream)))

        (unless (numberp num1)
          (let ((msg (make-condition 'not-readable-number 
                                     :object-seen num1)))
            (with-simple-restart (number-was-fixed "")
              (signal msg))
            (setf num1 (get-corrected-value msg))))

        (unless (numberp num2)
          (let ((msg (make-condition 'not-readable-number 
                                     :object-seen num2)))
            (with-simple-restart (number-was-fixed "")
              (signal msg))
            (setf num2 (get-corrected-value msg))))

        (handler-case
            (format t 
                    "The ratio of ~A to ~A is ~A~%" 
                    num1 num2 (/ num1 num2))
          (arithmetic-error (c)
            (format t "An arithmetic error occured:~%~A~%" c)
            (format t "Continuing forward.~%"))))))

;; Returns the next object in the stream
(defun get-next-number (stream)
  (read stream))

(defparameter *recognized-words*
  '(("zero" . 0)
    ("one" . 1)
    ("two" . 2)
    ("three" . 3)
    ("four" . 4)
    ("five" . 5)
    ("six" . 6)
    ("seven" . 7)
    ("eight" . 8)
    ("nine" . 9)
    ("ten" . 10)
    ("eleven" . 11)
    ("twelve" . 12)
    ("thirteen" . 13)))

;; Try to take a string or symbol representation and convert it to a
;; number
(defun fix-number-technique-1 (c)
  (let ((bad-object (get-object-seen c))
        string-rep)
    (when (symbolp bad-object)
      (setf string-rep (symbol-name bad-object)))
    (when (stringp bad-object)
      (setf string-rep (copy-seq bad-object)))

    (when string-rep
      ;; Maybe it's just a number in double-quotes.  Let's ask if we
      ;; can read a number from that.
      (let ((candidate (read-from-string string-rep nil nil)))
        (when (numberp candidate)
          (setf (get-corrected-value c) candidate)
          (invoke-restart 'number-was-fixed)))

      ;; If we get here, it wasn't that.  Maybe it's one of our
      ;; recognized words.

      (let ((match (assoc string-rep 
                          *recognized-words* :
                          test 'string-equal)))
        (when match
          (setf (get-corrected-value c) (cdr match))
          (invoke-restart 'number-was-fixed))))))

Before leaving the subject of exceptions, there are a few more things to point out.

Whenever programming with exceptions, the programmer should be aware that certain function calls may invoke an exception, that could unwind through the function you’re writing to reach an exception handler above.  In C++, if you designed a function to write out data to disc, it might first open the file, then call some functions to generate the output, then finally close the file.  If one of those output-generating functions throws an exception which is caught at a higher level, the result will be a dangling open file with no way to recover its handle to close it.  Problems of this sort are generally handled in one of two ways:

  1. The programmer ensures that any cleanup work that might be necessary is located inside the destructor of an auto (stack-allocated) object.  When the stack is unwound, auto objects have their destructors called, and cleanup occurs there.
  2. The programmer surrounds the block that might throw an exception in a try/catch block with a default catch, which does the appropriate cleanups and then rethrows.

An example of the second case might be this:
 

{
    FILE *ofile = ::fopen(pathname, "w");

    try {
        write_out_some_data(ofile);
    } catch (...) {
        ::fclose(ofile);
        throw;
    }

    ::fclose(ofile);
}

Most C++ programmers, though, will probably prefer the first approach, to clean up in destructors.  Instead of using the fopen() and fclose() functions, they would use iostream classes that close their files in the destructor, so that the cleanup happens automatically even if an exception is thrown from a called function and caught from a calling function.

In Lisp, though, objects don’t have destructors.  Lisp objects also don’t have the properties of auto variables, they aren’t built on the stack of the function and destroyed when the function exits.  Lisp objects are destroyed by garbage collection once all references to them are known to be gone, so they can persist long after the function that allocated them has exited, or they can vanish partway through execution of the function.  How, then, do Lisp programmers use exceptions safely?  In Lisp, the usual technique is a variation of method 2 above.  A form called unwind-protect is used to clean up.  Note, however, that unwind-protect is not exactly like the C++ model above.  The unwind-protect form is always executed, whether an exception is thrown or not.  What it does is to say that, no matter how the protected form is exited, the cleanup code runs before the stack is allowed to unwind past that point.  In the tiny C++ sample above, we have two invocations of ::fclose().  One in the exception handler, and one in the normal execution case.  In Lisp, the code would appear only once, as the unwind-protect cleanup forms execute in both cases.  An example might be:
 

(defun write-out (pathname)
  (let ((output-stream (open-output-file pathname)))
    (unwind-protect
         (progn
           (write-output-1 output-stream)
           (write-output-2 output-stream))
      (close-output-file output-stream))))

It is a common Lisp construct to create functions or macros with a name like (with-*) to indicate that there is an unwind-protect on a resource within.  For instance, in the code I’ve been presenting in this series on exceptions, I’ve been using a method (with-open-file).  This can be implemented as a macro using unwind-protect and the (open) and (close) functions of lisp.  The with-open-file method ensures that the file is always cleanly closed before the form exits, whether it exits by normal flow or by a non-local return through a catch or restart.

Exception handling in Lisp, as seen from C++, Part 5

Now, we’re ready to set up the code to correct and continue from certain errors.  This isn’t going to be the final form of this example, I’m putting in unnecessary extra code in hopes that it aids in clarity.  I’ll make some final changes to streamline it a bit in a later post.
 

;; Some examples of using conditions in Lisp code.
;;
(declaim (optimize (debug 3) (safety 3)))

(define-condition not-readable-number (error)
  ((object-seen         :initarg :object-seen
                        :initform nil
                        :reader get-object-seen)
   (was-corrected       :initform nil
                        :accessor get-was-corrected)
   (corrected-value     :initform nil
                        :accessor get-corrected-value))
  (:documentation "The condition that will be used to communicate
exceptions when trying to read numbers.")
  (:report (lambda (c s)
             (format s "The value \"~A\" could not be interpreted as a number."
                     (get-object-seen c)))))

(defun condition-demo (pathname &key fix-bad-numbers)
  (format t "Starting to analyse file ~A~%" pathname)
  (handler-case
      (with-open-file (s pathname :direction :input)
        (if fix-bad-numbers
            (handler-bind ((not-readable-number 
                            #'fix-number-technique-1)) 
              (analyse-stream s))
            (analyse-stream s)))

    (file-error (c)
      (format t "A file error was encountered while trying to analyse ~A.~%The error returned was:~%~A~%" pathname c))
    (end-of-file ()
      (format t "Successfully reached the end of the file.~%"))
    (condition (c)
      (format t "A non-file error was encountered while trying to analyse ~A.~%The error returned was:~%~A~%" pathname c))))

;; Goes through the stream, dividing entries one by the next
(defun analyse-stream (stream)
  (do ()
      (nil)
    (let ((num1 (get-next-number stream))
          (num2 (get-next-number stream)))

        (unless (numberp num1)
          (let ((msg (make-condition 'not-readable-number 
                                     :object-seen num1)))
            (with-simple-restart (number-was-fixed "")
              (signal msg))
            (unless (get-was-corrected msg)
              (error msg))
            (setf num1 (get-corrected-value msg))))

        (unless (numberp num2)
          (let ((msg (make-condition 'not-readable-number 
                                     :object-seen num2)))
            (with-simple-restart (number-was-fixed "")
              (signal msg))
            (unless (get-was-corrected msg)
              (error msg))
            (setf num2 (get-corrected-value msg))))

      (format t 
              "The ratio of ~A to ~A is ~A~%" 
              num1 num2 (/ num1 num2)))))

;; Returns the next object in the stream
(defun get-next-number (stream)
  (read stream))

(defparameter *recognized-words*
  '(("zero" . 0)
    ("one" . 1)
    ("two" . 2)
    ("three" . 3)
    ("four" . 4)
    ("five" . 5)
    ("six" . 6)
    ("seven" . 7)
    ("eight" . 8)
    ("nine" . 9)
    ("ten" . 10)
    ("eleven" . 11)
    ("twelve" . 12)
    ("thirteen" . 13)))

;; Try to take a string or symbol representation and convert it to a
;; number
(defun fix-number-technique-1 (c)
  (let ((bad-object (get-object-seen c))
        string-rep)
    (when (symbolp bad-object)
      (setf string-rep (symbol-name bad-object)))
    (when (stringp bad-object)
      (setf string-rep (copy-seq bad-object)))

    (when string-rep
      ;; Maybe it's just a number in double-quotes.  Let's ask if we
      ;; can read a number from that.
      (let ((candidate (read-from-string string-rep nil nil)))
        (when (numberp candidate)
          (setf (get-was-corrected c) t)
          (setf (get-corrected-value c) candidate)
          (invoke-restart 'number-was-fixed)
          (return-from fix-number-technique-1)))

      ;; If we get here, it wasn't that.  Maybe it's one of our
      ;; recognized words.

      (let ((match (assoc string-rep *recognized-words* :test 'string-equal)))
        (when match
          (setf (get-was-corrected c) t)
          (setf (get-corrected-value c) (cdr match))))))
  (invoke-restart 'number-was-fixed))

Let’s look over this code in order and see what it does.  We’ve defined a condition, ‘not-readable-number’, that we will signal (i.e. throw in C++ parlance) when we encounter an unreadable number.  In the previous post, we saw the output generated when that condition was signaled and caught in the generic catch-all in the condition-demo function.

Starting with condition-demo.  We print an informational message, then set up the outer condition handlers with handler-case.  The handler-case function sets up for mandatory stack unwinding before the conditions are handled, so it looks like the C++ catch statement.  There, we’ve set up three handlers.  They are file-error, end-of-file, and condition.  As mentioned earlier, all conditions must ultimately derive from the condition class.  Note that end-of-file is not derived from the file-error class, so the file-error condition handler, which appears earlier in the list than end-of-file, will not itself collect end-of-file conditions thus depriving the end-of-file handler of its conditions.  The final entry, condition, catches any conditions not caught by earlier entries in the list of handlers.

Underneath the handler-case, we check to see if the invocation has asked to attempt to fix bad numbers.  If so, we use handler-bind to attach the not-readable-number condition to a function, fix-number-technique-1.  In C++ terms, the handler-bind function sets up something similar to an interrupt handler, but one that runs in the context of the location where the interrupt was signaled, rather than unwinding the stack as happens in a throw or handler-case.  Then, we continue as before, calling analyse-stream.

In analyse-stream, we read the number, but then check to see if the thing we read in was really a number, and not some other lisp object.  If it was not a number, we construct a not-readable-number condition and signal it underneath a ‘number-was-fixed’ restart.  If we did not bind a handler to that restart, because the caller did not ask to fix bad numbers, then this signaled condition is caught by the final “condition” handler in the top handler-case.  The stack unwinds, a message is printed, and the function exits.  However, if the handler was bound to the condition, a function is called with a single argument, the condition object that was signaled.  If this handler function invokes a restart, flow continues after the restart form.  If it does not invoke a restart, but instead returns normally, then it is deemed not to have handled the condition, and the next outer condition handler gets to attempt to handle it.  In order for the condition to be deemed handled, it must make a non-local transfer of control, either by invoking a restart or by signaling a new condition.

The fix-number-technique-1 function is there, for clarity.  This particular one can interpret as numbers the bare words zero, ZERO, one, One, TwO, etc.  It also recognizes those as strings, enclosed in double-quotes.  Finally, it can recognize numeric representation of numbers that have been accidentally enclosed in double-quotes causing them to look like strings to the reader.  If the function can find a replacement number, it modifies the condition object by filling in the corrected number, and setting a field to notify the user that a corrected number is available.  It then always calls the restart, which makes flow continue just after the with-simple-restart form.  There, we examine the condition object, and if it contains a corrected value, we use it, otherwise we throw the condition again, this time as an error.  The significant difference between signal and error is that, when there is no handler available for the condition in question, execution continues after a signal function, but does not continue through an error function.

This last point is important to emphasize.  In C++, you expect that if you throw an exception without a handler available for it, the program will ABEND with a message about an unhandled exception.  In Lisp, if you use signal to raise a condition, and there is no enclosing handler available for it, execution continues after the signal.  This can surprise those who aren’t expecting it.  If the aim is to ensure that execution will not continue past the form, the condition should be raised with error rather than with signal.

In the next post, we’ll clean this up a bit, making use of some subtler behaviours we’ve skipped for now.