Common Lisp the Language, 2nd Edition

7.2. Generalized Variables

In Lisp, a variable can remember one piece of data, that is, one Lisp object. The main operations on a variable are to recover that object and to alter the variable to remember a new object; these operations are often called access and update operations. The concept of variables named by symbols can be generalized to any storage location that can remember one piece of data, no matter how that location is named. Examples of such storage locations are the car and cdr of a cons, elements of an array, and components of a structure.

For each kind of generalized variable, typically there are two functions that implement the conceptual access and update operations. For a variable, merely mentioning the name of the variable accesses it, while the setq special form can be used to update it. The function car accesses the car of a cons, and the function rplaca updates it. The function symbol-value accesses the dynamic value of a variable named by a given symbol, and the function set updates it.

Rather than thinking about two distinct functions that respectively access and update a storage location somehow deduced from their arguments, we can instead simply think of a call to the access function with given arguments as a name for the storage location. Thus, just as x may be considered a name for a storage location (a variable), so (car x) is a name for the car of some cons (which is in turn named by x). Now, rather than having to remember two functions for each kind of generalized variable (having to remember, for example, that rplaca corresponds to car), we adopt a uniform syntax for updating storage locations named in this way, using the setf macro. This is analogous to the way we use the setq special form to convert the name of a variable (which is also a form that accesses it) into a form that updates it. The uniformity of this approach is illustrated in the following table.

setf is actually a macro that examines an access form and produces a call to the corresponding update function.

Given the existence of setf in Common Lisp, it is not necessary to have setq, rplaca, and set; they are redundant. They are retained in Common Lisp because of their historical importance in Lisp. However, most other update functions (such as putprop, the update function for get) have been eliminated from Common Lisp in the expectation that setf will be uniformly used in their place.

[Macro]
setf {place newvalue}*

(setf place newvalue) takes a form place that when evaluated accesses a data object in some location and ``inverts'' it to produce a corresponding form to update the location. A call to the setf macro therefore expands into an update form that stores the result of evaluating the form newvalue into the place referred to by the access form.

If more than one place-newvalue pair is specified, the pairs are processed sequentially; that is,

(setf place1 newvalue1 
      place2 newvalue2) 
      ... 
      placen newvaluen)

is precisely equivalent to

(progn (setf place1 newvalue1) 
       (setf place2 newvalue2) 
       ... 
       (setf placen newvaluen))

For consistency, it is legal to write (setf), which simply returns nil.

The form place may be any one of the following:

• The name of a variable (either lexical or dynamic).

• A function call form whose first element is the name of any one of the following functions:
  aref car svref nth cdr get elt caar getf symbol-value rest cadr gethash symbol-function first cdar documentation symbol-plist second cddr fill-pointer macro-function third caaar caaaar cdaaar fourth caadr caaadr cdaadr fifth cadar caadar cdadar sixth caddr caaddr cdaddr seventh cdaar cadaar cddaar eighth cdadr cadadr cddadr ninth cddar caddar cdddar tenth cdddr cadddr cddddr

X3J13 voted in March 1988 (AREF-1D)   to add row-major-aref to this list.

X3J13 voted in June 1989 (DEFINE-COMPILER-MACRO)   to add compiler-macro-function to this list.

X3J13 voted in March 1989 (FUNCTION-NAME)   to clarify that this rule applies only when the function name refers to a global function definition and not to a locally defined function or macro.

• A function call form whose first element is the name of a selector function constructed by defstruct.

X3J13 voted in March 1989 (FUNCTION-NAME)   to clarify that this rule applies only when the function name refers to a global function definition and not to a locally defined function or macro.

• A function call form whose first element is the name of any one of the following functions, provided that the new value is of the specified type so that it can be used to replace the specified ``location'' (which is in each of these cases not truly a generalized variable):

  Function Name	Required Type 
  --------------------------------
  char	string-char 
  schar	string-char 
  bit	bit 
  sbit	bit 
  subseq	sequence 
  --------------------------------
  

X3J13 voted in March 1989 (CHARACTER-PROPOSAL)   to eliminate the type string-char and to redefine string to be the union of one or more specialized vector types, the types of whose elements are subtypes of the type character. In the preceding table, the type string-char should be replaced by some such phrase as ``the element-type of the argument vector.''

X3J13 voted in March 1989 (FUNCTION-NAME)   to clarify that this rule applies only when the function name refers to a global function definition and not to a locally defined function or macro.

In the case of subseq, the replacement value must be a sequence whose elements may be contained by the sequence argument to subseq. (Note that this is not so stringent as to require that the replacement value be a sequence of the same type as the sequence of which the subsequence is specified.) If the length of the replacement value does not equal the length of the subsequence to be replaced, then the shorter length determines the number of elements to be stored, as for the function replace.

• A function call form whose first element is the name of any one of the following functions, provided that the specified argument to that function is in turn a place form; in this case the new place has stored back into it the result of applying the specified ``update'' function (which is in each of these cases not a true update function):
  Function Name Argument That Is a place Update Function Used ---------------------------------------------------------------- char-bit first set-char-bit ldb second dpb mask-field second deposit-field ----------------------------------------------------------------

X3J13 voted in March 1989 (CHARACTER-PROPOSAL)   to eliminate char-bit and set-char-bit.

X3J13 voted in March 1989 (FUNCTION-NAME)   to clarify that this rule applies only when the function name refers to a global function definition and not to a locally defined function or macro.

• A the type declaration form, in which case the declaration is transferred to the newvalue form, and the resulting setf form is analyzed. For example,

  (setf (the integer (cadr x)) (+ y 3))
  

  is processed as if it were

  (setf (cadr x) (the integer (+ y 3)))
  

• A call to apply where the first argument form is of the form #'name, that is, (function name), where name is the name of a function, calls to which are recognized as places by setf. Suppose that the use of setf with apply looks like this:

  (setf (apply #'name x1 x2 ... xn rest) x0)
  

  The setf method for the function name must be such that

  (setf (name z1 z2 ... zm) z0)
  

  expands into a store form

  (storefn zi zi ... zi zm)
  

  That is, it must expand into a function call such that all arguments but the last may be any permutation or subset of the new value z0 and the arguments of the access form, but the last argument of the storing call must be the same as the last argument of the access call. See define-setf-method for more details on accessing and storing forms.

  Given this, the setf-of-apply form shown above expands into

  (apply #'storefn xi xi ... xi rest)
  

  As an example, suppose that the variable indexes contains a list of subscripts for a multidimensional array foo whose rank is not known until run time. One may access the indicated element of the array by writing

  (apply #'aref foo indexes)
  

  and one may alter the value of the indicated element to that of newvalue by writing

  (setf (apply #'aref foo indexes) newvalue)
  

X3J13 voted in March 1989 (FUNCTION-NAME)   to clarify that this rule applies only when the function name apply refers to the global function definition and not to a locally defined function or macro named apply.

• A macro call, in which case setf expands the macro call and then analyzes the resulting form.

X3J13 voted in March 1989 (FUNCTION-NAME)   to clarify that this step uses macroexpand-1, not macroexpand. This allows the chance to apply any of the rules preceding this one to any of the intermediate expansions.

• Any form for which a defsetf or define-setf-method declaration has been made.

X3J13 voted in March 1989 (FUNCTION-NAME)   to clarify that this rule applies only when the function name in the form refers to a global function definition and not to a locally defined function or macro.

X3J13 voted in March 1989 (FUNCTION-NAME)   to add one more rule to the preceding list, coming after all those listed above:

• Any other list whose first element is a symbol (call it f). In this case, the call to setf expands into a call to the function named by the list (setf f) (see section 7.1). The first argument is the new value and the remaining arguments are the values of the remaining elements of place. This expansion occurs regardless of whether either f or (setf f) is defined as a function locally, globally, or not at all. For example,

  (setf (f arg1 arg2 ...) newvalue)
  

  expands into a form with the same effect and value as

  (let ((#:temp1 arg1)     ;Force correct order of evaluation 
        (#:temp2 arg2) 
        ... 
        (#:temp0 newvalue)) 
    (funcall (function (setf f)) 
             #:temp0 
             #:temp1 
             #:temp2 ...))
  

  By convention, any function named (setf f) should return its first argument as its only value, in order to preserve the specification that setf returns its newvalue.

X3J13 voted in March 1989 (SYMBOL-MACROLET-SEMANTICS)   to add this case as well:

• A variable reference that refers to a symbol macro definition made by symbol-macrolet, in which case setf expands the reference and then analyzes the resulting form.

setf carefully arranges to preserve the usual left-to-right order in which the various subforms are evaluated. On the other hand, the exact expansion for any particular form is not guaranteed and may even be implementation-dependent; all that is guaranteed is that the expansion of a setf form will be an update form that works for that particular implementation, and that the left-to-right evaluation of subforms is preserved.

The ultimate result of evaluating a setf form is the value of newvalue. Therefore (setf (car x) y) does not expand into precisely (rplaca x y), but into something more like

(let ((G1 x) (G2 y)) (rplaca G1 G2) G2)

the precise expansion being implementation-dependent.

The user can define new setf expansions by using defsetf.

X3J13 voted in June 1989 (SETF-MULTIPLE-STORE-VARIABLES)   to extend the specification of setf to allow a place whose setf method has more than one store variable (see define-setf-method). In such a case as many values are accepted from the newvalue form as there are store variables; extra values are ignored and missing values default to nil, as is usual in situations involving multiple values.

A proposal was submitted to X3J13 in September 1989 to add a setf method for values so that one could in fact write, for example,

(setf (values quotient remainder) 
      (truncate linewidth tabstop))

but unless this proposal is accepted users will have to define a setf method for values themselves (not a difficult task).

[Macro]
psetf {place newvalue}*

psetf is like setf except that if more than one place-newvalue pair is specified, then the assignments of new values to places are done in parallel. More precisely, all subforms that are to be evaluated are evaluated from left to right; after all evaluations have been performed, all of the assignments are performed in an unpredictable order. (The unpredictability matters only if more than one place form refers to the same place.) psetf always returns nil.

X3J13 voted in June 1989 (SETF-MULTIPLE-STORE-VARIABLES)   to extend the specification of psetf to allow a place whose setf method has more than one store variable (see define-setf-method). In such a case as many values are accepted from the newvalue form as there are store variables; extra values are ignored and missing values default to nil, as is usual in situations involving multiple values.

[Macro]
shiftf {place}+ newvalue

Each place form may be any form acceptable as a generalized variable to setf. In the form (shiftf place1 place2 ... placen newvalue), the values in place1 through placen are accessed and saved, and newvalue is evaluated, for a total of n+1 values in all. Values 2 through n+1 are then stored into place1 through placen, and value 1 (the original value of place1) is returned. It is as if all the places form a shift register; the newvalue is shifted in from the right, all values shift over to the left one place, and the value shifted out of place1 is returned. For example:

(setq x (list 'a 'b 'c)) => (a b c) 

(shiftf (cadr x) 'z) => b 
   and now x => (a z c) 

(shiftf (cadr x) (cddr x) 'q) => z 
   and now x => (a (c) . q)

The effect of (shiftf place1 place2 ... placen newvalue) is equivalent to

(let ((var1 place1) 
      (var2 place2) 
      ... 
      (varn placen)) 
  (setf place1 var2) 
  (setf place2 var3) 
  ... 
  (setf placen newvalue) 
  var1)

except that the latter would evaluate any subforms of each place twice, whereas shiftf takes care to evaluate them only once. For example:

(setq n 0) 
(setq x '(a b c d)) 
(shiftf (nth (setq n (+ n 1)) x) 'z) => b 
   and now x => (a z c d) 

but 

(setq n 0) 
(setq x '(a b c d)) 
(prog1 (nth (setq n (+ n 1)) x) 
       (setf (nth (setq n (+ n 1)) x) 'z)) => b 
   and now x => (a b z d)

Moreover, for certain place forms shiftf may be significantly more efficient than the prog1 version.

X3J13 voted in June 1989 (SETF-MULTIPLE-STORE-VARIABLES)   to extend the specification of shiftf to allow a place whose setf method has more than one store variable (see define-setf-method). In such a case as many values are accepted from the newvalue form as there are store variables; extra values are ignored and missing values default to nil, as is usual in situations involving multiple values.

[Macro]
rotatef {place}*

Each place form may be any form acceptable as a generalized variable to setf. In the form (rotatef place1 place2 ... placen), the values in place1 through placen are accessed and saved. Values 2 through n and value 1 are then stored into place1 through placen. It is as if all the places form an end-around shift register that is rotated one place to the left, with the value of place1 being shifted around the end to placen. Note that (rotatef place1 place2) exchanges the contents of place1 and place2.

The effect of (rotatef place1 place2 ... placen) is roughly equivalent to

(psetf place1 place2 
       place2 place3 
       ... 
       placen place1)

except that the latter would evaluate any subforms of each place twice, whereas rotatef takes care to evaluate them only once. Moreover, for certain place forms rotatef may be significantly more efficient.

rotatef always returns nil.

X3J13 voted in June 1989 (SETF-MULTIPLE-STORE-VARIABLES)   to extend the specification of rotatef to allow a place whose setf method has more than one store variable (see define-setf-method). In such a case as many values are accepted from the newvalue form as there are store variables; extra values are ignored and missing values default to nil, as is usual in situations involving multiple values.

Other macros that manipulate generalized variables include getf, remf, incf, decf, push, pop, assert, ctypecase, and ccase.

Macros that manipulate generalized variables must guarantee the ``obvious'' semantics: subforms of generalized-variable references are evaluated exactly as many times as they appear in the source program, and they are evaluated in exactly the same order as they appear in the source program.

In generalized-variable references such as shiftf, incf, push, and setf of ldb, the generalized variables are both read and written in the same reference. Preserving the source program order of evaluation and the number of evaluations is particularly important.

As an example of these semantic rules, in the generalized-variable reference (setf reference value) the value form must be evaluated after all the subforms of the reference because the value form appears to the right of them.

The expansion of these macros must consist of code that follows these rules or has the same effect as such code. This is accomplished by introducing temporary variables bound to the subforms of the reference. As an optimization in the implementation, temporary variables may be eliminated whenever it can be proved that removing them has no effect on the semantics of the program. For example, a constant need never be saved in a temporary variable. A variable, or for that matter any form that does not have side effects, need not be saved in a temporary variable if it can be proved that its value will not change within the scope of the generalized-variable reference.

Common Lisp provides built-in facilities to take care of these semantic complications and optimizations. Since the required semantics can be guaranteed by these facilities, the user does not have to worry about writing correct code for them, especially in complex cases. Even experts can become confused and make mistakes while writing this sort of code.

X3J13 voted in March 1988 (PUSH-EVALUATION-ORDER)   to clarify the preceding discussion about the order of evaluation of subforms in calls to setf and related macros. The general intent is clear: evaluation proceeds from left to right whenever possible. However, the left-to-right rule does not remove the obligation on writers of macros and define-setf-method to work to ensure left-to-right order of evaluation.

Let it be emphasized that, in the following discussion, a form is something whose syntactic use is such that it will be evaluated. A subform means a form that is nested inside another form, not merely any Lisp object nested inside a form regardless of syntactic context.

The evaluation ordering of subforms within a generalized variable reference is determined by the order specified by the second value returned by get-setf-method. For all predefined generalized variable references (getf, ldb), this order of evaluation is exactly left-to-right. When a generalized variable reference is derived from a macro expansion, this rule is applied after the macro is expanded to find the appropriate generalized variable reference.

This is intended to make it clear that if the user writes a defmacro or define-setf-method macro that doesn't preserve left-to-right evaluation order, the order specified in the user's code holds. For example, given

(defmacro wrong-order (x y) `(getf ,y ,x))

then

(push value (wrong-order place1 place2))

will evaluate place2 first and then place1 because that is the order they are evaluated in the macro expansion.

For the macros that manipulate generalized variables (push, pushnew, getf, remf, incf, decf, shiftf, rotatef, psetf, setf, pop, and those defined with define-modify-macro) the subforms of the macro call are evaluated exactly once in left-to-right order, with the subforms of the generalized variable references evaluated in the order specified above.

Each of push, pushnew, getf, remf, incf, decf, shiftf, rotatef, psetf, and pop evaluates all subforms before modifying any of the generalized variable locations. Moreover, setf itself, in the case when a call on it has more than two arguments, performs its operation on each pair in sequence. That is, in

(setf place1 value1 place2 value2 ...)

the subforms of place1 and value1 are evaluated, the location specified by place1 is modified to contain the value returned by value1, and then the rest of the setf form is processed in a like manner.

For the macros check-type, ctypecase, and ccase, subforms of the generalized variable reference are evaluated once per test of a generalized variable, but they may be evaluated again if the type check fails (in the case of check-type) or if none of the cases holds (in ctypecase or ccase).

For the macro assert, the order of evaluation of the generalized variable references is not specified.

Another reason for building in these functions is that the appropriate optimizations will differ from implementation to implementation. In some implementations most of the optimization is performed by the compiler, while in others a simpler compiler is used and most of the optimization is performed in the macros. The cost of binding a temporary variable relative to the cost of other Lisp operations may differ greatly between one implementation and another, and some implementations may find it best never to remove temporary variables except in the simplest cases.

A good example of the issues involved can be seen in the following generalized-variable reference:

(incf (ldb byte-field variable))

This ought to expand into something like

(setq variable 
      (dpb (1+ (ldb byte-field variable)) 
           byte-field 
           variable))

In this expansion example we have ignored the further complexity of returning the correct value, which is the incremented byte, not the new value of variable. Note that the variable byte-field is evaluated twice, and the variable variable is referred to three times: once as the location in which to store a value, and twice during the computation of that value.

Now consider this expression:

(incf (ldb (aref byte-fields (incf i)) 
           (aref (determine-words-array) i)))

It ought to expand into something like this:

(let ((temp1 (aref byte-fields (incf i))) 
      (temp2 (determine-words-array))) 
  (setf (aref temp2 i) 
        (dpb (1+ (ldb temp1 (aref temp2 i))) 
             temp1 
             (aref temp2 i))))

Again we have ignored the complexity of returning the correct value. What is important here is that the expressions (incf i) and (determine-words-array) must not be duplicated because each may have a side effect or be affected by side effects.

X3J13 voted in January 1989 (SETF-SUB-METHODS)   to specify more precisely the order of evaluation of subforms when setf is used with an access function that itself takes a place as an argument, for example, ldb, mask-field, and getf. (The vote also discussed the function char-bit, but another vote (CHARACTER-PROPOSAL)   removed that function from the language.) The setf methods for such accessors produce expansions that effectively require explicit calls to get-setf-method.

The code produced as the macro expansion of a setf form that itself admits a generalized variable as an argument must essentially do the following major steps:

• It evaluates the value-producing subforms, in left-to-right order, and binds the temporary variables to them; this is called binding the temporaries.

• It reads the value from the generalized variable, using the supplied accessing form, to get the old value; this is called doing the access. Note that this is done after all the evaluations of the preceding step, including any side effects they may have.

• It binds the store variable to a new value, and then installs this new value into the generalized variable using the supplied storing form; this is called doing the store.

The place-specifier forms ldb, mask-field, and getf admit (other) place specifiers as arguments. During the setf expansion of these forms, it is necessary to call get-setf-method to determine how the inner, nested generalized variable must be treated.

In a form such as

(setf (ldb byte-spec place-form) newvalue-form)

the place referred to by the place-form must always be both accessed and updated; note that the update is to the generalized variable specified by place-form, not to any object of type integer.

Thus this call to setf should generate code to do the following:

• Evaluate byte-spec and bind into a temporary
• Bind the temporaries for place-form
• Evaluate newvalue-form and bind into the store variable
• Do the access to place-form
• Do the store into place-form with the given bit-field of the accessed integer replaced with the value in the store variable

The treatment of mask-field is similar to that of ldb.

In a form such as:

(setf (getf place-form ind-form) newvalue-form)

the place referred to by the place-form must always be both accessed and updated; note that the update is to the generalized variable specified by place-form, not necessarily to the particular list which is the property list in question.

Thus this call to setf should generate code to do the following:

• Bind the temporaries for place-form
• Evaluate ind-form and bind into a temporary
• Evaluate the newvalue-form and bind into the store variable
• Do the access to place-form
• Do the store into place-form with a possibly new property list obtained by combining the results of the evaluations and the access

If the evaluation of newvalue-form alters what is found in the given place-such as setting a different named property in the list-then the change of the property denoted by ind-form will be to that altered list, because the access step is done after the newvalue-form evaluation. Nevertheless, the evaluations required for binding the temporaries are done before the evaluation of the newvalue-form, thereby preserving the required left-to-right evaluation order.

Note that the phrase ``possibly new property list'' treats the implementation of property lists as a ``black box''; it can mean that the former property list is somehow destructively re-used, or it can mean partial or full copying of it. A side effect may or may not occur; therefore setf must proceed as if the resultant property list were a different copy needing to be stored back into the generalized variable.

The Common Lisp facilities provided to deal with these semantic issues include:

• Built-in macros such as setf and push that follow the semantic rules.

• The define-modify-macro macro, which allows new generalized-variable manipulating macros (of a certain restricted kind) to be defined easily. It takes care of the semantic rules automatically.

• The defsetf macro, which allows new types of generalized-variable references to be defined easily. It takes care of the semantic rules automatically.

• The define-setf-method macro and the get-setf-method function, which provide access to the internal mechanisms when it is necessary to define a complicated new type of generalized-variable reference or generalized-variable-manipulating macro.

Also important are the changes that allow lexical environments to be used in appropriate ways in setf methods.

[Macro]
define-modify-macro name lambda-list function
[doc-string]

This macro defines a read-modify-write macro named name. An example of such a macro is incf. The first subform of the macro will be a generalized-variable reference. The function is literally the function to apply to the old contents of the generalized-variable to get the new contents; it is not evaluated. lambda-list describes the remaining arguments for the function; these arguments come from the remaining subforms of the macro after the generalized-variable reference. lambda-list may contain &optional and &rest markers. (The &key marker is not permitted here; &rest suffices for the purposes of define-modify-macro.) doc-string is documentation for the macro name being defined.

The expansion of a define-modify-macro is equivalent to the following, except that it generates code that follows the semantic rules outlined above.

(defmacro name (reference . lambda-list) 
  doc-string 
  `(setf ,reference 
         (function ,reference ,arg1 ,arg2 ...)))

where arg1, arg2, ..., are the parameters appearing in lambda-list; appropriate provision is made for a &rest parameter.

As an example, incf could have been defined by:

(define-modify-macro incf (&optional (delta 1)) +)

An example of a possibly useful macro not predefined in Common Lisp is

(define-modify-macro unionf (other-set &rest keywords) union)

X3J13 voted in March 1988 (GET-SETF-METHOD-ENVIRONMENT)   to specify that define-modify-macro creates macros that take &environment arguments and perform the equivalent of correctly passing such lexical environments to get-setf-method in order to correctly maintain lexical references.

[Macro]

defsetf access-fn {update-fn [doc-string] |

                   lambda-list (store-variable)

                   [[{declaration}* | doc-string]] {form}*}

This defines how to setf a generalized-variable reference of the form
(access-fn ...). The value of a generalized-variable reference can always be obtained simply by evaluating it, so access-fn should be the name of a function or a macro.

The user of defsetf provides a description of how to store into the generalized-variable reference and return the value that was stored (because setf is defined to return this value). The implementation of defsetf takes care of ensuring that subforms of the reference are evaluated exactly once and in the proper left-to-right order. In order to do this, defsetf requires that access-fn be a function or a macro that evaluates its arguments, behaving like a function. Furthermore, a setf of a call on access-fn will also evaluate all of access-fn's arguments; it cannot treat any of them specially. This means that defsetf cannot be used to describe how to store into a generalized variable that is a byte, such as (ldb field reference). To handle situations that do not fit the restrictions imposed by defsetf, use define-setf-method, which gives the user additional control at the cost of increased complexity.

A defsetf declaration may take one of two forms. The simple form is

(defsetf access-fn update-fn doc-string)

The update-fn must name a function (or macro) that takes one more argument than access-fn takes. When setf is given a place that is a call on access-fn, it expands into a call on update-fn that is given all the arguments to access-fn and also, as its last argument, the new value (which must be returned by update-fn as its value). For example, the effect of

(defsetf symbol-value set)

is built into the Common Lisp system. This causes the expansion

(setf (symbol-value foo) fu) -> (set foo fu)

for example. Note that

(defsetf car rplaca)

would be incorrect because rplaca does not return its last argument.

The complex form of defsetf looks like

(defsetf access-fn lambda-list (store-variable) . body)

and resembles defmacro. The body must compute the expansion of a setf of a call on access-fn.

The lambda-list describes the arguments of access-fn. &optional, &rest, and &key markers are permitted in lambda-list. Optional arguments may have defaults and ``supplied-p'' flags. The store-variable describes the value to be stored into the generalized-variable reference.

The body forms can be written as if the variables in the lambda-list were bound to subforms of the call on access-fn and the store-variable were bound to the second subform of setf. However, this is not actually the case. During the evaluation of the body forms, these variables are bound to names of temporary variables, generated as if by gensym or gentemp, that will be bound by the expansion of setf to the values of those subforms. This binding permits the body forms to be written without regard for order-of-evaluation issues. defsetf arranges for the temporary variables to be optimized out of the final result in cases where that is possible. In other words, an attempt is made by defsetf to generate the best code possible in a particular implementation.

Note that the code generated by the body forms must include provision for returning the correct value (the value of store-variable). This is handled by the body forms rather than by defsetf because in many cases this value can be returned at no extra cost, by calling a function that simultaneously stores into the generalized variable and returns the correct value.

An example of the use of the complex form of defsetf:

(defsetf subseq (sequence start &optional end) (new-sequence) 
  `(progn (replace ,sequence ,new-sequence 
                   :start1 ,start :end1 ,end) 
          ,new-sequence))

X3J13 voted in March 1988 (FLET-IMPLICIT-BLOCK)   to specify that the body of the expander function defined by the complex form of defsetf is implicitly enclosed in a block construct whose name is the same as the name of the access-fn. Therefore return-from may be used to exit from the function.

X3J13 voted in March 1989 (DEFINING-MACROS-NON-TOP-LEVEL)   to clarify that, while defining forms normally appear at top level, it is meaningful to place them in non-top-level contexts; the complex form of defsetf must define the expander function within the enclosing lexical environment, not within the global environment.

The underlying theory by which setf and related macros arrange to conform to the semantic rules given above is that from any generalized-variable reference one may derive its ``setf method,'' which describes how to store into that reference and which subforms of it are evaluated.

Given knowledge of the subforms of the reference, it is possible to avoid evaluating them multiple times or in the wrong order. A setf method for a given access form can be expressed as five values:

• A list of temporary variables

• A list of value forms (subforms of the given form) to whose values the temporary variables are to be bound

• A second list of temporary variables, called store variables

• A storing form

• An accessing form

The temporary variables will be bound to the values of the value forms as if by let*; that is, the value forms will be evaluated in the order given and may refer to the values of earlier value forms by using the corresponding variables.

The store variables are to be bound to the values of the newvalue form, that is, the values to be stored into the generalized variable. In almost all cases only a single value is to be stored, and there is only one store variable.

The storing form and the accessing form may contain references to the temporary variables (and also, in the case of the storing form, to the store variables). The accessing form returns the value of the generalized variable. The storing form modifies the value of the generalized variable and guarantees to return the values of the store variables as its values; these are the correct values for setf to return. (Again, in most cases there is a single store variable and thus a single value to be returned.) The value returned by the accessing form is, of course, affected by execution of the storing form, but either of these forms may be evaluated any number of times and therefore should be free of side effects (other than the storing action of the storing form).

The temporary variables and the store variables are generated names, as if by gensym or gentemp, so that there is never any problem of name clashes among them, or between them and other variables in the program. This is necessary to make the special forms that do more than one setf in parallel work properly; these are psetf, shiftf, and rotatef. Computation of the setf method must always create new variable names; it may not return the same ones every time.

Some examples of setf methods for particular forms:

• For a variable x:

  () 
  () 
  (g0001) 
  (setq x g0001) 
  x
  

• For (car exp):

  (g0002) 
  (exp) 
  (g0003)  
  (progn (rplaca g0002 g0003) g0003) 
  (car g0002)
  

• For (subseq seq s e):

  (g0004 g0005 g0006) 
  (seq s e) 
  (g0007) 
  (progn (replace g0004 g0007 :start1 g0005 :end1 g0006) 
         g0007) 
  (subseq g0004 g0005 g0006)
  

[Macro]

define-setf-method access-fn lambda-list
                   [[ {declaration}* | doc-string ]]  {form}*

This defines how to setf a generalized-variable reference that is of the form (access-fn...). The value of a generalized-variable reference can always be obtained simply by evaluating it, so access-fn should be the name of a function or a macro.

The lambda-list describes the subforms of the generalized-variable reference, as with defmacro. The result of evaluating the forms in the body must be five values representing the setf method, as described above. Note that define-setf-method differs from the complex form of defsetf in that while the body is being executed the variables in lambda-list are bound to parts of the generalized-variable reference, not to temporary variables that will be bound to the values of such parts. In addition, define-setf-method does not have defsetf's restriction that access-fn must be a function or a function-like macro; an arbitrary defmacro destructuring pattern is permitted in lambda-list.

By definition there are no good small examples of define-setf-method because the easy cases can all be handled by defsetf. A typical use is to define the setf method for ldb:

;;; SETF method for the form (LDB bytespec int). 
;;; Recall that the int form must itself be suitable for SETF. 
(define-setf-method ldb (bytespec int) 
  (multiple-value-bind (temps vals stores 
                        store-form access-form) 
      (get-setf-method int)         ;Get SETF method for int 
    (let ((btemp (gensym))          ;Temp var for byte specifier 
          (store (gensym))          ;Temp var for byte to store 
          (stemp (first stores)))   ;Temp var for int to store 
      ;; Return the SETF method for LDB as five values. 
      (values (cons btemp temps)    ;Temporary variables 
              (cons bytespec vals)  ;Value forms 
              (list store)          ;Store variables 
              `(let ((,stemp (dpb ,store ,btemp ,access-form))) 
                 ,store-form 
                 ,store)                     ;Storing form 
              `(ldb ,btemp ,access-form)     ;Accessing form 
              ))))

X3J13 voted in March 1988 (GET-SETF-METHOD-ENVIRONMENT)   to specify that the &environment lambda-list keyword may appear in the lambda-list in the same manner as for defmacro in order to obtain the lexical environment of the call to the setf macro. The preceding example should be modified to take advantage of this new feature. The setf method must accept an &environment parameter, which will receive the lexical environment of the call to setf; this environment must then be given to get-setf-method in order that it may correctly use any locally bound setf method that might be applicable to the place form that appears as the second argument to ldb in the call to setf.

;;; SETF method for the form (LDB bytespec int). 
;;; Recall that the int form must itself be suitable for SETF. 
;;; Note the use of an &environment parameter to receive the 
;;; lexical environment of the call for use with GET-SETF-METHOD. 
(define-setf-method ldb (bytespec int &environment env) 
  (multiple-value-bind (temps vals stores 
                        store-form access-form) 
      (get-setf-method int env)     ;Get SETF method for int 
    (let ((btemp (gensym))          ;Temp var for byte specifier 
          (store (gensym))          ;Temp var for byte to store 
          (stemp (first stores)))   ;Temp var for int to store 
      ;; Return the SETF method for LDB as five values. 
      (values (cons btemp temps)    ;Temporary variables 
              (cons bytespec vals)  ;Value forms 
              (list store)          ;Store variables 
              `(let ((,stemp (dpb ,store ,btemp ,access-form))) 
                 ,store-form 
                 ,store)                     ;Storing form 
              `(ldb ,btemp ,access-form)     ;Accessing form 
              ))))

X3J13 voted in March 1988 (FLET-IMPLICIT-BLOCK)   to specify that the body of the expander function defined by define-setf-method is implicitly enclosed in a block construct whose name is the same as the name of the access-fn. Therefore return-from may be used to exit from the function.

X3J13 voted in March 1989 (DEFINING-MACROS-NON-TOP-LEVEL)   to clarify that, while defining forms normally appear at top level, it is meaningful to place them in non-top-level contexts; define-setf-method must define the expander function within the enclosing lexical environment, not within the global environment.

[Function]
get-setf-method form

get-setf-method returns five values constituting the setf method for form. The form must be a generalized-variable reference. get-setf-method takes care of error-checking and macro expansion and guarantees to return exactly one store variable.

As an example, an extremely simplified version of setf, allowing no more and no fewer than two subforms, containing no optimization to remove unnecessary variables, and not allowing storing of multiple values, could be defined by:

(defmacro setf (reference value) 
  (multiple-value-bind (vars vals stores store-form access-form) 
      (get-setf-method reference) 
    (declare (ignore access-form)) 
    `(let* ,(mapcar #'list 
                    (append vars stores) 
                    (append vals (list value))) 
       ,store-form)))

X3J13 voted in March 1988 (GET-SETF-METHOD-ENVIRONMENT)   to add an optional environment argument to get-setf-method. The revised definition and example are as follows.

[Function]
get-setf-method form &optional env

get-setf-method returns five values constituting the setf method for form. The form must be a generalized-variable reference. The env must be an environment of the sort obtained through the &environment lambda-list keyword; if env is nil or omitted, the null lexical environment is assumed. get-setf-method takes care of error checking and macro expansion and guarantees to return exactly one store variable.

As an example, an extremely simplified version of setf, allowing no more and no fewer than two subforms, containing no optimization to remove unnecessary variables, and not allowing storing of multiple values, could be defined by:

(defmacro setf (reference value &environment env) 
  (multiple-value-bind (vars vals stores store-form access-form) 
      (get-setf-method reference env)     ;Note use of environment
    (declare (ignore access-form)) 
    `(let* ,(mapcar #'list 
                    (append vars stores) 
                    (append vals (list value))) 
       ,store-form)))

[Function]
get-setf-method-multiple-value form

get-setf-method-multiple-value returns five values constituting the setf method for form. The form must be a generalized-variable reference. This is the same as get-setf-method except that it does not check the number of store variables; use this in cases that allow storing multiple values into a generalized variable. There are no such cases in standard Common Lisp, but this function is provided to allow for possible extensions.

X3J13 voted in March 1988 (GET-SETF-METHOD-ENVIRONMENT)   to add an optional environment argument to get-setf-method. The revised definition is as follows.

[Function]
get-setf-method-multiple-value form &optional env

get-setf-method-multiple-value returns five values constituting the setf method for form. The form must be a generalized-variable reference. The env must be an environment of the sort obtained through the &environment lambda-list keyword; if env is nil or omitted, the null lexical environment is assumed.

This is the same as get-setf-method except that it does not check the number of store variables; use this in cases that allow storing multiple values into a generalized variable. There are no such cases in standard Common Lisp, but this function is provided to allow for possible extensions.

X3J13 voted in March 1988 (GET-SETF-METHOD-ENVIRONMENT)   to clarify that a setf method for a functional name is applicable only when the global binding of that name is lexically visible. If such a name has a local binding introduced by flet, labels, or macrolet, then global definitions of setf methods for that name do not apply and are not visible. All of the standard Common Lisp macros that modify a setf place (for example, incf, decf, pop, and rotatef) obey this convention.