Common Lisp the Language, 2nd Edition

11.9. An Example

Most users will want to load and use packages but will never need to build one. Often a user will load a number of packages into the user package whenever using Common Lisp. Typically an implementation might provide some sort of initialization file mechanism to make such setup automatic when the Lisp starts up. Table 11-1 shows such an initialization file, one that simply causes other facilities to be loaded.

X3J13 voted in March 1989 (LISP-PACKAGE-NAME)   to specify that the forthcoming ANSI Common Lisp will use the package name common-lisp-user instead of user.

When each of two files uses some symbols from the other, the author of those files must be careful to arrange the contents of the file in the proper order. Typically each file contains a single package that is a complete module. The contents of such a file should include the following items, in order:

1. A call to provide that announces the module name.

2. A call to in-package that establishes the package.

3. A call to shadow that establishes any local symbols that will shadow symbols that would otherwise be inherited from packages that this package will use.

4. A call to export that establishes all of this package's external symbols.

5. Any number of calls to require to load other modules that the contents of this file might want to use or refer to. (Because the calls to require follow the calls to in-package, shadow, and export, it is possible for the packages that may be loaded to refer to external symbols in this package.)

6. Any number of calls to use-package, to make external symbols from other packages accessible in this package.

7. Any number of calls to import, to make symbols from other packages present in this package.

8. Finally, the definitions making up the contents of this package/module.

The following mnemonic sentence may be helpful in remembering the proper order of these calls:

Put in seven extremely random user interface commands.

Each word of the sentence corresponds to one item in the above ordering:

           Put             Provide 
           IN              IN-package 
           Seven           Shadow 
           EXtremely       EXport 
           Random          Require 
           USEr            USE-package 
           Interface       Import 
           COmmands        COntents of package/module

The sentence says what it helps you to do.

The most distressing aspect of the X3J13 vote to eliminate provide and require (REQUIRE-PATHNAME-DEFAULTS)   is of course that it completely ruins the mnemonic sentence.

Now, suppose for the sake of example that the phlogiston and alchemy packages are single-file, single-package modules as described above. The phlogiston package needs to use the alchemy package, and the alchemy package needs to use several external symbols from the phlogiston package. The definitions in the alchemy and phlogiston files (see tables 11-2 and 11-3) allow a user to specify require statements for either of these modules, or for both of them in either order, and all relevant information will be loaded automatically and in the correct order.

For very large modules whose contents are spread over several files (the lisp package is an example), it is recommended that the user create the package and declare all of the shadows and external symbols in a separate file, so that this can be loaded before anything that might use symbols from this package.

Indeed, the defpackage macro approved by X3J13 in January 1989 (DEFPACKAGE)   encourages the use of such a separate file. (By the way, X3J13 voted in March 1989 (LISP-PACKAGE-NAME)   to specify that the forthcoming ANSI Common Lisp will use the package name common-lisp instead of lisp.) Let's take a look at a revision of I. Newton's files using defpackage.

The new version of the initialization file avoids using require; instead, we assume that load will do the job (see table 11-4).

The other files have each been split into two parts, one that establishes the package and one that defines the contents. This example uses a simple convention that for any file named, say, ``foo'' the file named ``foo-package'' contains the necessary defpackage and/or other package-establishing code. The idiom

(unless (find-package "FOO") 
  (load "foo-package"))

is conventionally used to load a package definition but only if the package has not already been defined. (This is a bit clumsy, and there are other ways to arrange things so that a package is defined no more than once.)

The file alchemy-package is shown in table 11-5. The tricky point here is that the alchemy and phlogiston packages contain mutual references (each imports from the other), and so defpackage alone cannot do the job. Therefore the phlogiston package is not mentioned in a :use option in the defpackage for the alchemy package. Instead, the function use-package is called explicitly, after the package definition for phlogiston has been loaded. Note that this file has been coded with excruciating care so as to operate correctly even if the package current when the file is loaded does not inherit from the common-lisp package. In particular, the standard load-package-definition idiom has been peppered with package qualifiers:

(cl:unless (cl:find-package "PHLOGISTON") 
  (cl:load "phlogiston-package"))

Note the use of the nickname cl for the common-lisp package.

The alchemy file, shown in table 11-6, simply loads the alchemy package definition, makes that package current, and then defines the ``real contents'' of the package.

The file phlogiston-package is shown in table 11-7. This one is a little more straightforward than the file alchemy-package, because the latter bears the responsibility for breaking the circular package references. This file simply makes sure that the alchemy package is defined and then performs a defpackage for the phlogiston package.

The phlogiston file, shown in table 11-8, simply loads the phlogiston package definition, makes that package current, and then defines the ``real contents'' of the package.

Let's look at the question of package circularity in this example a little more closely. Suppose that the file alchemy-package is loaded first. It defines the alchemy package and then loads file phlogiston-package. That file in turn finds that the package alchemy has already been defined and therefore does not attempt to load file alchemy-package again; it merely defines package phlogiston. The file alchemy-package then has a chance to import phlogiston:make-fire-bottle and everything is fine.

On the other hand, suppose that the file phlogiston-package is loaded first. It finds that the package alchemy has not already been defined, and therefore it immediately loads file alchemy-package. That file in turn defines the alchemy package; then it finds that package phlogiston is not yet defined and so loads file phlogiston-package again (indeed, in nested fashion). This time file phlogiston-package does find that the package alchemy has already been defined, so it simply defines package phlogiston and terminates. The file alchemy-package then imports phlogiston:make-fire-bottle and terminates. Finally, the outer loading of file phlogiston-package re-defines package phlogiston. Oh, dear. Fortunately the two definitions of package phlogiston agree in every detail, so everything ought to be all right. Still, it looks a bit dicey; I certainly don't have the same warm, fuzzy feeling that I would if no package were defined more than once.

Conclusion: defpackage goes a long way, but it certainly doesn't solve all the possible problems of package and file management. Neither did require and provide. Perhaps further experimentation will yield facilities appropriate for future standardization.