@node External protocols
@chapter External protocols

@menu
* Packages::
* Basic features::
* Ordinary reader features::
* Readtable features::
* Parse result construction features::
* CST reader features::
@end menu

@node Packages
@section Packages

@menu
* Package for basic features::
* Package for ordinary reader features::
* Package for readtable features::
* Package for parse result construction features::
* Package for CST features::
@end menu

@node Package for basic features
@subsection Package for basic features

The package for basic features such as customizable source location
construction is named @t{@basepackage{}}.  Although this package does
not shadow any symbol in the @t{@clpackage{}} package, we still
recommend the use of explicit package prefixes to refer to symbols in
this package.

@node Package for ordinary reader features
@subsection Package for ordinary reader features

The package for ordinary reader features is named @t{@readerpackage{}}.
To use features of this package, we recommend the use of explicit
package prefixes, simply because this package shadows and exports names
that are also exported from the @t{@clpackage{}} package.  Importing
this package will likely cause conflicts with the @t{@clpackage{}}
package otherwise.

@node Package for readtable features
@subsection Package for readtable features

@cindex readtable

The package for readtable-related features is named @t{@rtpackage{}}.
To use features of this package, we recommend the use of explicit
package prefixes, simply because this package shadows and exports names
that are also exported from the @t{@clpackage{}} package.  Importing
this package will likely cause conflicts with the @t{@clpackage{}}
package otherwise.

@node Package for parse result construction features
@subsection Package for parse result construction features

@cindex parse result

The package for features related to the creation of client-defined parse
results is named @t{@prpackage{}}.  To use features of this package, we
recommend the use of explicit package prefixes, simply because this
package shadows and exports names that are also exported from the
@t{@clpackage{}} package.  Importing this package will likely cause
conflicts with the @t{@clpackage{}} package otherwise.

@node Package for CST features
@subsection Package for CST features

@cindex parse result
@cindex concrete syntax tree

The package for features related to the creation of concrete syntax
trees is named @t{@cstpackage{}}.  To use features of this package, we
recommend the use of explicit package prefixes, simply because this
package shadows and exports names that are also exported from the
@t{@clpackage{}} package.  Importing this package will likely cause
conflicts with the @t{@clpackage{}} package otherwise.

@node Basic features
@section Basic features

@cindex client
@cindex source location

In this section, symbols written without package marker are in the
@t{@basepackage{}} package (@pxref{Package for basic features}).

This package provides the mechanism that enables clients to customize
the behavior of the reader.  Furthermore this package provides a
protocol for customizing a particular aspect of the behavior, namely the
construction of source positions and source ranges.  @sysname{} uses
source positions and source ranges in signaled conditions and parse
results (@pxref{Parse result construction features}).

@defclassa {stream-position-condition,@basepackage{}}

This condition type is the supertype of all conditions which are
signaled by @sysname{} functions.  An instance of this condition type
stores an approximate position in an input stream and an offset from
that position.  The condition is associated with the stream content at
the designated position and offset.  The position uses a representation
which is controlled by the respective client by adding a method on the
@t{source-position} generic function.  The offset indicates a distance
in characters which must be added to the approximate position to produce
the exact position.
@end deftp

@defgena {stream-position,@basepackage{}} condition

This generic function can be called by clients in order to obtain the
approximate position in the input stream to which @var{condition}
pertains.  The type and interpretation of the returned object depend on
the client, namely the presence of client-specific methods on the
@t{source-position} generic function.  The information returned by the
functions @t{position-offset} and @t{range-length} can be used to refine
the approximate position and compute a range in the input stream
respectively.

Applicable methods exist for all conditions of type
@t{stream-position-condition}.
@end deffn

@defgena {position-offset,@basepackage{}} condition

This generic function is called in order to compute the exact position
(or start of a range) in the input stream to which @var{condition}
pertains by refining the approximate position obtained by calling
@t{stream-position}.  The returned value is an integer (possibly
negative) which indicates the offset in characters from the approximate
position to the exact position.  Since the representation of the
approximate position is chosen by the client, applying the offset to
that position in a suitable way is also the responsibility of the
client.  Assuming the object returned by @t{(stream-position
@var{condition})} is suitable for arithmetic, the exact position is
@math{stream-position + position-offset}.

Applicable methods exist for all conditions of type
@t{stream-position-condition}.
@end deffn

@defgena {range-length,@basepackage{}} condition

This generic function is called in order to compute the length of the
range in the input stream to which @var{condition} pertains.  The
returned value is a non-negative integer which indicates the length of
the range in characters.  Therefore, assuming the object returned by
@t{(stream-position @var{condition})} is suitable for arithmetic, the
range covers input the positions @math{[start, start + range-length]}
where @math{start = stream-position + position-offset}.

Applicable methods exist for all conditions of type
@t{stream-position-condition}.
@end deffn

@defvara {*client*,@basepackage{}}

This variable is used by several generic functions which are called by
@funref{read,@readerpackage{}}.  The default value of the variable is
@t{nil}.  Clients that want to override or extend the default behavior
of some generic function of @sysname{} should bind this variable to some
standard object and provide a method on that generic function,
specialized to the class of that standard object.
@end defvar

@defgena {source-position,@basepackage{}} client stream

This generic function is called in order to determine the current
position in @var{stream}.  @sysname{} does not inspect or manipulate the
objects returned by this generic function beyond storing them in
signaled conditions and passing them as arguments to the
@t{make-source-range} generic function.  A client is therefore free to
define methods on this generic function that return arbitrary objects.

The default method on this generic function calls
@speclink{f,file-position}.
@end deffn

@defgena {make-source-range,@basepackage{}} client start end

This generic function is called in order to turn the source positions
@var{start} and @var{end} into a range representation suitable
for @var{client}.  The returned representation designates the range
of input characters from and including the character at position
@var{start} to but not including the character at position
@var{end}.  The default method returns
@code{(cons @var{start} @var{end})}.
@end deffn

@node Ordinary reader features
@section Ordinary reader features

@cindex parse result
@cindex source tracking

In this section, symbols written without package marker are in the
@t{@readerpackage} package
(@pxref{Package for ordinary reader features})

The features provided in this package fall into two categories:
@itemize
@item
The functions @funref{read,@readerpackage{}},
@funref{read-preserving-whitespace,@readerpackage{}},
@funref{read-from-string,@readerpackage{}} and
@funref{read-delimited-list,@readerpackage{}} which, together with
standard special variables, replicate the interface of the standard
@commonlisp{} reader (except functions related to readtables which
@sysname{} provides separately, @pxref{Readtable features}).  These
functions are discussed in the section @ref{@commonlisp{} reader
compatible interface}.

@item
The second category is comprised of the @varref{*client*,@basepackage{}}
special variable and a collection of protocols which allow customizing
the behavior of the reader by defining methods specialized to a
particular client on the generic functions of the protocols.
@end itemize

@float Figure,fig:read-call-sequence-ordinary
@center @image{read-call-sequence-ordinary,15cm}
@caption{Functions and typical function call sequences.  Solid arrows
represent calls, dashed arrows represent returns from function calls.
Labels above arrows represent arguments and return values.}
@end float

@ref{fig:read-call-sequence-ordinary} illustrates the categorization
into the @commonlisp{} reader compatible interface and the extensible
behavior protocol as well as typical function call patterns that arise
when the functions @funref{read,@readerpackage{}},
@funref{read-preserving-whitespace,@readerpackage},
@funref{read-from-string,@readerpackage} and
@funref{read-delimited-list,@readerpackage} are called by client code.

@menu
* @commonlisp{} reader compatible interface::
* Reader behavior protocol::
* Reader state protocol::
* Labeled objects and references::
* S-expression creation::
* Readtable initialization::
@end menu

@node @commonlisp{} reader compatible interface
@subsection @commonlisp{} reader compatible interface

The following functions are like their standard @commonlisp{}
counterparts with the two differences that their names are symbols in
the @t{@readerpackage{}} package and that their behavior can deviate from
that of the standard reader depending on the value of the
@varref{*client*,@basepackage{}} variable.

@deffuna {read,@readerpackage{}} @optional (input-stream @t{*standard-input*})@
  (eof-error-p @t{t})@
  (eof-value @t{nil})@
  (recursive-p @t{nil})

This function is the main entry point for the ordinary reader.  It is
entirely compatible with the standard @commonlisp{} function
@speclink{f,read}.
@end deffn

@deffuna {read-preserving-whitespace,@readerpackage{}} @optional (input-stream @t{*standard-input*})@
  (eof-error-p @t{t})@
  (eof-value @t{nil})@
  (recursive-p @t{nil})

This function is entirely compatible with the standard @commonlisp{}
function @speclinkl{f,read,read-preserving-whitespace}.
@end deffn

@deffuna {read-from-string,@readerpackage{}} string @optional@
  (eof-error-p @t{t})@
  (eof-value @t{nil})@
  @pkey@
  (start @t{0})@
  (end @t{nil})@
  (preserve-whitespace @t{nil})

This function is entirely compatible with the standard @commonlisp{}
function @speclink{f,read-from-string}.
@end deffn

@deffuna {read-delimited-list,@readerpackage{}} char @optional@
  (input-stream @t{*standard-input*})@
  (recursive-p @t{nil})

This function is entirely compatible with the standard @commonlisp{}
function @speclink{f,read-delimited-list}.
@end deffn

@node Reader behavior protocol
@subsection Reader behavior protocol

By defining methods on the generic functions of this protocol, clients
can customize the high-level behavior of the reader.

@float Figure,fig:read-call-sequence-customization
@center @image{read-call-sequence-customization,15cm}
@caption{Functions and typical function call sequences terminating on
the right hand side of the diagram at customizable generic functions
which implement aspects of the reader algorithm and standard reader
macros.  Solid arrows represent calls, dashed arrows represent returns
from function calls.  Labels above arrows represent arguments and return
values.}
@end float

@ref{fig:read-call-sequence-customization} illustrates how the
customizable generic functions described in this section are called
through the client interface and the implementation of the reader
algorithm.

@defgena {call-as-top-level-read,@readerpackage{}} client thunk input-stream@
  eof-error-p eof-value preserve-whitespace-p

This generic function is called by @funref{read,@readerpackage{}} if
@t{read} is called with a false value for the @var{recursive-p}
parameter.  It calls @var{thunk} with the necessary context for a global
@t{read} call.  @var{thunk} should read and return an object without
consuming any whitespace following the object.  If
@var{preserve-whitespace-p} is false, this function reads up to one
character of whitespace after @var{thunk} returns.  By default, this
function returns the object or @var{eof-value} returned by @var{thunk}
as its sole value.

@quotation Note
This generic function may return more values in addition to the one
described above.  Clients may use this feature to communicate additional
information between methods (@pxref{Parse result construction
features}).  Client defined methods on this generic function should
accept such additional values when calling @var{thunk}, a next method or
@genref{read-common,@readerpackage{}} and themselves return the
additional values.
@end quotation

The default method on this generic function performs two tasks:
@enumerate
@item
It establishes a context in which labels (@t{#N=}) and references
(@t{#N#}) work.

@item
It realizes the requested @var{preserve-whitespace-p} behavior.
@end enumerate
@end deffn

@defgena {read-common,@readerpackage{}} client input-stream @
  eof-error-p eof-value

This generic function is called by @funref{read,@readerpackage{}},
passing it the value of the variable @varref{*client*,@basepackage{}}
and the corresponding parameters.  By default, this generic function
returns the objects as its sole value.

@quotation Note
This generic function may return more values in addition to the one
described above.  Clients may use this feature to communicate additional
information between methods (@pxref{Parse result construction
features}).  Client defined methods on this generic function should
accept such additional values when calling a next method or
@t{read-maybe-nothing} and themselves return the additional values.
@end quotation

Client code can add methods on this function, specializing them to the
client class of its choice.  The actions that
@funref{read,@readerpackage{}} needs to take for different values of the
parameter @var{recursive-p} have already been taken before @t{read}
calls this generic function.
@end deffn

@defgena {read-maybe-nothing,@readerpackage{}} client input-stream@
  eof-error-p eof-value

This generic function can be called directly by the client or by the
generic function @genref{read-common,@readerpackage{}} to read an object
or consume input without returning an object.  If called directly by the
client, the call has to be in the dynamic scope of a
@genref{call-as-top-level-read,@readerpackage{}} call.  The function
@t{read-maybe-nothing} either

@itemize
@item
encounters the end of input on @var{input-stream} and, depending on
@var{eof-error-p} either signals an error or returns the values
@var{eof-value} and @t{:eof}

@item
or reads one or more whitespace characters an returns the values @t{nil}
and @t{:whitespace}

@item
or reads an object.  If @link{aspect-read-suppress,@t{*read-suppress*}}
is @emph{true}, the function returns nil and @t{:suppress}.  Otherwise
it returns the object and @t{:object}.

@item
or consumes a macro character and the characters consumed by the
associated reader macro function if that reader macro function does not
return a value.  In this case the function returns @t{nil} and
@t{:skip}.
@end itemize

@quotation Note
This generic function may return more values in addition to the ones
described above.  Clients may use this feature to communicate additional
information between methods (@pxref{Parse result construction
features}).  Client defined methods on this generic function should
accept such additional values when calling a next method and themselves
return the additional values.
@end quotation
@end deffn

@defgena {note-skipped-input,@readerpackage{}} client input-stream reason

This generic function is called whenever the reader skips some input
such as a comment or a form that must be skipped because of a reader
conditional.  It is called with the value of the variable
@varref{*client*,@basepackage{}}, the input stream from which the input
is being read and an object indicating the reason for skipping the
input.  The default method on this generic function does nothing.
Client code can supply a method that specializes to the client class of
its choice.

When this function is called, the stream is positioned immediately
@emph{after} the skipped input.  Client code that wants to know both the
beginning and the end of the skipped input must remember the stream
position before the call to @funref{read,@readerpackage{}} was made as
well as the stream position when the call to this function is made.
@end deffn

@defvara {*skip-reason*,@readerpackage{}}

This variable is used by the reader to determine why a range of input
characters has been skipped.  To this end, internal functions of the
reader as well as reader macros can set this variable to a suitable
value before skipping over some input.  Then, after the input has been
skipped, the generic function
@genref{note-skipped-input,@readerpackage{}} is called with the value of
the variable as its @var{reason} argument.

As an example, the method on
@genref{note-skipped-input,@readerpackage{}} specialized to
@classref{parse-result-client,@prpackage{}} relays the reason and
position information to the client by calling the
@genref{make-skipped-input-result,@prpackage{}} generic function
(@pxref{Parse result construction features}).
@end defvar

@defgena {read-token,@readerpackage{}} client input-stream @
  eof-error-p eof-value

This generic function is called by @genref{read-common,@readerpackage{}}
when it has been detected that a token should be read.  This function is
responsible for accumulating the characters of the token and then
calling @genref{interpret-token,@readerpackage{}} in order to create and
return a token.
@end deffn

@defgena {interpret-token,@readerpackage{}} client input-stream token@
  escape-ranges

This generic function is called by @genref{read-token,@readerpackage{}}
in order to create a token from accumulated token characters.  The
parameter @var{token} is a string containing the characters that make up
the token.  The parameter @var{escape-ranges} indicates ranges of
characters read from @var{input-stream} and preceded by a character with
single-escape syntax or delimited by characters with multiple-escape
syntax.  Values of @var{escape-ranges} are lists of elements of the form
@t{(@var{start} . @var{end})} where @t{@var{start}} is the index of
the first escaped character and @t{@var{end}} is the index
@emph{following} the last escaped character.  Note that @t{@var{start}}
and @t{@var{var}} can be identical indicating no escaped characters.
This can happen in cases like @t{a||b}.  The information conveyed by the
@var{escape-ranges} parameter is used to convert the characters in
@var{token} according to the @emph{readtable case} of the current
readtable before a token is constructed.
@end deffn

@defgena {check-symbol-token,@readerpackage{}} client@
  input-stream token escape-ranges@
  position-package-marker-1 position-package-marker-2

This generic function is called by the default method on
@genref{interpret-token,@readerpackage{}} when the syntax of the token
corresponds to that of a symbol.  This function checks the syntactic
validity of the symbol token and signals an error in case of a syntax
error.  If there are no syntax errors (or error recovery has been
performed, @pxref{Recovering from errors}), this function returns three
values:
@enumerate
@item
@var{token} or a value derived from @var{token} by error recovery
operations.

@item
@var{position-package-marker-1} or a value derived from
@var{position-package-marker-1} by error recovery operations.

@item
@var{position-package-marker-2} or a value derived from
@var{position-package-marker-2} by error recovery operations.
@end enumerate

The parameter @var{input-stream} is the input stream from which the
characters were read.  The parameter @var{token} is a string that
contains all the characters of the token.  The parameter
@var{escape-ranges} indicates ranges within @var{token} that were
preceded by a character with single-escape syntax or delimited by
characters with multiple-escape syntax.  The parameter
@var{position-package-marker-1} contains the index into @var{token} of
the first package marker, or @t{nil} if the token contains no package
markers.  The parameter @var{position-package-marker-2} contains the
index into @var{token} of the second package marker, or @t{nil} if the
token contains no package markers or only a single package marker.

The default method on this generic function checks the positions of the
package markers taking into account escape ranges.  The method signals
errors and allows error recovery as described above.
@end deffn

@defgena {interpret-symbol-token,@readerpackage{}} client@
  input-stream token position-package-marker-1@
  position-package-marker-2

This generic function is called by the default method on
@genref{interpret-token,@readerpackage{}} when the syntax of the token
corresponds to that of a valid symbol.  The parameter @var{input-stream}
is the input stream from which the characters were read.  The parameter
@var{token} is a string that contains all the characters of the token.
The parameter @var{position-package-marker-1} contains the index into
@var{token} of the first package marker, or @t{nil} if the token
contains no package markers.  The parameter
@var{position-package-marker-2} contains the index into @var{token} of
the second package marker, or @t{nil} if the token contains no package
markers or only a single package marker.

The default method on this generic function calls
@genref{interpret-symbol,@readerpackage{}} with a symbol name string and
a package indicator.
@end deffn

@defgena {interpret-symbol,@readerpackage{}} client input-stream@
  package-indicator symbol-name internp

This generic function is called by the default method on
@genref{interpret-symbol-token,@readerpackage{}} as well as the default
@t{#:} reader macro function to resolve a symbol name string and a
package indicator to a representation of the designated symbol.  The
parameter @var{input-stream} is the input stream from which
@var{package-indicator} and @var{symbol-name} were read.  The parameter
@var{package-indicator} is a either

@itemize
@item
a string designating the package of that name

@item
the keyword @t{:current} designating the current package

@item
the keyword @t{:keyword} designating the keyword package

@item
@t{nil} to indicate that an uninterned symbol should be created
@end itemize

The @var{symbol-name} is the name of the desired symbol.

The default method uses @speclink{f,find-package} (or
@link{aspect-package,@t{*package*}} when @var{package-indicator} is
@t{:current}) to resolve @var{package-indicator} followed by
@speclink{f,find-symbol} or @speclink{f,intern}, depending on
@var{internp}, to resolve @var{symbol-name}.

A second method which is specialized on @var{package-indicator} being
@t{nil} uses @speclink{f,make-symbol} to create uninterned symbols.
@end deffn

@defgena {call-reader-macro,@readerpackage{}} client input-stream char@
  readtable

This generic function is called when the reader has determined that
some character is associated with a reader macro.  The parameter
@var{char} has to be used in conjunction with the
@var{readtable} parameter to obtain the macro function that is
associated with the macro character.  The parameter
@var{input-stream} is the input stream from which the reader macro
function will read additional input to accomplish its task.

The default method on this generic function simply obtains the reader
macro function for @var{char} from @var{readtable} and calls it,
passing @var{input-stream} and @var{char} as arguments.  The
default method therefore does the same thing that the standard
@commonlisp{} reader does.
@end deffn

@defgena {find-character,@readerpackage{}} client designator

This generic function is called by the default @t{#\} reader macro
function to find a character.  @var{designator} is either
@itemize
@item
a @t{string} that is the name of the character to be found with single
and multiple escapes removed, but with the case of all characters as it
was in the input.

@item
or a character designating itself.
@end itemize
The function has to either return the character designated by
@var{designator} or @t{nil} if no such character exists.

If @var{designator} is a @t{string}, it is the responsibility of the
client to disregard the case of characters in @var{designator}, for
example by producing an uppercase string from @var{designator} before
looking up the designated character.

A default method on this generic function that is not specialized to any
particular client but is specialized to @var{designator} being a
@t{string} recognizes the mandatory character names listing in
@uref{https://novaspec.org/cl/13_1_Character_Concepts#sec_13_1_7,
Section 13.1.7 Character Names of the @commonlisp{} specification}.
Another default method on this generic function that is not specialized
to any particular client but is specialized to @var{designator} being a
@t{character} just returns @var{designator}.
@end deffn

@defgena {make-structure-instance,@readerpackage{}} client name initargs

This generic function is called by the default @t{#S} reader
macro function to construct structure instances.  @var{name} is a
symbol naming the structure type of which an instance should be
constructed.  @var{initargs} is a list the elements of which
alternate between string designators naming structure slots and values
for those slots.

It is the responsibility of the client to coerce the string
designators to symbols as if by
@code{(intern (string slot-name) (find-package 'keyword))}
as described in the @commonlisp{} specification.

There is no default method on this generic function since there is no
portable way to construct structure instances given only the name of
the structure type.
@end deffn

@defgena {evaluate-expression,@readerpackage{}} client expression

This generic function is called by the default @t{#.} reader macro
function to perform read-time evaluation.  @var{expression} is the
expression that should be evaluated as it was returned by a recursive
@funref{read,@readerpackage{}} call and potentially influenced by
@var{client}.  The function has to either return the result of
evaluating @var{expression} or signal an error.

The default method on this generic function simply returns the result
of @code{(cl:eval @var{expression})}.
@end deffn

@defgena {check-feature-expression,@readerpackage{}} client@
  feature-expression

This generic function is called by the default @t{#+} and
@t{#-} reader macro functions to check the well-formedness of
@var{feature-expression} which has been read from the input stream
before evaluating it.  For compound expressions, only the outermost
expression is checked regarding the atom in operator position and its
shape -- child expressions are not checked.  The function returns an
unspecified value if @var{feature-expression} is well-formed and
signals an error otherwise.

The default method on this generic function accepts standard
@commonlisp{} feature expressions, i.e. expressions recursively
composed of symbols, @t{:not}-expressions,
@t{:and}-expressions and @t{:or}-expressions.
@end deffn

@defgena {evaluate-feature-expression,@readerpackage{}} client@
 feature-expression

This generic function is called by the default @t{#+} and
@t{#-} reader macro functions to evaluate
@var{feature-expression} which has been read from the input stream.
The function returns either @emph{true} or @emph{false} if
@var{feature-expression} is well-formed and signals an error
otherwise.

For compound feature expressions, the well-formedness of child
expressions is not checked immediately but lazily, just before the child
expression in question is evaluated in a subsequent
@t{evaluate-feature-expression} call.  This allows expressions like
@code{#+(and my-cl-implementation (special-feature a b)) @var{form}} to
be read without error when the @t{:my-cl-implementation} feature is
absent.

The default method on this generic function first calls
@genref{check-feature-expression,@readerpackage{}} to check the
well-formedness of @var{feature-expression}.  It then evaluates
@var{feature-expression} according to standard @commonlisp{} semantics
for feature expressions.
@end deffn

@node Reader state protocol
@subsection Reader state protocol

@cindex reader state
@cindex aspect, reader state
@cindex quasiquotation
@cindex consing dot

The reader state protocol consists of generic functions which the reader
and the client call to query and modify the values of reader state
@dfn{aspects}.  Each aspect is named by a symbol and holds a current
value and has a stack of shadowed values like a special variable.  Most
aspects roughly correspond to a particular reader control variable
defined in the @commonlisp{} specification.  In addition to those,
@sysname{} uses aspects for representing the validity of the consing dot
as well as the quasiquotation depth and validity in a given context.  In
total, @sysname{} defines the following aspects:

@anchor{table:minimal-reader-state-aspects}
@table @code
@item cl:*readtable*

@anchor{aspect-readtable}
Like the @speclinkl{v,readtable,cl:*readtable*} special variable, this
aspect controls the readtable object in which the reader looks up the
syntax types of characters, the case conversion mode as well as reader
macros.  By default, values of this aspect must satisfy the
@genref{readtablep,@rtpackage{}} predicate.

@item cl:*package*

@anchor{aspect-package}
Like the @speclinkl{v,package,cl:*package*} special variable, this
aspects controls the package which the reader uses when it looks up or
interns symbols in the current package.  By default, values of this
aspect must be package designators.

@item cl:*read-suppress*

@anchor{aspect-read-suppress}
Like the @speclinkl{v,read-suppress,cl:*read-suppress*} special
variable, this aspect controls whether the reader skips over expressions
without detailed parsing.

@item cl:*read-eval*

@anchor{aspect-read-eval}
Like the @speclinkl{v,read-eval,cl:*read-eval*} special variable, this
aspect controls whether the reader evaluates expressions in @t{#.}
constructs.

@item cl:*features*

@anchor{aspect-features}
Like the @speclinkl{v,features,cl:*features*} special variable, this
aspect controls the evaluation of features in feature expressions in
@t{#+} and @t{#-} constructs.  By default, values of this aspect must be
proper lists of symbols.

@item cl:*read-base*

@anchor{aspect-read-base}
Like the @speclinkl{v,read-base,cl:*read-base*} special variable, this
aspect controls the interpretation of tokens by the reader as being
integers or ratios.  By default, values of this aspect must be of type
@t{(integer 1 36)}.

@item cl:*read-default-float-format*

@anchor{aspect-read-default-float-format}
Like the
@speclinkl{v,read-default-float-format,cl:*read-default-float-format*}
special variable, this aspect controls the floating-point format that
the reader uses for floating-point numbers without exponent marker or
the default exponent marker.

@item @readerpackage{}::*quasiquotation-state*

@anchor{aspect-quasiquotation-state}
@quotation Warning
Clients should not query, bind or set the value of this aspect at this
time.
@end quotation

This aspect controls whether backquote and unquote are allowed in the
current context.

@item @readerpackage{}::*quasiquotation-depth*

@anchor{aspect-quasiquotation-depth}
@quotation Warning
Clients should not query, bind or set the value of this aspect at this
time.
@end quotation

This aspect tracks the backquote nesting depth in the current context.

@item @readerpackage{}::*consing-dot-allowed-p*

@anchor{aspect-consing-dot-allowed-p}
@quotation Warning
Clients should not query, bind or set the value of this aspect at this
time.
@end quotation

This aspect controls whether the consing dot is allowed in the current
context.

@end table

@defclassa{state-value-type-error,@readerpackage{}}

Errors of this type are signaled when an attempt is made to establish an
object as the value for a reader state aspect and the supplied object is
not of the type required by the aspect.

Since this condition type is a subtype of @speclink{t,type-error}, the
offending value and the expected type can be retrieved via the readers
@speclink{f,type-error-datum} and
@speclinkl{f,type-error-datum,cl:type-error-expected-type} respectively.
The aspect for which the value was supplied can be retrieved via the
reader @t{@readerpackage{}:aspect}.
@end deftp

@defgena{valid-state-value-p,@readerpackage{}} client aspect value

This generic function is called by the reader to determine whether
@var{value} is a valid value for the reader state aspect designated by
@var{aspect}.  The generic function returns true if, according to
@var{client}, @var{value} is a valid value for the reader state aspect
designated by @var{aspect}.  @var{aspect} must designate a reader state
aspect that is recognized by @var{client}.  At least the aspects listed
in the @ref{table:minimal-reader-state-aspects,,minimal reader state
aspects table} must be recognized by any client.

With the exceptions of @link{aspect-readtable,@t{cl:*readtable*}} and
@link{aspect-package,@t{cl:*package*}}, the default methods on this
generic function recognize state aspects and implement type restrictions
informed by the @commonlisp{} specification:
@multitable @columnfractions .3 .7
@headitem Aspect                             @tab Type
@item     @t{cl:*readtable*}                 @tab @t{(satisfies @rtpackage{}:readtablep)}
@item     @t{cl:*package*}                   @tab @t{(or cl:package cl:symbol cl:string cl:character)} (package designator)
@item     @t{cl:*read-suppress*}             @tab @t{t} (generalized Boolean)
@item     @t{cl:*read-eval*}                 @tab @t{t} (generalized Boolean)
@item     @t{cl:*features*}                  @tab @t{list} (proper list)
@item     @t{cl:*read-base*}                 @tab @t{(integer 2 36)} (radix)
@item     @t{cl:*read-default-float-format*} @tab @t{(member short-float single-float double-float long-float)}
@end multitable
@end deffn

@defgena{state-value,@readerpackage{}} client aspect

Return the current value of the reader state aspect designated by
@var{aspect}.

@var{aspect} must designate a reader state aspect that is recognized by
@var{client}.  At least the aspects listed in the
@ref{table:minimal-reader-state-aspects,,minimal reader state aspects
table} must be recognized by any client.

The @link{aspect-package,@t{cl:*package*}} aspect mandates further
explanation: When the client uses only the default methods of the reader
state protocol, the return value of this generic function for the
@t{cl:*package*} aspect is of type @speclink{t,package} which is a
strict subtype of the type of valid values for this aspect.  In other
words, the defaults coerce package designators to package objects.
@end deffn

@defgena{(setf state-value),@readerpackage{}} new-value client aspect

Set the current value of the reader state aspect designated by
@var{aspect} to @var{new-value}.

@var{aspect} must designate a reader state aspect that is recognized by
@var{client}.  At least the aspects listed in the
@ref{table:minimal-reader-state-aspects,,minimal reader state aspects
table} must be recognized by any client.

@var{new-value} is the desired new value for the designated aspect.
@var{new-value} has to be a valid value for @var{aspect} in the sense
that
@code{(@genref{valid-state-value-p,@readerpackage{}} @var{client} @var{aspect} @var{value})}
must return @emph{true}.

The @link{aspect-package,@t{cl:*package*}} aspect mandates further
explanation: When the client uses only the default methods of the reader
state protocol, the method on this generic function which handles the
@t{cl:*package*} aspect coerces @var{new-value} from designators to
package objects so that a subsequent
@genref{state-value,@readerpackage{}} call returns the designated
package object.
@end deffn

@defgena{call-with-state-value,@readerpackage{}} client thunk aspect value

Call @var{thunk} with the reader state aspect designated by @var{aspect}
bound to @var{value}.

@var{aspect} must designate a reader state aspect that is recognized by
@var{client}.  At least the aspects listed in the
@ref{table:minimal-reader-state-aspects,,minimal reader state aspects
table} must be recognized by any client.

The following properties must hold:

@itemize
@item
@var{value} has to be a valid value for @var{aspect} in the sense that
@code{(@genref{valid-state-value-p,@readerpackage{}} @var{client} @var{aspect} @var{value})}
must return @emph{true}.

@item
During the call to @var{thunk} and absent any intervening calls to
@genref{call-with-state-value,@readerpackage{}}, the expression
@code{(@genref{state-value,@readerpackage{}} @var{client} @var{aspect})}
must evaluate to @var{value}.
@end itemize

When @sysname{} calls this generic function with @t{cl:*package*} as the
value of @var{aspect}, the @var{value} is a always a string designator
and never a package object.  The default method on this generic function
coerces such string designators to package objects so that a subsequent
@genref{state-value,@readerpackage{}} call returns the designated
package object.
@end deffn

@cindex quasiquote
@cindex backquote
@cindex unquote

Backquote and unquote syntax is forbidden in some contexts such as
multi-dimensional array literals (@t{#A}) and structure literals
(@t{#S}).  @sysname{} tracks and controls whether backquote, unquote or
both should be allowed in a given context using the aspects
@link{aspect-quasiquotation-state,@t{@readerpackage{}::*quasiquotation-state*}}
and
@link{aspect-quasiquotation-depth,@t{@readerpackage{}::*quasiquotation-depth*}}
mentioned above.  Since custom reader macros may also have to control
this state, @sysname{} provides the following convenience macro:

@defmaca {with-quasiquotation-state,@readerpackage{}} client context@
  quasiquote-forbidden-p unquote-forbidden-p @pbody body

@quotation Warning
This macro is experimental and its name is not exported for now.
@end quotation

Control whether backquote syntax, unquote syntax or both are allowed in
@t{read} functions called during the execution of @var{body}.
@var{context} is a symbol identifying the current context which is used
for error reporting.  A typical value is the name of the reader macro
function in which this macro is used.  @var{quasiquote-forbidden-p}
controls whether backquote syntax should be forbidden.  The value
@t{:keep} causes the binding to remain unchanged.
@var{unquote-forbidden-p} controls whether unquote syntax should be
forbidden.  The value @t{:keep} causes the binding to remain unchanged.
@end deffn

@defmaca {with-forbidden-quasiquotation,@readerpackage{}} context@
  @optional@
  (quasiquote-forbidden-p @t{t})@
  (unquote-forbidden-p @t{t})@
  @pbody body

@quotation Warning
This macro is deprecated as of @sysname{} 0.11 and will be removed in a
future version.  This macro is replaced by the macro
@macref{with-quasiquotation-state,@readerpackage{}} but that macro is
experimental and its name is not exported for now.
@end quotation

Disallow backquote syntax, unquote syntax or both in @t{read}
functions called during the execution of @var{body}.
@var{context} is a symbol identifying the current context which is
used for error reporting.  A typical value is the name of the reader
macro function in which this macro is used.
@var{quasiquote-forbidden-p} controls whether backquote syntax
should be forbidden.  The value @t{:keep} causes the binding to
remain unchanged.  @var{unquote-forbidden-p} controls whether
unquote syntax should be forbidden.  The value @t{:keep} causes
the binding to remain unchanged.
@end deffn

@node Labeled objects and references
@subsection Labeled objects and references

@cindex labeled object

@sysname{} includes implementations of the @t{#=} and @t{##} reader
macros and they are present in the default readtable.  One way to
customize the behavior of the reader around the @t{#=} and @t{##} syntax
is replacing the reader macro functions with custom ones but with this
approach the client code has to reimplement a lot of functionality.  As
a finer grained and more composable mechanism for customization,
@sysname{} provides a protocol for implementing and customizing the
behavior of the @t{#=} and @t{##} reader macros, with or without
modifying the readtable.  The remainder of this section describes that
protocol.

To start with a bit of terminology, we call the object created by
reading @t{#@var{N}=@var{expression}} a @dfn{labeled object}. We call
@var{N} the @dfn{label} of the labeled object and the result of reading
@t{@var{expression}} the @dfn{object} of the labeled object.  We say
that @t{#@var{N}=@var{expression}} @dfn{defines} the labeled object and
@t{#@var{N}#} @dfn{references} the labeled object.  We call the
reference @dfn{circular} if @t{#@var{N}#} occurs within
@var{expression}.  Labeled objects are internal to the reader and only
exist during @funref{read,@readerpackage{}} calls: before such a call
returns an object, each labeled object within the returned object is
replaced by its respective final object.  Callers of
@funref{read,@readerpackage{}} and related functions will therefore only
ever see the object, never the labeled object@footnote{Reader macro
functions which call @funref{read,@readerpackage{}} may receive labeled
objects under certain circumstances (@pxref{Circular objects and custom
reader macros}).}.

On a technical level, a labeled object is represented as a data type
with a current state and a single (possibly unbound) slot containing the
object.  The following diagrams depicts the possible states of a labeled
object together with input patterns and corresponding transitions:

@float Figure,fig:labeled-object-states
@center @image{labeled-object-states,15cm}
@caption{Possible states of a labeled object and input patterns which
correspond to state transitions.}
@end float

Put differently, a labeled object can be in the following states:

@multitable @columnfractions .5 .3
@headitem State                                          @tab Object slot
@item undefined                                          @tab --
@item defined                                            @tab unbound
@item final                                              @tab the object
@item referenced @emph{(not strictly needed)}            @tab the object
@item circular                                           @tab unbound
@item final (circular)                                   @tab the object
@item referenced (circular) @emph{(not strictly needed)} @tab the object
@end multitable

The distinction between the states @t{final} and @t{referenced} on the
one hand and @t{final (circular)}, and @t{referenced (circular)} on the
other hand is not required for implementing labeled objects.  Those two
pairs of states are therefore collapsed to just @t{final} and @t{final
(circular)} in the remainder of this section.  The following figure and
paragraphs describe generic functions and methods which implement the
creation, registration, lookup and manipulation of labeled objects
according to the reduced set of states:

@float Figure,fig:labeled-object-api-states
@center @image{labeled-object-api-states,12cm}
@caption{Reduced set of states of a labeled object and protocol
functions with corresponding state transitions.}
@end float

In addition to the generic functions referenced in the above diagram,
the generic functions @genref{fixup-graph-p,@readerpackage{}},
@genref{fixup-graph,@readerpackage{}}, @genref{fixup,@readerpackage{}}
and @genref{new-value-for-fixup,@readerpackage{}} are part of the
protocol.  Those functions are used to replace labeled objects with
their respective final objects within an object that is about to be
returned to the caller of @funref{read,@readerpackage{}} @footnote{This
fixup processing has to be delayed under certain circumstances
(@pxref{Circular objects and custom reader macros}).}.  To this end, the
@t{#=} reader macro function must inspect and update the state of the
labeled object it is processing after reading @t{@var{expression}} by
calling @genref{finalize-labeled-object,@readerpackage{}}.
@genref{finalize-labeled-object,@readerpackage{}} decides whether
@genref{fixup-graph,@readerpackage{}} (see below) must be called: If
after reading @t{@var{expression}} the labeled object is in state
@t{:circular}, @t{@var{expression}} must have contained circular
references and the result of reading it contains labeled objects that
have to be replaced with their respective final objects.
@genref{fixup-graph,@readerpackage{}}, @genref{fixup,@readerpackage{}}
and @genref{new-value-for-fixup,@readerpackage{}} perform this
replacement.  The replacement is performed by recursively@footnote{The
implementation of the traversal is not purely recursive since a purely
recursive implementation could lead to function call nesting of an
unbounded depth.} traversing objects which are reachable from the final
object of the labeled objects, for example by visiting the slots of
standard objects, and replacing labeled objects with their respective
final object.

In certain cases, the computational complexity of this traversal and
replacement can be rather high, depending on when and how exactly the
traversal is performed: consider an expression of the form
@t{#1=(1 #1# #2=(2 #2# @dots{}))}.  The nested labeled objects in this
expression are all circular and thus require fixing up.  The @t{read}
call for the innermost labeled object, say @t{#100=@dots{}}, returns
first and the fixup processing for the labeled object could be performed
immediately.  The problem is that each of the labeled objects would
be processed in the same manner which would lead to a computation
complexity of @math{O(N M)} where @math{N} is the number of labels and
@math{M} is the number of nodes in the object graph rooted at the object
which is returned by the outermost @t{read} call.  One way to avoid this
problem would be to perform fixup processing only for the outermost
@t{read} call.  The problem with that approach is that only a small
sub-graph of the whole object graph may be circular in which case most
of the work for traversing the whole graph would be wasted.  To address
both problems, @sysname{} allows clients to track the nesting of labeled
objects and fix up sub-graphs which contain multiple nested objects in
one go (see @genref{fixup-graph-p,@readerpackage{}}).

@defgena{call-with-label-tracking,@readerpackage{}} client thunk

This generic function is called by the default method on
@genref{call-as-top-level-read,@readerpackage{}} in order to establish a
context for tracking @t{#=} label definitions and @t{##} label
references around a call to @var{thunk}.

The default method on this generic function establishes a context in
which the default @t{#=} and @t{##} reader macro functions can make the
appropriate calls to @genref{note-labeled-object,@readerpackage{}},
@genref{forget-labeled-object,@readerpackage{}},
@genref{find-labeled-object,@readerpackage{}}.
@end deffn

@defgena{note-labeled-object,@readerpackage{}} client input-stream label@
  parent

This generic function is called by the default @t{#=} reader macro
function to note the definition of a labeled object with label
@var{label} while reading from @var{input-stream}.  The function
creates, registers and returns a representation of the labeled object.
The returned object is registered in the sense that a subsequent call to
@genref{find-labeled-object,@readerpackage{}} with arguments
@var{client} and @var{label} returns the same object unless
@genref{forget-labeled-object,@readerpackage{}} has been called to
unregister the object.

@var{parent} is either @t{nil} or a (previously created) surrounding
labeled object.  The parent labeled object is provided to allow the
client to potentially defer fixup processing for the new labeled object
if the processing for the surrounding labeled object subsumes the
processing for the new labeled object.

Note that, when reading an expression of the form
@t{#@var{N}=@var{object}}, this function is called after reading
@t{#@var{N}=} from @var{input-stream} but before reading
@t{@var{object}}.  Consequently, the created and returned labeled object
is defined but does not have an object associated with it.

The default method on this generic function calls
@genref{make-labeled-object,@readerpackage{}} with @var{client},
@var{input-stream} and @var{label} to create an object of an unspecified
type.  The method registers and returns the created object.  Client code
should manipulate the object only via the generic functions described in
this section and in particular not rely on the object being of a
particular type (since methods on
@genref{make-labeled-object,@readerpackage{}} specialized to certain
client classes could return unexpected objects).  The default method
requires the context established by the default method on
@genref{call-with-label-tracking,@readerpackage{}}.
@end deffn

@defgena{forget-labeled-object,@readerpackage{}} client label

This generic function is called by the default @t{#=} reader macro
function when @sysname{} reads an invalid labeled object of the form
@t{#@var{N}=#@var{N}#} and the caller chooses to recover from the
resulting error (@pxref{Recovering from errors}).  In that situation,
the remainder of the input is processed as if there had been no labeled
object with label @var{N}.  This function makes the labeled object
undefined so that a subsequent
@genref{find-labeled-object,@readerpackage{}} call for @var{label} will
return @t{nil}.

The default method on this generic function requires the context
established by the default method on
@genref{call-with-label-tracking,@readerpackage{}}.
@end deffn

@defgena{find-labeled-object,@readerpackage{}} client label

This generic function is called by the default @t{##} reader macro
function to look up the previously registered representation of a
labeled object for @var{label}.  The function returns @t{nil} if no such
object has been registered for @var{label} and the registered object
otherwise.

The default method on this generic function requires the context
established by the default method on
@genref{call-with-label-tracking,@readerpackage{}}.
@end deffn

@defgena{make-labeled-object,@readerpackage{}} client input-stream label@
  parent

This generic function is called by
@genref{note-labeled-object,@readerpackage{}} to create and return a
representation of a labeled object with label @var{label}.  @var{parent}
is either @t{nil} or a previously created, surrounding labeled object
which allows the client to potentially defer fixup processing for the
new labeled object if the processing for the surrounding labeled object
subsumes the processing.

The default method on this generic function creates and returns an
object of an unspecified type.  Client code should manipulate the object
only via the generic functions
@genref{labeled-object-state,@readerpackage{}},
@genref{finalize-labeled-object,@readerpackage{}} and
@genref{reference-labeled-object,@readerpackage{}} and in particular not
rely on the object being of a particular type (since methods on this
generic function specialized to certain client classes could return
unexpected objects).
@end deffn

@defgena{labeled-object-state,@readerpackage{}} client object

This generic function is called by the default @t{#=} reader macro
function to determine the state of @var{object}.  This function returns

@itemize
@item
@t{nil} if @var{object} is not a labeled object

@item
two values if @var{object} is a labeled object: one of the keywords
@t{:defined}, @t{:circular}, @t{:final}, @t{:final/circular} and the
final object stored in @var{object} if the first value is either
@t{:final} or @t{:final/circular} or @t{nil} otherwise.
@end itemize

The following table lists all possible return value shapes:
@multitable @columnfractions .3 .3 .3
@headitem @var{object} is a labeled object @tab First value         @tab Second value
@item no                                   @tab @t{nil}             @tab
@item yes                                  @tab @t{:defined}        @tab @t{nil}
@item yes                                  @tab @t{:circular}       @tab @t{nil}
@item yes                                  @tab @t{:final}          @tab @var{final-object}
@item yes                                  @tab @t{:final/circular} @tab @var{final-object}
@end multitable

@quotation Note
This generic function may return more values in addition to the ones
described above.  Clients may use this feature to communicate additional
information between methods (@pxref{Parse result construction
features}).  Client defined methods on this generic function should
accept such additional values when calling next methods and themselves
return the additional values.
@end quotation

The default method on this generic function is applicable to labeled
object representations returned by the default methods on
@genref{note-labeled-object,@readerpackage{}} and
@genref{make-labeled-object,@readerpackage{}}.
@end deffn

@defgena{finalize-labeled-object,@readerpackage{}} client labeled-object@
  object

This generic function is called by the default @t{#=} reader macro
function after reading a complete labeled object in order to store
@var{object} in @var{labeled-object} and change the state of
@var{labeled-object} to either @t{:final} or @t{:final/circular}.  The
function returns two values: the finalized @var{labeled-object} and the
new state of @var{labeled-object}.

The default method on this generic function is applicable to labeled
object representations returned by the default methods on
@genref{note-labeled-object,@readerpackage{}} and
@genref{make-labeled-object,@readerpackage{}}.
@end deffn

@defgena{reference-labeled-object,@readerpackage{}} client input-stream@
  labeled-object

This generic function is called by the default @t{##} reader macro
function to process a reference to @var{labeled-object} while reading
from @var{input-stream}.  @var{labeled-object} must be a representation
of a labeled object and has, in the context of the @t{##} reader macro
function, likely been obtained by calling
@genref{find-labeled-object,@readerpackage{}}.  Depending on the state
of @var{labeled-object}, this function returns either
@var{labeled-object} itself or an object that can be returned to the
caller as-is.  In case @var{labeled-object} is returned, it will be
replaced by its associated object later, when
@genref{fixup-graph,@readerpackage{}} is called.

The default method on this generic function is applicable to labeled
object representations returned by the default methods on
@genref{note-labeled-object,@readerpackage{}} and
@genref{make-labeled-object,@readerpackage{}}.
@end deffn

As briefly mentioned above, the generic functions
@genref{fixup-graph,@readerpackage{}} and
@genref{fixup,@readerpackage{}} traverse and inspect objects in the
object graph reachable from an object that is about to be returned to
the caller of @funref{read,@readerpackage{}}.  In order to distinguish
ordinary objects from labeled objects that act as placeholders in the
object graph and must be replaced with their respective final objects,
@genref{fixup,@readerpackage{}} methods call
@genref{labeled-object-state,@readerpackage{}} on all encountered
objects.  @genref{labeled-object-state,@readerpackage{}} returns @t{nil}
for all objects that are not labeled objects and @t{:final} for labeled
objects which must be replaced with their final object.

@defgena {fixup-graph-p,@readerpackage{}} client root-labeled-object

This generic function is potentially called by a method on
@genref{finalize-labeled-object,@readerpackage{}} to determine whether
the object graph reachable from the object of @var{root-labeled-object}
should be fixed up by calling @genref{fixup-graph,@readerpackage{}} with
@var{client} and @var{labeled-object}.

Multiple default methods on this generic function jointly implement the
following behavior:

@itemize

@item
If @var{root-labeled-object} has a parent labeled object,
@var{root-labeled-object} should not be fixed up immediately (since the
fixup processing for ancestor labeled objects will subsume the fixup
processing for @var{root-labeled-object}).

@item
If @var{root-labeled-object} does not have a parent labeled object but
has child labeled objects, @var{root-labeled-object} should be fixed up
immediately.

@item
If @var{root-labeled-object} does not have a parent labeled object and
is in state @t{:final/circular}, @var{root-labeled-object} should be
fixed up immediately.
@end itemize
@end deffn

@defgena {fixup-graph,@readerpackage{}} client root-labeled-object@
  @pkey object-key

This generic function is potentially called after the reader has
constructed an object graph which is reachable from the object of
@var{root-labeled-object} and noticed circular references within this
graph to fix up circular references before the object of
@var{root-labeled-object} is returned to the caller (of
@funref{read,@readerpackage{}} or related functions).

@var{object-key} is a function that accepts a labeled object and returns
the object of the labeled object.

The default method on this generic function creates a hash table for
tracking already processed objects and calls
@genref{fixup,@readerpackage{}} with @var{client}, the object of
@var{root-labeled-object} and the hash table to recursively process
objects in the object graph which is reachable from the object of
@var{root-labeled-object}.
@end deffn

@defgena {fixup,@readerpackage{}} client object traversal-state

This generic function is potentially called to apply circularity-related
changes to the object constructed by the reader before it is returned to
the caller.  @var{object} is the object that should be modified.
@var{traversal-state} is an unspecified object that is used, among other
things, to track already processed objects (see below).  A method
specialized to a class, instances of which consist of parts, should
modify @var{object} by scanning its parts for labeled object markers,
replacing found labeled object markers with the respective final object
and recursively calling @genref{fixup,@readerpackage{}} for all parts.

To recognize labeled objects which have to be replaced, methods should
call @genref{labeled-object-state,@readerpackage{}} on each part of
@var{object} and interpret the returned values as follows: if @t{nil} is
returned, the part should not be replaced but recursively processed.  If
@t{:final} is returned as the first value, the part should be replaced
with the final object that is returned as the second value.  Parts are
replaced by mutating @var{object}.

This generic function is called for side effects -- its return value is
ignored.

Default methods specializing the @var{object} parameter to
@speclink{t,cons}, @speclink{t,array}, @speclink{t,standard-object} and
@speclink{t,hash-table} process instances of those classes by calling
@genref{new-value-for-fixup,@readerpackage{}} for each ``place'' in
@var{object} and storing the returned object in the ``place''.

An unspecialized @t{:around} method queries and updates
@var{traversal-state} to ensure that each object reachable from
@var{object} is processed exactly once.  The method also limits the
depth of nested function invocations (but importantly not the nesting
depth or structure of the processed object graph, in fact the opposite:
the limit on function invocation nesting is a consequence of allowing
arbitrary object graphs to be processed).

@quotation Warning
Due to the limit on function call nesting, methods on this generic
function must not rely on the traversal from the root object to
@var{object} resulting in an uninterrupted chain of nested function
calls.  In particular, bindings of special variables and
@speclink{f,unwind-protect} cleanups that are established in a call
which processes one object may not be in scope in calls which process
objects that are logically at a deeper nesting level.
@end quotation
@end deffn

@defgena {new-value-for-fixup,@readerpackage{}} client labeled-object @
  current-value final-value

This generic function is called by the generic function
@genref{fixup,@readerpackage{}} when a new value for a ``place'' within
a circular object must be produced.  This function is used when fixup
processing is applied to both ordinary objects and parse results.

@var{labeled-object} is the labeled object marker which will be
replaced by the computed value to finalize the circular object.

@cindex parse result
@var{current-value} is the current value of the ``place'' in the
circular object.  The value of this parameter is the same as
@var{labeled-object} when fixup processing is applied to an ordinary
object but is different when fixup processing is applied to a parse
result.  In the latter case, @var{current-value} is a parse result and
@var{labeled-object} is the is ordinary object represented by the parse
result.

@var{final-value} is the ordinary object that should replace
@var{labeled-object} in the circular ordinary object regardless of
whether fixup processing is performed on ordinary objects or parse
results.  In the latter case, methods on this generic function must
return a suitable parse result that represents @var{final-value} and can
replace @var{current-value}.

The default method on this generic function simply returns
@var{final-value}.

@cindex concrete syntax tree
A method specialized to @t{(client cst-client)} and @t{(current-value
cst:cst)} returns a CST that represents @var{final-value}.  Depending on
the client, the returned CST is either another occurrence of the CST
that represents the definition of @var{final-value} or a new CST that
explicitly represents the labeled object reference.
@end deffn

@node S-expression creation
@subsection S-expression creation

@cindex quotation

The following generic functions allow clients to construct
representations of quoted and quasiquoted forms as well as
@speclink{f,function} special forms.

@defgena {wrap-in-quote,@readerpackage{}} client material

This generic function is called by the default @t{'}-reader macro
function to construct a quotation form in which @var{material} is
the quoted material.

The default method on this generic function returns a result
equivalent to @code{(list 'common-lisp:quote @var{material})}.
@end deffn

@cindex quasiquotation

@cindex quasiquote
@cindex backquote

@defgena {wrap-in-quasiquote,@readerpackage{}} client form

This generic function is called by the default @t{`}-reader macro
function to construct a quasiquotation form in which @var{form} is
the quasiquoted material.

The default method on this generic function returns a result
equivalent to @code{(list '@readerpackage{}:quasiquote @var{form})}.
@end deffn

@cindex unquote

@defgena {wrap-in-unquote,@readerpackage{}} client form

This generic function is called by the default @t{,}-reader macro
function to construct an unquote form in which @var{form} is the
unquoted material.

The default method on this generic function returns a result
equivalent to @code{(list '@readerpackage{}:unquote @var{form})}.
@end deffn

@defgena {wrap-in-unquote-splicing,@readerpackage{}} client form

This generic function is called by the default @t{,@@}-reader macro
function to construct a splicing unquote form in which @var{form} is the
unquoted material.

The default method on this generic function returns a result
equivalent to
@code{(list '@readerpackage{}:unquote-splicing @var{form})}.
@end deffn

@cindex function

@defgena {wrap-in-function,@readerpackage{}} client name

This generic function is called by the default @t{#'}-reader macro
function to construct a form that applies the @t{function} special
operator to the @var{name} expression.

The default method on this generic function returns a result equivalent
to @code{(list 'common-lisp:function @var{form})}.
@end deffn

@node Readtable initialization
@subsection Readtable initialization

@cindex readtable

The standard syntax types and macro character associations used by the
ordinary reader can be set up for any readtable object implementing the
readtable protocol (@pxref{Readtable features}).  The
following functions are provided for this purpose:

@deffuna {set-standard-syntax-types,@readerpackage{}} readtable

This function sets the standard syntax types in @var{readtable} (See
@uref{https://novaspec.org/cl/2_1_Character_Syntax#sec_2_1_4, Section
2.1.4 Character Syntax Types in the @commonlisp{} specification}).
@end deffn

@deffuna {set-standard-macro-characters,@readerpackage{}} readtable

This function sets the standard macro characters in @var{readtable} (See
@uref{https://novaspec.org/cl/2_4_Standard_Macro_Characters,Section 2.4
Standard Macro Characters in the @commonlisp{} specification}).
@end deffn

@deffuna {set-standard-dispatch-macro-characters,@readerpackage{}}@
  readtable

This function sets the standard dispatch macro characters, that is
sharpsign and its sub-characters, in @var{readtable} (See
@uref{https://novaspec.org/cl/2_4_Standard_Macro_Characters#sec_2_4_8,Section
2.4.8 Sharpsign in the @commonlisp{} specification}).
@end deffn

@deffuna {set-standard-syntax-and-macros,@readerpackage{}} readtable

This function sets the standard syntax types and macro characters in
@var{readtable} by calling the above three functions.
@end deffn

@node Readtable features
@section Readtable features

@cindex readtable

In this section, symbols written without package marker are in the
@t{@rtpackage{}} package (@pxref{Package for readtable features}).

This package exports two kinds of symbols:

@enumerate
@item
Symbols the names of which correspond to the names of symbols in the
@t{common-lisp} package.  The functions bound to these symbols are
generic versions of the corresponding standard @commonlisp{} functions.
Clients can define custom readtables by defining methods on these
generic functions.

@item
Symbols bound to additional functions and condition types.
@end enumerate

@defgena {readtablep,@rtpackage{}} object

This function is the generic version of the standard @commonlisp{}
function @speclink{f,readtablep}.  The function returns @emph{true} if
@var{object} can be used as a readtable in @sysname{} via the protocol
functions in the @t{@rtpackage{}} package.  The default method returns
@t{nil}.
@end deffn

TODO

@node Parse result construction features
@section Parse result construction features

@cindex client
@cindex parse result

In this section, symbols written without package marker are in the
@t{@prpackage{}} package (@pxref{Package for parse
result construction features}).

This package provides clients with a reader that behaves similarly to
@speclink{f,read} but returns custom parse result objects controlled by
the client.  Some parse results correspond to things like symbols,
numbers and lists that @speclink{f,read} would return, while others, if
the client chooses, represent comments and other kinds of input that
@speclink{f,read} would discard.  Furthermore, clients can associate
source location information with parse results.

Clients using this package pass a ``client'' object for which methods on
the generic functions described below are applicable to
@funref{read,@prpackage{}},
@funref{read-preserving-whitespace,@prpackage{}} or
@funref{read-from-string,@prpackage{}}.  Suitable client classes can be
defined by using @classref{parse-result-client,@prpackage{}} as a
superclass and at least defining a method on the generic function
@genref{make-expression-result,@prpackage{}}.

When a client constructs parse results, some of the generic functions
for customizing the behavior of the reader (@pxref{Reader behavior
protocol}, @pxref{Labeled objects and references}) return additional
values:

@multitable @columnfractions .25 .15 .3 .3
@headitem Generic function                             @tab Situation    @tab Ordinary values                   @tab Extended values

@item @genref{call-as-top-level-read,@readerpackage{}} @tab object       @tab @emph{object}                     @tab @emph{object}, @emph{parse result}, @emph{orphan results}
@item @genref{read-common,@readerpackage{}}            @tab object       @tab @emph{object}                     @tab @emph{object}, @emph{parse result}
@item @genref{read-maybe-nothing,@readerpackage{}}     @tab object       @tab @emph{object}, @emph{kind}        @tab @emph{object}, @emph{kind}, @emph{parse result}

@item @genref{call-as-top-level-read,@readerpackage{}} @tab end of input @tab @var{eof-value}                   @tab @var{eof-value}, @emph{orphan results}
@item @genref{read-common,@readerpackage{}}            @tab end of input @tab @var{eof-value}                   @tab @var{eof-value}
@item @genref{read-maybe-nothing,@readerpackage{}}     @tab end of input @tab @var{eof-value}, @t{:eof}         @tab @var{eof-value}, @t{:eof}
@item @genref{labeled-object-state,@readerpackage{}}   @tab @emph{any}   @tab @emph{state}, @emph{final object} @tab @emph{state}, @emph{final object}, @emph{parse result}, @emph{inner labeled object}
@end multitable

Note how @genref{call-as-top-level-read,@readerpackage{}} and
@genref{read-common,@readerpackage{}} return fewer values for the ``end
of input'' situation.  This difference in return value count allows the
caller to recognize the ``end of input'' situation even if
@var{eof-value} is an object that could be read such as @t{nil}.  Using
such an @var{eof-value} makes sense for clients which construct parse
results since top-level @funref{read,@prpackage{}} calls return these
parse results so that there is no risk of confusing the chosen
@var{eof-value}, even if something like @t{nil}, with having read a
similar object.

@float Figure,fig:read-call-sequence-parse-result
@center @image{read-call-sequence-parse-result,15cm}
@caption{Functions and typical function call sequences.  Solid arrows
represent calls, dashed arrows represent returns from function calls.
Labels above arrows represent arguments and return values.  Differences
from the non-parse result version are highlighted with bold text.}
@end float

@ref{fig:read-call-sequence-parse-result} shows typical function call
patterns, including ordinary and additional return values, that arise
when the functions @funref{read,@prpackage{}},
@funref{read-preserving-whitespace,@prpackage{}},
@funref{read-from-string,@prpackage{}}
@c and @funref{read-delimited-list,@prpackage{}}
are called by client code.

@deffuna {read,@prpackage{}} client @optional@
  (input-stream @t{*standard-input*})@
  (eof-error-p @t{t})@
  (eof-value @t{nil})

This function is the main entry point for this variant of the reader.
It is in many ways similar to the standard @commonlisp{} function
@speclink{f,read}.  The differences are:

@itemize
@item
A client instance must be supplied as the first argument.

@item
The first return value, unless @var{eof-value} is returned, is an
arbitrary parse result object created by the client, not generally the
read object.

@item

@cindex orphan result
The second return value, unless @var{eof-value} is returned, is a list
of ``orphan'' results.  These results are return values of
@genref{make-skipped-input-result,@prpackage{}} and arise when skipping
input at the toplevel such as comments which are not lexically contained
in lists: @t{#|orphan|# (#|not orphan|#)}.

@item
The function does not accept a @var{recursive} parameter since it sets
up a dynamic environment in which calls to
@funref{read,@readerpackage{}} behave suitably.
@end itemize
@end deffn

@deffuna {read-preserving-whitespace,@prpackage{}} client@
  @optional (input-stream @t{*standard-input*})@
  (eof-error-p @t{t})@
  (eof-value @t{nil})

This function is similar to the standard @commonlisp{} function
@speclinkl{f,read,cl:read-preserving-whitespace}.  The differences are
the same as described above for @funref{read,@prpackage{}} compared to
@speclink{f,read}.
@end deffn

@deffuna {read-from-string,@prpackage{}} client string@
  @optional@
  (eof-error-p @t{t})@
  (eof-value @t{nil})@
  @pkey@
  (start @t{0})@
  (end @t{nil})@
  (preserve-whitespace @t{nil})

This function is similar to the standard @commonlisp{} function
@speclink{f,read-from-string}.  The differences are:

@itemize
@item
A client instance must be supplied as the first argument.

@item
The first return value, unless @var{eof-value} is returned, is an
arbitrary parse result object created by the client, not generally the
read object.

@item

@cindex orphan result
The @emph{third} return value, unless @var{eof-value} is returned, is a
list of ``orphan'' results (Described above).
@end itemize
@end deffn

@defclassa {parse-result-client,@prpackage{}}

This class should generally be used as a superclass for client classes
using this package.
@end deftp

@defgena {make-expression-result,@prpackage{}} client result@
  children source

This generic function is called in order to construct a parse result
object.  The value of the @var{result} parameter is the raw object read.
The value of the @var{children} parameter is a list of already
constructed parse result objects representing objects read by recursive
@t{read} calls.  The value of the @var{source} parameter is a source
range, as returned by @genref{make-source-range,@basepackage{}} and
@genref{source-position,@basepackage{}} delimiting the range of
characters from which @var{result} has been read.

This generic function does not have a default method since the purpose
of the package is the construction of @emph{custom} parse results.
Thus, a client must define a method on this generic function.
@end deffn

@defgena {make-skipped-input-result,@prpackage{}} client@
  stream reason children source

This generic function is called after the reader skipped over a range of
characters in @var{stream}.  It returns either @t{nil} if the skipped
input should not be represented or a client-specific representation of
the skipped input.  The value of the @var{children} parameter is a list
of already constructed parse result objects which represent object read
by recursive @t{read} calls (Such as the feature expression and the
ignored expression in @t{#+(and (or) some-feature) skipped-expression}).
The value of the @var{source} parameter designates the skipped range
using a source range representation obtained via
@genref{make-source-range,@basepackage{}} and
@genref{source-position,@basepackage{}}.

Reasons for skipping input include comments, the @t{#+} and @t{#-}
reader macros and @link{aspect-read-suppress,@t{*read-suppress*}}.  The
aforementioned reasons are reflected by the value of the @var{reason}
parameter as follows:

@multitable @columnfractions .5 .5
@headitem Input                            @tab Value of the @var{reason} parameter
@item Comment starting with @t{;}          @tab @t{(:line-comment . 1)}
@item Comment starting with @t{;;}         @tab @t{(:line-comment . 2)}
@item Comment starting with @math{n} @t{;} @tab @t{(:line-comment . @math{n})}
@item Comment delimited by @t{#|} @t{|#}   @tab @t{:block-comment}
@item @t{#+@emph{false-expression}}        @tab @t{(:sharpsign-plus . @emph{false-expression})}
@item @t{#-@emph{true-expression}}         @tab @t{(:sharpsign-minus . @emph{true-expression})}
@item @t{*read-suppress*} is true          @tab @t{*read-suppress*}
@item A reader macro returns no values     @tab @t{:reader-macro}
@end multitable

The default method returns @t{nil}, that is the skipped input is
not represented as a parse result.
@end deffn

@node CST reader features
@section CST reader features

@cindex parse result
@cindex concrete syntax tree

In this section, symbols written without package marker are in the
@t{@cstpackage{}} package (@pxref{Package for CST features}).

@deffuna {read,@cstpackage{}} @optional@
  (input-stream @t{*standard-input*})@
  (eof-error-p @t{t})@
  (eof-value @t{nil})

This function is the main entry point for the CST reader.  It is
mostly compatible with the standard @commonlisp{} function
@speclink{f,read}.  The differences are:

@itemize
@item
The return value, unless @var{eof-value} is returned, is an instance of
a subclass of @t{concrete-syntax-tree:cst}.

@item
The function does not accept a @var{recursive} parameter since it sets
up a dynamic environment in which calls to
@funref{read,@readerpackage{}} behave suitably.
@end itemize
@end deffn

@deffuna {read-preserving-whitespace,@cstpackage{}}@
  @optional (input-stream @t{*standard-input*}) (eof-error-p @t{t})@
  (eof-value @t{nil})

This function is similar to the standard @commonlisp{} function
@speclinkl{f,read,cl:read-preserving-whitespace}.  The differences are
the same as described above for @funref{read,@cstpackage{}} compared to
@speclink{f,read}.
@end deffn

@deffuna {read-from-string,@cstpackage{}} string@
  @optional@
  (eof-error-p @t{t})@
  (eof-value @t{nil})@
  @pkey@
  (start @t{0})@
  (end @t{nil})@
  (preserve-whitespace @t{nil})

This function is similar to the standard @commonlisp{} function
@speclink{f,read-from-string}.  The differences are the same as
described above for @funref{read,@cstpackage{}} compared to
@speclink{f,read}.
@end deffn
