Representing, Similarity, and the Storage
of Information
David Banach
Representing is an activity, a
process through which a subject cognizes the world. Most theories of
representation take one element or component of this activity and identify it
as the representation by attributing to that element, in isolation, properties
it has only in the context of the act of representing. The purpose of this
paper is to argue that such views are false and that representation can only
play its normal roles in cognition if it is seen as an activity or process
rather than a static element in the cognitive process.
I will look at three basic views of
representation and argue that they fail because they abstract out one component
of the representing process and try to explain how this component can play its
role in cognition in virtue of its intrinsic properties. First, I will consider
views of iconic representation: views that see representations as objects that
represent in virtue of their similarity to the object they represent. I will
argue that similarity is neither sufficient nor necessary for representation,
although it can play an important role in representation. Second, I will look
at views of symbolic representation: views that see objects as representing in
virtue of some isomorphism effected by a function that maps elements of the
representation onto elements of its object. I will take language and
information storage as the paradigmatic examples of this type of representation
and will argue that neither the objects that have the isomorphism nor the
functions that do the connecting (nor the physical embodiments of these
functions) can be seen as representations. Third, the views of representation
involved in Parallel Distributed Processing models and in Paul Churchland's
analysis of neural networks as phase space sandwiches will be considered. These
views identify representation as a pattern of activation distributed over a
network or a phase space that is connected to other representations through a
network of weighted connections. I will argue that, although these views
properly analyze the structure of the representing process, they mistakenly
identify a single part of this process as the representation. Finally, I will
try to suggest what a view of representation that avoids these mistakes might
look like and how it might solve some of the puzzles about the role of
representation in cognition that these mistakes tend to generate. I will argue
that representing is a process of connecting two different modes of interaction
with the world and that its content, and its role in cognition, lies in the
connections made within this process, not in the two things connected, nor in
the function that connects them.
I. Iconic Representation
The simplest, and perhaps the most
fundamental, view of representation is the iconic, which views a representation
as an object that represents in virtue of its similarity to its object. The
epistemological difficulties with such a view are commonplace. (How is the
similarity between the representation and the object ascertained? Which of the
many similarities are important and how are they picked out?) I aim to point
out a more fundamental problem, which I shall argue gives rise to the
epistemological problems: Similarity is not essentially connected to
representation. It is neither sufficient nor necessary for representation.
Similarity is not sufficient for
representation. This point seems obvious. (Although the reason why it is true
is, perhaps, not so obvious.) Not all things that bear a similarity to one
another are representations of one another. A leaf and the flame emitted from
burning copper wire are similar in color, yet no one would take one as
representing the other. Even in cases that are more plausible, such as the
resemblance of a photograph to its subject, if similarity is sufficient for
representation then one could equally claim that the subject is a
representation of the photograph. As Putnam has pointed out (Putnam 1981, pp.
1-5) a physical object cannot represent an object simply in virtue of its
similarity to it. If an ant traced out a perfect caricature of Winston
Churchill in its wanderings, we would not say that the ant has produced a
representation of Churchill. Why not, and why would anyone think it plausible
to say that an iconic representation represents in virtue of its similarity to
the object?
We do not recognize the trail of the
ant or the green leaf as representations because we realize that these things
require interpretation by an observer with certain abilities and knowledge in
order to serve as representations. They don't do it themselves, in virtue of
their similarity to certain objects. In order to represent an object, the icon
must call up other properties of the object
besides those to which it is similar. The lines in the sand must call to
mind the three dimensional rotund shape or the other features we happen to
associate with Churchill. The icon cannot do this itself. It requires an
interpreter with certain abilities, knowledge, and associations to connect the
properties of the icon that are similar
to the other properties of the object.
(Witness the dog who passes by the life-like statue in the museum
without ever seeing it as representing a man.)
The argument can be put in a
slightly more rigorous form. A representation must call up more properties than
those to which it is similar. It cannot simply represent the properties that it
shares with its object. This is true for two reasons: (1) It does not represent
these properties; it instantiates them.[1] A yellow disk as an icon for the sun
does not represent just the yellowness and
roundness of the sun, it is yellow and it is round. It can only be said
to represent the sun if it calls up other properties besides the particular
yellowness and roundness that it possesses, and these it cannot call up on its
own, simply in virtue of its similarity to the sun. This requires an
interpreter who knows these other properties and associates them with the
properties instantiated in the yellow disk. (2) If icons represent in virtue of
their similarity then they can never represent individuals, but only universal
sets of properties. No icon is similar in all respects to the object it
represents. (Even if it were, it would not represent its object; it would be it.) Hence, it cannot represent only
that one object, but also every other object and possible object to which it is
similar. If the yellow disk represents merely in virtue of similarity, then it
cannot represent the sun; it represents all yellow, round objects. [2]
Hence, similarity alone cannot be
sufficient for representation. Icons represent only in virtue of calling up other
properties of the object besides those to which they are similar, and they
cannot do this apart from the interpreter with certain abilities, knowledge,
and dispositions. Why did anyone think they could? The answer to this question
is revealing with respect to the epistemological problems that arise from these
views of representation. The problem arises, I believe, from taking certain
types of physical icons as a paradigm of representation, incompletely analyzing
how they work, and then applying this analysis to the paradigmatic mental
representations, visual images. Hence, I call this view the Physical-Visual
Model of Representation.
Imagine looking at an object and a
physical representation of that object. Take, for example, a statue and the
woman it was modelled after. In this case both the object and the
representation are contained in the same perception. We perceive their similarities,
and we can perceive the correspondences between the statue and the women. This
allows us to see, for example, how the elbow of ivory maps onto the elbow of
flesh. Thus we can take the ivory as representing the flesh; we project the
properties of the woman onto the statue guided by the perceived similarities.
The statue represents the woman only in virtue of the complete interpretation
of the situation by the observer and the access that he has to both the statue
and the woman. This is a paradigmatic
case of physical representation, but it is one that is bound to be incompletely
analyzed. It is natural to leave out the part that the observer plays in this
situation and attribute the representative qualities of the statue to its
similarity to the woman. After all, it is the similarities that guide the
projection of the properties of the woman onto the statue. The fact that this
projection is an act of the observer, dependent upon his ability to interact
perceptually with both the statue and the woman, is easily overlooked. It is
this simplified analysis of physical representation that is taken as the
paradigm and applied to mental representation.
The mental image (or the piece of
language in modern theories) is seen as a representation or picture. It is
thought to function autonomously, apart from the action of an interpreting
subject and its interaction with an external object, just as a physical
representation seems to. The mental representation is seen as a mental object
that represents in virtue of its own properties. Instead of seeing
representation as a process of interaction between subject and object, the
representation is reified into an object and the visual perspective on the
external world turned into a view of this mental object, not a view of the
external world. Through a perspective
we see only the reified representation, not the world; thus, mental
representation becomes a veil of ideas separating the subject from any
independent access to the external object. If the mental image is seen as representing merely in virtue of its
similarity and as the only medium
though which we see reality, it is obvious that we can never determine whether
our representation is really similar to external reality. This is where the
epistemological problems that attend this view of representation arise. More
importantly, it is also clear that it is also impossible for such mental images
to serve as representations of external reality. Since an icon can only
represent by calling up other properties of the object besides those to which
it is similar, and since this can be done only by an interpreter who has access
to the object and its other properties apart from the representation (as is the
case of physical icons), the physical-visual model of representation makes
mental representation impossible.[3]
II. Symbolic Representation and the
Storage of Information
If similarity is not sufficient for
representation, neither is it necessary. It seems that many forms of symbolic
representation involve no similarity (in any obvious sense) between the symbol
and what it represents. When I use X's and O's to represent football players, it
is not similarity, in any normal sense, that allows these symbols to represent.
Rather, there are a set of conventions or rules that allow one to move from the
symbol to its meaning.
Obviously, language is a
paradigmatic example of symbolic representation. This type of representation is
also often attributed to various types of physical systems that encode
information. This involves taking information and placing it in a new medium in
which it takes another form and from which it can be retrieved. Examples of
this are the storage of a visual image on a photographic negative, the storage
of information on computer disks, the
reproduction of music on tapes and on compact disks, and the encoding of
genetic information on DNA strands. All of these are popular models for
representation. It seems in all of these as if the information is stored by
representing it in a different form and then using that representation to guide
the reproduction of the original information. It is attractive to claim that
the song is on the compact disk,
that the words are on the computer
disk, represented in a different way to allow for convenient storage. It is
attractive to claim that information storage involves the representation of the
information.
It is not clear why. These types of
systems are primarily involved in reproduction of various images, sounds, and
organisms, rather than any representation of the type involved in cognition.
Perhaps a type of personification is involved. We know that we could not
reproduce an image or a sound unless we had a good representation of it that
guided our activity. Hence, it is plausible to see the pattern of marks on a
compact disk through which the player reproduces the original sound as
analogous to the mental representation or blueprint that guides the builder's
reproduction of a house. It is not at all clear why one should assume that the
reproduction of certain physical forms in such machines must be mediated by a
representation. Nor is it any more clear that the various physical processes
through which human beings engage in similar types of reproduction (e.g.,
memory) involve the mediation of physical objects or states that are stored
representations of the reproduced form.[4]
I will argue that information storage and retrieval does not involve
representation. Computer disks, tapes, compact disks, and configurations of
neurons in the brain do not contain representations. They are components
(modular and, sometimes, portable and interchangeable components) of physical
systems that reproduce certain sounds, objects, or patterns of activity, but
reproduction is not representation, nor does it require representation.[5]
A proponent of the iconic view of
representation might argue that the compact disk and its pattern of marks does
represent or contain the song in virtue of an isomorphism, or highly abstract
similarity of form between the pattern and the song. Of course this is just a
more sophisticated version of the iconic view that things represent simply in
virtue of similarity and is prey to the same arguments given above. But it is
instructive to see in some detail how those arguments can be applied in this
case.
Note that the position stated above
actually involves two separate possibilities: (1) that the disk actually
contains the song itself; or (2) that the disk contains a representation of the
song. In the first case it might be claimed that the song is actually an
abstract set of functional relations that are also present in the patterns of
marks on the disk. The song, which is
the abstract pattern is both in the sounds and on the disk. Note that this is a
case of complete similarity: the pattern of marks on the disk does not
represent the song it is the song.
Again, an icon cannot represent the
traits that it shares with the object; it simply instantiates those traits.
Even this claim of identity is
implausible in the case of the compact disk for two reasons: (1) Even if the
song were merely an abstract functional pattern, it is not clear that the disk
shares this pattern in any clear sense. The song exists as a pattern of sounds
in time. The pattern on the disk cannot instantiate these temporal relations.
More likely, the reproduction of the sound, or even the pattern of electrical
impulses that cause the reproduction in the amplifier or speaker, instantiate this pattern. (Hence, we believe
the reproduction is the same song as the original, not a representation.) (2) More importantly, the isomorphism that
the pattern on the disk bears to the song is completely relative to the program
or set of functions in the disk player that can reproduce the song from the
patterns on the disk. To say that there is an isomorphism between the two is to
say nothing more than that there exits or could exist an algorithm that could
map the elements of one onto the elements of the other in some determinate way.
Of course this is true of just about any two objects, given an intricate enough
algorithm.[6] Once one sees that there is nothing
unique or special about the pattern on the disk (apart from the fact that it
might be especially compact or economical) that sets it apart from the infinite
number of other patterns that, given the right algorithm or type of player,
could also reproduce the song, it is much less plausible to take the pattern as
being the song, or even as
representing it.
Of course, the above discussion of
the possibility of the pattern on the disk being the song is irrelevant to the
question of whether such information storage systems involve representation: If
it is the song, it doesn't represent
it. Some of the same arguments, however, will apply to the thesis that the
pattern on the disk represents the song. They are really just versions of the
arguments presented above against the thesis that icons represent in virtue of
their similarity. To represent, the pattern on the disk must be able to call up
the other properties of the song, besides those it possesses; it must be able
to reproduce the song. This it cannot do by itself in virtue of its similarity
to the song. It can do this only as
part of a causal system, the player, which embodies a function that can
reproduce the other properties of the song from the pattern on the disk. Again,
an analogue to an interpreter with specific abilities and dispositions is
required. The song itself, or a representation of it, is not present and intact
in the new medium, to which anyone who has had a computer disk without the
appropriate software to read it can attest. The very same physical object with
all the same properties would not be an encoding of information if the causal
system, in this case compact disk players did not exist. The very same pattern
that, on a compact disk, encodes MC Hammer's "You can't Touch This"
might have existed on the etchings of a meteor in the orbit of Mars before the
existence of human beings. Yet, it stretches the imagination, to say the least,
to claim that this would have been a representation of the croonings, or
rappings, of MC Hammer.
Further, the very same pattern might
allow a reproduction of many other songs given a different algorithm or type of
player. The isomorphism that the pattern on the disk possesses is hopelessly
indeterminate. It could be isomorphic to almost any song given an intricate
enough algorithm that would allow the mapping of the elements of the song onto
the elements of the pattern on the disk. If it is a representation at all in
virtue of its intrinsic properties, it is a hopelessly indeterminate or
universal one.
A causal system that reproduces
information does not represent. It may be useful to an interpreter who can
interpret the reproduced information and use it to represent, but it does not
do the job itself. A system that reproduces objects does not represent them,
nor does it require representations of them. A chair factory does not represent
chairs; it makes them. Thus, a system for encoding information does not itself
represent, although it may be useful for a creature that can represent.
III. PDP and Phase Space Sandwich Models
Parallel Distributed Processing and
Paul Churchland's phase space sandwich interpretation of neural networks
provide analyses in which representation is presented as firmly embedded in a
physical system that is capable of calling up or connecting the representation
with the other properties of the object. But, perhaps taken in by the allure of
the iconic model of representation, they identify only a part of this process
as the representation. I will argue that they correctly identify the essential
features of the representing process, but that they mis-identify the parts of
that process that are the representations.
PDP models (Rumelhart 1987) identify
distributed patterns of activation over a matrix of units as representations.
Each such set of units is connected to others by sets of weighted connections
by which a representation, or pattern of activation, on one set of units is
transformed into another pattern, a different representation, on the other set
of units. For example, a set of units connected to smell receptors might be
connected by a set of weighted connections to a set of units in the visual part
of the brain. Input from the nose might stimulate a certain pattern of
activation in the olfactory units; this might be identified with the
representation or olfactory image of the smell of a rose. The connections to
the visual units pass on this pattern transforming it into another pattern
along the way, which might be identified with the representation, or visual
image, of the rose. Thought might be conceived as the transformation of one
representation into another effected by these weighted connections.
The representation is identified as
a particular pattern or configuration of activation over certain units. Why?
The pattern does bear a certain similarity or isomorphism to the object it
arises from and to the pattern it will give rise to in the other set of units.
It is exactly analogous to the pattern on the disk that is situated in an
information reproduction system: Memory is a system through which images are
reproduced according to the information "represented" in the input
patterns according to the algorithms embodied in the weighted connections. But
we have already seen what is wrong with those two models of representation.
In Churchland's phase space sandwich
model (Churchland 1986, 1987), we have large layers of interconnected neurons
instead of the smaller sets of units in PDP models. In between these layers are
sandwiched sets of interconnections between the two layers. For example, the
input from the eyes might produce a certain pattern of activation on a layer of
neurons in the visual cortex. This might represent some object in visual phase
space. The pattern in the phase space might even be similar to the object; it
might be a topological transformation of it. The visual cortex might be
connected to the motor cortex by a set of connections that embody a tensor
function that transforms the pattern in the visual space into one in the motor
space. This pattern represents a position or a movement in motor phase space
that directs the motions of some limb. Representations are positions or
patterns in the phase space, ones that are actually visually similar
topological transformations of the shape of the objects they represent.
(Churchland 1987, pp. 279-280) Computation is the coordinate transformation of
the representation on one phase space into another by means of a tensor
function embodied in a set of connections sandwiched between the two layers.
Philosophers, since the time of
Berkeley, have been poking fun at the idea that there are little pictures or
images of objects in the head, and now it turns out that there actually are
little topologically transformed icons in our brains. The motivation for
identifying these as representations is clear; hopefully, as clear as the
problems with doing so. It is not these patterns that represent merely in
virtue of their similarity to some objects. We have already seen how icons and
isomorphic patterns involved in information reproduction systems cannot be
representations
Even if one recognizes that the
patterns of activation in sets of units or in layers of neurons cannot serve as
representations simply in virtue of their similarity or isomorphism to the
objects they represent, one might still be tempted to identify some other part
of the process as the representation. In particular, since it seems that it is
the connections between the units or layers that actually do all the work, it
might be plausible to see them as representations. It is these in which our
knowledge is stored, these that allow us to remember, these that allow us to
coordinate our motor activities with our visual perceptions. These might be
seen as concepts, embodied dispositions to connect certain images, actions, or
modes of interaction with the world. Kant's view of concepts as rules by which
we connect or synthesize our experience seems similar to this view of
representation. On this view,
representations are not the icons or images that we form, but the
embodied dispositions or sets of connections that allow us to form these images
in our mind. The representation is not the statue the artist makes, but the
concept through which the artist creates it, or the concept through which the
audience interprets it.
This view, however, cannot be
correct. Concepts, or sets of connections that effect the transformation of one
pattern to another, cannot be representations. Representations serve two
cognitive functions: (1) They present us with information, or images of the
world.
Re-presentations are still presentations. A representation must be present to
us in a way that re-presents the
world to us. (2) Representations are the bearers of knowledge. Representations
are that through which we know; they must be capable of being true or
false. Concepts, so interpreted, can do
neither.
The neural connections that
associate the smell of a rose with a visual image of a rose are not present to
us. We don't see them in any sense, and they don't re-present to us any
information. We see the effects of these connections, but we don't see them. If they are not present to us,
they cannot re-present anything to us. (You can see why it was more plausible
to identify the patterns of activation as the representations: they, at least,
are present to us through the mental images that are associated (or identical)
to them.) Note that the problem here is not directly connected to the
relationship of consciousness and physical states. Even if we assume that
neural states can be present to us (as they might seem to be in the case of the
distributed representation of the smell of a rose), there is nothing in the
neural connections between units that presents us with anything. The problem is
that the connections themselves are not important; it is what they do that is
important. This is why we describe them dispositionally, as abilities to
perform certain transformations. The physical features that embody these
abilities, though necessary for the representing process, are not the
representation. The hardware that connects a set of sounds to a pattern of
marks on a compact disc is not the reproduction of the sound; we don't see and
aren't interested the hardware. We are interested in what it does, in the
connection it establishes between the pattern on the disc and the music we
hear. In the same way, the connections could be said to do the representing, to
do the connecting, but they themselves in isolation from the process are not
representations.[7]
IV. Conclusion
Representation is not iconic
resemblance; it is not symbolic isomorphism effected by an algorithm or
function; it is not the algorithm, function, or concept that does the
connection. These views all mistake a
part of the representing process for the whole, and then run into problems when
they try to find within the intrinsic properties of the part, in isolation,
properties that it only has in relation to the whole.
The suggestion I wish to make about
what representation is is fairly
simple, but has some important consequences. Representation is the process of connecting one mode of interaction
with the world to another. It is not the pattern of activation on one set of
units, nor the connections through which the transformation is made, that is
the representation. It is the process of connecting one pattern to another, the
transformation itself, that is the re-presentation of the one pattern by the
other. Representation is always a connection between one mode of interaction
with the world and another, a projection of one set of properties onto another.
It is the re-presentation of one
thing by another.
Note that this implies that the
phenomenological interpretation we gave of the PDP model was false. It was
assumed that we first feel an olfactory image, when the olfactory units are
activated, then we see a visual image after the pattern of activation has been
transformed into a pattern on the visual units. I am suggesting that we never
perceive simple properties like olfactory images or visual images.
Talk of perception of simple images
is an abstraction; perception is always of connections or projections of one
set of properties onto another. We identify separate properties only by
abstracting them from the connection in which we find them. We interact with
the rose through our sense of smell causing a pattern of activation to arise in
a set of olfactory units that immediately
causes a set of patterns to arise in the visual units. What we perceive is this
process of connection, not any one component of it. We perceive the visual
image of the rose as projected onto the smell of the rose; we perceive the
smell as re-presenting the visual image. They are perceived together,
inseparably. We do not first smell, then see. We see only through the smell; we
smell only as re-presenting the sight. The separate images are abstractions
separated only after the fact. This is more clear in the case of perceptual
representation than in the case of memory. A two dimensional image is projected
upon my retinas and this is connected with my knowledge of three dimensional
structures, the names they are called, and the other properties associated with
them. I see a rose. I see a certain type of object, a rose. I see that it has a
certain shape. I see that it is soft. I see that it has a certain fragrance, as
the visual image calls this to mind. All of these are part of the perception of
the rose. My perception is the
projection of these other properties onto my visual interaction with the rose.
My perception is the visual re-presentation of the other properties of the
rose. I don't first see a two-dimensional shape and then associate it with
other images. My perception is the process of connections and cannot be
identified with any of the components abstracted from it.
This intrinsic connection of one
part of our conscious experience to other parts is sometimes referred to as
Intentionality. The properties of a representation always point beyond
themselves to the properties of that which they represent. It is often thought
that this property of our conscious experience presents insuperable problems
for most models of mental representation. One of the distinct advantages of the
model I suggest is that it sees Intentionality as an intrinsic, but not
inexplicable, property of acts of representing. Since an act of representing
is, and must be, an act of connecting one mode of interaction with the world to
another, it is not surprising that our consciousness of these acts involves
properties that point beyond themselves to other properties. Representing is a re-presenting of one set of
properties by another. Views that see representations as self-contained icons
or bits of information will continue to find the intrinsic Intentionality of
mental representation a mystery. The problem of Intentionality cannot be solved
by building more information into the system. It will be only be solved by
recognizing that we must model representation by an activity of connecting bits
of information, rather than by the self-contained bits of information
themselves. Intentionality is not a mysterious property that the components of our
models must somehow acquire; it is something they must do through their
activity.
Of course this cannot be argued for
rigorously in the space allotted. Let me close by suggesting some of the
epistemological implications such a view might have. The problem with the
iconic view was that the icons were both the only way of contacting the world
and the bearers of our knowledge. There was no way of telling if they were
really similar to reality or not. The images became veils separating us from
reality.
In particular, it was seen that any of
our particular images could not reflect or be similar to reality since it was a
result of our peculiar constitution. Redness is not in things it is the effect
they have on us. There was no way to verify our representations, and there was
good reason to think they could not be similar to external objects.
On the view I am suggesting,
particular images or effects the object has on us are not representations. A
representation is a process of connecting two such effects, and its content
lies in this connection, not in the
particular features of the images connected. The content of my perception of
the rose as red and as fragrant is not the "feel" of these particular
images, but the connections that are established between them by my act of
representing. While the particular features of the images reflect only the
effect the object has on me, the connections within my representation can
reflect the connections between the real dispositions in the object that give
rise to these effects. While there is
no determinate isomorphism between individual icons and the world, there can be
an isomorphism between the connections made between modes of interaction within
an act of representing and the properties in the object that participated in
the modes of interaction that are
connected. Connections can be objective while images cannot.
Further, if representation is seen
as a process of connecting different modes of interaction with the world, the
veil of ideas disappears. A representation is not an internal icon that is
presented to the mind's eye. A representation is an act in which various
aspects of our interaction with the world are connected. It is an organized
means of interacting with the world. On the view I suggest, representations can
serve their cognitive function. They can become windows rather than veils, and
the mind a vista onto the world rather than a windowless room.
REFERENCES
Churchland, Paul M.
1986. "Cognitive Neurobiology: A Computational Hypothesis for Laminar
Cortex." Biology and Philosophy, V.1 (1), 1986, pp. 25-51.
1987. "Some Reductive Strategies in
Cognitive Neurobiology." Mind, 1987, pp. 279-309.
Putnam, Hilary.
1976. "Realism and Reason." In Meaning and the Moral Sciences;
London: Routledge and Kegan Paul, 1978.
1977. "Models and Reality." In Realism
and Reason. Cambridge: Cambridge University Press, 1983, pp. 1-25.
1981. Reason, Truth and History.
Cambridge: Cambridge University Press, 1981.
1988. Representation and Reality.
Cambridge: MIT Press. 1988.
Rumelhart, David
and James McClelland. 1987. Parallel Distributed Processing. Boston: MIT
Press, 1987.
[7] The epistemological
problem that arises from this view of representation should be fairly obvious
from a consideration of Kant's version of it. Concepts constitute the reality
in which we live and perceive. They cannot in any clear sense be said to
represent that reality in a way that can be true or false. Hence, concepts, or
connections, cannot fulfill the second of the cognitive roles representations
ought to play. (Of course, this view can lead to internal views of
justification and truth, but even in these it is not the concepts that are true
but the coherent system of connections that is the result of the action of the concepts.)