COMPUTERS: dead or alive?
by John Shotter, Dept. of Psychology, University of Nottingham
 

The author has worked for some years now upon the problem of what is required to make computers 'understand' ordinary language. At the moment he feels fairly certain that this can only be achieved in a very limited sense. Human beings not only USE systems of symbols via which to communicate and within which to think, they also CREATE new systems, and in doing so transform themselves in undreamt of ways. The basis of this creative process is not individual, but social, activity. Such activity depends, not only upon having heads (and whatever is in them), but also upon having arms and legs and other bits of body too. It also depends upon being in the world as we are. The general implications of these ideas for computer 'intelligences' are explored in this article.

It seems only to be commonsense that the body is worked by the mind in some way. Thus if a 'mind' could be built, it would only be a rather trivial and' certainly secondary task to provide it with a body to animate. It seems only to be common sense also to say that the brain, in its better moments, works in an orderly fashion, that it has an organisation which one day will be revealed and written down. In the field of "computer simulation" in Psychology, attempts are being made to build "models" of the brain, in the hope that insights into the nature of many fundamental human abilities will be attained. Insights into such abilities as: 'the recognition of patterns, the solution of problems, the learning of languages, in short, how we come to understand the world that we do, and act appropriately in it.

The above paragraph begins with what one might call "The two dogmas of computer intelligence;" (1) That only what goes on in individual minds is worth attention, and (2) what goes on can be written down within a system of permanent physical symbols, In this article I want to assert (there is no space for supportive arguments) a quite different view as to what is of  importance in being intelligent and in achieving a genuine under [end 1^st-col p.24]-standing of a situation. If my view is correct, then there are severe limitations upon the abilities of computers; but not only that, there are important implications bearing upon the effects caused by organization any human activities to take on the precision and order of those exhibited by computers.

The current issue of Science Journal (Oct. 1968) is concerned with "Machines like Men". In the leader, Marvin Minsky, Professor of Electrical Engineering at M.I.T. makes the authoritarian pronouncement: "Do not be bullied by authoritarian pronouncements about what machines will never do. Such statements are based upon pride, not fact. There has emerged no hint, in any scientific theory of machines, of limitations not shared by man." (A corollary of this is, that anything which ordinary individuals can do, an organisation can also do, but more efficiently and to order.) In fact, bold claims, concerning the capabilities of machines, have been made for the past ten years or more. Recently, they have begun to take on an apocalyptic flavour. Stuart Sutherland, Professor of Psychology at the University of Sussex declares that as computers are not "selfish" or "irrational" they may be "the first true saints". (How the choice is made in selecting in the world of men a distinction to mark with the word "saint", he does not say. Would his computers "Know" whether to call themselves 'saint' or not?). Anyway, he sees them soon transcending our [end 2^nd-col p.24] limited understanding for " ...if we call design a machine more intelligent than ourselves, then a fortiori that machine will be able to design one more intelligent than itself," and so on. "Once," Minsky says, "a certain threshold is passed, this could lead to a spiral of acceleration. and that it may be hard to perfect a reliable 'governor' to restrain it". In the mechanical millennium then, that is to come, such developments, Sutherland has forecasted, will lead us "...to treat information-processing machines in the same way that we treat humans".

But some people may think that this happened long ago - certainly those who have ever come across Dr. Andrew Ure's: Philosophy of Manufacturers (!885) will. It sets forth procedures of all types for converting people into parts for the "vast automation", i.e. the efficient factory. What is new then? In line with the second dogma of computer intelligence - that saying something in writing is as good as doing it - is the belief that ideas can he manufactured in the same way as other more physical products of man: "Manufacture the instructions, their realisation follows mechanically." But does it in fact do so ? At some point these instructions have to be understood. Can that process ever be written down ?

(from "What the Tortoise said to Achilles." by Lewis Carroll, paraphrased from Peter Winch: The Idea of a Social Science, Routledge and Kegan Paul, !958).

Achilles and the Tortoise are discussing three propositions A, B and Z, where Z is looked upon as following logically from A and B. But the Tortoise asks Achilles to treat him as if, while accepting that A and B were true, he did not yet accept the truth of the hypothetica! proposition (C) : 'If A and B be true, Z must be true', and force him, logically, to accept Z as true.

Achilles begins by asking the Tortoise to accept C, which he does; Achilles then writes [end 1^st-col p.25] in his notebook:

A
B
C (If A and B are true, Z must be true).
Z

Then he says to the Tortoise: "If you accept A and B and C. yoU must accept Z."

When the Tortoise asks why he must, Achilles replies: "Because it follows logically from them. If A and B and C are true, Z must be true (D). You don't dispute that I imagine ?" The Tortoise agrees to accept D if Achilles will write it down. The dialogue continues with Achilles having to write down more and more ancillary hypothetical propositions. At one point the Tortoise asks : "Suppose I still refuse to accept Z ?" Achilles triumphantly replies : "Then Logic would take you by the throat, and force you to do it." But nothing happens; "Whatever Logic is good enough to tell me is worth writing down. So enter it in your book, please. .." says the Tortoise. And he continues to refuse to accept Z, and Achilles sadly continues filling his notebook, E, F, G, H,...

The moral of this is, of course, that the actual process of drawing an inference, which is after all at the heart of Logic, is something which cannot itself be written down as a logical formula. Learning to infer is not just a matter of being shown patterns of marks on a piece of paper and being told that they are explicit logical relations - in any case there are an infinity of them, so one cannot be shown every one, only a selection of examples -, it is a matter of coming to grasp how to do something, and agreeing to do it. It is the same in learning anything where there are "accepted" ways of going on - where there are explicit or implicit 'rules' regulating the activity. It is so even with the greatest of all human achievements, the learning of language.

In general then, to understand instructions, even a moron has to make a genuine attempt, in the presence of someone who already understands, to use them. He not only has to repeat examples that he has been shown, but go on [end 2n-col p.25] to use them in new situations in the same way. If he tries, but makes mistakes, he is corrected, and so the process goes on until he can pass all the tests that someone who already understands can devise.

Attempting to act correctly, in order to achieve an understanding of how to act correctly, is essential; and the form of one's knowledge then depends upon the activities that were its basis.

It is on this point that the great divide comes: On the one hand are the exponents of "mechanical untelligences" who think that the form of one's activities is determined by "dispositions of the mind" or other clearly formulable and discoverable principles of mental organisation. (They determine not only the 'logic' of our actions ; they also de- termine the form of our language, thus ultimately its rules of formation can be made explicit and programmed into a computer along with everything else). But on the other hand is the viewpoint taken in this article, that any such explicit formulation depends upon the prior existence of, as Wittgenstein called it, a "form of life,"and the whole of our body is involved in that.

To understand this notion, consider quite a small, circumscribed locality, such that an individual could get to know a great deal about it, personally. Clearly, he would only pay attention to those aspects of it relevant to his needs; but all humans must have similar basic needs, and thus we might expect that other individuals would share with the first a personal knowledge of that locality. Again, it would only be knowledge relevant to their concerns. We now have the basis for the creation of, as Wittgenstein calls it, a "language-game". Without going into the process of how "language-games come to be "played" (see, Lewis Murnford: The Myth of the Machine, Secker & Warburg, 1967), we can see how the symbols in them come to have a significance. They are grasped by being used in the context of doing something within that circumscribed environment. And the system of symbols, that is, the language-game, takes on an order, as if it were functioning according to 'rules', because of the orderliness [end 1^st-col p.26] of the activities, all connected to one another, in the underlying "form of life."

But this is strange: to the extent that all individu1lls share the same basic personal knowled~ of the situation, how can they ever be told anything new ? Well consider now a very circumscribed and highly stable "form of life" indeed, that revolving around a small number of distinctive marks on a piece of paper, i.e. Logic. All participants understand how to arrange and rearrange the elements into acceptsble patterns, and all start with thc same basic set, the axioms. Now the individual's task here is to work out theorems, and this takes time and effort. If someone were to give you the results of his work; after checking that the conclusions really were true or 'acceptable', you could use them to ease your own labours. Might not some of the uses of language be similar to this ? But what you have just been told, while new to you, is not all that surprising, for it bears a "family resemblance" (Wittgenstein's term again) to what you already knew; it is not from a completely unfamiliar "form of life".

In general however, "forms of life" are obviously nowhere nearly so circumscribed and stable as logic or chess (say). People are not all confined within a small enclosure; logic and suchlike activities represent extreme fomls. In everyday life, new activities are continually being invented somewhere, and thus new language-games grow up around them (as around space-research and so on). The activity comes first, the new language-game afterwards. How do linguistic novices cope here ? Surely they get told something genuinely new in such situations as these? Well, they can be given empty words, they can even come to not only remember them but, with other remembered verbal relations, work out 'acceptable' new forms. (Playing with words.) Is this equivalent to the type of understanding that they would gain by participating in the new activity? And is the significancc of the words, then, in the two situations, the same? To attain genuine understanding they must be shown how to do things, before they can be told anything of significance. (Empty words are often a spur to search for signifi- [end 2^nd-col p.26] cance. I'm not here saying that there is a one best education method.) Once a genuine understanding of the language has been attained it can be used not only to provide descriptions, but to elicit help, issue commands, to make plans, wishes, promises, etc. There is a continual interplay between language and action; in many situations one's actions alone are sufficient for others to grasp their significance. While actions alone are significant, could words alone have ever been so? Is what can be written down then the only "certain" knowledge? What could Goethe have meant then when he said: "All that is fact is already theory"?

The discussion above seems to suggest that there is not such a thing as the English Language which we all know in varying degrees, only collections of language-games based upon forms of life. Perhaps having a Language is like having a box of tools of a certain type and a skill at putting them to use; one can make many things. With a different Language, or a different set of tools, one may be able to achieve some of the same things, but with differing degrees of facility, and some things not at all. However, does onc really need these ready-made tools? They were made once, it is perfectly possible now to create new tools as needed. So it is with the components of Language - a creation of our ancestors. But before there were written dictionaries and "grammars" language was somewhat less stable than it is now, as Samuel Johnson well knew.

I have dwelt so long upon the nature of language as it is right at the heart of all the problems to do with knowledge, understanding and the limitations of rational organizations. If Wittgenstein's views are correct then, not withstanding Minsky's warnings about "authoritarian pronouncements", it is quite clear where the limitations of computers lie: Their immediate limitation is the inabilty to participate in some activity in the world with other computers; and even if they could be made to do that (and to do it they would not only need sensors and manipulators⁽¹⁾, but would also need to be 'concerned' to satisfy certain needs) they would still have to create amongst themselves systems of symbols via which to "talk" about their "form of life" (and later within which to 'think' about it.) In short, they are essentially limited in their creative ability: they can create (some might say "produce") systematically, via a logical language-game (but a human "form of life" is its basis); but they cannot create transcendentally - not via any sort of language at all but via new actions that can later be the basis for a new language-game. And such limitations as these are inherent in any rigid organisation.

Minsky, and other such "high priests" of the age old (it was human machines that built the pyramids) "church of regimentation" by capitalising upon our current belief that the only "certain" truths are those based upon "scientific theories" have bemused our intelligence. They can dismiss as vague, subjective, uncertain, based on pride, or even as inessential, those essential aspects of our abilities that cannot be written down within the confines of a logical-system, as they are the basis from which such a system is created. Yet Minsky's own assertions are based upon unprovable beliefs. (see "The two dogmas. ..", second paragraph above). If one believes that all 'thought' takes place within a logical-system (a belief demonstrated to be mistaken by philosophers from Heraclitus to William James), that can one day be written down, then it is not at all surprising to find men - or at least those activities (or is it 'products' of activities ?) of men that can publicly and unequivocably encode within a system of permanent physical entities - to be limited in the same way as other of our physical creations. What Minsky says is true; but it is banal. If men's abilities can be unambiguously encoded into one physical system of expression (a set of marks on paper), surely they can easily be embodied in another (a machine). While it is true that the machine could quickly manufacture [end 2^nd-col p.27] patterns of symbols that might take an eternity to arrive at, in principle they tell us nothing that we could not have eventually worked out for ourselves. With sufficient organization - a task that raises essential human problems, as well as ones of technique - people can be brought, ultimately, to do everything that machines can do, as Dr. Andrew Ure pointed out long ago.

The National Computer Centre estimates that in 1970 there will be 5,000 computers in use in Britain; associated with putting them to use will be about 75,000 people. In 1975 the number will have increased to 10,000 and the associated people to around 200,000. But this is to say nothing of all those who work without computers. but whose sole function it is to organise and "rationalise" the activities of the rest of us. Such an increase is bound to effect the way that we understand our world.

The fantasy of the little, all-alone man twisting and turning in the face of the inexorable onslaughts of some mindless entity, quite un- responsive to human appeals, has always been good for a laugh: could it, or has it today, gone beyond a joke and become tragic? Perhaps just because man now seriously feels in danger of mysteriously losing his humanity is he examining its nature closely - concerned to identify precisely what it is that is worth fighting for. However, while on the political front the individual's right to choose how to regulate his own personal life is now becoming a world-wide issue, on the technological front, where the actual practical changes that channel and shape OUR "forms of life", everything proceeds unheedingly apace. It is even argued by many, even those concerned with man's inhumanity to man, that such developments are doubly necessary if we are "to solve" the problems that trouble us. But the upshot of such an urge to organise and rationalise, totally, inevitably brings a living death to all concerned. (To operate a machine efficiently [end 1^st-col p.28] you must become like a machine.) If we voluntarily submit to the demands to do only what is "productive" or "useful" or "in the national interest' or in accordance with any pre-established formal requirement, then it will amount to making our "forms of life" as fixed and as closed as a big game of logic. Under such circumstances computers could be made to do all that men couJd do. We could relieve ourselves of the necessity to do anything at all - perfect freedom!? But gradually, as new men would no longer participate in the activities that computers would carry out, man would loose all understanding of the world in which he exists. As mindless, loveless, amoeboid lumps, we would find the world unintelligible and alien.

But are we not now beginning to achieve this state, and voluntarily too, upon the advise of experts ? Their logics are unassailable ; but what of the beliefs upon which they are based ? They do not make those available ; who has the knowledge, the grasp of the whole needed to question them ? But as the experts are only wielding their skills-and have they not a duty to do so, as most of them have been trained at the state's expense ?-the results are not their responsibility. Well whose are they then ? Other experts' ? Suddenly everywhere experts are needed to understand the "complexities" in terms of their own special techniques. Yet what are they doin~ is not necessarily relevant at all to human concerns. The experts' concerns are to achieve certainty within the terms of their technique. But such procedures are inevitably socially divisive. For it amounts to elite groups, from positions of special knowledge, believed to be productive of "the one best answer" as they are .'scientific", imposing upon the rest of us a way of life, and thus a view of the world, that is only suited to their purposes, not ours. And to the extent. that we are denied access to "worthless" and "chaotic" activities when we feel the need, we are denied access to that wav of life within which cre- ative cultural evolution may take place. We are denied our essential humanity. "And we shall (truly) come to treat people as machines." Prophecies such as that have a tendency to be self-fulfilling.

1. As yet, like Logic, they lack the ability "to seize us (How would they recognise us?) by the throat" and force us to accept their conclusions.