Searle, John. R. (1980) Minds, brains, and programs. Behavioral and Brain Sciences 3 (3): 417-457
This article can be viewed as an attempt to explore the consequences of two propositions. (1) Intentionality in human beings (and animals) is a product of causal features of the brain I assume this is an empirical fact about the actual causal relations between mental processes and brains It says simply that certain brain processes are sufficient for intentionality. (2) Instantiating a computer program is never by itself a sufficient condition of intentionality The main argument of this paper is directed at establishing this claim The form of the argument is to show how a human agent could instantiate the program and still not have the relevant intentionality. These two propositions have the following consequences (3) The explanation of how the brain produces intentionality cannot be that it does it by instantiating a computer program. This is a strict logical consequence of 1 and 2. (4) Any mechanism capable of producing intentionality must have causal powers equal to those of the brain. This is meant to be a trivial consequence of 1. (5) Any attempt literally to create intentionality artificially (strong AI) could not succeed just by designing programs but would have to duplicate the causal powers of the human brain. This follows from 2 and 4.
Subscribe to:
Post Comments (Atom)
-
Bekoff, M., & Harnad, S. (2015). Doing the Right Thing: An Interview With Stevan Harnad . Psychology Today . Twelve years after steppi...
-
Harnad, S. (2003) The Symbol Grounding Problem . Encylopedia of Cognitive Science. Nature Publishing Group. Macmillan. or: Harnad, S...
-
2b. Harnad, S. (2008) The Annotation Game: On Turing (1950) on Computing, Machinery and IntelligenceHarnad, S. (2008) The Annotation Game: On Turing (1950) on Computing,Machinery and Intelligence . In: Epstein, Robert & Peters, Grace (...
ReplyDelete“the answers to the Chinese questions and the English questions are equally good. But in the Chinese case, unlike the English case, I produce the answers by manipulating uninterpreted formal symbols. As far as the Chinese is concerned, I simply behave like a computer; I perform computational operations on formally specified elements. For the purposes of the Chinese, I am simply an instantiation of the computer program [...] from the point of view of someone reading my "answers" -- the answers to the Chinese questions and the English questions are equally good.”
I might be misunderstanding here, but isn’t it a bit tricky to argue that Searle's system would be proficient or could develop proficiency in Chinese? How could such a formalism without an interface to the outside world not be subject to the symbol grounding problem if it’s essentially just a circuit of rules? Surely his claim that Schank's narrative program doesn't understand the stories it infers details from holds up, but if searles system isn't theoretically capable of passing for a native chinese speaker (is this essentially the same as passing T2?), would his argument that ~some grounded t2 passing system~ couldn't possibly understand/have intentionality hold without some update to his framework?
(I’m worried I might have some fundamental misunderstanding of Searle's system so please tell me about it if that’s the case)
Yes, this is T2. Yes it may not be possible for computation alone to pass T2 (but Searle is taking this as the premise of "Strong AI" [computationalism] and showing that even if the premise is true, T2 would not understand. (And this was written in 1980, before the symbol grounding problem was noticed; in fact, it is what led to the symbol grounding problem being noticed!)
Delete“[...] the formal symbol manipulations by themselves don't have any intentionality”
ReplyDeleteSearle is providing a very good argument against the case that “cognition is just computation”. The strength of his argument rely on the simplicity of the thought experiment. Just imagine yourself in a room. You receive a series of Chinese symbols, and your role is to provide other Chinese symbols as a response of the ones you receive. Note you do not speak Chinese at all, and that for you these symbols are just meaningless squiggles. Nevertheless, you have a set of instructions in your native language (English or whichever it is, and you understand it perfectly) that indicates to you how to give the right symbols (output) according to which symbols you received, and you can recognize the shape of the symbols to do so. The instructions are so well constructed that your answers to the Chinese symbols are totally undistinguishable from the ones of a native Chinese speaker.
Following this, the main claim of Searle is you don’t need to understand Chinese to act as a Chinese speaker would do, in the same way understanding is not required nor created by computation. The more general claim is therefore that cognition cannot be **just** computation, because cognition entails understanding.
I think that the strength of the argument rests on the very close analogy to what a Turing machine can do. The person in the room can 1) read symbols according to their shape 2) write symbols 3) all according to a certain set of rules that the person knows. Of course, the argument is not telling us that “there is no computation at all implicated in cognition”. It is just saying that formal symbol manipulation is not enough.
Is the CRA now a demonstration of the “hard problem”? While one could reasonably think so (after all, it seems like it “feels like something” to understand something). But I do not think that Searle is taking this direction. “If you can exactly duplicate the causes, you could duplicate the effects. And indeed it might be possible to produce consciousness, intentionality, and all the rest of it using some other sorts of chemical principles than those that human beings use.” Searle rather seems to be inclined to the necessity of to pass T3 (and could even argue for T4) in order to explain understanding, and (of course) cognition in general.
The premise of T2 for a computationalist is that there exists an algorithm (symbol manipulation rules, software) -- or perhaps many (because of underdetermination and weak equivalence) -- such that any physical system (hardware) that is able to execute the T2 algorithm (for a lifetime) is understanding language (whether it's the Chinese T2 or the English T2) when it is executing the algorithm: ("cognition = symbol-manipulation"). Searle points out that this is not so, because he could do it for Chinese T2 without understanding Chinese.
DeleteBut T2 is not just question-answering. It's anything you could email with Gabe about, for a lifetime. It's everything a person can say, and understand (including this skywriting!),
And it's true (but irrelevant) that in order to be able to execute the symbol-manipulation rules for a lifetime, Searle would have to be told (and to memorize) the rules in English (unlike the Mac that is running the same Chinese T2 program). The point is not about whether Searle understands the rules; it's about whether he understands Chinese just because he is executing the rules.
Searle's Chinese Room Argument came before the Symbol Grounding Problem. He did not know or say that understanding requires T3 grounding -- in fact, he assumed that T2, passed by computation alone, was the only Turing Test, and thought that hence he had refuted not only "Strong AI" (computationalism: cognition = computation) but the Turing Test itself.
As you note, all Searle showed was that cognition cannot be all computation; he thought he had shown that cognition cannot be computation at all [false]. He also thought he had shown that the Ttest is invalid, whereas he had only shown that a purely computational T2 (if it was possible at all) would be invalid. He also thought he had shown that cogsci should instead focus on studying the brain -- which would have meant T4, if Searle had considered all the possible levels of the Ttest. He also misunderstood the importance of robotic (sensorimotor) capacity, agreeing with the computationlists that sensorimotor function is just the peripheral I/O devices of a Turing Machine.
My intuition/initial response to Searle’s Chinese Room case is aligned with the Robot Reply. The Robot Reply says that if we gave a computer an appropriate robotic body and sensors to enable interaction with the world, similarly to humans, then there would be understanding because the robot would be able to learn. I agree with this reply for the most part, although I would argue that it’s not so much having a robotic body similar to ours that would make understanding possible but rather the experiences that could provide learning for the ‘robots’. However, it is possible that the AI could only have similar experiences to us if they were constructed atomically similar as us, but we must be clear that is not the human-like body that produces the understanding.
ReplyDeleteSearle responds to the Robot Reply by saying that the addition of ‘perceptual’ and ‘motor’ capacity adds nothing to understanding. I would agree that it is not ‘motor’ capacity that provides understanding but rather experiential knowledge that creates a body of understanding. The knowledge which is gained from engaging with other AI or humans would increase understanding and provide meaning for the language of formal symbols used. Searle must learn meaning, categorization, grouping, translation, etc. in Chinese to able to understand the language. In a similar way, a robot must be programmed with the ability to learn —along with skills of categorization, ascribing meaning, association, etc. — in order to understand.
One does not understand based solely on a word or the word’s reference. Reference does not equal meaning (Morning/evening star example). I would argue that it is context and experiences which ascribes meaning/understanding to a word. If an AI machine were able to have similar experiences to us and ascribe meaning based on those experiences, it is my intuition that they would have Understanding.
If by "experiences" you mean Input/Output (I/O), then Gabe can have the same I/O as any of us (why not?). But if you are talking about felt experiences, then you are switching to the "hard" problem (and begging the question). (And we have no way of knowing whether Game feels anything at all.)
DeleteYes, the T2-passing algorithm would have to include (verbal) learning capacity: But so what? Searle is executing that too, and it doesn't make him understand Chinese. He cannot, however, "execute" a robot's sensorimotor functions, because they are not computational. All the worse for "Strong AI."
Grounding connects words to their referents (robotically), but grounding is not meaning. It feels like something to understand a word or sentence; it also feels like something to mean whatever a word or sentence means, when you are saying or thinking it. Your idea of "experience" seems to smuggle in feeling somehow.
Context does not sort any of this out. (What do you mean by context?)
Searle argues that computers are not capable of understanding and that cognition cannot be computation. I agree that cognition cannot simply be computation, although I do believe that computation is definitely a part of cognition. I disagree with Searle’s belief that computation is not at all involved in the process of understanding. Humans compute things in order to reach certain mental states which give rise to certain understandings and feelings; therefore, computation cannot be completely overlooked when discussing the process of understanding.
ReplyDeleteI am inclined to believe that simply passing T2 would not be enough to demonstrate complete understanding. I believe there is something about interacting with the world that adds to human’s understanding of many different things. I do not believe that only demonstrating powerful verbal performance capacity is enough. The fact that the Turing Machine is able to give appropriate responses and interact in a socially acceptable manner does not quite encompass understanding.
In Searle’s paper, he talks about how cognitive scientists should focus on the brain (T4). I disagree that we should focus on passing T4. I believe that the focus now should be on passing T3. I don’t believe that the robot needs to be structurally indistinguishable from humans (required to pass T4) in order to demonstrate understanding. I do believe that a robot would need to be functionally indistinguishable from humans to demonstrate understanding though (required to pass T3).
Although I do not believe passing T4 is necessary for a robot to demonstrate understanding, I do believe that passing T4 would help people in general accept the idea of robots being capable of understanding things like we do. People, like grandmas, seem to be focused on the fact that things that are not made up of biologically similar materials to us cannot think or understand like we do.
I wonder what you mean by "complete understanding"? The question with T2 is whether there is any understanding at all. It feels like something to understand.
DeleteAnd passing T2 does not just mean "demonstrating powerful verbal performance capacity... to give appropriate responses and interact in a socially acceptable manner." It means being able to do anything Gabe can do, via email, indistinguishably from any of us (via email), for a lifetime.
Given today’s understanding of the interconnectedness of all areas of the brain, the discussion of subsystems in the “systems reply” which are discrete from one another seems ludacris. (p.6) I also agree with Searle that the content of the counterargument is self-defeating given its reliance on differentiating the English subsystem from the Chinese subsystem, and thereby drawing a distinction between forms of understanding. However, I disagree with the equivalence between the so-called “Chinese subsystem” and the stomach. While they do each take inputs, process them, and then produce outputs, they are not both operating at the level of language.
ReplyDeleteI do not see how computationalism led to discussions of thermostats having beliefs.
I’m not sure that the example of water pipes simulates neuron firings in the way that was intended by proponents of the “brain simulator reply”. I imagine they would have meant physical models that more closely resembled the brain. That said, Searle is absolutely right that this would be an entirely different approach than that of reverse engineering. If the intention is to build a machine which can demonstrate to us how the mind thinks, we can’t be expected to first know the brain structure and operations well enough to simulate the neuron firings.
I’m not certain that I understand what Searle means by “causal powers”, for instance, in the sentence: “only something that has the same causal powers as brains can have intentionality”(p.12).
I also don’t understand why “everything is a digital computer” (p.14).
For the issue of the equivalence between the "Chinese subsystem" and the stomach, it does not matter if they do not both operate at the level of language as the fact that they do only take in inputs and release an output is what's important.
DeleteThermostats having beliefs is just irrelevant nonsense, and it's not computationalism because a thermostat is not just a computer manipulating symbols.
DeleteBy "causal powers" Searle means whatever causal mechanism is able to pass T2 as well as T3 and T4.
“Everything is a digital computer” is another piece of nonsense. He probably just meant the Strong Church/Turing Thesis (that computation can simulate just about anything). But from that it does not follow that everything is a computer.
In his paper, Searle presents an argument against the claims of strong AI, which assert that cognition is computation. In his Chinese room argument, Searle denounces strong AI because he believes computation and formal symbol manipulation alone does not produce understanding or intentionality. It seems to me that the biggest difference, between Searle manipulating English and manipulating Chinese, is the intentional states associated with each. This form of argument parallels ideas found in Thomas Nagel’s “What Is It Like to Be a Bat?” – the core of consciousness in organisms is that it is something it is like to be that organism. It feels like something to understand Chinese, and conversely it feels like something to not understand it.
ReplyDeleteFocusing on a specific passage in this text, Searle raises the objection which he calls “The Robot Reply”. In this objection, the computer program is placed inside a robot that has the capacity to move, eat, drink, etc. This objection seems similar to T3, but after further inspection I don’t think it is. My question is what is the difference between the robot that Searle is discussing in this objection, and Gabe (our MIT robot)?
Forget about "intentionality": it's just another weasel-word for a felt state.
DeleteThe robot reply ought to be referring to T3, but it's not. It's just repeating the computationalist premise that cognition is just computation (as executed by a computer) and that the sensorimotor input and output to the computer is just input/output. It overlooks the fact that sensorimotor activity is indeed physical activity, hence eligible as being a causal part of the causal mechanism of cognition.
It seems that Searle argues that understanding is not important for answering a set of questions or carrying out a program. He repetitively emphasizes that he does not need to understand Chinese for him to be able to answer Chinese-written questions with Chinese-written responses (following a set of English “instructions”), such that his responses can be indistinguishable from responses of a native Chinese speaker. Although I am not questioning his ability to do so, I am questioning the negligence for understanding. Searle’s emphasis seems to be on the output (i.e., responses) and what he explains is weak behavioural equivalence: it doesn’t matter how the output is generated, but it only matters if the output can be indistinguishable from that of a native Chinese speaker. But why is this the case? Am I wrong to think that understanding is a major component of cognition and in particular, thinking? I personally believe that understanding and thinking go hand in hand: I think so that I can understand. So even though a non-native and native Chinese speaker can generate Chinese-written responses, the non-native speaker is simply following a set of instructions to generate the responses, but the native speaker actually understands the input. So can we say that any cognizing is going on at all with the non-native Chinese speaker?
ReplyDeleteOne of the reasons I think Searle is making such a big deal about understanding is to demonstrate that just passing the Turing test, which would be producing an indistinguishable output, is not enough to prove computationalism. So, I think in Searle's case, for the non-native Chinese speaker, it is demonstrating that the action of producing the output of Chinese demonstrates that cognition is not computation, although cognition can still occur.
DeleteSearle is not aguing understanding is not needed for understanding Chinese. He is just arguing that even if we accept the premise that computation alone could pass the Chinese T2, it would not be understanding Chinese!
Delete“Imagine a robot with a brain-shaped computer lodged in its cranial cavity, imagine the computer programmed with all the synapses of a human brain, imagine the whole behavior of the robot is indistinguishable from human behavior, and now think of the whole thing as a unified system and not just as a computer with inputs and outputs.”
ReplyDeleteThe problem that I have with Searle’s argument is that he is basing it on the fact that we do not understand consciousness (he calls it intentionality, but I think it ultimately comes down to consciousness). He goes on to give multiple examples of possible systems whereby there would be an input and an output system of symbol manipulation to emulate human thinking. He gives one example, the combination reply, which is explained in the above quote. In this example, he goes on to conclude that the robot would not be conscious (or have intentionality). To support this conclusion, he proposes the following thought experiment “Suppose we knew that the robot's behavior was entirely accounted for by the fact that a man inside it was receiving uninterpreted formal symbols from the robot's sensory receptors and sending out uninterpreted formal symbols to its motor mechanisms”. This thought experiment is designed to show us that even in the combination reply example, the robot isn’t understanding anything, it isn’t intentional, it is merely following a set of symbol manipulation rules designed to produce an output in response to an input.
However, the exact same argument could be made of human brains. The fact of the matter is, we still don’t know how consciousness arises, whether there is a “soul” that is somehow causally related to the brain, whether our consciousness is a by-product of brain activity with no causal influence, etc. His arguments regarding intentionality presuppose that, in addition to understanding, humans have free will, and that they are not simply executing an output in response to an input, but I disagree with that premise. Humans do operate as an input-output system. Our minds cannot causally influence our brains because, in order for that to be possible, a non-physical system “mind” would have to have a causal influence on a physical system “brain”. Thus, brain activity cannot arise from intentionally, it can, however, arise as a response to external stimuli, internal sensations, or from previous brain activity itself. Therefore, I think it is reasonable to consider the brain as an input-output system with no intentionality.
Nevertheless, this doesn’t explain the difference between English speaking and Chinese “speaking” in Searle’s thought experiment. That is, the man in the room can understand English and, while he can effectively “communicate” in Chinese, he does not understand a single word of it. Although, if we consider the brain as an input-output system, we aren’t able to explain subjective experiences, like the feeling of understanding words or perceiving colors. The perception of color, I believe, involves light reflecting off of an object, let’s say, which gives that light a particular wavelength (input), which stimulates photoreceptors on our retina (input), which then transmits a signal (output) to other receptors (input). Those receptors send signals (output) to the visual cortex (input), and we perceive the color (output). Now although we can explain the input-output system here, we are unable to explain the “understanding” of the “symbol” or “output”, which translates into our perception. Thus, I think, in a similar fashion, just because it isn’t clear to us how the man in the Chinese room example could understand Chinese, that doesn’t mean that he could not understand Chinese.
"Intentionality" is an empty weasel-word for a felt-state, as in what it feels like to mean an apple when you say "apple." Forget it.
DeleteThe "combination reply" is an incoherent conflation of T2, T3 and T4.
Searle and computationalists and T2 are focusing on language understanding as their stand-in for cognition, not color vision (which a computer also cannot do).
While I am replying to this comment a month later, after I've been informed that he is referring to the symbol grounding problem. I still believe that Searle's Chinese room argument has inherent flaws. It's his thought experiment is predicated on the computationalist idea that cognition is just computation, but I don't think his thought experiment is possible. Almost 40 years later and we still aren't that close to producing a Turing machine that can effectively communicate with a person indistinguishably to a human using only computation.
DeleteIn Searle's Chinese room, Searle can effectively communicate with a Chinese person using only a rule for symbol manipulation of Chinese characters without even being able to understand Chinese, but it's very likely that to be able to effectively pass the Turing test something more would be required, that is, the grounding of symbols. I just don't think that it's realistically possible to effectively communicate in an unknown foreign language using only a rule-based manipulation of arbitrary symbols.
Re: “I just don't think that it's realistically possible to effectively communicate in an unknown foreign language using only a rule-based manipulation of arbitrary symbols.”
DeleteI would tend to agree with you. I think that is part of why translation tools are not as effective. A lot of meaning is derived from the context and reading between the lines (to understand things like satire and irony, especially in written text. No existing computerized translator gets to the point where it is indistinguishable from a native speaker. One could argue that eventually, with artificial intelligence, we could get better but I remain sceptical. The purpose of communication is to transmit information, which happens through the semantic meanings of words and not their syntactic manipulation, so I would argue that symbol manipulation – even if it were to seem as good as native speaker – would remain a surface-level similarity and lack the underlying meaningfulness that makes communication so critical to humans.
“Such intentionality as computers appear to have is solely in the minds of those who program them and those who use them.”
ReplyDeleteI think this line of Searle’s article stands out the most to me. In my opinion, this is the clearest reason why computationalism cannot stand. Because unless a human programmer or a person creates the computer and gives it all of the “rules” that it then uses, nothing can occur. I also appreciated that in Searle’s article he compares the intentionality of a computer to the intentionality of an automatic door but just because robots can “behave” like humans we extend some of our intentionality onto them.
"Because unless a human programmer or a person creates the computer and gives it all of the “rules” that it then uses, nothing can occur."
DeleteWhat about rules that allow other rules to be inferred with experience? If a core and fundamental set of rules is created with enough flexibility to allow scenarios not covered to be learned with time, then perhaps it takes more than a human can understand to replicate a human at the highest levels?
Consider for a moment, a rule that allowed uncertain scenarios to be observed and recorded such that after observing them with enough frequency a logical estimation can be made at a correct response. This is one of the fundamental concepts fo the AI/ML boom: we needn't consider every outcome but rather build something with the ability to learn from those outcomes. Perhaps fewer rules is in-fact better at achieving high functioning computation?
Searle means that the T2 computer doesn't understand Chinese; the understanding of its messages is just in the heads of its human interrogators.
DeleteThe capacity to learn is part of T2. But if part of the computation is a learning algorithm, that still does not save computationalism from Searle's argument that he could execute the computations without understanding Chinese.
In Searle's paper, he responds to the System reply (Berkeley) with the following idea: "let the individual internalize all of these elements of the system." The practical application of this argument seems to be that a person would internalize and "memorize" the rules of the system in order for the rules to no longer be external. I'm not sure that this is a good refutation to the system reply, given that those rules require an understanding of the Chinese language. In other ways, they were written by someone or something that understands the Chinese language. Maybe a more adequate reply might be to say that writing the rules doesn't require an "understanding" of Chinese in the sense that those rules are also made on the basis of things like grammatical rules for a given language. He's right that the whole system doesn't truly understand Chinese, but I don't think his claim that there can be nothing outside of the individual in the system is right.
ReplyDeleteWhy would you think memorizing any algorithm would "require an understanding of the Chinese language"? Computation is just manipulating squiggles and squoggles.
DeleteI really enjoyed Searle’s response to the system’s argument. One of his memorable lines was: “If we are to conclude that there must be cognition in me on the grounds that I have a certain sort of input and output and a program in between, then it looks like all sorts of noncognitive subsystems are going to turn out to be cognitive. For example, there is a level of description at which my stomach does information processing” (6).
ReplyDeleteI think that this is an interesting point, because we are often taught to think of humans and the brain holistically. For example, even doing something as simple as looking out of the window will light up several regions in the brain. Your visual areas will be activated, your frontal and parietal cortices will be put into action connecting what you are seeing to your memories and previous experiences etc. What Searle says is really important because it hones in on the fact that there is only 1 component of a system that truly understands. I also think that this concept draws some parallels with the Turing machine, because it is the executive unit that actually directs the computation as it interprets the algorithm and then directs the head and tape.
Another point I found interesting was that Searle believes that the hardware is important in understanding. This would be a cross between T4 & T5, because he doesn’t state that the entire human body needs to be present. In trying to visualize this I am imagining a human brain hooked up by wires to a robotic body. Although, this model would likely “understand”, it doesn’t really help us understand how we understand as we haven’t been able to reverse engineered anything.
See above, about how Searle conflates the Church/Turing Thesis with computation itself.
DeleteIn insisting on brain power, Searle is not speaking of the brain as the hardware for implementing software.
“Whatever else intentionality is, it is a biological phenomenon, and it is as likely to be as causally dependent on the specific biochemistry of its origins as lactation, photosynthesis, or any other biological phenomena. No one would suppose that we could produce milk and sugar by running a computer simulation of the formal sequences in lactation and photosynthesis, but where the mind is concerned many people are willing to believe in such a miracle because of a deep and abiding dualism: the mind they suppose is a matter of formal processes and is independent of quite specific material causes in the way that milk and sugar are not.”
ReplyDeleteI completely agree with Searle that the processes of the mind depend on the specific “machinery” found in the brain (neurons, synapses, neurotransmitters, etc.). With the advancement of neuroscience research, we are now aware that specific patterns of neuronal activity are what govern our behavior. The processes of the mind are dependent on the brain, which leads me to conclude that (in my view, at least) the mind *is* the brain. Dualism’s continuing influence on cognitive science and artificial intelligence ends up misleading people into believing that the mind does not require specific hardware to execute its formal processes.
I have previously brought up the notion of intentionality (for the Horswill reading), as I cannot wrap my head around a machine being able to understand and think as a human would without intending its thoughts and actions. Searle, however, defines intentionality as “a product of causal features of the brain.” Although my previous thoughts on intentionality did not revolve around the brain’s causal features, Searle’s definition encompasses a part of the strong/weak AI argument that I find incredibly interesting.
Searle thought his argument showed that cognition is not computation but brain function. Read 3b to understand that he really only showed that cognition cannot be only computation.
DeleteSearle's response to the combination reply from Berkeley and Stanford relied on the Homunculus as a counterargument, but the Homunculus can actually be used in this way. If a person were inside the robot with a computer shaped brain that acted as we did, the person would have no understanding of what was occurring outside their little compartment inside the robot. They would just be receiving symbols, following their instructions, and outputting other symbols. That person could have a person inside their head doing the exact same thing, many layers deep, and even if the mini-Homunculuses knew–that is understood–what was happening immediately outside the largest Homunculus, they still wouldn't understand what is occurring outside the robot. They would only be processing the symbols they receive. I rather liked this reading, and I find I agree with Searle.
ReplyDeleteUnfortunately, I feel like Searle would kick me if he knew I was made at MIT. "We would certainly make similar assumptions about the robot unless we had some reason not to, but as soon as we knew that the behavior was the result of a formal program, and that the actual causal properties of the physical substance were irrelevant we would abandon the assumption of intentionality." This, along with what he says in his final statements, makes me believe Searle wants T4 before he will accept understanding. He says the machines "stuff" needs to be made of the same "stuff" we are. A program voids intentionality in Searle's eyes because that just boils down to input and output, but we've been talking about computationalism so much it's hard for me to not think of cognition in this way, though I do find myself agreeing with Searle. It's just jarring to read if there is a program inside a machine, regardless if it acts exactly like us, it has no intentionality. I should start wearing shin guards I guess.
The CRA (Searle's Periscope) works against Chinese T2 -- but only if T2 is passed by computation alone. That (because of the hardware-independence of computation) allows Searle to say: I cannot understand Chinese, and « le système c'est moi ».
DeleteAll bets are off if what is going on is not just hardware-independent symbol manipulation and Searle is only part of what it going on.
"Intentionality in human beings (and animals) is a product of causal features of the brain."
ReplyDeleteSearle uses the word "intentionality" as a descriptor of "understanding" and distinguishing trait between machine computations and human cognition. As he states in the paper, "how the brain produces intentionality cannot be that it does it by instantiating a computer program." Essentially, Searle argues that whatever it means to understand something, machines cannot understand things in the same sense as humans; understanding is a cognitive state that machines cannot assume through only mechanical processes. In the same vein, one cannot test this characteristic quality of "intentionality" using the Turing Test (at T2) which only examines a machine or entity's output. The Chinese Room Argument illustrates this point: though the man in the room can take input and produce intelligible output, successfully performing computations, he does not actually understand Chinese.
So what exactly is "intentionality" and "understanding"? Searle did not explicitly define the two consequential concepts that his entire argument revolved around. Perhaps he intended to explain understanding through his Chinese Room Argument and other metaphors by presenting examples of when intentionality was absent, but I would have really appreciated a straight-forward, kid-sib friendly definition. It seems that he uses "intentionality" and "understanding" to describe consciousness, but as we discussed in class, substituting explanations for other words does not provide any clarification.
Nevertheless, if anyone has a better understanding of intentionality, please let me know. I believe that Searle attempts to describe a kind of "directedness" of human thought. Mental representations are directed at an object that actually exists in the external world; if I think "tree", the tree is not merely "squiggles and squaggles" but an entity that exists outside of my head. Therefore, a candidate for the Turing Test (at least at the T2 level) and the man in the room do not "understand" because they may be unaware of what the symbols represent in the world.
Forget the weasel-word "intentionality." The word "understanding" will do. And it needs no definition. "You can understand this sentence" (let's all it S1). And it feels like something to understand this sentence, and you know what it feels like. Whereas the sentence "Trump is a brainless, clueless, sociopathic clown" (S2) is another sentence. You understand S2 (whether or not you agree!), and it feels different from what is feels like to be understanding S1.
Delete"Intentionality" or "aboutness" is supposedly a "property" of "mental terms." The property is that you have some object in mind when you say them. The object is the intended object of what you are saying, the thing you mean, your "intended meaning". What you say is "directed at" that object, and that's what makes in mental.
This is all idle, trivial verbiage. Forget about it. Just keep in mind that it feels like something to think, say, or mean something. And a "mental state" is simply a felt state. An unfelt state is not a mental state.
The candidate does not understand "the cat is on the mat" if it does not feel what it feels like to understand "the cat is on the mat." This is even more apparent if the candidate does not feel at all.
"Yes, but could an artifact, a man-made machine think? Assuming it is possible to produce artificially a machine with a nervous system, neurons with axons and dendrites, and all the rest of it, sufficiently like ours, again the answer to the question seems to be obviously, yes. If you can exactly duplicate the causes, you could duplicate the effects."
ReplyDeleteIn this paper, Searle rejects the notion of Strong AI that computation via formal symbol manipulation is all that underlies our capacity for cognition. Through the CRA, he argues that the brain's causal ability to produce intentionality can only be explained and result instead from some biological phenomenon. While I agree with Searle that computation cannot be all of cognition, I disagree with his view that computation does not play a role at all. Reading the above quote, I am reminded of our discussions last week about the issue of under- and overdeterminance. While I agree with the idea that by duplicating the causes, you could also duplicate the effects (cognitive science is a matter of reverse-engineering after all), I don't see how Searle's focus on T4-like physiological structures helps us better understand our capacity for intentionality if we instead focus all of our efforts trying to correctly model very single neuron in our brain, where it is located and its neurochemical and electrical activities. While Searle argues that the symbols used in programs themselves are meaningless, I can't discern how the firing of a single neuron isn't also just a mechanistic part of a larger internal process.
You are right that Searle only shows that thinking cannot be all computation (though he thinks he shows that thinking cannot be computation at all.
DeleteHe is right to demand the causal mechanism of thinking, and right that the brain must be such a causal mechanism. But he does not give a hint of what that mechanism is; just that it's not (just) computation.
Since Searle seems to be arguing that computation isn’t enough to explain cognition, it seems to me that it is a problem that he asks us to accept as an argument against computationalism, a theoretical human whose brain can approximate certain computer functions to a level of sophistication that (I don’t think) we’ve observed as a human trait – that is, he asks us to accept for the sake of his thought experiment a human who can internalize a seemingly infinite number of algorithms that beget output of meaningless symbols from input of meaningless symbols. A human brain cannot achieve this without the ability to symbol-ground the symbols, and so – I’m sure I’m missing something - I have a hard time understanding why the Chinese Room Argument is compelling. It seems that to me that the only way a person could do this would be to learn to (maybe accidentally) understand Chinese. If a person cannot do this in the way that Searle imagines a computer could do it (internalize & execute a ‘program’ without understanding it) – then imagining a person as a placeholder for a computer to demonstrate that computation does not require intentionality, doesn’t seem to me to be enough for Searle’s conclusions to logically follow.
ReplyDeleteIn other words, if, for a human to execute the computer program in practice, they necessarily ‘do something special’ (ie understand / symbol-ground), then the Chinese Room Argument seems arbitrary - the theoretical human who internalizes the rules of responding in Chinese does not elucidate anything for me- , and the argument does not convince me that a computer isn’t also doing ‘understanding’.
Also, I have two questions as a follow-up to the last seminar:
1. I just want to clarify for myself what we mean by symbol-grounding. It seems to me that the important difference between T2 and T3 is T3’s ability to symbol-ground (by virtue of T3’s ability to interact with the physical world). If, for example, a computer with a camera (can detect and respond to light signals), and represent them in some way in its program with symbols – is this symbol grounding?
2. (This one is also a reaction to the Searle paper) - If human cognition can’t be explained by computation (algorithms/programs) – what else is there? (Besides a religious notion of a ‘soul’, or ascribing a 'soul' to biological material)?
a. The hypothesis of computationalism is that T2 could be passed by computation alone.
Deleteb. The software is not infinite.
c. It is not relevant how long it would take a human to memorize it: the point is the same as if it had been the software for playing tic-tac-toe, but in some other, non-spatial code. The computer can play tic-tac-toe against a person. But Searle could memorize the software and execute it without knowing he was playing tic-tac-toe.
1. The words used by T2 are grounded by T3's capacity to identify and interact with the referents of the words it uses: "The cat is on the mat. That's a cat. That's a mat. There the mat is on the cat. I will take it off." But full T3-scale, lifelong (Gabe). (Appearance does matter with T3, however, up to a point, because to do many of the things humans can do, indistinguishably from humans, you do need some kind of body (though it could be an extra-terrestial one!). "Grounding" a khepera robot who can only walk and talk and whose only symbols are "wall," "bump" and "fall" would be symbol grounding, but trivial. Symbol grounding only becomes interesting when it is at or near T3-scale.
2. Besides hardware-independent computation there are lots of potential mechanical, physical, physiological and biochemical candidates. (But the hard problem is equally hard for all of them.) Because of the Strong Church/Turing Thesis, these are all computer-simulable, just like the virtual vacuum-cleaner...
"The only motivation for saying there must be a subsystem in me that understands Chinese is that I have a program and I can pass the Turing test; I can fool native Chinese speakers. But precisely one of the points at issue is the adequacy of the Turing test."
ReplyDeleteHere it sounds that Searle is actually rebutting the Turing Test by questioning the adequacy of it. Even by passing TT, Searle does not want to admit that the system, be it himself or a computer program, actually understands Chinese. And he mentions the word "intentionality" a lot in his arguments, saying that by simply manipulating symbols there's no intentionality involved, thus it doesn't contribute to anything related to understanding.
Further, he claims:
"Whatever else intentionality is, it is a biological phenomenon, and it is as likely to be as causally dependent on the specific biochemistry of its origins as lactation, photosynthesis, or any other biological phenomena."
This sounds like he is pursuing at least T4 or above to me, where it requires not only behavioral equivalence, but also structural equivalence. If based on this, I might argue that we cannot even confirm that all human-beings share the same cognitive mechanism, or we are equivalently "understanding" the world, given that everyone of us has different biological characteristics, which may or may not affect our ways of perception, our cognitive capacity, and so on, which is in fact not that reasonable.
Searle's CRA refutes computationalism "cognition is only computation" but he thinks it refutes the Turing Test and proves that cognition is not computation at all, hence the only way to explain cognition is by studying the brain. The CRA does not do all these other things.
Delete"If by 'digital computer' we mean anything at all that has a level of description where it can correctly be described as the instantiation of a computer program, then again the answer is, of course, yes, since we are the instantiations of any number of computer programs, and we can think."
ReplyDeleteThis also sounds quite contradictory to me. He first acknowledges that we all are computers and thus machines can think, but immediately denies that any program can think and thus strong AI is false. Is it because that strong AI is strictly restricted to only one program instead of a machine? In other words, even though it still does not appear too convincing to me, is he saying that just a intangible software can never think but there must be at least a tangible, physical device?
"As long as the program is defined in terms of computational operations on purely
formally defined elements, what the example suggests is that these by themselves have no interesting connection with understanding."
"In the sense in which people "process information" when they reflect, say, on problems in arithmetic or when they read and answer questions about stories, the programmed computer does not do -information processing." Rather, what it does is manipulate formal symbols."
In fact, from the very beginning of this paper until finishing reading it, I cannot help but keep focusing on one primary thought: I really cannot believe that only with pure symbol manipulation, one who does not understand a language at all can perform indistinguishably from any native speaker. Probably I'm questioning exactly the fundamental assumption of Searle's Chinese Room Argument. It is by no means that any non Chinese speakers could answer questions of a Chinese story just like any Chinese speakers would do only according to some formally defined rules. If languages were that easy, the field of natural language processing would most likely have been fully developed years ago, or at least far more advanced than what it is now. Therefore, I simply cannot take the idea that without "understanding" or "learning" anything, one can behave equivalently only with any previously defined rules.
That's exactly why Turing's hypothesis that machines can learn turned out to be incredibly influential, which also seems to be opposed by Searle. But I'm totally with Turing. I would even argue that AI in effect only comes into being when machine can learn by itself, instead of being programmed and what it does is only to follow the instructions. From my perspective, learning is in essence what "intelligence" means. Thus not only that any machine/system/device can never achieve behavioral equivalence by simply following rules/symbols, but that it's exactly learning and understanding that would help it to pass TT.
Lastly, "My response to the systems theory is quite simple: let the individual internalize all of these elements of the system." I would argue that it’s the beginning of understanding, and indeed it’s exactly what humans do. So I don't see why the internalization process cannot be somehow categorized as the process, or at least a small step, of learning or understanding.
Searle conflates computation, computationalism and the Strong Church/Turing Thesis (that almost everything is computer-simulable).
DeleteBeing able to learn Chinese (as a first or as a second language) would be part of both T2 and T3 (but only a part of it).
A computer that is passing Chinese T2 is not learning Chinese. It is executing the software that is producing the capacity to understand and say (anything) in Chinese; lifelong. Like Gabe (in English).
“Because the formal symbol manipulations by themselves don't have any intentionality; they are quite meaningless; they aren't even symbol manipulations, since the symbols don't symbolize anything. In the linguistic jargon, they have only a syntax but no semantics. Such intentionality as computers appear to have is solely in the minds of those who program them and those who use them, those who send in the input and those who interpret the output”
ReplyDelete“And the point is not that it lacks some second-order information about the interpretation of its first- order symbols, but rather that its first-order symbols don't have any interpretations as far as the computer is concerned. All the computer has is more symbols.”
I was first confused by the term “intentionality”. According to the previous quotes, I thought that it meant to attach a referent in the outside world to the symbol one is manipulating. Professor Harnard commented on a previous post that it meant to have “some objects in mind when you say them”. Searle in his room cannot attribute referents to the Chinese symbols because he doesn’t understand Chinese; and consequently, he doesn’t have intentionality when he “speaks/writes” it.
Therefore, there must be something that a Chinese speaker’s brain has that a formal system alone can’t provide to Searle. Searle says: “Whatever else intentionality is, it is a biological phenomenon, and it is as likely to be as causally dependent on the specific biochemistry of its origins”.
As I understand it, by saying that intentionality “originates” from the biological, it leaves the possibility that intentionality is something else than the brain itself. I thought Searle’s point was to take us away from dualism. Why can’t the biochemistry of the brain simply be intentionality? Implying that intentionality originates in the brain and depends on its biology again puts us on a quest for something that is intangible as a soul or a spirit is. I would have pushed this claim against dualism a bit further, by denying that intentionality is anything else than the biochemistry of the brain itself.
I think when Searle says that intentionality "originates" from the biological, he means that the quality of the phenomenon exists it things that are alive. The brain, too, exists in the biological -- and all phenomena that arise from the brain (such as feeling, consciousness) are part of the "hard problem". Searle is referring to his previous statements in why a piece of metal on the wall (a thermostat) is not alive -- it does not have beliefs. Thus, in line with his own arguments about what it means to be "living," Searle would be correct in saying that "intentionality" (which is about feeling not doing) originates from the biological. The biochemistry of the brain could be "intentionality", but it would not by any means be "simple" (or possible) -- if you could create a brain that undergoes every biochemical system, and mimics every cause-effect process, perhaps then you would have what we call "intentionality." But to create such a "biological" entity, the brain would have to pass T3 (and, consequently, T2).
DeleteLike you mentioned, and Prof Harnard mentioned above, it is even more useful to abandon the word "intentionality" to begin with, using instead "understanding" (having some objects in mind when you say them).
Forget the weasel-word "intentionality" (it is a conflation of intended-doing and intended-meaning anyway). Replace it here by "understanding (Chinese)" and remember that understanding is not just something that you do (T2); it also feels like something to understand. That's what makes understanding (or anything) a "mental" state: it is a felt state. So if Searle says he doesn't understand Chinese (or Urdu), he knows what he is talking about! Computation alone cannot produce understanding.
DeleteOn the other hand, when he says only the brain can do it, he is overplaying his hand: Maybe so, maybe not. As with the question of whether T3 (Gabe) understands, Searle's Periscope does not work.
Searle 1980
ReplyDeleteAfter reading Searle, one thing that struck me in particular was how he easily dismissed the strong-AI objection to his interpretations of the Chinese symbol case. While he does not necessarily disagree with the objection, he dismisses it as a farfetched claim;
“It is simply more formal symbol manipulation that distinguishes the case in English, where I do understand, from the case in Chinese, where I don’t” (Searle 1980)
In this argument, Searle deems semiotic interpretation, or at least organization as purely necessary for understanding, rather than sufficient for it to occur. I believe that this is possibly a mischaracterization of what semiotic interpretation is and more importantly, a possible underestimation of the cognitive capacities which are necessary for such a process to effectively occur. Understanding might indeed be necessary for the process of semiotic interpretation, and as such in the Chinese experiment, a machine’s ability to process the symbols as prescribed by the “programming” do in some way demand for certain cognitive processes that verge, or at the very least intersect with understanding, like categorization for example.
Matt, you need to speak more kid-sibly!
DeleteForget "semiotic interpretation." Searle understand English and not Chinese -- unless he learns it; and to memorize and execute a (hypothetical) T2-passing algorithm is not to understand Chinese (any more than to memorize and execute a recipe for extracting the roots of quadratic equations is to understand what it means to extract the roots of quadratic equations.
“According to strong AI, the computer is not merely a tool in the study of the mind; rather, the appropriately programmed computer really is a mind, in the sense that computers given the right programs can be literally said to understand and have other cognitive states.” Searle intends to refute this claim by introducing the Chinese Room thought experiment. He argues that the “instantiation of a formal program with the right input and output” (which in his experiment takes the form of Searle, a monolingual English speaker, generating native-like responses to questions formulated in Chinese through the manipulation of Chinese characters according to a specific set of rules), is not sufficient for understanding. So, cognition does not (only) consist of manipulating symbols, after all. However, Searle doesn’t clarify what it is that we do in addition to symbol manipulation, which would be able to generate ‘real’ understanding (i.e. grasping the meaning of Chinese symbols, rather than seeing them as arbitrary squiggles and squoggles). Searle only states that for a program/mechanism to be capable of understanding, it must be able to duplicate the causal powers of the brain, which suggests that the answer, according to Searle, lies within our brains.
ReplyDeleteI agree with Searle that neither Searle himself, performing the Chinese Room Experiment, nor a computer program executing the same algorithm, are capable of genuine understanding. I do wonder however, where the other mind’s problem fits into this. We can only infer that other human beings have minds (and feelings) from looking at their behavior. Hence, when a computer program displays behavior that is indistinguishable from ours, shouldn’t we attribute it a thinking mind as well?
Searle does say what he thinks is doing the job: the brain. He's right, but that doesn't tell us how the brain is doing it -- just that it cannot be just computation. He certainly does not show that only the brain can do it (whatever it is).
DeleteGabe (T3) is doing it, because (hypothetical) T3 can do other things people can do, but (hypothetical) T2 alone cannot.
Hi Erica, this response is coming late, but I thought I'd focus on your skywriting, as it seems to have ties to one of the replies against Searle's CRA, the Other Minds Reply.
DeleteThe critics who give this reply to Searle, argue along the following lines: How can we ever know in our daily lives if someone understands Chinese? Well, we know this because they behave as though they understand Chinese: they speak Chinese. And, since Searle in his Chinese room is behaving as though he understands Chinese (he is speaking Chinese), we must say that he understands Chinese.
Searle suggests that this is an inadequate response. How we decide that someone understands Chinese is not the point. Rather, what is important is what is *involved* in Chinese understanding. This is unexplainable by input-output program alone: Searle’s argument is just that computation is not enough for understanding.
I now understand my confusion about the other-minds-problem and Searle’s CRA. The OMP can be ‘circumvented’ for computational symbol-systems (the T2 implemented by Searle) because by definition computation is implementation-independent. This allows us to ‘penetrate’ the computational system’s mind and ‘experience’ the states it is in (Searle’s Periscope). Since in doing so we realize that no understanding of Chinese ever arises – it’s just meaningless squiggles and squoggles – we can conclude that computational systems don’t understand. So, cognition cannot be just computation, because it feels like something to us to understand or to not understand.
DeleteSince human beings aren’t implementation-independent computational systems, Searle’s Periscope fails, making the OMP the problem that it is.
You talk about Searle's CRA of speed. I am not quite sure I understand why Searle argues that speed dismisses the CRA. I did not read anything in the text on speed. Would you please explain the importance of this remark? Why does Searle believe speed is an important factor to discuss?
ReplyDeleteIn your video, you say something along the lines of: a T3 robot could not be simulated by Searle without being the system… he would have to be the robot... If he learned Chinese well enough, then he would understand Chinese. In Searle’s definition of ‘understanding’ he states “understanding implies both the possession of mental (intentional) states and the truth (validity, success) of these states. For the purposes of this discussion we are concerned only with the possession of the states.” How does this imply that Searle (in the CRA) would ‘understand’ Chinese? I feel Searle would still argue this is not at the level of understanding, rather he is still manipulating symbols at a higher capacity. Would you explain this refutation of Searle's thought experiment in further detail? When learning a new language, what is the point we can consider we truly 'understand'?
Searle does not cite speed one way or the other. Some of his System-Reply critics reject his reply that he can become the whole system by memorizing and executing the T2 algorithm because there is not enough time to memorize it and Searle is too slow to execute it. (That criticism is weak, and misses the point of Searle's thought experiment: That computation alone could pass T2 is itself just imaginary; Searle can imagine himself as speedier, but with no reason to believe he would "accelerate" to understanding from it.)
DeleteI find myself in agreement with most of Searle's arguments. Searle argues that a program or computation is just a formal operation made on symbols. For the program or the machine these symbols do not hold an intrinsic meaning (no semantics, only syntax). The machine does not perceive, act, learn or understand these symbols. From Searle's writing and my comprehension of it, understanding requires the presence of these elements of perception, acting and learning. Meaning requires a learned association with between a symbol and a referrent, which in my opinion, in addition to the elements listed by Searle also require memory of this understanding that was formed in the past.
ReplyDeleteThe Turing test (t2) does not account for understanding. Similarly the notion of strong AI, that mental operations merely consist of computation over formal elements, also ignores the notion of understanding, as thus does not encompass all of cognition. As such minds are not to brains what programs are to hardware.
However, there is one distinct point Searle made that is unclear to me. Searle says that the "mental-nonmental distinction cannot be just in the eye of the beholder but must be intrinsic to the systems". What does he mean by this? Is he just referring to the fact that a system is deemed mental only if it is capable of understanding?
1. Yes, a causal connection is needed between words and their referents (and that requires T3), but it is an understatement to call that just an "association." (Association explains nothing except maybe nonsense-syllable pairing.)
Delete2. Searle is referring to the other-minds problem. "Mental" means "felt." Understanding is felt. But there is no way you can tell whether Gabe (or anyone other than you) is feeling anything. Searle uses "Searle's Periscope" (the hardware-independence of computation) to show that a computer passing T2 would not understand.
During class, I really struggled with the idea of what it means to truly feel and how a T3 could potentially achieve that. Having been at the first class of the semester and having you go over Searle's Chinese Room thought experiment, I thought that it proved that T3 would not be capable of feeling. If Searle, a human being, was in place of the machine, and even he couldn't understand chinese, then how can we say that a machine could do so? It seemed as though having a machine with all the senses as a human does was similar to Searle's Chinese Room, because Searle was human himself in this experiment and had all the senses to process as well. However, the more I thought about it, and after the paper, I realized that it was in fact modeling a T2, as Searle was limited to the room with only 1 input and 1 output. If that is the case, then cognition could not be achieved. And if strong AI is decided by the Turing test alone, then it definitely couldn't prove that computation = cognition. However, it is hard to see Searle's argument come into play for T3. If Searle's room had now opened to more senses and he could hear the Chinese, see the Chinese and someone saying it, feel the objects that the Chinese was refering to, then wouldn't Searle be able to eventually understand Chinese? I guess that goes into supervised and unsupervised learning and what you would need to comprehend Chinese, but just based on feeling, I think that Searle would eventually be able to feel what it is like to know Chinese.
ReplyDeleteHes, Searle showed only that a (purely computational) T2 could not understand (because it feels like something to understand). "Searle's Periscope" does not work for T3, which is not purely computational. ("Stevan Says" T3 would feel, though we can't be sure, as we cannot with real people either, because of the other-minds problem.)
Delete“(...) let the individual internalize all of these elements of the system. He memorizes the rules in the ledger and the data banks of Chinese symbols, and he does all the calculations in his head. The individual then incorporates the entire system. There isn't anything at all to the system that he does not encompass. (...) If he doesn't understand, then there is no way the system could understand
ReplyDeletebecause the system is just a part of him.”
The difference between a human executing english vs machine executing english is about “understanding”. He states that the machine does not “understand” because it is just following instructions. However, a human understands because that human have a knowledge about the inputs that he/she is manipulating, and knowledge about the symbols that he/she is putting out. Then, considering this argument, if we were to give the knowledge about symbols to a machine it would be able to understand. He claims that a machine can only have this possession of mental states, only if causal powers a given to machine which I think from the explanation should be able to pass t4.
Why would it be that we can only create “understanding” by creating the causal powers of the brain? Maybe we do not necessarily need the “causal powers of the brain” to make a machine understand because of underdetermination. However to do that, we should be able to explain what this knowledge (about symbols) involves.
Even though, it is difficult to express what this knowledge of symbols involve, it is not any difficult than “creating the causal powers of a brain”. I think this, because he claims that creating the causal powers of the brain would be able to produce intentional states. And, I think that the set of “intentional states” would be the same thing as the “consciousness”. (in other words, consciousness is a set of “intentional states”) Then understanding problem is the same as “machine’s not having consciousness”.
Also to speculate a bit on these states/mental states:
“In the Chinese case I have everything that artificial intelligence can put into me by way of a program, and I understand nothing; in the English case I understand everything,”
Not all states that a human posses are mental states. It looks like these “mental states” are a subset of the “states” an understanding machine like human can possess. Then what qualifies to be a mental state. I am guessing it is the one that involves consciousness. Does a human at any given point possess hold all the mental states that they can possess. Probably not, if we see mental states as function of consciousness and inpus, ouputs to human beings. This does not necessarily mean that humans are not “understanding”. But, given the expamle of a human who can answer a few questions that are asked to them during their sleep without “consciousness” is enough to show that humans can have states that are not necessarily “mental”. But then, it is difficult to explain the “understanding” behind this event. Does this human qualify as “having understood the question” and answered (then the consciousness is lacking), the human did not understand the question (hence how can a human work on a “non-mental state”. I think the answer is: somekind of computation must be playing a role in how a system that “understand” can work)
can you please delete the previous one, as I mistakenly published it without a user name.
Delete"...if we were to give the knowledge about symbols to a machine it would be able to understand." But how do you do that?
DeleteA "mental" state is a felt state. An unfelt state is not a mental state (though it is still a state).
Understanding is a felt state. There is no such thing as "unfelt understanding."
Most of the internal states and processes that generate our capacity to do all the things we can do are unfelt, even though the doing itself, which they generate, is felt. Calling the unfelt internal causes of felt states "mental" is not informative.
After reading Searle’s paper, I think the most important question I have is what exactly does “understand” mean, or at least, what does Searle mean by “understanding”. Just like what he has mentioned, with which I totally agree, there are different degrees (kinds, levels) of “understanding”. After, he claims that there are clearly cases where “understanding” does not apply, for example, the automatic door. It seems that even though the result of the action is the same as a human opens a door but the door does not “understand” why it opens, but it “knows” it opens when some condition shows up (if...then…). Intuitively, it is illogical for me to say that “complexity” of the thing that is being understood or not can explain why the difference between something is understood or not exists. However, I was wondering when the complexity decreases, the way one can “understand” something seems to be lessened, and if we take it to the extreme, there is this very basic thing that can only be (I want to avoid using the word “understand”, but I don’t know which word works better, adopt, know, etc. , for now I will just use “know”) “known” in this one single way. Then, if a machine can “know” this, it has to be the same way as how a brain “knows” it, then, could I say the machine thinks the same way as a human does?
ReplyDeleteAt the end of the paper, Searle says that “Whatever else intentionality is, it is a biological phenomenon, and it is as likely to be as causally dependent on the specific biochemistry of its origins as lactation, photosynthesis, or any other biological phenomena”. (Aside from the fact that I am not quite sure I really understand what he means by “intentionality”,) it seems that he is suggesting that to test if a machine can think, T4 has to be used. But then, it reminds me of the other post that I made for 1b, “This seems to bring us back to the question of ‘the existence of zombie’. If there were such an ‘artificial human’ that has every material a human has, with the input that it receives, will it not create human-like ‘intelligence’ in itself?”
Understanding (or believing or knowing or meaning or wanting or seeing or willing) something is something you (1) are able to do (T2 + T3) plus (2) what it feels like to do it.
Delete"Degree of complexity" does not exlain any of it.
"Intentionality" is a weasel word. See my other replies on this.
There are lots of feelingless zombies (rocks, rockets, and probably also microbes and plants), but vertebrates and invertebrates are sentient (though no one know how or why: that's the hard problem). "Stevan Says" T3's are not zombies, but we cannot explain why not.
I don't follow what you mean in terms of "complexity", given the context that you refer to it after discussing whether or not an automatic door can understand that it's open. However, I suppose what you meant is the hierarchy of sentient beings and the relative strength of their understanding. For instance, a person's understanding of the social construct of politics versus a dog's ability to understand the concept of politics. Regardless, I think the more significant question comes back to how to resolve the easy problem (understanding how and why we can do the things we do - focused on external behaviours) and the hard problem (understanding how and why we have felt experiences).
DeleteI think it may be harder to wrap my head around felt states (avoiding the weasel word) because we can't observe them. Although the easy problem is still quite difficult, ultimately we expect to find a physical thing that plays a causal role in how and why we do the things we do. When it comes to felt experiences, I have no idea where that internal state may stem from. I'm accustomed to thinking of it as meta-physical/not observable.
The other minds problem is similar in that we can't observe other's thinking, we can only see their actions and infer that they have feelings without (yet?) knowing if they do feel.
"Suppose we design a program that doesn't represent information that we have about the world, such as the information in Schank's scripts, but simulates the actual sequence of neuron firings at the synapses of the brain of a native Chinese speaker when he understands stories in Chinese and gives answers to them. The machine takes in Chinese stories and questions about them as input, it simulates the formal l structure of actual Chinese brains in processing these stories, and it gives out Chinese answers as outputs. We can even imagine that the machine operates, not with a single serial program, but with a whole set of programs operating in parallel, in the manner that actual human brains presumably operate when they process natural language. Now surely in such a case we would have to say that the machine understood the stories”
ReplyDeleteIt is almost comical how the presented replies all add so much complexity and complication (i.e. peripheral sensors, neuronal firing) to the Chinese room examples in order to poke holes in the simple statement that the program does not feel or understand. Searle’s response are very simply put - if he in the Chinese room using his natural biological brain to implement the program does not understand, these elaborate constructions do not matter. From the eyes of a non-philospher, Searle’s counter arguments seem so ingenious (i.e. the analogies of the water pumps), but the gist of what he is saying is probably the innate response most humans would have: no, programs don’t understand like we do; no, a simulation is not the same as the real thing; no, the way I process a story is not the same as a programmed computer. It is interesting to me that those so deep in the AI field/proponents of strong AI would be trying to push those instinctual conclusions away.
Easy to say what most people, especially our grannies, believe. Not so easy to show whether and why they are right.
Delete"Because the formal symbol manipulations by themselves don't have any intentionality; they are quite meaningless; they aren't even symbol manipulations, since the symbols don't symbolize anything. In the linguistic jargon, they have only a syntax but no semantics. Such intentionality as computers appear to have is solely in the minds of those who program them and those who use them, those who send in the input and those who interpret the output."
ReplyDeleteI agree with Searle's arguments. A machine that performs symbol manipulations, gives verbal responses, and passes T2 does not mean it truly understands anything, like Searle said in the quote above, it is all syntax and no semantics. There is a feeling of understanding something that strong AI /computationalism does not explain.
I just wonder what he would have to say about the human brain before the emergence of language. How would thinking be described without using formal and natural language, without semantics as an explanation?
Understanding is not just verbal, and not just humans understand (though we are the only ones who produce language (and other species can only understand a small fragment of what we say, and their understanding is probably not linguistic).
DeleteOnce Searle has memorized the T2-passing program and is executing it on Chinese I/O he is not "inside" or a part of anything. He is doing everything that is being done (computation) and he is not understanding Chinese.
ReplyDelete“As long as the program is defined in terms of computational operations on purely formally defined elements, what the example suggests is that these by themselves have no interesting connection with understanding. They are certainly not sufficient conditions, and not the slightest reason has been given to suppose that they are necessary conditions or even that they make a significant contribution to understanding.” I understand the first part of the final sentence, which says that programs are not sufficient in constituting language. As he shows that all the programs do is manipulating formal symbol, and such manipulation of symbols is showed as inadequate for understanding Chinese via the illustration of CRA.However, it is not very clear to me from where Searle derives the conclusion that computational operation is not needed in understanding at all. To valid this argument, doesn’t Searle need to demonstrante that in some condition that does not involve computational operation, the understanding of Chinese can still occur? But it seems to me that Searle never talks about such kind of positive cases about understanding Chinese, so I wonder if the latter half of conclusion is still valid.
ReplyDeleteAnother problem that I have is that about a variation of CRA. What is Searle is given an English-Chinese dictionary which could make him understand what every word(and sometimes even with a picture) is referring to in Chinese？I suppose it would be harder in this case to deny that he understands Chinese, and isn’t this state also achievable for programs as well? Is this a plausible way to make the programs understand Chinese?
Searle does not show that cognition does not include some computation (e.g., maths, logic, stats); he just shows that computation cannot be all of computation.
DeleteTeaching Searle Chinese is irrelevant; the TT-passing computer Searle is replacing was not taught Chinese...
One of Searle’s argument is to say that Strong AI is wrong since “what matters are program, and programs are independent or their realization in machines; indeed, as far as AI is concerned, the same program could be realized by an electronic machine , a Cartesian mental substance, or a Hegelian world spirit”. His solution is that instead , we should be focusing on the brain, and that no other type of material than the brain will tell us something about understanding.
ReplyDeleteIn class we seemed to disagree with this point by arguing for example that Searle’s periscope wouldn’t work with T3 and that Searle doesn’t give us an answer on what the mechanism of thinking is.
I tend to agree with Searle. How would studying anything else than the brain, tell us something about how the brain works ? Even by replicating the brain functions, by creating a T3 or T4 Turing machine, the material , foundation is different which presents a problem. Why would T3 be able to feel? Adding sensorimotor capacities to a machine doesn’t tell us anything about its ability to feel or not.
The hard problem is inherent to human’s brains and creating machines that simulate human behaviour won’t give us more answers about how is it that we feel. Studying the brain will.
"I have tried to show that a system could have input and output capabilities that duplicated those of a native Chinese speaker and still not understand Chinese, regardless of how it was programmed."
ReplyDeleteThere is something unique about "understanding" that cannot be simply translated into a computer. There is a feeling and mental state associated with understanding that we don't even know how to describe with words, and that's not possible for a computer to have at this point. It could be argued that it has to do with being alive, rather than being human or needed language because animals have demonstrated that they also understand even though they aren't human and don't use language like us. In other words, the hardware is different but the function of understanding is occurring. Therefore, you could argue that because a computer isn't alive in the same way all other organisms are (biologically), it is not possible of understanding, and passing the T2 test doesn't really mean anything significant beyond being able to perform well with given inputs. So, although I think that more creative thinking could change my mind, I side with Searle at the moment.
I’m assuming that with ‘being alive’ you mean to be 'conscious'. I agree with you that computers (as well as toasters, vacuum cleaners and clocks) are not conscious. In addition to being very limited in what they can do, what they lack is the ability to ‘feel’. Humans and animals are conscious, it feels like something for them to be poked, to be hungry, or to understand. I also agree with you that understanding is not just something we do, but also something we feel. Just as it feels like something for Searle to not understand Chinese, it feels like something for Searle to understand English. Language is an important facet of how we can tell whether other people are conscious, what is going on in their minds (although we can never fully be sure that other people indeed have a mind or whether they are just unconscious zombies – the other minds problem). But as you state, the ability to use and understand language is not a requirement for consciousness, since animals (as far as we know) do not communicate with each through language, yet they are conscious. Language, as sophisticated as it may be, is just a behavior, something humans ‘do’.
DeleteAs we have discussed in class, a machine/robot (which is different from a computer) must possess sensory-motor capacities (i.e. be a T3 robot) in order to pass T2. This robot would have to be indistinguishable from humans in everything that it can do, which is much more than to just “perform well with given inputs”. In addition, for a robot to pass T2 is not an insignificant feat, since it would mean that we successfully solved the easy problem of Cognitive Science, moving us (possibly) closer to solving the question of how and why it is that we feel and why it is that we are conscious whereas a computer or toaster (or T3 robot?) is not.
The Chinese Room Argument places an emphasis on weak behavioural equivalence. But by saying that weak equivalence is enough, are we actually learning about cognition at all? Although a non-native and native Chinese speaker can generate a similar output that makes them indistinguishable from one another, focusing on the output tells us more about behaviour. Rather, understanding how the outcome is produced can tell us more about cognition and potentially even distinguish us from machines.
ReplyDeleteCorrect me if I am wrong or if I misinterpreted your critique, but I believe the point that Searle is making is specifically that weak equivalence of a human cognizer is not sufficient because "for any program you like it is possible for something to instantiate that program and still not have any mental states". Because it is possible to create things that replicate human cognition on the behavioural level as you say does not mean that this thing is actively thinking, because this thing does not show evidence that it has "the same causal powers as brains". If we were able to establish that, for sure, a computer has mental states and these are what cause other things to come about, then we would have a succesful model of cognition, but it doesn't seem like we are able to do that. As long as the formal symbol manipulation is what causes anything else (like Searle can generate responses to symbols he doesn't comprehend), we will not have a successful model of cognition.
DeleteThis comment has been removed by the author.
ReplyDeleteFirst of all, I completely agree with you on the first two paragraphs, and yes, I agree that computation does look like cognition (but I still do not think computation is cognition).
DeleteI think computers would be able to make connections between their “internal representation” of “rain” and “wet” and “cold” and all the associated words. (Mental lexicon, lemma, word-form nodes, co-occurrence.) But even if they can, does that necessarily mean that “they cognize”? Does a computer that can do all this really feel it like a human does? What is your definition for an internal representation? Is it derived from the sensorimotor experience (or maybe even the simulation of the sensorimotor experience) that is grounding the word? Is the actual meaning of “rain” necessarily *just* “’wet’ and ‘cold’ and all other things associated/connected with it”? Would not a kid who does not yet know the meaning of “wet” or “cold” still have the capacity to have a personalized meaning of “rain”? (Gestalt psychology - the whole is greater than the sum of its parts (?).)
Re: Minds, brains, and programs
ReplyDelete“As regards the second claim, that the program explains human understanding, we can see that the computer and its program do not provide sufficient conditions of understanding since the computer and the program are functioning, and there is no understanding.”
Searle's "Minds, Brains, and Programs," shows that a computer can have input and output capabilities that perfectly duplicate those of a native Chinese speaker, and yet still not understand Chinese, regardless of how it has been programmed. From my interpretation of Searle's Chinese room argument, I agree that the importance of the logistics behind the manipulation of symbols is irrelevant. I think the argument tries to explain and convince readers that manipulation of symbols does not imply understanding of those symbols. This relates to a question I posed regarding an earlier reading. My question was: since language is an ever-changing thing, wouldn’t Searle's Chinese Room one day fail the Turing Test if Searle was told to never break the symbol manipulation rules given to him from the start? I wondered this with the belief that since certain words could have new meanings and certain new words may be added to the Chinese language, a machine would have to learn and adapt accordingly with the times, which may imply an involvement of some sort of consciousness. Although, I now understand that even if we are to include the possibility that there may be some third party that could update the system with more adapted rules with time, we still would not conclude that there is consciousness, understanding or feeling of the Chinese language. Searle argues that the batches of Chinese writings and instructions given to the person in the room mimic programs given to a computer.
Something which I find very interesting regarding Searle's CRA and computation in general is the fact that computation is implementation-independent. In Searle's CRA he embodies a T2-passing computer and does only computational manipulations of symbols such as a computer would do. Searle is able to "be" (in a sense) the T2-passing computer because computation is implementation-independent. Being implementation-independent means that it doesn't matter what hardware the software is being run on. Thus, it doesn’t matter what the computation is being done by because any hardware could have done it (in this case, Searle is the hardware).
ReplyDeleteI see this as being connected to the idea of underdetermination which essentially says that you can get the same ends through different means. If we take the example of reverse engineering we can see that it doesn't matter what the thing looks like or is made out of as long as it performs in the same way. Take the case of a reverse engineered or fake heart: a fake heart is not 'imitating' a heart (as a T2 would be 'imitating' a human) BUT it is *doing* what a real heart does. When the fake heart is doing everything that the real heart can do it doesn't matter what the fake heart looks like or what it is made of (as long as the body doesn't reject it, that is).
"Strong AI only makes sense given the dualistic assumption that where the mind is concerned, the brain doesn’t matter. In strong AI what matters are programs and programs are independent of their realization in machines "
ReplyDeleteI wonder how proponent of strong AI would justify this claim? Why wouldn’t the hardware matter? For example a calculator can generate calculation but can’t generate verbal outputs like a Turing machine even if we tried to install a verbal software on top of the hardware!
I think it’s naïve to ask ourselves the question: “Could a machine think”. First of all, why does he put the “brains” in the same category as “machines”. I feel like it cancels the human behind the brain. Humans are not robots. Also, to think it’s a human property. Can machines do similar actions to ours that could lead us to understand that “thinking” can be cut down in parts and then be scientifically analyzed.
ReplyDeleteSearle’s Chinese room argument against the computational theory of cognition focuses on strong AI.
ReplyDelete“Appropriately programmed computer literally has cognitive states and that the programs thereby explain human cognition”. What are cognitive states and how are they outlined? If like most things in the human experience cognitive states are seen to be dimensional rather than categorical, the dilemma of cognitive states themselves is that they are themselves human constructs, if we were to draw the lines between cognitive states they would be extremely construed.
Looking at Schank’s program that “the aim of the program is to simulate the human ability to understand stories” further definitions come into question. How is understanding measured, is it enough to be able to answer questions about a story to be able to understand it in a human way? How would this account for feelings which we have established accompany every human experience. Additionally, the word simulates here is problematic as simulate itself implies that it is only imitating the true thing rather than being something in itself. So what is lacking to make the machines understanding only a simulation, not a true understanding of the story? Computers do not imitate. They perform the same function. If you are saying that a machine does not truly understand the story, what would it additionally have to do to qualify for having “understanding”?