Harnad, S. (2001) What's Wrong and Right About Searle's Chinese RoomArgument? In: M. Bishop & J. Preston (eds.) Essays on Searle's Chinese Room Argument. Oxford University Press.
Searle's Chinese Room Argument showed a fatal flaw in computationalism (the idea that mental states are just computational states) and helped usher in the era of situated robotics and symbol grounding (although Searle himself thought neuroscience was the only correct way to understand the mind).
"Consider reverse-engineering a duck: A reverse-engineered duck would have to be indistinguishable from a read luck both structurally and functionally: It would not only have to walk, swim and quack (etc.) exactly like a duck but it would also have to look exactly like a duck, both externally and internally. No one could quarrel with a successfully reverse-engineered duck like that: no one could deny that a complete understanding of how that candidate works would also amount to a complete understanding of how a real duck works. Indeed, no one could ask for more."
ReplyDeleteIf the above describes D4 and shortly after, Stevan says D3 requires only functional indistinguishability, I am having trouble understanding how Gabe is not the “human” equivalent of D4 (as we keep calling him a robot or T3). Certainly, Gabe is functionally indistinguishable from another human being (meaning he “passes” T3), but he is also structurally indistinguishable from us. So, wouldn’t he also “pass” T4? What I mean is that any reverse-engineered human is structurally and functionally indistinguishable from me. As Stevan says: “no one could deny that a complete understanding of how that candidate works would also amount to a complete understanding of how a real [human] works.” And by Stevan’s definition of a reverse-engineered human passing T3 as only functionally indistinguishable from me, I would think that a better representation of a T3 would be one that looks like a (“traditional”) robot but shares that same functions as I do.
T3 is total indistinguishability of sensorimotor performance capacity. T4 is total indistinguishability of sensorimotor performance capacity and internal structure and function. If we disregard our knowledge that Gabe is really a human, we really have no idea what Gabe’s internal structures and functions are. In his paper, Professor Harnad states “. . . aiming for functional equivalence alone still leaves a lot of structural degrees of freedom open. (Those degrees of freedom would shrink still further if we became more minute about function -- moulting, mating, digestion, immunity, reproduction -- especially as we approached the level of cellular and subcellular function. So there is really a microfunctional continuum between D3 and D4”. There are a number of ways Gabe can be functionally equivalent while have some structural similarities to humans (such as his appearance). The main difference between T3 and T4, I believe, is when one starts replicating cellular, neurochemical, and hormonal equivalence. These structural capacities, among others, would be T4 not T3.
DeleteIn my view, just because we can't observe the inner functionality of our pet robot Gabe, doesn't mean that it is not T4. If we were never to test the internal structures of Gabe, we would have no reason to suspect that he were not a human thus making him T4-apparent. Is the appearance of T4 not sufficient by most standards and arguably the most important test of being T4?
DeleteGabe is just T3, not T4. (Actually he's a real person, but he is playing the role of a T3 for us, not a T4. External appearance does not matter for either T3 or T2. External and internal appearance matter for T4.)
Delete“If cognition is a hybrid, computationalism is still wrong.”
ReplyDeleteI am unsure about what the other part of the hybrid would be. Could this be equated with the hard question of cognitive science that it is how we feel that is missing from this hybrid “equation” of cognition?
The other part is any other physical causal mechanism other than just hardware executing a computation.
Delete“And even in conscious entities unconscious mental states had better be brief! We're ready to believe that we "know" a phone number when, unable to recall it consciously, we find we can nevertheless dial it when we let our fingers do the walking. But finding oneself able to exchange inscrutable letters for a lifetime with a pen-pal in this way would be rather more like sleep-walking, or speaking in tongues (even the neurological syndrome of "automatic writing" is nothing like this; Luria 1972). It's definitely not what we mean by "understanding a language," which surely means CONSCIOUS understanding.”
ReplyDeleteI’m not sure that I completely understand why “understanding a language” would need to be conscious. While it is true that dialing a phone number, that we have consciously forgotten, through the use of unconscious motor memory is an example of an unconscious mental state, I don’t see why understanding a language is not also considered an unconscious mental state. Similar to motor memory, understanding a language requires no conscious effort to evoke the meaning of a word or sentence. These things are implicitly known, similar to how we unconsciously remember how to walk.
Additionally, I’m not sure I understand why unconscious mental states need to be brief in conscious beings. Surely all of our conscious mental states are influenced to some degree by unconscious mental states, for instance, when speaking to a potential sexual partner, our conscious mental state may consist of how attracted we are to this person, what we are going to say next, or how we will woo this person. However, our unconscious mental states might consist of our implicit understanding of this person’s body language, which would greatly affect how we decide to consciously woo this person, and this would not be a brief unconscious mental state.
I suspect that the author took this into account, which is why I’m wondering what was actually meant by this. It is mentioned that even “automatic writing” is dissimilar to the unconscious state of remember a phone number through motor memory, but it is not explained why. For these reasons, I am having trouble agreeing with the idea that understanding a language means consciously understanding it.
"Conscious" means felt. So does "mental." Unfelt brain processes are internal (to the brain) but not mental (although psychology often loosely conflates the two.)
DeleteIt feels like something to understand this sentence. And it feels like something else to understand this sentence. Searle can't do that in Chinese. (We're talking about what if feels like to understand and mean what you say; we are not talking about understanding or feeling how we manage to understand and mean: we leave that to cogsci and those trying to solve the easy problem.)
"Implementation-independent implementations of computer programs" is a particularly confusing turn of phrase. Just to be sure, is this referring to how software/cognition could theoretically (according to computationalism) operate in the same fashion with different kinds of hardware/implementation?
ReplyDeleteIs the sentence “it is merely special pleading to try to save computationalism by stipulating ad hoc (in the face of CRA) that implementational details do matter after all, and that the computer’s is the “right” kind of implementation, whereas Searle’s is the “wrong” kind” referring to the Robot Reply?
I’m not certain I understand how the Systems Reply may be partially correct. It appears that it’s right in suggesting that computationalism is at least somehow related to cognition, despite not being the entire answer. However, I don’t see any examples offered, aside from the “subsystems” one for how that could be?
Computation is independent of its physical implementation (hardware); the same software can be implemented on many different kinds of hardware. But it does have to be implemented somehow, otherwise it is not computation.
DeleteThe Systems Reply is wrong for T2, passed by a computer. But it would be right for a T3 robot, if your ran the computational part and left out the sensorimotor functions or other noncomputational functions of a T3 robot, (e.g., temperature): the T3-passing would depend on all its functions, not just its computational ones.
“[Searle’s] critique has helped open up the vistas that are now called "embodied cognition" and "situated robotics," and they have certainly impelled me toward the hybrid road of grounding symbol systems in the sensorimotor (T3) world with neural nets.”
ReplyDeleteHarnad’s review of the famous Chinese room argument (CRA) is mainly relativizing its claim: rather than showing that cognition cannot be computation whatsoever, the CRA can only indicate that computation alone is not enough to explain cognition (without Searle knowing it).
In addition to that, according to Harnad, the CRA, because of its design based on Chinese verbal interaction, is only undermining T2, not the higher levels T3 and/or T4 (Searle himself was arguing – without noticing – for the validity of T4 by stating the importance of the brain’s structure to explain cognition – even if we are not obligated to agree on the necessity of T4 for cognitive science).
This last point is important, because if T3 and/or T4 are not proven wrong, it means that they can still be useful for cognitive science. Indeed, T3 (sensorimotor indistinguishability) seems to be leading the way toward what Harnad calls the “grounding” of symbol systems – even though Harnad do not details in this paper what this expression entails exactly. On the face of it, the expression seems to imply “connecting” symbols manipulation (because it is not getting rid of computation all together) with real ground activity, but it is not clear for now how this connection would be done without falling again in the peripheric sensorimotor I/O computationalism - which leads to the very same problem Searle was knocking out with its argument.
Sensorimotor grounding means connecting words (symbols) to their referents in the real world. (There is no "peripheral sensorimotor I/O computationalism": computationalism is symbol-manipulation: symbols in and symbols out. If it takes more than symbol-manipulation, it's not computationalism (and cognition is not just computation.)
DeleteI am a bit confused. In a previous skywriting, you indicated that "Searle also misunderstood the importance of robotic (sensorimotor) capacity, agreeing with the computationalists that sensorimotor function is just the peripheral I/O devices of a Turing Machine."
DeleteI was wondering what "peripheral I/O devices of a Turing Machine" means in this context. Are these devices extensions in which the data is "translated" into symbols to allow their manipulation by a Turing machine? And then the Turing machine would translate again these symbols into motor actions? And if so, in which way would it be different from the "grounding of symbol systems" described as the connection between symbols and their real-world referent?
Think of eyes and ears. They are not peripherals connecting to a computer. Much of the brain is sensorimotor too; and the retina is actually part of the brain. And there's no reason why a lot of sensorimotor processing can't be done in analog form, never turning it into computation. Acoustic amplitude goes into frequency, but it is still proportional, even though frequency is discrete.
DeleteAnd the most important point is that analog processing, unlike computation, is not hardware-independent.
If Gabe passes T3, any or all his analog components could be crucial to his cognitive capacities, hence cognition. His cognitive state is not just the state of a computer CPU in his head (if he has one). It could be any of the noncomputational stuff too.
One of the first thing that this article does is the redefinition of the stances that CRA is refuting under Computationalism (as opposed to what Searle described as Strong AI). Searle says that proponents of Strong AI believe in three propositions: 1- the mind is a computer program, 2 – the brain is irrelevant and 3- the Turing Test is decisive. The article changes those to match the beliefs of computationalists as opposed to proponents of Strong AI.
ReplyDeleteI’m confused regarding the fundamental differences between Strong AI and Computationalism, and why it matters in this specific debate. In Searle’s paper, he describes strong AI as “the appropriately programmed computer really is a mind, in the sense that computers given the right programs can be literally said to understand and have other cognitive states”. I’m inferring from this description that the main difference between Strong AI and Computationalism is that computationalists think all cognition is just computation, while that isn’t a tenet of Strong AI. Why is this distinction relevant in the CRA?
In the second piece, “the mind is a computer program” (which seemingly has no implications regarding what exactly it’s doing whether that’s computation only or not – it just states that the mind the same thing as a computer program) is changed to “mental states are just implementations of the right computer programs”. Then, we are told (same article) that this is equal to “mental states are just computational states”. I’m not sure I buy that those two things follow each other. Why does the “right computer program” have to be computational in nature?
Hi Veronique,
DeleteI agree with you. I am also not entirely sure why the redefinition of Strong AI needed to be done in order to understand Searle’s argument. Is it to help understand where Searle sees the biggest flaw in computationalism (ie. That computational states are implementation-independent rather than the brain as a whole being irrelevant)? It seems clear already that Searle thinks the brain is not irrelevant when he states that “actual human mental phenomena might be dependant on actual physical/chemical properties of actual human brains” (Searle, 13). Does the redefinition simply imply that even if computational states are implementation-dependant they are still computational states? This then proves Harnad’s point that Searle took his CRA too far when he implied that cognition couldn’t be computation at all (because they are in fact defined as computational states).
1. Computationalism is exactly the same thing as "Strong AI."
Delete2. Nothing has been redefined; they have been described in other words -- words closer to what Turing and the other definers of computation used, and meant.
"For although we can never become any other physical entity than ourselves, if there are indeed mental states that occur purely in virtue of being in the right computational state, then if we can get into the same computational state as the entity in question, we can check whether or not it's got the mental states imputed to it."
ReplyDeleteThough I do not fully understand the concept, Searle's Periscope presents an alluring response to the other minds problem. The theory implies that if computationalism were true, such that every cognitive state can be physically realized through computational states, one could arguably "know" another entity's mental life. For if computational/mental states are implementation-independent, an entity's thoughts could theoretically be replicated or imitated in another.
I wonder if Searle's Periscope could somehow be applied to our long debate about whether or not we would kick Gabe. It seems that the question of whether or not one can discern whether another entity has consciousness will always be unanswerable until we truly figure out how the mind works; however, in this context, the principals of Searle's Periscope would assert that if one can replicate Gabe's computational state, then can either confirm or deny that he/it is thinking in the same way as us.
Having said that though, I believe that cognition requires far more than computation to experience consciousness and so, the other minds problem remains a mystery. As Harnad states, all we have now are "biologically prepared inferences and empathy based on similarities to our own appearance, performance, and experiences." The extent to which another entity behaves similarly to us ultimately determines how we respond to it.
Searle's Periscope only works for T2, if passed through computation alone. It is ineffective for T3 (hence Gabe). It does not solve the other-minds problem; it just describes what would be a solution if computationalism were true.
DeleteFictions can easily solve problems. Telepathy would solve the other-minds problem too. And psychokinetic dualism (in which the mind is a 5th force in the universe) would solve the hard problem -- just as the intervention of a benign omnipotent deity would solve the climate problem...
With regards to Searle’s Periscope (in order to check an entity’s mental state, we need to “get into the same computational state as the entity in question”):
ReplyDeleteIf we’re humouring that computationalism is in fact true – if cognition is just computation – I would think that the following premises are easily-accepted: our minds are so complex that everyone is in a unique mental state all the time, because our mental states are cumulative/dependent on previous mental states (as the Turing machine), which are all informed by a completely unique set of sensorimotor input we’ve received over a lifetime.
So would not ‘getting into the same computational state’ require having had the exact same program, and sensorimotor inputs into that program, consecutively, as the other entity – in other words, isn’t it still true that in order to experience another entity’s mental state, we need to “become” that other entity?
Is a system not "protected" by virtue of being a unique execution of a (maybe unique) program?
At any given moment, a Turing Machine is in whatever state it is. The state may have been shaped by its history, but the state is what it is now. The history is not still lingering there, like the bacterial starter of a (vegan) yoghurt. Or, rather, if there are traces of prior history, they are now part of the current state. So if cognition were computation (no matter how "complex"), then the Turing Machine's current state could be duplicated on another TM. (Of course, like a biological clone of me, concocted just now by a 3-D printer standing to my left, our states would immediately begin to diverge, as I looked at him on my left and he looked at me on his right... That's when history starts putting an end to our identical starting state.)
DeleteBut enough sci-fi: cogsci's task is hard enough already...
"an executing program cannot be ALL there is to being an understanding system, but wrong that an executing program cannot be PART of an understanding system"
ReplyDelete"Note that T2 covers all cognitive capacities that can be tested by paper/pencil tests (reasoning, problem-solving, etc.); only sensorimotor (i.e., robotic) capacities (T3) are left out."
I guess I'm still not clear enough on why programs cannot be all of an understanding system. I still dwell on the idea that when it comes to cognition, our primary concern lies mostly on the computation part, instead of the execution part. Thus the apparatus that are not involved in computation are just peripherals to the whole system.
Furthermore, I totally agree with this claim:
"But in the human case, "quacking" is a rather more powerful and general performance capacity, and some consider its full expressive power to be equivalent to, or at least to draw upon, our full cognitive capacity."
Therefore I still cannot fathom why T2 is not enough to prove that a machine passing TT does not have the same cognitive capacity as human-beings.
This argument is extremely intriguing. But I'm just wondering how "we can get into the same computational state" so as to refute computationalism?
"Can we ever experience another entity's mental states directly? Not unless we have a way of actually BECOMING that other entity, and that appears to be impossible -- with one very special exception, namely, that soft underbelly of computationalism: For although we can never become any other physical entity than ourselves, if there are indeed mental states that occur purely in virtue of being in the right computational state, then if we can get into the same computational state as the entity in question, we can check whether or not it's got the mental states imputed to it."
A computer has been passing T2 in Chinese for years through email, daily. Searle memorizes the algorithm (including the current computational state of the computer) and takes over in executing it, starting with tomorrow's email input.
DeleteYou have the intuition that cognition must be computation. Now you have to understand that it is that intuition that is on trial in the Chinese Room, and the outcome is that cognition cannot be just computation, for the same reason Searle, executing Chinese T2 for the next forty years, will not understand Chinese.
This may help: What makes you think that understanding language is not dependent on sensorimotor capacity (which is also cognition, after all, but not computation)?
“Whatever cognition actually turns out to be -- whether just computation, or something more, or something else -- cognitive science can only ever be a form of "reverse engineering" (Harnad 1994a) and reverse-engineering has only two kinds of empirical data to go by: structure and function (the latter including all performance capacities).”
ReplyDeleteSearle finishes his paper with the idea that intentionality, or as Harnad says “conscious mental states”, originates in the biological. If cognition is in that sense embodied, I have a difficulty understanding why structure doesn’t matter more in our assessment of cognition in other beings (robots or not). By structure I mean sharing a similar biochemistry that is required for the functioning of the brain. I think this touches Harnad’s point on “structure/function coupling constraints”. The Turing Test even at a TT3 level assesses functional equivalence, but some similarities in structure are required for functional equivalence to occur.
In response to Chloe,
DeleteSearle makes the argument that intentionality originates in the biological, but this is not for certain. The question of how similar in structure a being (like a robot…) must be to humans in order to demonstrate intentionality still remains. Yes, Professor Harnad mentions the structure/functioning coupling, but he also mentions that despite this coupling there is still a lot of structural degrees of freedom. We can hypothesize that the more functionally accurate we want a robot (or other being) to be, that the more it must structurally resemble a human biologically at a cellular level. I believe that in regard to cognition, structure could be important, but I am not inclined to believe that a being/robot must be structurally identical to demonstrate cognition. I am not inclined to believe that we need to put emphasis on structure over function.
Structure is T4. But not all of T4 is relevant for cognition (respiration and temperature control, for example). The best arbiter of what -- if any -- of T4 is necessary for cognition is T3. If you need part of T4, you won't be able to pass T3 without it.
Delete“Mental states are just implementation-independent implementations of computer programs. This is not self-contradictory. The computer program has to be physically implemented as a dynamical system in order to become the corresponding computational state, but the physical details of the implementation are irrelevant to the computational state that they implement -- except that there has to be SOME form of physical implementation. Radically different physical systems can all be implementing one and the same computational system.”
ReplyDeleteIf I’m understanding this correctly, mental states are implementation-independent implementations because they have to be implemented by some sort of physical system, but the actual characteristics of said physical system are irrelevant. The implementation system could be a computer or a mechanical flamingo as long as the computational state corresponds to a specific mental state. This contradicts Searle’s argument because according to Searle, mental states must be implemented by the brain’s specific biochemical processes.
While I do agree that Searle may have argued a few points rather clumsily, I’m still not sold on the idea that mental states are implementations of a computer program. Weak AI appeals to me; *modelling* mental states as computational states and using principles of engineering and computer science to reverse engineer certain forms of cognition seems much more acceptable to me than fully accepting the Strong AI hypothesis that the mind simply *is* a computer (or any other implementing physical system other than the brain).
Hi Emmanuelle!
DeleteSearle's implicit argument of "implementation-independence" is equivalent to the software/hardware distinction ("software is hardware independent"). Using your example, if an implementation system of a computer and a mechanical flamingo have the same "software," or can "do" the same thing, this is an example of weak AI or behavioral equivalence: it does not matter the "how" as long as they "do" the same thing. I think I am agreeing with you here: successfully implementing "mental states" in this case, would be solving the hard problem (while also implying that cognition is computation).
For the same reasons, weak AI appeals to me too. But I do not think weak AI would model mental states, but rather states of "doing." I think it is impossible to model mental states, because mental states (along with consciousness, intentionality, understanding) are about feeling -- the impossible-to-model problem that Turing sets aside from the very beginning of his methodology.
We agree that the mind is not a computer, so narrowing our focus, towards models of doing (weak AI) is the more plausible and productive methodology of cognitive science (thanks Turing!). Prof Harnard emphasizes this in his paper: there is no better *empirical* basis for studying cognition that the TT, in other words weak AI, functional equivalence, behavioral equivalence.
"Mental states" are for "feelers" to speculate (perhaps with eternal lack of productivity)...
See 3b for what Searle's argument does and does not show. (He was wrong that the only option cognition=computation was what cognition=brainfunction.
DeleteWeak AI uses computation to simulate cognition (as it simulates a vacuum-cleaner); it cannot actually produce cognition, which requires sensorimotor hardware. Weak AI can only simulate doing, just as a simulated airplane only simulates flying. It requires hardware to do or fly. And though more than one kind of hardware can do things, doing is not implementation-independent.
"Mental" states are felt states. Unlike vacuum cleaners and dust, felt states are not visible: only their doings are visible. So there is no way to be sure a system feels.
For doing, you need a body. That's not a computer. Nor is it Weak AI (which is just an instance of the Strong Church/Turing Thesis).
Harnard 3b
ReplyDeleteIn continuation from my earlier post regarding Searle, Harnard proposes an argument, or more so a necessary qualification that proves to be unexplainable to proponents of Searle – that of the “unconscious MENTAL states” which are present in a conscious entity. Harnard specifies that Searle only plainly considered a concept of understanding that was explicit in constructing the parameters of the Chinese Room Argument. Perhaps there are things that are tacitly understood by underlying mental states (which Searle would not agree are even present – and as such preempts himself from ever considering them). In this scenario, Harnard is right to say that there is no way to confirm or disconfirm their presence.
The “Other Minds” problem is something then that I find distinct interest in, and I believe Harnard’s insistence on the Turing Test (In its various levels) is indeed a measuring tool that many people subconsciously implement in evaluating the capacities of other living or non-living (perhaps “Thinking” would be more appropriate) entities.
So then how do we ‘become’ and therefore learn what ‘state’ such entities are in to ultimately prove if they fall on the upper echelon of the TT’s?
"Mental" means felt. There's no direct solution to the other-minds problems; but other humans and most animals are similar enough to us to make it highly probable that, like us, they too are sentient (i.e., that they feel).
Delete(Matthew, can I ask you to skywrite in a less formal, more kid-sibly way?)
'Moreover, it is only T2 (not T3 or T4 REFS) that is vulnerable to the CRA, and even that only for the special case of an implementation-independent, purely computational candidate. The CRA would not work against a non-computational T2-passing system; nor would it work against a hybrid, computational/noncomputational one (REFS), for the simple reason that in neither case could Searle be the entire system; Searle's Periscope would fail.'
ReplyDeleteHarnard makes clear from his analysis that Searle went too far when refuting the idea that computation is all of cognition. Going through this reading, I felt this exert most clarified to me the shortcomings of Searle's arguments. For example, the restricted applicability of CRA to e-mail T2 with only verbal input/output capacities does not reflect the extent of our cognition (ironically aspects such as, 'intentionality'). Along with this, completing disregarding any chance that computation could play a role in favor of strong equivalence goes back to what Turing argues (but in Harnard's words), "stop breaking your head over things you can't solve" - i.e. overdeterminance and 'feeling'. While I do appreciate that Searle broke the stronghold of computationalist approaches to cognition, it is clear that Searle's CRA similarly has significant theoretical pitfalls. I am intrigued to see where the discussion goes from here.
It leads to modern cognitive science, and the attempt to reverse-engineer our T3 capacities.
DeleteI have to admit, at some points in reading Searle's piece I too found myself wondering how he wouldn't have picked up at least some Chinese while computing all the messages. It seems Systematists did the same. I did away with that idea because I told myself thoughts in that stream were missing the point of Searle's argument. That would mean Searle's instructions had rules of some kind to allow for learning on the job, which was not the case. But beyond that, it would invalidate his entire argument. So to nitpick on that point would be obtuse. It was interesting to see how you dealt with this point in a more concrete manner, laying out unconscious states in nonconscious and conscious entities. Though it took me a few reads, I understand (I think) your analogy between conscious and unconscious understanding of a language. One could memorize the instructions transcribe the Chinese responses without knowing the language because it's just meaningless symbols in patterns. But to know Chinese would require a conscious awareness of how the language works, what sounds and looks right, and what doesn't. One could decide something doesn't look right if they knew the instructions–unconscious understanding–but they would only know that because it appears wrong to them. Someone who legitimately knows Chinese would understand the reasoning behind it.
ReplyDeleteSomeone who understands words would not just have to say them in a way that makes sense (to anyone who understands them) for a lifetime (as T3 Gabe can) but would have to be able to point to the things they refer to (as T3 Gabe can, but T2 Searle can't, in Chinese). Also, if someone understands words they would have to feel what it feels like to understand the words (as Searle can in English, but not in Chinese; can T3 Gabe?).
DeleteIn response to Gabe:
DeleteI have a few points to add. Searle’s Chinese Room Argument is simply T2; he argues that the batches of Chinese writings and instructions given to the person in the room mimic programs given to a computer. Additionally, he argued that cognition cannot be all computation (which rids the idea that Searle had a consciousness and understanding of the Chinese language). As I understand it, there is a difference between truly understanding versus responding to input.
Since T3 is not just computation, it would get us out of this problem because the symbols that Searle manipulated in the room weren’t grounded. The Chinese word for “red” had no meaning of red for him at all. Colour vision would solve this along with experiences with other red objects existing in one’s environment. The symbols that are computed in computation would have to mean something and overall, these meanings come from sensorimotor capacity. Therefore, symbol manipulation is not vulnerable to the Chinese Room Argument.
It’s good to point out that a machine could decide something doesn’t look right if they knew the instructions, but someone who legitimately understands the Chinese language would know the reasoning behind why it is incorrect. T3 would allude to a conscious understanding of a language, while T2 would allude to an unconscious one.
I enjoyed reading and agree with Steven's reformulation of the three tenants of strong AI. These proved to be clearer than Searle's propositions. While I agree that an unconscious understanding cannot be applied to an unconscious machine, the notion of unconscious understanding has made me question what understanding is. For instance when we develop a certain procedural memory of a certain act, like opening a door, in colloquial speak we can claim to understand how the door works, although only peripherally (we do not imply that we understand the physics or mechanics behind the door functioning). And this understanding (of say even walking) becomes almost unattended to. I'm guessing that for a machine repeating the same action does not lead to even an unconscious understanding, given that the machine does not have a sensorimotor referrent to the symbols it manipulates. But for a T3 passing robot with sensory motor properties is it safe to say that the robot understand things, like how to walk? or open a door?
ReplyDeleteUnderstanding language and being able to do things is not the same thing, even when the thing you are doing is passing T2.
DeleteAnd being able to do things is not the same as knowing how you do it. (Only cogsci can tell us that.)
Understanding language is not just doing T2, or even T3; it feels like something to understand language. "Consciousness" is just one of many weasel-words for feeling. "Unconscious understanding" would be unfelt understanding. But since understanding language feels like something, this would have to be an unfelt feeling. (This is one of the reasons for avoiding weasel words.)
What shows the system as a whole understands during the CRA while Searle (the person) does not (Systems Reply)? Searle rebuts the Systems Reply by saying he had memorized all the symbols. Harnad says that the System Reply is right… “CRA shows that cognition cannot be ALL just computational, it certainly does not show that it cannot be computational AT ALL”. How does Harnad's response exhibit computation is at least somewhat involved in the CRA (especially in Harnad’s revised CRA experiment)?
ReplyDeleteThe System Reply is wrong for purely computational T2. (Once the algorithm is memorized, Searle is the whole system, and he says he does not understand Chinese: believe him.) But for T3 (hybrid computational/noncomputational) the System Reply would be right, because Searle could only implement the computational part of a T3 robot like Gabe).
DeleteCognition is part computational (at the very least) because people can do mathematics. (Also, the brain, and evolution, would be silly not to make use of the power of computation where it works -- which is for anything that can be done by a Turing Machine, but not a sensorimotor robot -- or a sentient one.) "Sentient" is a synonym for feeling, but not a weasel word; it is less like to be misunderstand as only meaning emotion.)
"[Searle's] critique has helped open up the vistas that are now called "embodied cognition" and "situated robotics," and they have certainly impelled me toward the hybrid road of grounding symbol systems in the sensorimotor (T3) world with neural nets." (Harnad, 2001). Harnad points out that Searle's CRA shows us that cognition cannot be (just) computation. There must be something more - a hybrid possibly between computation and some other physical casual mechanism. This leads us to a T3 which includes sensorimotor mechanisms. Harnad states that Searle's CRA has lead us to "embodied cognition" and "situated robotics". The ideas of embodied cognition (our motor system influences our cognition) and situated robotics (the robot exists in an environment which they can manipulate or change) are very appealing to me and I believe they will take us in a good direction in understanding the "hybrid road of T3". However, I am unsure how exactly symbol systems will be grounded in the sensorimotor (T3) world? Is it simply that we need to create neural networks of recognizing patterns, machine perception, interpreting sensory data etc. that will allow us to ground these symbol systems - or is something more needed?
ReplyDeleteComputation will be part of T3 because of the many things we do that can be done by computation, such as logic and mathematics; and anything that can be done by a digital algorithm rather than analog activity and processing.
Delete"Can we ever experience another entity's mental states directly? Not unless we have a way of actually BECOMING that other entity, and that appears to be impossible -- with one very special exception, namely, that soft underbelly of computationalism..."
ReplyDeleteI'm a bit confused about this part in the article. What is meant by the "soft underbelly of computationalism"? How is that the exception to the inability to experience another entities mental state?
On another note, in the lecture (and a bit in this article) there was some discussion about whether computationalism even passes T2. I found this quite interesting and I agree with the notion that it does not fully. While a machine can perhaps display an output of a verbal message, the words or answer means absolutely nothing to it (they're just squiggles and squoggles). In our world, words have meaning that we have attached to it. So I guess computationalists would argue that there is verbal equivalence, but only at a syntactic level. The machine is just executing a program. There is no feeling of understanding what these symbols represent.
The implementation-independence of computation is the Searle's Periscope on the other minds problem: If computationalism -- cognition is just computation -- were true, Searle would be understanding Chinese when he implements the T2 computation. But since he would not be understanding Chinese, computationalism is false: cognition is not just computation.
DeleteDuring class, we talked about how T3 is what we truly would need to understand something. Something as abstract as a CPU with peripheries is necessary for us to ground the language at least for an x amount of content. Although, I agree with all this and I do think the next question is now what peripheries are truly necessary and how long do we need them for before the rest can be done solely by CPU, my real trouble is back to the Turing test and your proof as to why the turing test doesn't work with T2. Let us assume that T2 exists.
ReplyDeleteI know that you don't feel like T2 can exist without T3, but let us assume that there is a machine that can write to us for eternity and we would not know that it is solely a machine. Now you said, that in this case, the fatal flaw would be when it is shown a picture (of a marble lets say) and asked to describe it. This is where I don't necessarily understand. In this case, I believe that a CPU could answer this question. Using machine learning software such as Ilastik, you can input multiple images of x and get the computer to learn through trial and error what these images contain. Later, if you were to give the computer another series to images that it has never seen before, it can (if trained well) be able to decipher where x is and what it's properties are. In this trial and error case, we are using reinforcement learning. We don't have a ground truth, but we are simply identifying a few images while the computer is live updating and trying to find a pattern in all of this.
In this case, with this training, if T2 was given a picture of marble that it has never seen before, it can be able to tell you that the picture is in fact a marble, and that it is blue and yellow and round. If we were to perform finite element analysis, it could even tell you what stresses that marble would feel if it were to roll down a certain hill. It can do all of this without peripheries.
note: by peripheries I mean sensorimotor applications
DeleteCPU plus sensorimotor peripherals is a hybrid computational/analog system. Searle can implement the software part, just as in the CRA, without understanding Chinese. With the peripherals, T3 Gabe is a hybrid computational/analog system (for which Searle's Periscope does not work -- it just works for the computational part, no matter what the computational part is, including learning-algorithms).
DeleteWhat you are imagining as CPU + sensorimotor peripherals is in reality (the brain) mostly sensorimotor and analog through and through. But even if you want to imagine a complete separability of the computational and analog part, once you remove the analog part (the "peripherals"), what's left is not thinking (cognition), because it cannot feel.
In general, intuitions about cognition as computer + perpherals (or as a "brain in a vat") are misleading because they simply presume that the thinking part is computational, and autonomous as such, whereas in reality it is just that the computational part (whatever it is) is computational, whereas thinking (cognition) is hybrid. (Think about it! Remember that computationalism is just a conjecture, not a conclusion.)
“This is the point where reasonable men could begin to disagree. But let us take it to be T2 for now, partly because that is the version that Turing described, and partly because it is the one that computationalists have proved ready to defend. Note that T2 covers all cognitive capacities that can be tested by paper/pencil tests (reasoning, problem-solving, etc.); only sensorimotor (i.e., robotic) capacities (T3) are left out. And the pen-pal capacities are both life-size and life-long: the candidate must be able to deploy them with anyone, indefinitely, just as a real pen-pal could; we are not talking about one-night party-tricks (Harnad 1992) here but real, human-scale performance capacity, indistinguishable from our own.”
ReplyDeleteI am curious how would Searle respond to another model of T3 Turing times with possible variations of CRA. Is it possible to come up with a thought experiment where the computer can have the T3 sensorimotor capacities and output the right Chinese characters on the key-board yet by his standard still fail to understand Chinese. It is not yet been clear to me what sensorimotor capacities exactly are required to be a T3 model machine, but I wonder what is the implication for Searle of having a T3 machine passing the TT. Would this change shift Searle’s position in his answer to the question “Can the machine understand Chinese?” Will he oppose to it because it is not made out of the same stuff?
Sensorimotor function is not computation. Like heating or vacuuming, it can be simulated computationally, but it cannot be implemented computationally: it needs the right hardware. For that reason, Searle's Periscope cannot penetrate it.
ReplyDelete"Moreover, it is only T2 (not T3 or T4 REFS) that is vulnerable to the CRA, and even that only for the special case of an implementation-independent, purely computational candidate. The CRA would not work against a non-computational T2-passing system; nor would it work against a hybrid, computational/noncomputational one (REFS), for the simple reason that in neither case could Searle BE the entire system; Searle's Periscope would fail"
ReplyDeleteLet's suppose we have a T3 sensorimotor system, that can have a robotic interaction with the world and do what T2 is doing. Even if Searle is not the entire system, which means that T3 can do more than just manipulating symbol, how does its sensorimotor functions ads something to the notion of understanding ? What could a T3 do more than a T2 in a CR? T3 is still just manipulating symbols and interacting with the world.
The T3’s sensorimotor capacity enables it to ground symbols by connecting them to referents in the world. This is already much more than what a T2 does, which is to manipulate symbols based on their arbitrary shapes. The CRA showed that understanding doesn’t arise through computation alone. That’s where the T3 with its symbol grounding ability comes in. The reading of 5a states that symbol grounding is a necessary condition for understanding the meaning of symbols (although it may not be everything there is to it).
DeleteYou’re asking what a T3 could do more than a T2 in a CR. In contrast to a T2 (sidenote: according to Harnad, this can be limited further to only T2’s that are implementation-independent and purely computational), a T3 would not be vulnerable to the CRA. If I understood this correctly, the reason why is because the ‘mind’ (for lack of a better word) of the T3 robot would be ‘impenetrable’ to an outsider (i.e. Searle’s Periscope would fail). Or in other words: it is only because the computational T2 is implementation-independent that we can, through Searle’s Periscope, *be* that system. Since a T3 robot is implementation-dependent and not only computational, we can’t experience the states it is in. Hence, we have no way of proving nor disproving whether the T3 robot really understands.
I think your interpretation of the capacity of T3 to overcome the downfalls of the T2 system as described in the CRA is accurate. I'm wondering what's the role of the degrees of freedom mentioned in Harnad's article in that? In the article, we read: "To walk like a duck, something roughly like two waddly appendages are needed, and to swim like one, they'd better be something like webbed ones too. But even with these structure/function coupling constraints, aiming for functional equivalence alone still leaves a lot of structural degrees of freedom open." I'm not able to grasp exactly what the degrees of freedom are such that a functional equivalent (i.e. weak equivalent?) has something left to offer? And are those things somehow involved in the ability to pass or not the Turing test? Really, I’m just wondering what the last sentence quoted means, and if it has implications for the Turing test.
DeleteHi Marianne!
Delete“ To walk like a duck, something roughly like two waddly appendages are needed, and to swim like one, they'd better be something like webbed ones too. But even with these structure/function coupling constraints, aiming for functional equivalence alone still leaves a lot of structural degrees of freedom open.”
This part also got me thinking about the the difference between a t3 and t4, and the implications of this difference to the Turing test. Because regarding Gabe, a t3 passing robot, we said he was able to do anything a human can, however he opens his skull we see that is a machine inside. And t4 is all the capabilities that Gabe has plus the complete structural similarity to a human.
In the paper, Prof Harnad states that “there is a really microfunctional continuum between d3 and d4”. So, depending on the level of physical inspection on a t3, you can get super close, (if not directly) to the same type of inspection that you would do on a t4. You can keep looking into more and more subtle details: starting to look at if the imitated duck has webbed appendages, then looking into if these appendages on the inside has a similar mechanism as a real duck does, looking at if the muscles are similar to the real duck etc.
So the gap between t3, and t4 can be so small that we can almost ignore it. That is if we are looking in to finer and finer details on the body of a t3, and we cannot find the difference of the two. To illustrate, I guess we can say that we a detailed enough t3 test can even find structural differences in two different humans’ brains, since it can get super detailed regarding the anatomy of a brain.
However, for our purposes, we don’t dive into talking about the structural detail level of a t3 test. We use these tests as conventions of categorizing what is referred to by a t3 level robot and by a t4 level robot. So, I would say that the implications of structural degrees of freedom is not relevant to talking about Turing test. There is a relatively fine line between t3 and a t4. I think that was why Harnad said in the paper: “but let us leave that aside for now, and stick with D3 macrofunction.” So, we don’t dive into the details of a t3 test, but stick with the macrofunction of t3.
At first, I found myself disagreeing with Searle because I thought that after reading Chinese so many times he would at least be able to understand it, but after letting it sink in for quite some time I understand what Searle really meant by "understanding".
ReplyDeleteI came to the same concluding thought from Searle's paper as Harnad did in his that, "Granny has been given a much harder-headed reason to believe what she has known all along: That we are not (just) computers". This simply points how the complexity in what understanding is. This topic reminds me of my elementary and high school teachers reminding me that there is a difference between really learning something and memorizing something. Memorizing allows me to answer prompted questions and things that I'm familiar with in the short term, whereas learning (understanding) allows me to think about the topic in a more complex way, have my own ideas about it, contribute something new, and be ready to extrapolate my thinking to other conversations. While I think this is correct, I think that because computers are different in the way they perform their functions, maybe understanding like we do is also different. A computer doesn't have the chance to forget something they've been given through input, so although there's no feeling associated with their "learning", and their understanding can't be classified in the same way, maybe it's not necessary to do so.
I don’t think that computers understand at all. In fact, that’s what the CRA showed: an implementation-independent, computational T2 computer doesn’t understand, because Searle ‘incorporating’ this exact system doesn’t understand. For the purposes of Searle’s CRA, understanding is all or none, it’s not about whether there are different levels of understanding or different kinds of understanding: either the computer understands Chinese or it doesn’t. Searle expresses the same point very well in his paper:
Delete“I will argue that in the literal sense the programmed computer understands what the car and the adding machine understand, namely, exactly nothing. The computer understanding is not just (like my understanding of German) partial or incomplete; it is zero.”
Re: “I think that because computers are different in the way they perform their functions, maybe understanding like we do is also different.”
DeleteAs pointed out by Erica, I don’t think machines understand at all. I would also add that the whole purpose of the CRA is to show that computationalism is misguided. If you argue that computers understand differently from humans, then you are ultimately agreeing with Searle’s point that computers and computation do not explain cognition.
Your comment is just tweaking the other-minds problem – instead of asking *if* the machine has a mind, you’re asking if it has the *same mind* as humans. This just becomes a different question that the CRA did not set out to answer.
In his Chinese Room argument, Searle chooses to focus on one component of cognition, understanding, in order to show that cognition cannot be computation. His argument is as follows: Searle has been given a set of rules – an algorithm – such that, for all [Chinese] inputs he is given, he will give the correct [Chinese] output: Searle would pass the Chinese T2 test. But, Searle does not understand Chinese. So, Searle says, the Turing Test cannot be a good measure of whether something is cognizing.
ReplyDeleteSearle is specifically not making the claim that computers cannot cognize at all. Rather, he says that by symbol manipulation alone (by input-output alone), a machine cannot be said to be cognizing. But, Searle’s claim that this shows computation to be altogether irrelevant is fallacious. Searle’s CRA solely shows that computation cannot be the whole picture.
This comment has been removed by the author.
ReplyDelete“Moreover, it is only T2 (not T3 or T4 REFS) that is vulnerable to the CRA, and even that only for the special case of an implementation-independent, purely computational candidate. The CRA would not work against a non-computational T2-passing system; nor would it work against a hybrid, computational/noncomputational one (REFS), for the simple reason that in neither case could Searle BE the entire system; Searle's Periscope would fail.”
ReplyDeleteI understand the point that the CRA doesn’t disprove that computationalism may have a role in a larger system of cognition and am nodding my head at the idea of a hybrid system of computational and non-computational elements. I also understand that Searle in the CRA is not a hybrid system. However, I can’t quite understand WHY Searle wouldn’t be able to “BE the entire system”. I guess I am asking: what would it take (in theory) for Searle to be a hybrid system? If he has his own sensorimotor capabilities and he has absorbed the computational rules of manipulation for the chinese symbols in order to pass TT and that is not equivalent to a hybrid system, does that mean the system itself within Searle would have to have independent sensorimotor capabilities in order for Searle to be the hybrid system?
“For although we can never become any other physical entity than ourselves, if there are indeed mental states that occur purely in virtue of being in the right computational state, then if we can get into the same computational state as the entity in question, we can check whether or not it's got the mental states imputed to it.”
ReplyDeleteAll of Harnard’s argument on Searle’s periscope bases itself on accepting that a part of computational is true: mental states are computational states, and I mean a part because we are not saying here that mental states are just computational states. However, this claim was never fully proved before, nor by Turing, nor by Searle, nor by Harnard. We do, indeed sometimes think in a logical, computational manner to solve problem but that doesn’t mean that our mental states are computational states. Shouldn’t have Harnard discussed this part of the argument more before accepting this claim?
I don’t think that Searle’s contribution was purely negative. I think for us it’s easy to say that his mistake was easy to be spotted because we have all this information now that helps us get a better perspective on it, but while doing his experiments he didn’t know what was ahead of him.
ReplyDelete“An inert program alone could not understand anything, but surely an executing program might be part of what an understanding “system” like ourselves really does and is”. On a previous skywriting, I mentioned whether programs can develop/learn. Perhaps a developing program could, through using the program to converse in Chinese, eventually learn the system of symbols and eventually no longer need the program itself to converse. However, even by knowing the symbol system, would this be true understanding? Would the machine understand what the conversation was about or simply be responding symbols to symbols without any meaning?
ReplyDeleteIf you bring in the concept of circular definitions perhaps this could be an issue raised here. If the machine does not understand any Chinese to begin with, with definitions being made up of other words, there would be no base set to start the learning process.