Date: Sat 27 Aug 1988 00:05-EDT From: AIList Moderator Nick Papadakis Reply-To: AIList@mc.lcs.mit.edu Us-Mail: MIT LCS, 545 Tech Square, Rm# NE43-504, Cambridge MA 02139 Phone: (617) 253-6524 Subject: AIList Digest V8 #68 To: AIList@mc.lcs.mit.edu Status: R AIList Digest Saturday, 27 Aug 1988 Volume 8 : Issue 68 Philosophy: Connectionist model for past tense formation in English verbs Two Points (ref AI Digests passim) Can we human being think two different things in parallel? Rates of change ---------------------------------------------------------------------- Date: 24 Aug 88 18:17:58 GMT From: mind!harnad@princeton.edu (Stevan Harnad) Subject: Connectionist model for past tense formation in English verbs [Editor's note - Steve Pinker (MIT), and Alan Prince (Brandeis) co-authored an article in the journal 'Cognition', critiquing Rumelhart & McClelland's model for past tense formation in English verbs. This is Stevan Harnad's critique of that critique. - nick] On Pinker & Prince on Rules & Learning Steve: Having read your Cognition paper [28(1-2) 1988] and twice seen your talk (latest at cogsci-88), I thought I'd point out what look like some problems with the argument (as I understand it). In reading my comments, please bear in mind that I am NOT a connectionist; I am on record as a sceptic about connectionism's current accomplishments (and how they are being interpreted and extrapolated) and as an agnostic about its future possibilities. (Because I think this issue is of interest to the connectionist/AI community as a whole, I am branching a copy of this challenge to connectionists and comp.ai.) (1) An argument that pattern-associaters (henceforth "nets") cannot do something in principle cannot be based on the fact that a particular net (Rumelhart & McClelland [PDP Volume 2 1986 and MacWhinney 1987, Erlbaum]) has not done it in practice. (2) If the argument is that nets cannot learn past tense forms (from ecologically valid samples) in principle, then it's the "in principle" part that seems to be missing. For it certainly seems incorrect that past tense formation is not learnable in principle. I know of no poverty-of-the-stimulus argument for past tense formation. On the contrary, the regularities you describe -- both in the irregulars and the regulars -- are PRECISELY the kinds of invariances you would expect a statistical pattern learner that was sensitive to higher order correlations to be able to learn successfully. In particular, the form-independent default option for the regulars should be readily inducible from a representative sample. (This is without even mentioning that surely no one imagines that past-tense formation is an independent cognitive module; it is probably learned jointly with other morphological regularities and irregularities, and there may well be degrees-of-freedom-reducing cross-talk.) (3) If the argument is only that nets cannot learn past tense forms without rules, then the matter is somewhat vaguer and more equivocal, for there are still ambiguities about what it is to be or represent a "rule." At the least, there is the issue of "explicit" vs. "implicit" representation of a rule, and the related Wittgensteinian distinction between "knowing" a rule and merely being describable as behaving in accordance with a rule. These are not crisp issues, and hence not a solid basis for a principled critique. For example, it may well be that what nets learn in order to form past tenses correctly is describable as a rule, but not explicitly represented as one (as it would be in a symbolic program); the rule may simple operate as a causal I/O constraint. Ultimately, even conditional branching in a symbolic program is implemented as a causal constraint; "if/then" is really just an interpretation we can make of the software. The possibility of making such systematic, decomposable semantic intrepretations is, of course, precisely what distinguishes the symbolic approach from the connectionistic one (as Fodor/Pylyshyn argue). But at the level of a few individual "rules," it is not clear that the higher-order interpretation AS a formal rule, and all of its connotations, is justified. In any case, the important distinction is that the net's "rules" are LEARNED from statistical regularities in the data, rather than BUILT IN (as they are, coincidentally, in both symbolic AI and poverty-of-the-stimulus-governed linguistics). [The intermediate case of formally INFERRED rules does not seem to be at issue here.] So here are some questions: (a) Do you believe that English past tense formation is NOT learnable (except as "parameter settings" on an innate structure, from impoverished data)? If so, what are the supporting arguments for that? (b) If past tense formation IS learnable in the usual sense (i.e., by trial-and-error induction of regularities from the data sample), then do you believe that it is specifically unlearnable by nets? If so, what are the supporting arguments for that? (c) If past tense formation IS learnable by nets, but only if the invariance that the net learns and that comes to causally constrain its successful performance is describable as a "rule," what's wrong with that? Looking forward to your commentary on Lightfoot (in Behavioral and Brain Sciences), where poverty-of-the-stimulus IS the explicit issue, -- best wishes, Stevan Harnad -- Stevan Harnad ARPANET: harnad@mind.princeton.edu harnad@princeton.edu harnad@confidence.princeton.edu srh@flash.bellcore.com harnad@mind.uucp BITNET: harnad%mind.princeton.edu@pucc.bitnet UUCP: princeton!mind!harnad CSNET: harnad%mind.princeton.edu@relay.cs.net ------------------------------ Date: Thu, 25 Aug 88 10:51:01 +0100 From: "Gordon Joly, Statistics, UCL" Subject: Two Points (ref AI Digests passim). [a] More people died in the fire bombing of Dresden than in Hiroshima; the atom bomb is a more powerful image than naplam and hence we forget. [b] With regard to what Einstein said, Heisenberg's uncertainty princinple is also pertinent to "AI". The principle leads to the notion that the observer influences that which is observed. So how does this affect the observer who preforms a self analysis? Gordon Joly. ------------------------------ Date: 25 Aug 88 14:39:01 GMT From: hartung@nprdc.arpa (Jeff Hartung) Reply-to: hartung@nprdc.arpa (Jeff Hartung) Subject: Re: Can we human being think two different things in parallel? In a previous article, Ken Johnson writes: >>Can we human being think two different things in parallel? > >I think most people have had the experience of suddenly gaining insight >into the solution of a problem they last deliberately chewed over a few >hours or days previously. I'd say this was evidence for the brain's >ability to work at two or more (?) high-order tasks at the same time. >But I look forward to reading what Real Psychologists say. The above may demonstrate that the brain can "process" two jobs simultaneously, but is this what we mean by "think"? If so, this still doesn't demonstrate adequately that parallel processing is what is going on. It may be equally true that serial processing on several jobs is happening, only some processing is below the threshold of awareness. Or, there may be parallel processing, but with a limited number of processes at the level of awareness of the "thinker". On the other hand, if we take "thinking" to mean an activity which the "thinker" is aware of, at least in that it is going on, then there is strong evidence that there is only limited capacity to attand to multiple tasks simultaneously, but there is no final conclusion on this ability as far as I know. Many studies in the ability to attand to multiple tasks or perceptual stimuli simultaneously are still being done. --Jeff Hartung-- ARPA - hartung@nprdc.arpa hartung@sdics.ucsd.edu UUCP - !ucsd!nprdc!hartung !ucsd!sdics!hartung ------------------------------ Date: Fri, 26 Aug 88 16:25:24 EDT From: Subject: Rates of change In a previous article, John Nagle writes: >... Look at Marc >Raibert's papers. He's doing very significant work on legged locomotion. >Progress is slow; ... >Along the way >are endless struggles with hydraulics, pneumatics, gyros, real-time control >systems, and mechanical linkages. (I spent the summer of '87 overhauling >an electrohydraulic robot, and I'm now designing a robot vehicle. I can >sympathise.) >... It's depressing to think that it might take >a century to work up to a human-level AI from the bottom. Ants by 2000, >mice by 2020 doesn't sound like an unrealistic schedule for the medium term, >and it gives an idea of what might be a realistic rate of progress. > I think it's going to be a long haul. But then, so was physics. >So was chemistry. For that matter, so was electrical engineering. We >can but push onward. Maybe someone will find the Philosopher's Stone. >If not, we will get there the hard way. Eventually. Continued use of a bottom-up experimental approach to AI need not demand continued use of the current experimental MEDIUM which so constrains the rate of change. While today one may be better off working directly with mechanical systems, rather than with computational simulations of mechanical systems, it is unclear that this will be the case in 5 or 10 years. If a summer's overhaul could be a week's hacking, you have an order of magnitude acceleration. If your tools develop similarly, the _rate_ of change is sharply exponential. Science, like engineering, is limited by the feedback lags of its development cycles. Many (most?) of these lags are in information handling. Considering our increasing competence, are current challenges so much vaster than past as to require similar periods of calendar time? Mitchell Charity ------------------------------ End of AIList Digest ********************