Training Perception and Action to Do the Right Thing

Prof. Robert Goldstone (Indiana University Bloomington, USA)
16.03.2020 – 14.00 – (lecture, followed by an extended discussion)
RUB, GA 04/187 (Mercatorraum)


By one account, formal thought in mathematics and science requires developing deep construals that run counter to perception.  This approach draws an opposition between superficial perception and principled understanding.  In this talk, I advocate the converse strategy of grounding scientific and mathematical reasoning in perception and action.   Relatively sophisticated reasoning is typically achieved not by ignoring perception, but rather by adapting perception and action routines so as to conform with and support formally sanctioned responses.   Perception and action are more sophisticated than usually thought, particularly because they can be adapted to do the (cognitive) Right Thing.

The first case study for this thesis concerns arithmetic and algebraic reasoning, where we find that mathematical proficiency involves executing spatially explicit transformations to notational elements.  People learn to attend mathematical operations in the order in which they should be executed, and the extent to which students employ their perceptual attention in this manner is positively correlated with their mathematical experience.  People also produce mathematical notations that they are good at reading.  Based on observations like these, we have begun to design, implement, and assess virtual, interactive sandboxes for students to explore algebra.

The second case study involves students learning about science by exploring simulations.   We have developed a computational model of the process by which human learners discover patterns in natural phenomena.  Our approach to modeling how people learn about a system by interacting with it follows three core design principles: 1) perceptual grounding, 2) experimental intervention, and 3) cognitively plausible heuristics for determining relations between simulation elements.  In contrast to the majority of existing models of scientific discovery in which inputs are presented as symbolic, often numerically quantified, structured representations, our model takes as input perceptually grounded, spatio-temporal movies of simulated natural phenomena.