Research Overview

Natural languages are hybrid systems, products of both a common biological endowment (shared across languages) and a particular ecological niche (specific to a particular language). That shared endowment – the architecture of the human nervous system – serves as a powerful constraint on how languages vary and evolve. Nevertheless, the world’s languages exhibit remarkable diversity in sound, meaning, and structural organization. In my work, I take the view that human languages are the end point of complex processes of cultural evolution, occurring over generations, and that their features can thus be analyzed as adaptive solutions to a complex constraint-satisfaction problem.

In particular, my research seeks to understand 1) how cognitive principles of learning and memory serve to constrain cross-linguistic variation, 2) how social and historical contingencies select for certain designs, and 3) how different design choices can incur trade-offs between early acquisition and adult processing. These questions are approached from the dual perspectives of adaptation, which assesses the fit between a language’s feature and the mechanisms of mind and structure of the environment, and development, which assess the impact of transmission across generations of learners.

Projects:

Variation and Fitness

On the cultural transmission model (Tomasello, 2003), a language’s structural features are subject to selection pressures, and variation among languages results both from random drift and selective adaptation to variable circumstances. A central question is how to establish the ‘fitness’ of a particular linguistic feature. Information theory (Shannon, 1948) supplies an answer: Its theorems, and their correlates, specify how to construct a maximally efficient code (such that communication proceeds as rapidly and reliably as possible) and how to quantify the extent to which a given code deviates from this theoretical maximum. A feature’s fitness can thus be measured in terms of its communicative efficiency.

In this work, a guiding principle is that simple model systems can do much to illuminate more complex phenomena. For instance, we might wonder – why is it that when we want to express an idea, we chain words together in sequence? Why not instead simply invent more words to capture all the thoughts we might want to express? These questions get to the heart of one of language’s core design principles, which sets it apart from animal communication systems—that it is combinatorial (Hockett, 1960). To better understand why, we selected personal name grammars as a basis for investigating how the lexicon is structured to permit its growth and development, and how it balances the pressure to expand against cognitive and physiological limitations. The theoretical branch of this work revealed why combinatorial systems are a necessary solution to the problem of naming, and illustrated how information used for identification can be effectively distributed over a signal. The experimental component showed how an information theoretic framing can then be used to generate predictions across a variety of classic memory paradigms. Such empirical demonstrations help quantify the costs of deviating from principles of rational coding—costs that have real world social consequences.

A closely related project applied a similar logic to a different model system. In that investigation, we undertook a comparative analysis of nominal systems in German and English. This work illustrated how these two closely related languages have adopted alternative solutions to the shared problem of noun retrieval with concomitant costs and benefits. In particular, while German relies on grammatical gender to facilitate noun selection, English employs prenominal adjectives. Notably, one system is deterministic, the other probabilistic. Despite their differences, both systems act to efficiently smooth information over discourse, making nouns more equally predictable in context. The choice of solution reflects an interaction between the universal (cognitive constraints) and the particular (social and historical forces). While both languages have adopted solutions that strike a balance between efficiency and learnability, the precise nature of this compromise reflects the peculiar demands of different populations of speakers.​

The Limits of Information

Adopting an information theoretic model of communication requires, at a minimum, a reworking of traditional conceptions of language descended from structuralism (Ramscar & Baayen, 2013). On the structuralist account, language is a means of direct material exchange, in which messages are passed back and forth, much like a parcel delivery system (de Saussure, 1916). An information theoretic framework, by contrast, adopts the indirect model of telegraphy, in which the sender first translates the message into a signal that can be sent across a physical channel. Successful communication relies on the receiver’s ability to reconstruct the original message from the received signal using a common code.

A starting assumption, then, is that the forms of the language – speech, writing, and gesture – act as signals that discriminate among the repertoire of concepts and experiences available to the speaker. However, there is a significant wrinkle to consider: In the electronic systems that information theory was designed to describe, the sender and receiver share identical repertoires. In natural language, by contrast, conceptual and linguistic knowledge is never strictly identical across speakers, nor is it fixed across time (Labov, 1972). The extent to which communication succeeds depends, in part, on the extent to which the conceptual stores and the codes for encoding and decoding signals align. A rich literature attests to the variability in speakers’ repertoires, as well as the ways in which they may be motivated to align them in service of more effective communication—coordinating aspects of their speech, gaze, and gesture.

The picture of natural language provided by information theory is thus incomplete, recalling George Box’s injunction that “all models are wrong, but some are useful”. Notably missing from this account is how the system is first acquired in childhood, and how it can continue to flexibly develop and adapt over the lifespan. Computational models of learning are thus a necessary complement to the picture supplied by Shannon. My research program touches on each of these areas, asking:
​
  1. How the nature of learning in childhood guarantees the development of common predictive codes among speakers of the same language, 
  2. Whether sequences that are efficient for processing are similarly well-suited for acquisition,
  3. How interlocutors with different priors coordinate their communicative efforts to converge on the same message.

The Nature of Learning in Childhood

For communicative systems to function as such, they must be learnable and transmissible over generations of speakers. In the language acquisition literature, a central question is the extent to which the acquisition process is guided by deep-seated instincts and hardwired rules. To address this question, my research with Prof Michael Ramscar and colleagues has focused on what a simple classical conditioning model (Rescorla & Wagner, 1972) can and cannot explain about linguistic development. With a two-layer network, we have elegantly captured several of the key phenomena in early word learning, without recourse to innate constraints, or a more complex learning architecture. One of the virtues of this modeling approach is its interpretability: Given some set of assumptions about the state of the learner and the environmental input, the model can establish whether a complex behavior is, in principle, learnable without substantive priors, and if it is, what parameters influence its learning trajectory.

This approach has offered insight into questions like: How do kids learn the meanings of words in ambiguous contexts? Why do preschoolers over-regularize plural forms (saying "mouses" instead of "mice") and how do they recover from these errors? How do children abstract the properties of colors and numbers, and why are these categories so hard for them to learn? In a similar vein, we found evidence that the use and interpretation of noun compounds and inflected verbs are best explained in terms of learned semantic conventions, rather than deterministic rules. Taken together, these findings highlight how – in typical development – the richness of the input available to young learners drives canonical patterns of acquisition.
​
Our research suggests that associative learning mechanisms have far greater explanatory power than is often supposed. However, this cannot be the whole story: Human languages are markedly more complex and expressive than other animal communication systems. What explains human uniqueness? Prefrontal maturation is clearly an important piece of the puzzle : Young humans undergo a much longer period of cognitive immaturity than other animals, in which they are unable to actively direct their actions or attention. This extended developmental timetable appears to be crucial to the transmission of language and culture, as it enforces a sensitive period in which children sample their environments in broadly consistent ways and systematize variation in their input. At the same time, it exacts a steep cost, leaving these young learners vulnerable and relatively fixed in their behaviors.

Given this, one of the challenges of studying language as it develops, is that a given behavioral output at some point represents both 1) past learning history and 2) present cognitive function – contributions which are easily confounded. To tease these apart, we have shown how very young children, who cannot yet exert cognitive control online, can nevertheless learn to match their behavior to context in surprisingly sophisticated ways, in classic tasks like the A-not-B task and the Dimensional Change Card Sort. Counterintuitive as it may seem, the fact that children typically struggle with these tasks is reflective of a developing system well-adapted for learning.
​​

Learnability vs. Processing

Learning models can tell us something about how language is learned; information theoretic models can tell us something about how language, once learned, is processed. In bridging between these accounts, an important question is whether linguistic features that scaffold early learning are similarly well-suited for adult processing. In other words, are the demands of learnability and processing one and the same, or are they different?

To answer this, an ongoing project with Profs Michael Ramscar and Michael N. Jones seeks to highlight parallels between learning and information theory. Information theory dictates that well-designed verbal sequences should follow a characteristic tree-branching structure, which maintains a constant entropy rate over elements. Inverting this structure has the consequence of concentrating entropy over the first element. This much is is known. What has gone unrecognized, is that these alternative sequencings directly correspond to alternative schemas in discriminative learning (Osgood, 1949; Ramscar et al., 2010). In particular, differences in ‘optimal’ and ‘suboptimal’ coding map neatly onto ‘convergent’ and ‘divergent’ learning schemas, which have been shown to produce markedly different behavioral outcomes.

Practically, what this means, is that different sequence types should pose a trade-off for learning and processing. The existence of such a trade-off is corroborated across a range of experiments, which reveal that while optimally coded sequences are more efficiently processed and better recalled, ‘suboptimal’ sequences are actually better structured for semantic learning. Since languages are designed to be both useable and learnable, their structures must necessarily reflect a compromise between these desiderata; hence, the applicability of information theoretic principles depends closely on the communicative goal.

Adaptive Communication

​Languages are spoken and understood by people from very different backgrounds. Indeed two speakers of the same language may have significantly different representations of that language, even if they are perfectly capable of making themselves understood. An abiding question is how individual speakers develop these distinct representations as a function of their exposure over a lifetime, and how such differences play out in processing and production. For instance: Do differences in their daily reading materials mean that a doctor and a lawyer will process a newspaper article differently? Can readers flexibly adapt their expectations on the basis of genre, in the same way that some bilingual speakers can code-switch? 
​
  • Dye, M., Jones, M.N., & Ramscar, M. (in prep). Sensitivity to linguistic structure emerges as a function of expertise.
  • Samson, K., Dye, M., & Sherman, S.J. (2014). Is chocolate as bittersweet as nostalgia? Cross-modal priming effects for taste and emotion. Poster presented at the 36th Annual Meeting of the Cognitive Science Society, Quebec, Canada.
  • Dye, M., Cox, G., Frey S. (2012). Detours in understanding: The temporal dynamics of malapropisms. Paper presented at the International Society for the Empirical Study of Literature and Media, Montreal, QC, Canada.
  • Kao, J., Ryan, R., Dye, M. & Ramscar, M. (2010). An acquired taste: How reading literature affects sensitivity to word distributions when judging literary texts. Proceedings of the 32nd Meeting of the Cognitive Science Society, Portland, OR.
  • Ramscar, M., Matlock, T., & Dye, M. (2010). Running down the clock: The role of expectation in our understanding of time and motion. Language and Cognitive Processes 25(5): 589-615.

Learning in Context

Words are rarely experienced in isolation. Rather, they are heard and read in particular verbal and situational contexts, which are integral to our ability to learn a functional lexicon. In a series of experiments with Profs Michael N. Jones, Brendan Johns, and Richard Shiffrin, we have investigated how the particulars of our experience with a lexical item can affect subsequent recognition and comprehension processes. We have specifically been interested in assessing and piecing apart the contributions of recency, frequency, and contextual diversity, using both experimental manipulations and quantitative analyses of verbal stimuli. 
​​