DOES “NAME AGREEMENT” ACTUALLY HAVE ANY PSYCHOLOGICAL VALIDITY?
excerpt from the Oppenheim Lab
Following convention, researchers usually assume that the distribution of responses in picture naming norms reveals the distribution of lexical activations within each speaker’s head, but norms actually give us the wrong information to estimate these quantities. When using name agreement to estimate lexical competition, researchers typically assume (without acknowledging it) that sampling 50 people just once gives you the same results that you’d get if sampling 1 person 50 times. The implicit assumption is that lexical selection operates via something like a Luce Choice Rule, and each time you name a picture of a couch, you will choose couch with the independent probability described by p(couch) = a(couch) / sigma(a(couch, settee, divan, canape, chesterfield, davenport…)). In fact, much empirical research assumes this relationship, but strangely it has never before been tested (perhaps because the conclusions were acceptable enough that the premises never previously warranted scrutiny).
At the other end of the spectrum is the possibility that each person just has their favourite words for each concept, and they stick with them. ‘Norms’ just show us the mixture of such people in our population. Although the distinction makes little difference when using norms for their original purpose (picking reliable stimuli for an experiment), its importance is magnified when assuming that norms reflect within-subject ambiguity or variation. In other words, although name agreement can predict RT, its mechanism for doing so is less clear if the 50% of people who choose couch would never actually choose sofa in a million years.
The IRB rejected our initial plan to repeatedly test participants for a million years, so in this study we tested them just twice. The question was the same, though: how ‘sticky’ are participants’ responses in norming studies? Does choosing couch the first time you name the picture identify you as ‘a couch person’ who will always choose couch over its various synonyms, or do you actually choose between couch et al each time you choose, independently of anything you’ve done in the past (as typically assumed)?
In this study, we found evidence for both possibilities, suggesting some midpoint between them. If a person used couch the first time they named a picture, they were more likely to name it as couch a week later than if they had not. But norms from a population also predicted how likely participants were to deviate from their previous selections: they were more likely to switch between couch and sofa (about equally likely, for this example) than between sheep and lamb (sheep is much more likely)
It strikes us that these new by-item ‘stickiness’ norms may be useful to others, so here they are. The data column reports the proportion of 25 participants using the image’s most common name in both sessions.
An attentive reader will note a profound leap in logic in the previous study of the ‘effects’ of secondary name agreement. Following convention, I assumed that the distribution of responses in picture naming norms reveals the distribution of lexical activations within each speaker’s head, but norms actually give us the wrong information to estimate these quantities. When using name agreement to estimate lexical competition, researchers typically assume (without acknowledging it) that sampling 50 people just once gives you the same results that you’d get if sampling 1 person 50 times. The implicit assumption is that lexical selection operates via something like a Luce Choice Rule, and each time you name a picture of a couch, you will choose couch with the independent probability described by p(couch) = a(couch) / sigma(a(couch, settee, divan, canape, chesterfield, davenport…)). In fact, much empirical research assumes this relationship, but strangely it has never before been tested (perhaps because the conclusions were acceptable enough that the premises never previously warranted scrutiny).
At the other end of the spectrum is the possibility that each person just has their favourite words for each concept, and they stick with them. ‘Norms’ just show us the mixture of such people in our population. Although the distinction makes little difference when using norms for their original purpose (picking reliable stimuli for an experiment), its importance is magnified when assuming that norms reflect within-subject ambiguity or variation. In other words, although name agreement can predict RT, its mechanism for doing so is less clear if the 50% of people who choose couch would never actually choose sofa in a million years.
The IRB rejected our initial plan to repeatedly test participants for a million years, so in this study we tested them just twice. The question was the same, though: how ‘sticky’ are participants’ responses in norming studies? Does choosing couch the first time you name the picture identify you as ‘a couch person’ who will always choose couch over its various synonyms, or do you actually choose between couch et al each time you choose, independently of anything you’ve done in the past (as typically assumed)?
In this study, we found evidence for both possibilities, suggesting some midpoint between them. If a person used couch the first time they named a picture, they were more likely to name it as couch a week later than if they had not. But norms from a population also predicted how likely participants were to deviate from their previous selections: they were more likely to switch between couch and sofa (about equally likely, for this example) than between sheep and lamb (sheep is much more likely)
It strikes us that these new by-item ‘stickiness’ norms may be useful to others, so here they are. The data column reports the proportion of 25 participants using the image’s most common name in both sessions.