Mathematics and Education

A slow blog

Details: Greater Variability and the Right Tail

with one comment

Studies are only mentioned in response to particular assertions, thus their findings may be far more complex than might appear from their treatment here.

My comments appear on the right. My insertions are indicated by square brackets. For example, the online version of Tierney’s article includes links to other web pages and information about those pages is given in square brackets.

Excerpts from June 8 article Comments
. . . the new evidence supporting Dr. Summers’s controversial hypothesis about differences in the sexes’ aptitude for math and science. Ambiguity. Is “aptitude” meant to be synonymous with “ability”? Note that the researchers who produced the “new evidence” refer to “ability.” Outside of psychology, “ability” is sometimes considered immutable.A group of psychologists writes: 

Unfortunately, one cannot measure ability without also measuring achievement to some extent, so the distinction between these two constructs is somewhat blurred. We use the term ability as it was defined by Fleishman (1972): a general trait of an individual that is the product of learning and development. (Halpern et al., 2007, p. 3, italics added)

Mistake. The “new evidence” presented in this article is consistent with this hypothesis, but does not rule out other explanations. See Wai et al. 2010, p. 412 and further discussion in these comments.

How could these workshops reconcile the “existence of gender bias” with careful studies that show that female scientists fare as well as, if not better than, their male counterparts in receiving academic promotions and research grants? Ambiguity. “Bias” can refer to actions and practices (e.g., funding decision procedures) or to attitudes and perceptions. Bias in the latter can exist in the absence of bias in the former.

Omission. These “careful studies” are not exhaustive in scope, as discussed in part 2. Moreover, some of the findings suggest selection bias—the women who apply for promotions and grants may be, on average, more qualified than the men who apply.


For example:

In every field, women were underrepresented among candidates for tenure relative to the number of women assistant professors. Most strikingly, women were most likely to be underrepresented in the fields in which they accounted for the largest share of the faculty – biology and chemistry. (Gender Differences at Critical Transitions in the Careers of Science, Engineering and Mathematics Faculty, Finding 5-1, p. 148)

Ambiguity. “Faring as well” might mean the same qualifications are equally rewarded (absence of bias) or the proportions of men and women who apply for grants and promotions are the same as those who receive them (consistent with presence or absence of bias).

Omission. Promotions and grants are not the only arenas in which gender bias may occur and affect women. For example, fellowships, journal refereeing, awards, and honors are not mentioned, nor are workplace issues as course assignments and family leave.

Yet even if all these social factors were eliminated, [Summers] hypothesized, the science faculty composition at an elite school like Harvard might still be skewed by a biological factor: the greater variability observed among men in intelligence test scores and various traits. Ambiguity: “biological factor.” A group of psychologists writes:Terms like biological and innate are often misunderstood. Humans are both biological and social beings shaped by the complex interaction of biology and environment. When psychologists use the term innate, they are referring to a potential that is ‘‘ready’’ for development in a supportive environment. Innate does not mean immutable or unchangeable. (Halpern et al., 2007, p. 3) 

Ambiguity. Are “intelligence test scores and other traits” supposed to be relevant to the rest of the discussion? Is the reader supposed to assume that scores on math tests are correlated with these?

Omission. The greater male variability hypothesis is not supported by empirical evidence (for mathematics tests, see Guiso et al., 2008; Hyde et al., 2008; Hyde & Mertz, 2009; Penner, 2008). Also, note the findings of Feingold for verbal, mathematical, and spatial abilities:

[M]ales were more variable than females in some nations and females were more variable than males in other nations. (1994, p. 81)

Some have claimed he was proved wrong by recent reports of girls closing the gender gap on math scores in the United States and other countries.[The online version links to July 2008 Marginal Revolution blog post commenting on Hyde et al., Science, 2008. It does not comment on Guiso et al.’s May 2008 Science article about PISA, Penner’s 2008 article about TIMSS, or Andreescu et al.’s findings for the International Mathematics Olympiads and the Putnam exam.]


Omission. In addition to Hyde et al.’s 2008 Science article analyzing U.S. scores and Penner’s 2008 article analyzing TIMSS scores in 22 countries, a recent report is:Guiso et al., 2008, Science. Analysis of international dataset (PISA): mathematics and reading scores from 276,165 15-year-old students in 40 countries. Statistics included averages and proportions of girls and boys who scored in the 95th and 99th percentiles.
But even if those reports (which have been disputed) are accurate, they involve closing the gap only for average math scores — not for the extreme scores that Dr. Summers was discussing.[“have been disputed” links to a blog post December 2008 blog post of La Griffe du Lion. This discusses the variability reported by Guiso et al. and Hyde et al. which were both published in refereed journals. The post does not dispute the data, but their interpretation.]


Mistake. “[O]nly for average math scores” is incorrect. As noted above, Guiso et al. reported on “extreme scores” as well as variability.Also, Hyde et al. and Penner reported on variability. Because variability measures the shape of a distribution, it has implications for “extreme scores” at the right tail.
Some scientists and advocates for gender equity have argued that the remaining gender gap in extreme scores is rapidly shrinking and will disappear. . . . But some of the evidence for the disappearing gender gap involved standardized tests that aren’t sufficiently difficult to make fine distinctions among the brighter students. Omission. Some of Hyde and Mertz’s evidence concerned standardized tests that are sufficiently difficult to make fine distinctions among the brighter students, namely Andreescu et al.’s findings for the International Mathematics Olympiads and the Putnam exam, and the findings of Guiso et al. (PISA) and Penner (TIMSS).
Now a team of psychologists at Duke University has looked at the results of tests with more headroom. Ambiguity. Note that “headroom” and “sufficient difficulty” may mean different things. A test may be difficult but may not make fine distinctions among students.Omission. The Duke researchers analyzed SAT and ACT scores of middle school students. For middle school students, the SAT and ACT may have more “headroom” than state tests, however, they are not at the level of difficulty of the International Math Olympiads (high school) or Putnam (undergraduate). They may not even be at the level of PISA for 15-year-olds (examples on pp. 9–11 here) or the grade 12 TIMSS (examples here). Some SAT and ACT questions are more like the Middle School Math Olympiads (examples here).
In the early 1980s, there were 13 boys for every girl in that group, but by 1991 the gender gap had narrowed to four to one, presumably because of sociocultural factors like encouragement and instruction in math offered to girls. Unlikely explanation. Encouragement and instruction offered to girls in general seems a less likely cause than demographic shifts in the Duke talent search population. Such shifts have occurred for the Hopkins talent search (which was a model for the Duke search). See Hopkins demographics below.Hopkins demographics. In the Hopkins talent search samples, girls are more frequent among Asian Americans than among other ethnic groups mentioned—and the proportion of Asian Americans in the samples has increased over time. David Lubinski and Camilla Benbow write (apparently referring to U.S. talent searches in general): 

In American samples, these [gender] ratios have been fluctuating over the past decade at least partly as a function of increasing numbers of Asian students entering talent searches. For example, in Asian [American] samples, the proportion of males/females with SAT-M ≥ 700 is 4/1 (this ratio has also been observed in China); in Caucasian samples, the ratio is closer to 16/1. (1992, footnote 7, p. 66)

This is consistent with the findings of Andreescu et al. for the U.S. Math Olympiad team and Putnam participants.

Thus, the increase in the ratio of girls in the Duke talent search samples seems more likely to be connected with an increased percentage of Asian Americans in the sample than of “encouragement and instruction in math offered to girls.” For 1980–92, Asian Americans, many of them children of recent immigrants, were 35.3% of those with SAT-M scores of 700 and over (see Hopkins 1980–1992). In 2005, they were a majority (see Hopkins 2005).

Hopkins 1980–1992. Of the math qualifiers [SAT-M 700 and over], 63.4 percent (N= 637) are Caucasian, 35.3 percent (N=354) are Asian American, and 1.3 percent (N = 13) are African American or Hispanic. . . .55.0 percent (N = 110) of the females are Asian American, while 30.3 percent (N = 244) of the males are Asian American. . . . 

The first-generation-immigrant status of such a large majority of the parents of Asian American SET members raises interesting research questions. (Brody & Blackburn, 1996, p. 249, italics added)

Hopkins 2005. Descriptive studies of the SET [700 and over] population reveal a large Asian-American population in the group and currently [2005] more Asians qualify for SET in mathematics than any other ethnic group. . . . The Asian-American SET students come predominantly from homes where their parents were educated in their home country in Asia, at least for their undergraduate education, and they later migrated to the USA. In contrast, most parents of SET members who are Caucasians were raised and educated in the USA. (Brody, 2005, p. 90, italics added)

The Duke researchers report in Intelligence, “Our data clearly show that there are sex differences in cognitive abilities in the extreme right tail, with some favoring males and some favoring females.” Omission. The Duke researchers say that there are three possible kinds of explanations for their findings, none of which is ruled out by their study:Our findings are not inconsistent with previous explanations focusing on either biological . . . or social or cultural . . . aspects, but are likely best explained via frameworks that examine multiple perspectives simultaneously. (Wai et al., 2010, p. 419)  

Ambiguity. Outside of psychology, “ability” is sometimes considered immutable. As noted previously a definition used within psychology is: a general trait of an individual that is the product of learning and development (Halpern et al., 2007, p. 3, italics added).

Other studies [link to Wai et al., 2005] have shown that these differences in extreme test scores correlate with later achievements in science and academia. Reliance on one study. This conclusion is based on Phase 1 of Wai et al. This study is based on survey responses from participants identified as middle school students in the Hopkins talent search: Cohorts 1 and 2 of the Study of Mathematically Precocious Youth (SMPY). Three limitations of this study are detailed below.Unknown treatment. Benbow, Lubinski, & Suchy write of the SMPY cohorts: 

Cohort 3 received the most intensive treatment with the SMPY model, followed by cohort 2. Cohort 1 received the least amount of assistance from SMPY as its members were identified when SMPY was working out its procedures (the talent search, fast-paced programs, and so forth). Thus, the later cohorts not only received much more assistance from SMPY, they also benefited from the experience gained with the earlier cohorts. Finally, each cohort is successively more able [presumably “has higher SAT scores”]. It is, therefore, difficult to separate out and evaluate these confounding influences. (1996, p. 290, italics added)

Small sample. Correlation between SAT-M scores and achievements was studied for:

2,188 participants, identified 1972–74, SMPY Cohort 1, surveyed 1992–94.

778 participants, identified 1976–79, SMPY Cohort 2, surveyed 1996–99.

Atypical sample? SMPY Cohorts 1 and 2 were identified in the Hopkins talent search and may not be typical of U.S. scientists in their age cohort.


Andreescu, T., Gallian, J., Kane, J., & Mertz, J. (2008). Cross-cultural analysis of students with exceptional talent in mathematical problem solving. Notices of the American Mathematical Society, 55(10), 1248–1260.

Benbow, C., Lubinski, D., & Suchy, B. (1996). The impact of SMPY’s educational programs from the perspective of the participant. In Camilla Benbow & David Lubinski (Eds.), Intellectual talent (pp. 266–300). Baltimore: Johns Hopkins University Press.

Brody, L. (2005). The study of exceptional talent. High Ability Studies, 16(1), 87–96.

Brody, L., & Blackburn, C. (1996). Nurturing exceptional talent: SET as a legacy of SMPY. In Camilla Benbow & David Lubinski (Eds.), Intellectual talent (pp. 246–265). Baltimore: Johns Hopkins University Press.

Committee on Gender Differences in the Careers of Science, Engineering, and Mathematics Faculty; Committee on Women in Science, Engineering, and Medicine; National Research Council. (2009). Gender differences at critical transitions in the careers of science, engineering and mathematics faculty. Washington, DC: National Academy Press.

Feingold, A. (1994). Gender differences in variability in intellectual abilities: A cross-cultural perspective. Sex Roles, 30(1/2), 81–92.

Guiso, L., Monte, F., Sapienza, P., & Zingales, L. (2008). Culture, gender, and math. Science, 320, 1164–1165.

Halpern, D., Benbow, C., Geary, D., Gur, R., Hyde, J., & Gernsbacher, M. (2007). The science of sex differences in science and mathematics. Psychological Science in the Public Interest, 8(1), 1–51.

Hyde, J., & Mertz, J. (2009). Gender, culture, and mathematics performance. Proceedings of the National Academy of Sciences, 106(22), 8801–8807.

Hyde, J., Lindberg, S., Linn, M., Ellis, A., & Williams, C. (2008). Gender similarities characterize math performance. Science, 321, 494–495.

Lubinski, D. & Benbow, C. (1992). Gender differences in abilities and preferences among the gifted: Implications for the math-science pipeline. Current Directions in Psychological Science, 1(2), 61–66.

Penner, A. (2008). Gender differences in extreme mathematical achievement: An international perspective on biological and social factors. American Journal of Sociology, 114, Supplement, S138–S170.

Wai, J., Cacchio, M., Putallaz, M., & Makel, M. (2010). Sex differences in the right tail of cognitive abilities: A 30 year examination. Intelligence 38, 412–423.

Wai, J., Lubinski, D., & Benbow, C. (2005). Creativity and occupational accomplishments among intellectually precocious youths: An age 13 to age 33 longitudinal study. Journal of Educational Psychology, 97(3), 484–492.


Written by CK

August 30, 2010 at 12:46 pm

One Response

Subscribe to comments with RSS.

  1. “Unfortunately, one cannot measure ability without also measuring achievement to some extent, so the distinction between these two constructs is somewhat blurred.”

    The Guiso et al study mentioned here was based on data that showed that girls did more homework(about 20% so) and that they had better reading skills to go along with that advantage. Yet, they scored lower on maths test.
    One could make a solid case for boys having better ability to make up for those disadvantages in the achievement assessment.

    namae nanka

    May 24, 2014 at 10:01 pm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: