I recall that SJG makes the same point in
The Mismeasure of Man. Even if the differences between sexes (and that between races) were largely genetic, because the differences between individuals in each group are larger than the differences between groups, it would not explain how the various races and genders are represented in different occupations and socioeconomic classes.
"The fact that a believer is happier than a skeptic is no more to the point than the fact that a drunken man is happier than a sober one." -- George Bernard Shaw