Maybe I'm reading this wrong but isn't this exactly what Prof. M is disputing? Adaptationists insist there is a fitness value to all adaptations in your body or they wouldn't be there. This leads most adaptationists to posit selective pressures (natural selection) on the most flimsy of bases.
We could look at this through the eyes of evo-devo.
Of the genes involved in developmental pathways responsible for physical traits, how many show evidence of selection against deleterious mutations? My own expectations is that a vast majority of developmental genes show evidence of selection at the sequence level.
What Prof. M is saying, by my reading, is that having or not having a nose may be a major adaptation which helps define a species and is a major selective determinant in passing through the sieve of natural selection (fitness), but the (beautiful) angle of the bridge of his nose has little to no selective value to fitness and this adaptation merely drifts through the future population as a function of the probabilities in meiosis and fecundity.
IOW, there is more of a fitness mesa instead of a fitness peak.