Really it is all down to reproduction.
A good example is the fact that infant mortality has plummeted in the Western world over the last 100 years from about 1 in 10 to close to nothing. Most of this has been attributed to a serious reduction in infectious disease thanks to sanitation, antibiotics and vaccination. A child that survives has a lot higher chance of reproducing one that doesn't
Susceptibility to disease has a genetic basis (in particular our MHC class 1 and 2 proteins). So genetically susceptible individuals are now surviving and reproducing that otherwise would not have. And that is just infant mortality; 1 in 2 children died before the age of ten until the 18th century.
We coevolve with our infectious diseases. Some of that coevolution is now gone. Is this effect large? Who knows? Will it affect our gene pool? Definitely. Is it conciously directed? No, rather it is a side effect of modern medicine and sanitation. Is this removing a source of natural selection? Yes, and quite a large source.
If antibiotics were available when the Black Death began, genetically we would be somewhat different (though some are now suggesting it was actually a virus). Some research is suggesting a particular mutation in CCR5, which helps people survive bubonic plague also provides resistance to HIV1. Its been shown that the mutation is more prevalent in regions that suffered the plague in the past and is absent from those regions that have not been exposed.
I don't think the effect disease has on our genetic makeup should be underestimated, though naturally it is only one of many determinants.