Where does the gravity go?

You are, indeed, confused. The intertial mass of an object goes up as it speeds up. To infinity (but not beyond) as it approaches the speed of light.

Remember that light is "bent" in a gravitational field.

If you take heat as infra red radiation (which it was used as somewhere else ) then yes for sure.Now if it isn't in the form of IR photons then it is less clear to me but I don't see how that conclusion can be avoided. E does, after all, equal mc**2.