If calories-in/calories-out is wrong, if it's really the food categories rather than the calories consumed that matter, then where do the extra calories go in a low carb diet?
Digestion is an active, managed process that the body participates in. It's not just osmosis. Fats, in particular, are subject to active transport across the lumen and into the bloodstream. Simple sugars, like glucose and fructose, either diffuse simply across the lumen and into the blood, or enter through passive, gradient-driven transporters.
So where did the calories go? In the case of fats, they may have been ingested but not absorbed, meaning that they left the body in the feces. Why were they not absorbed? Because the body may have responded to high body fat levels by opting not to absorb them. The body can't respond in the same way to sugars in the diet; the lumen offers relatively little control to modulate sugar absorption into the blood.
That's all my speculation; I've never seen evidence suggesting that the body actually does regulate fat absorption based on how fat you already are. But in principle, it could be; we know that fat absorption is regulated in a way that sugar absorption isn't.
I think you're exactly right. And the research seems to indicate that as you diet, your body becomes more efficient both at extracting calories from your food and using calories during exercise; your muscles remodel into efficient slow-twitch so that exercise has less of an effect. And that there's a form of nearly-uncontrollable "superhunger" which some people experience under caloric restriction.
I think, for many people, our only hope is bottom-up genetic re-engineering of the human metabolism. The starvation event apparently at the dawn of our species has left us utterly unequipped to deal with a world of caloric plenty. Stuff like the PEPCK supermouse holds the key, I think.