Nwr explanation is much more complete because it says how the engery you are spending transfers to the system and increases the angular momentum.
Your explanation with energy is right, but it's form of ''you are spending energy and this energy goes to the system'' does not explain how it goes to the system.
As for the OP, I'll just add (since I agree with nwr) that saying there is no outside force is incorrect, since the force of gravity is correct. And it is the force of gravity that adds angular momentum when you change the position of the center of mass.
Remove the force of gravity, and you are correct that you will never be able to add angular momentum (you won't have an oscillatory system at all in fact)
Edited by slevesque, : No reason given.