Evolution does not state that an animal has to evolve from small to big. Evolution occurs to benefit the survival of an animal. I think a lot of people comprehend evolution as some magic process that magically changes animals. Evolution happens because of natural selection. If the environment all of a sudden becomes unsuitable for a large animal, the larger ones will start dying off, leaving only the smaller variations to breed. As most people know, most animals do have traits that create a lot of variance in size within a single species. Take the guppy for example, which lives in South America.
Guppies that live in an area with no predators will tend to grow quite large, because they dominate the smaller ones in the group. However, if you all of a sudden add a predator that prefers to eat large guppies, then the small ones will have dominance over the large ones. This eventually forms the guppies into two quite distinct groups. Mate these two groups together, and you won't get a medium sized guppy, you will recieve a guppy that is either small, or large, depending on which trait is dominant. And no, this does not necessarily have to happen of thousands of years, it has been done in about 10 years under scientific observation.