Of course people have something to say about it.
I have pointed out many times that not everything has a moral value, but people tend to assign a moral value to everything. We live in a day and age when people have an opinion on everything.
This is why I often encourage people not to have an opinion on everything. People need to understand that there are just some things that can really do without their opinion.
For example, when we bought our house we really appreciated that some people managed to refrain from giving their opinion whether it was a good decision or not. And in an ideal world, those of us who have nothing to do with the muslim community should refrain from having an opinion on whether they could or couldn't build the mosque right there. It's none of our business.
That said, I completely agree with the person earlier in this thread that said they ought to be able to hang a big poster of a cartoon drawing of the prophet mohammed. In fact, If I was living anywhere near NY I would have considered doing that myself.