I suspect all I would have to do is go to the Coffee House and talk about ghosts again, and my member rating would fall.
I'm not so sure this would be the case: I never bang on about woo woo being real but my score is lower than yours so you must be doing something right (independent of your position).