And if you are not hitting 0db the limiter wont be doing anything. So what you want to avoid is hitting 0db, regardless of the limiter. Either way would distort whatever (usually transients) going over 0db. If the signal is hitting 0 then itll sound terrible without a limiter or sound OK but limitted with a limiter.
What I meant is that all a limiter does is stop a signal going over a threshold.