I've never harbored any malice toward the character or the performance. I was 13 years old when I first saw TPM and distinctly recall finding his antics highly amusing. And for most of my time as a Star Wars fan, I've seen Jar Jar in the light GL seems to have intended; comic relief and an invocation of the mythological archetype of the fool or clown.
Coming from a visual arts background and knowing GL did so as well, I have always been sympathetic to the view of these films as art in addition to being entertainment (and highly entertaining at that!). In other words, I understand that these films can involve stark contrasts, sophisticated nuance, psychological motifs, poetic rhyming schemes and perhaps even an aiming toward certain truths while also serving more elemental emotional satiation. I think you're likely to trigger negative responses when you miss the forest for the trees (GL gives a ton of fucking interviews!) and see Star Wars as solely pew-pew entertainment. Therefore, if something's uncomfortable, displeasing or oppositional, you attack it for not conforming to a master objective of pleasure and emotional satisfaction.