We don't get to decide it, that's for sure, but there must be limits to how much importance we give to this or that grievance. There has to be a point at which we agree to diasgree. Otherwise we might as well put an end to this "society" thing and live in a cave, far away from everyone else (and far, far away from anything resembling garlic if I have my druthers). 

Symbols like the Confederate flag are obvious contention points, even if in and of themselves they might be seen as pretty innocuous by certain people. It's no good arguing that the swastika is an ancient Indian symbol of good fortune or that the *n* word simply means "black". Such symbols and words have acquired new meanings that nowadays trump any previous and less offending ones. I don't think anyone would dispute that their use must be considered more carefully than, say, the use of the word "damn" in public. 

That being said, and hoping that we can all refrain from willfully offending our neighbours, there is a point at which we must say "this is silly". There will never be a universal clear line between what is offending and what isn't, because that line will vary according to the individuals considered. But accepting that some things seen as mundane by many will offend us personally, and realizing that the right to never be offended doesn't exist, will allow us to live with others without being constantly upset. If we feel really strongly about a point, we can try to educate those around us. If we are in the right, eventually our view will become the norm; that's how society progresses. The discussion must be held before we decide that "X must be banned because Y is offended", however. 

You've summed up many of my feelings almost perfectly here.  

It's impossible to know what will offend every reasonable person, so it can be difficult to know how to proceed in everyday life.  What might be perfectly harmless to 99.44% of the population might be the very thing that sets that remaining .56% into a rage, but I can't possibly know who is in that tiny minority without foreknowledge, which we unfortunately don't have.  Almost anything today can trigger offense or trauma in someone, and that is OK until they begin pushing the issue too far.  

For example, I might write something on a message board that is perfectly innocent, something like "I thought the last episode of Game of Thrones was great!  A lot of excellent action!"  On the surface, nothing wrong with that.  However, maybe one of the scenes in the show depicted the beating and rape of a female character, which happens fairly often on the show (one of my few complaints about it, along with the persistent use of a certain vulgar term for female genitalia), or the killing of a gay character when he is found out.  Someone else on that message board, maybe a woman who has survived rape or a gay man, sees what I wrote and comes back at me asking if I enjoy seeing women brutalized or if I feel that should be how all gays are treated.  I ignore it, but then another person joins in, and another, and another, and all of a sudden, I'm painted as some sort of monster because I didn't feel the need to explain myself.

Not a big deal, you say; it's just an internet message board.  Maybe, but what if someone finds out my real name, maybe through Facebook or another social media source?  They could easily track down my phone number, e-mail address, LinkedIn profile, employer, business address, home address, etc.  Now all of a sudden, I'm involved in a real-world situation simply because I liked last night's Game of Thrones.  

These types of things happen all the time, such as the dentist who shot that lion in Africa or others who post pictures of their hunting trophies online.  Animal rights activists go after them with unrestrained fervor, all because they were offended by something that person did.  Should a person have his business boycotted or his family and his own life threatened because he happens to like to hunt, even if it is in poor taste to some?  The internet allows a mob mentality greater than any that was ever possible before, and sadly, some people use it to exact vengeance in terrible ways that go far beyond normal all because they didn't like what someone else did or said or posted.     

Re: Boycotts

I hated Hell -- er-- Home Depot already, but sometimes it was the last port in a storm.

Now that the CEO/founder/ownwer has come out in favor of Trump, I will be sure to sail past from now on.

Relevant:

I've always been mystified by people who make such a big deal about how mean boycotts are. The alternative is to pass a law forcing consumers to patronize the homophobe bakery that won't cater gay weddings.

FREEDOM! 

Fair point on the economic boycott, because no, we obviously cannot force individuals to frequent any establishment.

I was thinking more of protesting at the place of business, which always has the opportunity for violence when two opposite sides meet and someone who is really passionate about their position goes too far.