Does anyone else find this bullying difficult to watch?

I like the film overall, but I absolutely despise Mr. Reynolds and Mr. Lee (Edward Andrews, who sometimes plays really nasty guys). When Mr. Lee forces Tom to make the call giving up his role in the play, I find myself yelling at the TV, asking Tom why he doesn't just tell him off (although I use more colorful language than that). This was on TCM again today, and it was all I could do not to change the channel when either of these men spoke any lines. Yes, I realize what's going on with Mr. Reynolds (which is made more explicit in the play), but it's a real struggle for me to feel any sympathy for him, given his behavior. I can relate to Mrs. Reynolds' increasing anger and frustration, and I'm not sure how she managed to stay married to him as long as she did.