Probably because he has the secret gay gene. People used to like him in the Colbert Report where he parodied right wing people but it was still funny. At least Stewart hasn't really changed much.
I still remember when President Trump fired Comey, and Colbert announced it on his show that same day. Colbert's mostly liberal audience cheered and were in elation (because of what Comey did to Hillary days before the '16 election), but almost immediately Colbert said no! Stop that! And then he had to correct his audience into not cheering for that...
Hard to believe that was just a few short years ago, back when the world was still -relatively- normal. Today, Colbert is the biggest left wing nut on late night television, and he is still correcting his audience.
The early Colbert Report was brilliant. But the last few years, you could tell that he wasn't happy playing the character anymore, and pissed off that so many fans didn't realize he was actually doing satire. He was making fun of them, but he was also making the right wing look really cool and fun. It was kind of blowing up in his face. Hell, it was a few years before I even realized it was satire. It blew my mind. Ohhhhhhhh. He's actually making fun of us. Still, it was very clever so it didn't bother me none. I can take a good ribbing.
You are SPOT ON to what the problem is with these late-night hosts. They have stepped out of the COMEDY box and now it's just political CRUELTY. They're not funny anymore, they're just mean.
Even though I knew he was making fun of us (and make no mistake, I realize now he was laughing at us behind our backs because he thought we were so stupid we bought his shtick) his caricature was funny because it seemed loving and he just felt like a down to earth, regular guy. That was such a sham that now I wonder if he's actually some narcissistic psychopath.