It’s hard to side with a smut-peddler like Howard Stern, even when it comes to the protection of free speech.

But massive radio corporation Clear Channel Communication’s decision to yank Stern from its radio stations last week raises some troubling issues about federal regulation of speech.

What Stern did to cross the line and thus receive the boot — an interview with Rick Solomon, Paris Hilton’s former boyfriend, that mentioned penis size and anal sex — was vulgar and offensive by most standards.

At first, it is understandable how someone in Clear Channel might think the syndicated show had thoroughly violated “common decency” standards.

So a private company decided Stern was too offensive and made a decision to cut him — a decision any private company has the right to make.

What’s the free speech issue here?

The problem is, if Clear Channel was so concerned about upholding decency standards in its programming, why did it wait until now to suspend a show that has had such segments as “Fartman” and a person having sex on the air with a rubber doll?

And it’s difficult to argue Clear Channel was in danger of losing listeners — for Stern’s audience, a discussion of Solomon’s penis size was probably as tame and typical as the daily soybean report is on a farm program.

To find the real reason why Clear Channel suddenly has a moral backbone, you have to go back more than a month ago.

In late January, the FCC imposed its largest indecency fine ever when it handed a $755,000 fine against Clear Channel for the sexually explicit radio show, “Bubba the Love Sponge.”

A few weeks later, Janet Jackson’s “wardrobe malfunction” during the Super Bowl halftime show provoked hundreds of thousands of viewers to angrily complain to the FCC, which called the show a “new low” and prompted new measures to make sure that live broadcasts would never be so shocking.

So despite Clear Channel’s right to remove shows it finds offensive, don’t be fooled into thinking the company is concerned about children as much as it is about avoiding massive federal fines. It’s not hard to argue that in Stern’s case, his show has consistently dropped the bar in broadcasting standards for years and something needed to be done.

But any time a company suddenly changes its policy on what is acceptable speech — out of fear from government intervention — it raises the perpetual concern that the government might someday overstep its bounds and encroach on our personal responsibility.