I've met more men who don't, and who find it just as gross as women do. But, just as women laugh at unfunny, misogynistic joke to "keep the peace" and not be accused of being humorless, frigid bitches, those men also don't want to be bullied in the locker room.
Decent people need stronger backbones. They need to speak up, and realize there are still more of THEM than there are of the indecent and disgusting sort.