Saturday Night Live once ran a series of sketches with two characters named Hans and Frans. They regularly came on the air to make fun of people who engage in bodybuilding. Every time they introduced themselves, they said that they were there to pump the viewer up. Those who saw the sketches may be laughing about this at the moment, but someone who is worried about his body image may think that this pair, dated though they now are, represent the bulk of people who do bodybuilding. Do not worry. Hans and Frans represented only a small but vocal subset of this population.
The same trend of vocal but small minorities speaking for too many people of a given population can be seen in real life as well. Just look at any Internet comments section. Most people know not to read these posts filled of vitriol, but there are always people willing to feed the trolls. The people who speak in the Internet comments section do not represent the average way of thought on a particular topic. When a person first goes into a weight room, he will not be called a little girly man by most people in it. People in a gym are there to work out. They should not have time to judge you.