I think it's just because there is a large number who like those things, and people tend to group things together (something that is learned when you are younger).
but i also really like to play sports (not watching as much though..), and i'm actually pretty good at the majority of them, it's just stereotypes honestly.
Jokes!!!!!!! My friend irl loves rugby but i think that's more to do with the men wrestling how they do but still he likes it so yeah i think people just have bad perceptions sometimes.