When a whole bunch of people have the same reaction to you, it's usually you, not some mysterious quality the people have in common.
In particular, people frequently attribute to strangers the emotions they feel themselves. That would mean that the one who is hostile is probably you. The response you'd get for your hostility is some of the same you would be projecting.
Of course, I am saying this mostly because I have heard many young gay men speak like you. "I have nothing in common with gay culture" tends to mean, "I am not as gay as they are." "I am bisexual" tends to mean, "I am not comfortable being gay." "I should go back to chicks," finally means, "I know I can't make it work without men."
Conversely, there is a shock therapy that I've seen work almost invariably: befriend a drag queen, or a truly effeminate man. I mean, strike up a real friendship and hang out together. You'll lose your hostility, other gay men will be impressed that you got over your insecurity, and you may make a really good friend.