Gay men definitely have problems with keeping their dick or ass in their pants. It does go back to men being men. In heterosexual relationship, other than social norms, women act as a deterrent. Women aren't so eager to jump around and start fucking every hot piece of dick. I would imagine if the heterosexual relationship wasn't such a default paradigm for relationship, where you are supposed to marry and have kinds, men would be behaving the same way if women would let them. You do see this somewhat among straight men with money, good looks, or power. They do sleep around with lot of women because they can! This doesn't mean each and every straight men would have been doing this, but it would still be much more mainstream than being fringe as it is now. You also hear from time to time that there are more straight people in open relationships these days.
The good thing about being straight is though, you still have lot of options to choose from. Gay population is quite small. So this kind of behaviour does make life hard for people who are looking for some one with bit more control over their dick or ass. I am not sure if people really get satisfaction out of it after a certain age, or it's just a habit that's hard to get rid off. Yes I understand that sexual feelings don't go away completely as you get older, but from my own experience, it's just lot of work and comes with its own set of issues. That itself doesn't make it worthwhile experience for me.
People here are trying to portray that those gay men are exceptions, but it is the norm. Obviously you being fit, and probably good looking, attract large share of these gay men. You might see bit more exemplified version of this problem.