Sportsjockia, first, I'd ask myself why I am hanging around a 23 y.o. It's not the age difference between the two of you, but the fact that a 23 y.o. is virtually out of teenagehood....some are more mature, but many still have the expedient view of life that it will not end, and that they're interests go no further than their latest strong "desire."
Second, no doubt he is narcissistic. Whether he is a hustler is another issue. As a narcissist, he can simply love the attention that older guys pay towards him. Take a look at some of the profiles on RJ of older guys (some, not all, thank goodness) that say "...looking for a younger guy who wants to be with an older gentleman..." That's a set up...it tells the younger guy, he's going to be idolized.....for a while.
Actually, you're friend is nuts not to want to have a relationship with you, if looks/build are his only criterion. He probably doesn't give a whit that you are intelligent, and would be committed to him. I know that's not what you are asking, though.
So....I am willing to bet he is indeed a "boy toy" for these other guys.