I think it's because a lot of guys grow up not having any opportunities to act on it when they are younger, or spending a lot of time denying that they actually have that attraction. Plus depending on where you are when you are younger there may be a lot less opportunity to act on it. So these things add up and have a multiplicative effect.
As they get older, they still have some fear that they don't want to act on, or don't quite know how to act on, or aren't quite ready to admit to themselves that they want it, etc. Eventually, the mindset becomes "Well I've waited this long, I don't want this wait to be for nothing, I would feel terrible if I waited all this time and then had a shitty first experience. Since I've waited this long, I'll just wait longer until everything is just perfect."
Since nothing is ever perfect, they will probably just keep waiting, and the longer they wait, the more ideal of a sex situation they will require to justify the longer and longer wait.
The sad thing is that a lot of first time sexual encounters aren't great, because it takes practice and confidence to really get good at something. I'm glad i stoped waiting when I did and went through with it, so that now I know what i'm doing and know what I like, and had a lot of fun in the mean time instead of waiting for hte perfect situation. If I had, I'd still be a virgin and missed out on a lot of amazing ... "practice" for hte right guy! Just my opinion though!