The most important sex organ that you have is between your ears, not between your legs. If you can master that, you'll be fine.
Seems to me that you need to iron out the "peaks and valleys" in your libido. I believe there's a likely connection between your high libido when you're alone and your difficulty in getting/staying in the game when you're with someone. Clearly you don't have a physical problem; if you did then you wouldn't be able to get hard at all. It may be a sexual identity issue. SpeedoBuddy has some good questions, and the same ones that I would ask. You may be one of those guys who needs to completely wait until you find the right man and fall in love before you try and have sex. Or it could be just the opposite; you might need just a F*** buddy with whom you have complete acceptance and total horny-ness, and just no requirement or expectation to perform. If you could find someone who you can relate to, that you're very attracted to and just start out slowly until you've built your sexual confidence, you may find that you are secure enough in your ability that you can be totally "in the moment" and you can feel strong when sharing your energy with that person. I predict that if you do that, things will even out a bit.
Relax...you know...it's just sex!