I read a scientific article once correlating general good health and longevity with long legs. I can't find it though. I think there was also another which disputes that study.
Height is essentially an easy way to determine good nutrition and thus higher status, hence why it seems to be hardwired for females (and some gay males) to like taller men. It's evolution I think.
Unless it has certain environmental consequences (like in forest living tribes, notably the Mbuti Pygmy tribes of Congo), where height has an adverse effect on survivability and lower stature probably means a higher chance of finding food, evading predators and enemies, and living long enough to procreate. Hence the pressure to be shorter.
It also has an indirect effect on self-esteem, if your height is much lower (or way higher) than the average in your region. I hesitate to use this term... but... the so-called Napoleon Complex, probably exaggerated of course, but there is a definite correlation between height and social perception. Not because of genetic factors or whatnot, but because of social pressure to conform and the inability to do so. Taller people are shown to get better jobs
Not to mention sports and the military, most of which have minimum height requirements. Both of which can affect a man's sense of masculinity greatly.
I'm short by western standards (5'6") but actually taller than the average 5'4" of the Philippines, so I didn't have height issues growing up. I do wish I was taller sometimes. But meh.
I think I'm alright with my height. I do like taller (and bigger) guys generally, but far from exclusively. I dunno. I just like the protector image taller guys evoke. DISCLAIMER: The above psuedoscientific mumbo-jumbo is mostly composed of my own opinions and not based on actual scientific studies nor informed psychological evaluations, except where cited explicitly.