Love is frequently mistaken for something else... infatuation, lust, an alleviation of loneliness, fear of loneliness... all sorts of things.
Saying "I love you" is the clincher that seals the I'm-thinking-of-a-future-with-you deal. It's perceived as "the next step" often heavy with implications, just because there is love there doesn't mean a person is ready for all the things that come along with uttering the words.
More than anything love is often different things to different people, that can lead to a discontinuity of expectations, for people who aren't good at communicating their feelings, this sort of thing can be really scary. That's what I think.