If non sexual friendship is the extent you would like from him; TELL him you want to be friends ONLY. That is if you even enjoy his company in a non-sexual/& non-'romantic' connection kind of way, in any case.
I've been in a similar scenario a few times, but I usually let it be known (in the least hurtful way possible, I hope) by the second or third date/meet up or so, at the very latest, once I know for sure, & more than a few times after the first date also. As I mature and get to know exactly what I do and don't want, I tend to decide much quicker than in my (relatively) younger days.
The sooner you realise, the sooner you should make it clear and be conscious not to send out what may be perceived as mixed signals and behaviours. That will only lead him on and prolong a scenario you don't even really like yourself. It's a loose loose, illogical, self-sabotaging behaviour in the long run.
That unmistakable magnetic chemistry that can exist between people is very important. Don't ignore when it exists, nor when there is a definitive lack of it.