I wouldn't worry about this too much. Your BF is not a monk, or an hermit living in total seclusion. When you are there, he spends his time with you. That's what matters. You cannot expect him to stop seeing his friends just because he is an relationship with you or with anyone else for that matter.
You'll have to recognize that his history does not begin with you. His friends have been there for him for a while now. So, he continues seeing them, and socializing rather than being alone, unhappy, and possibly getting depressed over it. People are social animals after all.
Now, you have to extend a degree of trust to your BF. And he has to do the same to you. Living in fear that you are cheating each other would destroy every relationship even before it really starts.
Consider designing a plan to live together. Sure, no one wants to rush the things, but having an open endgame is probably not the wisest thing to do if you want this relationship to last.