From what I can gather from the OP, you broke up because he wanted to be single. Then he just comes crawling back after all this time (4 years? WTF) wanting to talk??? If I was you I would forget about him, he thinks that he can simply throw your feelings to one side, obviously a selfish ****, it appears he thinks that you will just be there when he wants to call. That is not the basis for a good relationship.
Furthermore, you said that he already has a new partner. What, is he going to start two timing him? Red warning lights should be flashing at this point! If he's willing to cheat on his present partner now, what makes you think he wouldn't do that to you? I know this comes across as harsh but you have obviously had a rough time with this guy and your feelings don't need to be mauled again by him. Take my advice and find a guy that will actually love you, he's there somewhere. Good Luck!