I'm sure I have not business commenting on this and I believe you need to do whatever works for you. But I read your story and felt the need to comment.
I'm a bit confused about whether or not there are still feelings left between you. If there are then is there a chance you still want to be with him and are sending mixed signals? Living with and "ex" who still cares about you is a bit selfish. In some way, you get some needs met because you come home everyday to someone who you know cares about you, but there are walls up all over the place. It's not fair to him either.
If there really are no feelings left between you, then the best thing is for both of you to move, and physically separate, on as soon as possible if not sooner.
Maybe I'm confused because I don't have a sense that I know all of the truth behind this situation. I hope that you both are being honest first and foremost, with yourselves. Then with each other. And I mean completely honest about what's going on in your head, your mind, your heart; stuff maybe you didn't think about before. If you do that, then you will both find the right path.
Take care I sincerely wish you the best. I just broke up with a casual boyfriend and it was tough.