Tell him why this can’t work without him moving past this misunderstanding. If he can’t, you will know that this relationship isn’t as solid as you may like ...