Am I in the wrong at all for this?

Do not feel bad about this. He used you, and you shouldn't be the victim anymore. Just confront him and tell him that you've moved on to his ex. Simply tell him off.