Self defense when someone won't let you leave?

So I was reading about Ronda Rousey's domestic violence incidents and came across a situation that I have pondered before, but don't know the answer to.

Lets say I'm out somewhere and go to the public restroom.
While in there someone gets into an aggressive verbal argument with me.
I decide to leave the situation, but they block the exit and refuse to move.

Would it be generally legal for me to use force to make them move and let me leave, assuming there were no other exits besides the one they are blocking?
I don't mean beating them badly, but say grabbing them & grappling them away from the door, then leaving without further violence (unless they attack further).
Or getting in a good body shot or leg kick and then slipping out past them.

I understand laws are different in different places. I'm just generally curious about the law regarding this type of situation, both in the US (i know it varies by state) and in the UK.

Does the answer change if I have a cell phone on me and could call for help? (Assume the person isn't actively attacking you, just preventing you from leaving.)