If he's a current client, and you want to continue on with that relationship: agree to settle on a figure, pay him, and continue on as before.
If you don't care whether he's a client or not: tell him to sue you, and quit working for him. It'll be up to him to prove a) you damaged the...