Should I keep my children from seeing him?

like it or not you have no authority to "set limits" on him or keep his kids from him. if you try that the courts will hand your to you in a paper bag. i personally know a woman who lost custody over this. don't do it. now, is he right? no he is not. being reasonable with him and talking it out is the best way to handle this