Judge in Scientology case won't remove himself