Legalities of employers confiing cell phones?

I work at a call center and got my phone confied. I was on break, I wasn't even using it...just had it in my lap. I left work (wasn't scheduled that early so they don't care if I come or go), I got my phone back but was told when I return for my shift later tonight I have to give it to him. Can he do anything if I leave it at home and don't bring it back to work?