You should read the contract you signed when you allowed them to put your number on their account.

Basically, they own it now and since it is associated with their business they can refuse to give it back to you. The reason makes sense. That number is now a point of contact for the company and if a customer calls you on that number after have left the company and maybe started working for a competitor you could potentially poach their customers.

I am not saying it would happen that they would keep your number but there is a chance and there might not be any recourse.