Same issue here - with a twist. The icon does not show up when someone leaves me a VM. I have to check VM every so often to pick up messages. But after I listen to the messages, the icon shows up. I do not have. Visual voicemail listed in my apps manger.
What device do you have?