I had this issue as well, but had no luck finding an answer in the Community. Just figured out myself that it wasn't the Voicemail app providing the notification, it was the Phone. Go to your apps, view the ALL tab, then select the Phone app and clear the data. I hope you have the same issue I did and this provides a solution.
Thank you! This solution worked for me!