Not sure if all the phones are the same but on a Moto Z Force (what I have), go to SETTINGS and click on APPS & NOTIFICATIONS then click on APP INFO then click on the three dots in upper right corner and click SHOW SYSTEM.  Scroll down to PRELOADED CONTACTS LOADER and then click STORAGE then CLEAR CACHE and CLEAR DATA then back arrow (top left corner) then click DISABLE.   Doing that finally got rid of them for me.