I can and do easily go all the way through glaze, clear, paint, and primer, right back to bare surface, with whatever grit I choose. I am using firm blocks btw.
As previously said it could be putting the primer on too thick and getting small pinpricks holes, this could be made worse with too fine a paper just polishing the finish and not cutting back the primer.