If that is oil have you given it a good clean, the surfaces should ideally be oil free.It's just oil I think. No scores.
Just so I'm clear. Open up the chuck so the middle doesn't stick out and place on a bit of wood. Use some of that abrasive paste on one side then run the drill and pull the head down into the chuck?
I'll give that a go. What removes oil, acetone?