I was going to mill it but filing it would seem a more controlled option. The biggest problem I had was work holding. Awkward shape and size meant it was slightly out!Either get a thicker copper washer or wind a grub screw in there and use the outer face as a guide to file the caliper square to the thread
But i am sure the copper washer was a little hard`. I think the biggest issue was the recess allows the washer to move as it is 10mm thread but the recess is more like 8mm. So it moves from side to side when tightening!