You need to carefully drill the rivets out.

Hold onto the broken part, you need to match the holes for the mounting plate with the new part.

use a small chisel to get get the rest off. You don't want the hole any bigger than 1/8".

Use the chisel to break the small flange off the rivet.

A nice sharp chisel will do the trick.

You can also carefully slice the flange in a couple of places using a dremel and a small cutoff wheel, then lightly chisel the rest off.