250 Clunk noise from brake pads

Hi,

my 250 makes a clunkish sound when changing direction (fwd, bwd), then braking (lightly). Only this formula produces the clunk.

I've found out that it's due to the front pads moving back and forth in the caliper. It has CL5+ pads, and the holes for the pins are significantly larger than the pin diameter.. Has anyone else experienced this problem, or am I perhaps missing some brake hardware that should take care of this?

IIRC, the sound wasn't there just after installation, so it's kind of of strange what could have changed.