I bought some refrigerator magnets with big plastic spring clips. The magnet pops off, so I can hang them with a hook, and they have only light force, so they just hold, but don't damage. So far they've worked fine.
You can get 1/8 inch diameter paper punches, if you want to pursue that. For smaller holes, Harbor Freight sells a metal punch with a 2mm hole as the smallest in the set, and you can also look at a leather punch, which has a similar size hole, but might be harder to use.
Bookmarks