Carbon fiber can be brittle depending on how thin the adapter is and what other fibers are laminated in
together with the carbon fiber.
If you're just drilling holes for screws then a sharp drill in a drill press will do fine, make sure the adapter
is well supported underneath to lessen tear out, set up a vacuum to suck away the dust
and make sure that the exhaust of your vac is directed outside unless your vac is HEPA rated.

Adding to what Labrat mentions in using super glues to seal off any tear out, after the super glue
cures wet sand the area. If you don't you'll be dealing with very fine needles of CF.

The bright side of the CF splinters is that they're black, unlike fiberglass.
You can see them to pull them out.