OK, aside from scratching the metal if they are not lined what about:
Cracking over time?
Exapnsion and contracting with temperature, humidity?
Critters that may be attracted to the cellulose or other materials?
And, as others pointed out. Outgassing?
Lenses have been made for a very long time but while I have seen lots of metal, plastic, leather caps I can't remember ever seeing one made from wood. Why is that?