I'm an atheist and I don't support the posting of the 10 Commandments on or in federal institutions and such. I'll admit, though, this situation is a bit of a curve ball with the historical twist thrown in.
Originally I was going to say not to mess with history and let the bronze list stay. But then it occured to me... taking into consideration that our laws were in effect before the placement of the bronze list, they would have been unconstitutional in the first place, not leaving them with any real merit to stay.
I would agree with Quark, though, that I'm not sure there's really much point in spending the time and money to have it removed. I find it more of an issue when they try to place such federal endorsements of religion in places still in operation, rather than leaving them on rarely viewed fossils.
And just to address Brunie... it's just a bronze plaque (I assume plaque), not a one-of-a-kind work of art. We have plenty of replications of the 10 Commandments in bronze, silver, gold, paper and whatever else they've taken the time to write them on. I wouldn't see its removal (note: it would be relocated, not destroyed) as a desecration of history. Maybe that's just me.