Nonetheless, people get theyselves worked up these things. Not terribly long ago, I essentially laid all this out when someone asked what the plural should be of the car known as the Prius. Easy, sez me: in English, the default way to make plurals is to add -(e)s, so, hey, Priuses. All this fancy talk about what the plural would be if this were a Latin word is just piffle, because we're not talking about Latin, we're talking about English.
Ask folks what the plural is of octopus, and yer more edumacated types will tell you it's octopi, because something-something-Greek-something-something. Or cactus, or focus, or schema, or -- heh-heh -- opus.
Now, I would not say that it's wrong to say that the plurals are octopi or cacti or foci or schemata or opera. But I will also staunchly defend octopuses and cactuses and focuses and schemas and opuses. Because these latter words all follow normal rules, and because if I were a native speaker of English, I would find those to be perfectly reasonable ways to make the plural forms. (Hey, wait, I am, and I do!)
Ok. Surely among the more conservative publications in terms of these things is the New York Times, yes? Yet behold this thing that they have printed:
(I make you a picture in case some editor at the Times get a gander at this, haha.)
Surely this is unusual for a publication, is it not? Google reports a mere 425,000 hits for this term, versus around 6 million for the term millennia (where the plural ends in -a because something-something-Latin-something-something).
Having read this far, tho, you will not be surprised to hear me say that I cheer the NYT for this. I'm sure they'll get letters from folks who want us all to know Latin declensions, but fie on those people. And I say to the word millennium, welcome to English, after all these years!
 This is not to be confused with the Mazda Millenia (sic), a singular car with a plural name, which we forgive because marketing has yet another set of rules, among them being "let's not let grammar get in the way of a good brand."
Update 1 Aug: Apparently I had a thought about millenniums in an earlier post. I guess this obsesses me, I just keep forgettting that it does.
Update 1 Aug: Per Ben, fixed all the spellings, oops. :-) This actually upped the count for the non-native plural (millennia) to more like 10 million.
Update 5 Oct: Mark Liberman posts on the Language Log about the plural of syllabus. Interesting comment in the thread from Henning Makholm speculating a bit about the use of -i as a productive plural marker in English, not just a fossil on foreign borrowings.