I have this notion, which is admittedly somewhat idiosyncratic, that when foreign words join the ranks of English, they don't get to bring along all their foreign-word baggage -- their plurals, their conjugations, or anything else that would require an English speaker to somehow have to intuit the rules for that word in its original tongue. No, because the word has joined English, which already has rules for these sorts of things, and in the fine print of the contract that the word signed upon joining the language, it explicitly says that it must abide by house rules.
Nonetheless, people get theyselves worked up these things. Not terribly long ago, I essentially
laid all this out when someone asked what the plural should be of the car known as the Prius. Easy, sez me: in English, the default way to make plurals is to add
-(e)s, so, hey,
Priuses. All this fancy talk about what the plural would be if this were a Latin word is just piffle, because we're not talking about Latin, we're talking about English.
Ask folks what the plural is of
octopus, and yer more edumacated types will tell you it's
octopi, because
something-something-Greek-
something-something. Or
cactus, or
focus, or
schema, or -- heh-heh --
opus.
Now, I would not say that it's
wrong to say that the plurals are
octopi or
cacti or
foci or
schemata or
opera. But I will also staunchly defend
octopuses and
cactuses and
focuses and
schemas and
opuses. Because these latter words all follow normal rules, and because if I were a native speaker of English, I would find those to be perfectly reasonable ways to make the plural forms. (Hey, wait, I am, and I do!)
Ok. Surely among the more conservative publications in terms of these things is the
New York Times, yes? Yet behold this thing that
they have printed:

(I make you a picture in case some editor at the
Times get a gander at this, haha.)
Surely this is unusual for a publication, is it not? Google reports a mere 425,000 hits for this term, versus around 6 million for the term
millennia (where the plural ends in
-a because
something-something-Latin-
something-something).[
1]
Having read this far, tho, you will not be surprised to hear me say that I cheer the
NYT for this. I'm sure they'll get letters from folks who want us all to know Latin declensions, but fie on those people. And I say to the word
millennium, welcome to English, after all these years!
[1] This is not to be confused with the Mazda Millenia (sic), a singular car with a plural name, which we forgive because marketing has yet another set of rules, among them being "let's not let grammar get in the way of a good brand." Update 1 Aug: Apparently I had a thought about millenniums in an earlier post. I guess this obsesses me, I just keep forgettting that it does.
Update 1 Aug: Per Ben, fixed all the spellings, oops. :-) This actually upped the count for the non-native plural (millennia) to more like 10 million.
Update 5 Oct: Mark Liberman posts on the Language Log about the plural of syllabus. Interesting comment in the thread from Henning Makholm speculating a bit about the use of -i as a productive plural marker in English, not just a fossil on foreign borrowings.