Yes, you hyphenate when you use numbers as adjectives - but that's slightly different. It also applies to some other kinds of compound adjectives.
So you would say:
"She jumped from the tenth floor."
but
"She jumped from a tenth-floor window."
As far as I know, there's no concrete answer about when to switch to figures, different places have different house styles. For example some places say if you have a number under ten and a number over ten in a sentence, it's better to be consistent. So you wouldn't write;
"The summer she turned ten was long and golden; the summer she turned 11 was entirely different."
You would write, "the summer she turned eleven" to maintain consistency with the ten.