It was only ten to nine; if they just took refuge in the library or the Owlery, which were both nearer — ’Harry, come on!’ shrieked Hermione from the centre of the knot of people now fighting to get out.
There are no more uses of "refuge" in the book.
Show samples from other sources
The park serves as a refuge for wildlife.
She took refuge from the sun under a beautiful oak tree.