Both Uses of

contrary

in

The Narrative of the Life of Frederick Douglass

She was a field hand, and a whipping is the penalty of not being in the field at sunrise, unless a slave has special permission from his or her master to the contrary—a permission which they seldom get, and one that gives to him that gives it the proud name of being a kind master.†
Chpt 1 *

Immediately after the holidays were over, contrary to all our expectations, Mr. Hamilton and Mr. Freeland came up to Easton, and took Charles, the two Henrys, and John, out of jail, and carried them home, leaving me alone.†
Chpt 10 *

contrary to = in opposition to

Definitions:

(1) (contrary as in: a contrary idea)
different (perhaps opposite or mutually exclusive)

In formal logic, contrary propositions or contrary arguments describe two things that cannot both be true. They might be described as mutually exclusive. For example, a pet cannot be both a cat and a dog. It doesn't need to be either, but if it is one, it cannot be the other. So the statement that a pet is a cat and the statement that the same pet is a dog, are contrary statements.
(2) (contrary as in: a contrary personality)
disagreeable -- typically in reference to someone's personality
(3) (contrary to as in: contrary to)
in opposition to

You could also think of this as meaning, despite, or in spite of, or in conflict with; but since a common meaning of contrary is opposite, you many find it easiest to think of contrary to as in opposition to.
(4) (on the contrary as in: on the contrary)
an expression used to intensify denial of an idea

More rarely, on the contrary is used to oppose an idea contained in what was just said rather than to oppose what was literally said. Here is an example: "I will not pay you for the work. On the contrary, I may sue you for damages." Suing for damages does not oppose not paying, but it does oppose the idea of paying.
(5) (to the contrary as in: to the contrary)
with an opposite or different effect; or something with an opposite or different effect
(6) (meaning too rare to warrant focus)