Hiking with their friend, the Old Lady, Babar and family decide to take refuge from a rainstorm in the Black Castle, which is reputed to be haunted. There, the kids encounter the friendly ghost, Baron Bardula, when they run off to explore the castle after the adults fall asleep.
After the baron shows them around the castle and regales them with stories of long ago, the children--who can see the ghost, although others can't--invite him to follow them back to Celesteville.
Upon the elephant family's return to the palace, mysterious things begin to happen: A lemonade pitcher floats in the air, and Arthur is pushed through the park in a shopping cart by an invisible hand. But when the ghost disrupts school and Celesteville's traffic, it's time for him to go back home.