Most of the wilderness lies between the crests of two parallel mountain ranges, the San Rafael Mountains and Sierra Madre Mountains, and includes the drainages of two watercourses: the Sisquoc River and Manzana Creek. Both flow to the northwest, eventually joining together and draining into the ocean near Santa Maria. Elevations within the wilderness vary from 1,166 feet (355 m) at the confluence of Manzana Creek and the Sisquoc River on the western boundary, to over 6,800 feet (2,100 m) at Big Pine Mountain, the highest point in Santa Barbara County. Dividing the drainages of the Manzana and Sisquoc is a ridge known as Hurricane Deck, a rugged 15-mile (24 km) slab of upthrust sandstone with a trail snaking along the top.
Rock formations in the wilderness are predominantly sedimentary, and are of Miocene and Cretaceous age. Both the Nacimiento and Big Pine Faults run through the wilderness, roughly parallelling the Sierra Madre and San Rafael Mountain crests respectively. Hurricane Deck is a single block of Miocene-age sedimentary rock. Immediately south of the wilderness, opposite the Big Pine and Camuesa Faults, is a large region of the Franciscan Formation. Mercury was formerly mined in this area, and abandoned mines along with tailings piles can be found.