There's a merchant (Gorell) in Bastions Keep that will sell you a plan to upgrade the staff, so basically there's a plan for nightmare, hell and inferno (when you first create the staff it works for normal mode). You have to make sure you are on the quest where Izual is already dead in act 4 before the merchant will sell it to you (for the respective difficulty), so you basically need to be on the quest to kill the prime evil. The plans cost 1 gold however to craft it it'll cost:
Nightmare: 200,000 gold + staff of herding
Hell: 500,000 gold + nightmarish staff of herding
Inferno: 1,000,000 gold + hellish staff of herding.
The higher staffs work in lower difficulties ie: inferno staff will open the secret level in all difficulties and you only need 1 staff as you can share it over your characters.
