In a 200 degree oven, finish drying bread out for 15-20 minutes while you prepare the rest of the ingredients.
In a large frying pan over medium heat, melt 1 stick of butter and peanut butter together. Add brown sugar and cook until melted.
Add bananas and stir for 1-2 minutes, until bananas start to break down. Remove from heat and set aside.
Whisk eggs, spices, and milk together in a large bowl.
Spread dried bread in a greased 9x13" glass dish. Pour banana mixture over the dried bread. Evenly spread egg mixture over casserole.
Cover tightly and place in the fridge overnight.
When you're ready to serve, bake casserole in a 350 degree oven for 35-40 minutes, until middle is set and top starts to brown.
Melt ½ cup peanut butter in the microwave in 30 second increments until it is mixable and pourable. Top casserole servings with drizzled peanut butter and bananas.