These are lesson learnt,

• scope to set to synchronized
• once the delivery notification occurs, it seems the orchestration continues its workflow, but it did trigger the exception handler block later
• if suspend the orchestration after delivery failure, use an orchestration instead of send port to handle the failed routing message, this is based on thread. I didn’t yet have time to figure out why exactly, I guess there is some properties get reset, thus make BizTalk unhappy.