I certainly don't have a lot of experience with the game yet, as I've only played one game with Koryu versus Sohei, but I would definitely recommend a buntai with some armor.
In our game the Sohei trampled the Koryu, eliminating every man without losing even one monk. Luck and dice seemed to even out fairly well and the skill level of the players was about equal, so, even with the Koryu's bujutsu they had a heck of a time getting through the Sohei's medium armor. The best that they achieved were stacking stun tokens on their opponents.
I would have hated to see the Koryu up against the Bushi!