The Renegade cavalry are supplied with an open right hand, with the intention that you supply the lance, so it would be straightforward to replace the lance with a rifle or sabre.
The only (28mm) western front British cavalry that I'm aware of are from Renegade and Battle Honours for troops with caps, and Old Glory for cavalry in helmets - I was surprised that Foundry didn't have any, since they have three different types of Germans (hussars, dragoons and uhlans). I hope the rumours of GWM cavalry are true - I'll get some of those if they come on the market!
One other option to consider is Brigade Games. Their Indian Lancers are very nice figures. Although Indian cavalry was present on the western front from November 1914, their actual service was primarily in a dismounted role. Still, one more option to consider!