There is an invaluable resource on Union armament by unit throughout the war, I believe put together by a military history enthusiast/re-enactor and donated to his local museum in Wisconsin. As far as I remember, it is free to download from the museum as a pdf. Can I suggest you search 'American Civil War Military Equipage Archival Collection, the Civil War Museum, Kenosha Wisconsin,'? It has some uniform details and regimental colour information as well.
I wasn't so lucky with information for the Secesh. My interest was the Georgia brigade of McLaw's division at Chancellorsville, which included the legions of Cobb and Phillips. I was able to track down each regiment through various online resources, sufficient to be pretty confident that the brigade used a mixture of Enfield P1853s and 'Richmond' rifles, or copies of the 1855 Springfields manufactured with machine-tools removed by Virginia from Harper's Ferry.
I am pretty late in responding to this thread, but I hope this may be of interest to the OP or someone else.