The Compleat Strategist on Easy 33rd Street between 5th Avenue and Madison Avenue was great when I visited a few years ago. It's quite centrally situated so should be easy to visit if you plan on going to the Empire State Building etc.
There was also a great comic store with some miniatures close to Times Square. On 7th if I recall correctly and a bit south of Times Square. It's a bit tricky to find as it's not on ground level, but at least back then they had massive comic related stickers on their windows.