I always rationalized this as the big guy still being somewhat encumbered by toting a 2-man weapon (and ammo etc.), so while he is strong, he's still not at peak melee efficiency (hence no bonus to FV), and when he does hit someone in combat (assuming he's not armed with a secondary weapon – knife, sword etc.), he's clumping them with a weapon not really designed to be used as a bludgeon – a large, improvised weapon perhaps, which is found in the equipment lists, and offers no Pluck modifier.
Looking at which melee weapons get Pluck modifiers, it seems to be – broadly speaking – a -1 for something sharp, and a -2 for something sharp and heavy, so I'd be reluctant to give a Pluck modifier to something that is neither, even if it is wielded by a big guy.