Exactly.
I was a mod on the Bolter and Chainsword for years and we (very) frequently had to change hotlinked pictures into links, as you basically leach someone elses bandwidth, without proper visiting their site. And on busy forums like the B&C and LAF, traffic can easily exceed what the owner of the picture has paid for, which can result in additional charges by their ISP.
Nowadays, with bandwidth not being such a dear commodity as it was a couple of years ago, it might not be as much of an issue anymore. But it's still a matter of good webiquette I suppose not to. I know I would balk at seeing my sites' bandwidth leached; I'd rather they visit my sites instead...
Oh; and doesn't Youtube encourage hotlinking to them by even providing embedding codes? I suppose they're ok with the whole thing as part of their policy.