Is the amp or the guitar typically the problem when you get a buzzing noise only when you stop touching the strings or other metal part on the guitar? Th amp is perfectly quiet when I touch the strings or the tailpiece on my Les Paul. The amp and guitar are at somebody else's house. I plan to try with a different guitar the next time I'm there. But what is the most common cause of this problem?

I think this is 'somewhat' normal for guitars that aren't that well shielded. The noise is grounding out through your body.

Lots of things could cause this.
I'm assumming you have a single coil stratish guitar with a bridge ground. By touching the guitar's metal parts you close a ground loop which is typical of single coil wiring practices. They are also poorly shielded as mentioned earlier.
The following link is very helpful and should help you minimize noise.
http://www.guitarnuts.com/index.php