The name records a resemblance rather than a relationship. True goats belong to the genus Capra, which includes the domestic goat and the ibexes. The mountain goat is Oreamnos americanus, the only living member of its genus, and it falls with the goat-antelopes: chamois, serows and gorals.
What it does share with them is the ability to work near-vertical rock. Each cloven hoof has a hard keratin rim that catches small irregularities, surrounding a softer pad that deforms against the surface and grips. The toes can spread and close, and dewclaws behind the foot help on descent. The animal is heavily muscled through the neck and forequarters, giving it the strength to pull itself upward on ledges only a few centimetres wide.
That cliff habit is defence as much as feeding. Precise placement within the goat-antelope group is still argued, since the fossil record for Oreamnos is thin and molecular results have not converged.