Laura Werlin’s The All American Cheese and Wine Book has this to say about the differences between sheep’s, cow’s, and goat’s milk:
The difference between the three milks is their composition or structure. The milks have different protein structures, different fatty acids, and different butterfat contents. This is part of the reason that each milk, when it becomes cheese, tastes different from the others. There are other factors too, including how each animal metabolizes what they eat. That translates directly to the milk. For example, goats don’t metabolize carotene the way that cows do. As a result, goat’s milk remains snow white in color, while cow’s milk has a yellowish or sometimes orangeish hue.
Wikipedia gives a handy breakdown of the nutritional content of cow’s, sheep’s, and goat’s milk.
When it comes to flavour, sheep’s milk is nutty and creamy, while goat’s milk has a somewhat stronger taste that’s well-suited to savoury dishes.