Reference Images Copyright

So when I am using images other than my own as  references  there is a nagging question.

The law is that copyright remains with the creator of the image unless you have an instruction from the creator saying otherwise.

Using the image as reference means I will no doubt recreate elements but how much is necessary so as not to infringe on copyright?  How derivative must the image be ? 

This has proven to be a difficult question to answer. The famous case of Rogers v Koons  demonstrates some of the key elements of infringement.  A really interesting commentary on the case is included on the American Society of Portrait Artist Site.
.