Docker: only copy what we need to the build image (#4562)

There are two reasons this is a good thing:

 * first, it means that you don't end up with stuff kicking around your working
   copy ending up in the build image by mistake (which can upset the pip
   install process)

 * second: it means that the docker image cache is more effective, and we can
   reuse docker images when iterating on the docker stuff.