This astonishing seminar, called “Accidental Pinhole and Pinspeck Cameras: revealing the scene outside the picture”, gives a refreshing view of the concept of shadows:

A shadow is also a form of accidental image. The shadow of an object is all the light that is missing because of the object presence in the scene. If we were able to extract the light that is missing (that is the difference between when the object is absent from the scene and when the object is present) we would get an image. That image would be the negative of the shadow and it will be approximatively equivalent to the image produced by a pinhole camera with a pinhole with the shape of the occluder. Therefore, a shadow is not just a dark region around an object. A shadow is the negative picture of the environment around the object producing it. 

I totally recommended you to watch the video seminar [Link], which is given by Antonio Torralba from the MIT university.