The brain it is..
(That's why I still have my doubts about the necessity of that front center speaker in 5.1)
The brain does the calculations, and it can do so, because we also have a left and right ear. So if a sound comes from the right, the right ear will hear this sound a fraction of time earlier before it reaches the left ear. Sound travels fast, so the difference of recorded signal between left and right is on micro phase-level. But the ear is fast enough to pass this information.
So, if both signals do have exactly the same content on volume- and on phase-level, the brain locates the source as coming from the centre of the panorama.
That's why we have 2 ears. To define where the sound is coming from. If we only would have one ear, we would need to turn our head around to focus the direction with the biggest amplitude, to locate the source.
If an ear would be just a hole in our head, we could only hear in 1 dimension : a line between left and right. But, because of the special shape of our ears, we can also distinguish if a sound coming from the sky or from the ground, and also if it's from the front or from behind.
This is why I wouldn't call Panorama an 'effect'. Is a volume-slider an effect ?
If you would invert the channel then your brain would be tricked giving a nice widened stereo image
This is done in the player option of OMPT of 'Pro-logic Surround' basically, together with some small delay between the channels.