Wow, despite studying probability for years I never quite realised that the mean minimised the L2 norm. This is despite knowing full-well the essentially equivalent fact that the mean is the orthogonal projection in L2!

The median minimising L1 is also very nice, and I did not know that. It also means that the concept of median generalises easily to higher dimensions.