Did you know that the question mark is a stylised q on top of a very tiny o? It was monkish shorthand for questio, which they used to write at the end of a sentence to indicate it was a question.
Oh, and the exclamation mark is a stylised io, which means "exclamation of joy" in Latin.
This little critter didn't get a name until the 19th century, when it was taught as the 27th letter of the alphabet. The kiddies felt Awkward saying "w, x, y and and", so they said "w, x, y and, per se, and". Which turned into ampersand.