I had honestly never wondered what happened in the bowels of a manually operated scoreboard. Figured there was a person in there, and he or she switched some numbers around, occasionally taking a break to let Manny Ramirez go to the bathroom.
↵But that was a mistake, as I found out after reading this fantastic article from Baseball Prospectus' Ian Miller. It's more involved than that, and it's also more interesting.