I probably should have read Computer Power and Human Reason years ago, when all this hype really started getting going. Honestly I should probably have read it in college when I was first getting into programming! But the next best time is now, so I borrowed: my mom's inscribed copy a few months ago (which to be fair she has also never read) and got started.
A lot of the first few chapters is fairly straightforward for anyone who has a basic grounding in computer science. Weizenbaum[1] is walking an odd line where he's trying to explain concepts like "turing machines" and "assembly languages" to a lay audience without sacrificing either rigor or a relatively academic tone. I'm curious how well this works for someone who genuinely has no prior exposure to this material; to me, all of this is review of the basics of the field, and so ended up feeling a little boring.
But then I got to this section which blew my socks clean off my feet (emphasis mine):
The idea that a person can write a program that embodies anything he "thoroughly understands" is at least equally problematical. Understanding something always means understanding it at a certain level. An actuary uses some fairly sophisticated mathematical tools whose fundamentals he almost certainly doesn't understand or care anything about. Everyone who makes change uses arithmetic, but very few people know or care much about the beautiful axiomatic system on which arithmetic is based. In effect, we all constantly use subroutines whose input-output behavior we believe we know, but whose details we need not and rarely do think about. To understand something sufficiently well to be able to program it for a computer does not mean to understand it to its ultimate depth. There can be no such ultimate understanding in practical affairs. Programming is rather a test of understanding. In this respect it is like writing; often when we think we understand something and attempt to write about it, our very act of composition reveals our lack of understanding even to ourselves. Our pen writes the word "because" and suddenly stops. We thought we understood the "why" of something, but discover that we don't. We begin a sentence with "obviously," and then see that what we meant to write is not obvious at all. Sometimes we connect two clauses with the word "therefore," only to then see that our chain of reasoning is defective. Programming is like that. It is, after all, writing too. But in ordinary writing we sometimes obscure our lack of understanding, our failures in logic, by unwittingly appealing to the immense flexibility of natural language and to its inherent ambiguity. The very eloquence that natural language permits sometimes illuminates our words and seems (falsely, to be sure) to illuminate our undeserving logic just as brightly. An interpreter of programming-language texts, a computer, is immune to the seductive influence of mere eloquence. And words like "obviously" are not represented in the primitive vocabularies of…