> The notion that I will become unemployable immediately after being handed the keys to a steam shovel, having spent decades digging with a spoon, strikes me as faintly ridiculous.
I’m fairly certain that most programmers will end up unemployed. My analogy would be ‘combine harvesters’ where all the people who reap are eventually replaced but one. There might be something to do for programmers for a while, but eventually there will be little left to do.
Take accounting for example. Accounting rules are all in the book which the programmer turns into accounting software, and the accountant does the accounts. Maybe the programmer does their programming quicker because they don’t need a book anymore, but eventually the accountant doesn’t need the software either. Then no one needs the accountant.
I think there’s a year or five where everyone gets more efficient, but then we don’t need them anymore more.
I’ll agree some of the young ones will figure out something new to do. Older ones won’t (like the engineers in the North of England in the 80s). Your steam shovel will immediately put thousands of spoon-diggers out of work. They won’t find a job, and I think AI will stay ahead of the workers.
We've had accounting software for 50 years and it hasn't replaced accountants (my dad had a CPA/MBA project that involved punchcards and mainframes). And accounting, or bookkeeping specifically, is one of the fields theoretically more exposed to automation - it's a large and largely uniform field with verifiable results (do accounts balance or not). Still - we have bookkeepers and accountants!
Okay, so you're using the word rational in a somewhat negative way then? It seems to imply for you a self-destructive disembodied state that requires one to ignore one's evolved instincts and biology. I suppose that I think it is rational to follow one's instincts and biology. If one were reasoning through what an embodied, evolved entity should do, one would rationally recommend that they default to following their instincts because those instincts will generally see them right on average.
Take a silly example. A couple come to you asking how to rekindle their love for each other. You tell them to have sex more. You tell them it amplifies hormones that will promote their relationship's health. Oh no, they tell you, we are rationalists and you are talking about a pre-rational biological response, we couldn't possibly. We were hoping you would recommend an insightful essay for us to read. You shrug. The couple end up divorcing.
Was it irrational for you to suggest that they take advantage of their biological instincts? I think it was very rational of you to make that suggestion, and they were irrational for ignoring it just because they had an irrational bias against biological mechanisms. Your ancestors have been solving problems for billions of years, and their best heuristics have been passed on to you in your biology.
Same here. It is eminently rational for you, an embodied, evolved being, to give your limbic system a seat at the decision-making table. Not the only seat—it's not infallible—but you needn't be ashamed of listening intently to its voice.
I'm just using the word rational to describe a decision process that uses logic and reasoning, I'm not trying to attach any positive or negative valence to it.
Our instincts can be adaptive and effective or the opposite, but they're never rational by definition.
Agree with you completely. A friend of mine, Navy test pilot and Astronaut, with an aeronautical enginering degree was playing the solitaire game spyder. I asked what his winning percentage was and he replied around 60%. I said mine was 98.5%, to which he replied you're not smarter than I am. I said maybe not, but success in the game spyder is independent of intelligence. It is essentially just a maze where persistence not intelligence prevails. Just keep going back and taking a different path until you win.
That's my take on AI, persistence in data accumulation at enormously fast speeds, but that's independent of intelligence.
Despite passing the Turing test, LLMs are presently just sophisticated auto-complete programs. However, it is quite possible that intelligence is an emergent phenomenon of neural net complexity. If LLM programmers begin to create models of self plus memory combined with feedback loops from external sensors, we may see AGI emerge following another 10-100x increase in network parameters, as the models approach the neuron/synaptic count of the human brain.
The pseudointelligence description reminds me of skinwalker legends. Appears human, can even do many of the things humans do, but it's just wearing your friend's skin (and it might kill you if you believe that it's in need of help and approach it in earnest)
You may make the jump from spoon digger to steam shovel operator but hundreds of your fellow spoon diggers will indeed lose their jobs. Note that I am not advocating for maintaining spoon digging as an occupation.
I'm a software engineer too and had the *exact* opinion you do, but three months ago. This summer has been a long overdue realization for me that these models are in fact intelligent (if it walks and barks like a dog...), AGI is here, and everything's about to change way faster than I expected it to.
People are worried about AGI, because the "General" in there means it's a replacement machine. An artificial intelligence generally equivalent to human intelligence.
No, that is not here yet. Which is good. And maybe it's not achievable via the architectures of today's LLMs. Which is irrelevant.
Because a lot of research and funding is going into the replacement machine (AGI), and LLMs are without doubt at least a useful stepping stone.
If you are not worried about AGI, when do you think it's time to start worrying? Once it's there and ready to be deployed with the intellectual power of a billion humans?
Dare I say the freshest, most unique story around is right here!
God has come to Earth; with a podcast, an agenda… and a personality.
The Godcast: A work of speculative fiction that blends spirituality, technology, science, geopolitics and pop culture as humanity confronts its biggest existential threats.
There are some things that I can thankfully still justify my job with. A program manager can certainly feed code into an llm and have it add features and refactor existing code. He'll get into trouble when it goes off the rails and tires to produce solutions that aren't relevant for what you're trying to accomplish. A risk of trusting an ai to refactor is of course that it will chop out lines of code that were actually important while saving things that don't help or are vestigial.
I'm not really worried about writing the thing now as much as writing a thing that can convince me that I'm writing the correct thing. So the unit test layer is increasingly necessary. I also am moving towards having a daemon in your containers monitoring the state and being able to notify the user that things have gone off the rails if a configuration is incorrect. So I'm increasingly black boxing everything I touch.
The other thing I’d point out about “know it when I see it” is that I genuinely struggle to describe Milan’s cathedral in a way to surely dispel any other candidate. But that doesn’t mean I don’t recognize it and don’t confuse other cathedrals for it.
My ability to convert neural connections to words isn’t really the measure of my knowledge even if it’s the only way for someone else to measure it.
The problem with coding agents is orchestrations. So, to really have a comparison, you need to first answer what, say, IntelliJ would look like if you spent $500B and 50M man hours on it.
I’m not entirely convinced it would not just have a button which generated a perfect bespoke e-commerce platform after you put some key info like the project’s name into a text field.
Because that’s the scale of the inputs we’re talking about.
I’m attempting some things I might not have considered otherwise, pre-AI…but I find I’m offloading boring jobs on it (looking for recommendations for apps to do some filmmaking thing, looking up how to use this or that app once I’ve gotten it.). I haven’t seen it perform any flashy feats of insight…maybe I’m afraid to ask, though.=p
> The notion that I will become unemployable immediately after being handed the keys to a steam shovel, having spent decades digging with a spoon, strikes me as faintly ridiculous.
I’m fairly certain that most programmers will end up unemployed. My analogy would be ‘combine harvesters’ where all the people who reap are eventually replaced but one. There might be something to do for programmers for a while, but eventually there will be little left to do.
Take accounting for example. Accounting rules are all in the book which the programmer turns into accounting software, and the accountant does the accounts. Maybe the programmer does their programming quicker because they don’t need a book anymore, but eventually the accountant doesn’t need the software either. Then no one needs the accountant.
I think there’s a year or five where everyone gets more efficient, but then we don’t need them anymore more.
This is finite world thinking. The world of bits is not like the world of atoms, there will always be new frontiers and more to accomplish.
I’ll agree some of the young ones will figure out something new to do. Older ones won’t (like the engineers in the North of England in the 80s). Your steam shovel will immediately put thousands of spoon-diggers out of work. They won’t find a job, and I think AI will stay ahead of the workers.
We've had accounting software for 50 years and it hasn't replaced accountants (my dad had a CPA/MBA project that involved punchcards and mainframes). And accounting, or bookkeeping specifically, is one of the fields theoretically more exposed to automation - it's a large and largely uniform field with verifiable results (do accounts balance or not). Still - we have bookkeepers and accountants!
The times they are a-changing.
“There’s no rational reason I should react so strongly and negatively to LLM-generated prose, but I do.”
Do you not think the uncanny valley is rational?
Definitely not, it's limbic, pre-rational.
Okay, so you're using the word rational in a somewhat negative way then? It seems to imply for you a self-destructive disembodied state that requires one to ignore one's evolved instincts and biology. I suppose that I think it is rational to follow one's instincts and biology. If one were reasoning through what an embodied, evolved entity should do, one would rationally recommend that they default to following their instincts because those instincts will generally see them right on average.
Take a silly example. A couple come to you asking how to rekindle their love for each other. You tell them to have sex more. You tell them it amplifies hormones that will promote their relationship's health. Oh no, they tell you, we are rationalists and you are talking about a pre-rational biological response, we couldn't possibly. We were hoping you would recommend an insightful essay for us to read. You shrug. The couple end up divorcing.
Was it irrational for you to suggest that they take advantage of their biological instincts? I think it was very rational of you to make that suggestion, and they were irrational for ignoring it just because they had an irrational bias against biological mechanisms. Your ancestors have been solving problems for billions of years, and their best heuristics have been passed on to you in your biology.
Same here. It is eminently rational for you, an embodied, evolved being, to give your limbic system a seat at the decision-making table. Not the only seat—it's not infallible—but you needn't be ashamed of listening intently to its voice.
I'm just using the word rational to describe a decision process that uses logic and reasoning, I'm not trying to attach any positive or negative valence to it.
Our instincts can be adaptive and effective or the opposite, but they're never rational by definition.
Agree with you completely. A friend of mine, Navy test pilot and Astronaut, with an aeronautical enginering degree was playing the solitaire game spyder. I asked what his winning percentage was and he replied around 60%. I said mine was 98.5%, to which he replied you're not smarter than I am. I said maybe not, but success in the game spyder is independent of intelligence. It is essentially just a maze where persistence not intelligence prevails. Just keep going back and taking a different path until you win.
That's my take on AI, persistence in data accumulation at enormously fast speeds, but that's independent of intelligence.
Dick Minnis
removingthecataract.substack.com
Despite passing the Turing test, LLMs are presently just sophisticated auto-complete programs. However, it is quite possible that intelligence is an emergent phenomenon of neural net complexity. If LLM programmers begin to create models of self plus memory combined with feedback loops from external sensors, we may see AGI emerge following another 10-100x increase in network parameters, as the models approach the neuron/synaptic count of the human brain.
The pseudointelligence description reminds me of skinwalker legends. Appears human, can even do many of the things humans do, but it's just wearing your friend's skin (and it might kill you if you believe that it's in need of help and approach it in earnest)
You may make the jump from spoon digger to steam shovel operator but hundreds of your fellow spoon diggers will indeed lose their jobs. Note that I am not advocating for maintaining spoon digging as an occupation.
I'm a software engineer too and had the *exact* opinion you do, but three months ago. This summer has been a long overdue realization for me that these models are in fact intelligent (if it walks and barks like a dog...), AGI is here, and everything's about to change way faster than I expected it to.
People are worried about AGI, because the "General" in there means it's a replacement machine. An artificial intelligence generally equivalent to human intelligence.
No, that is not here yet. Which is good. And maybe it's not achievable via the architectures of today's LLMs. Which is irrelevant.
Because a lot of research and funding is going into the replacement machine (AGI), and LLMs are without doubt at least a useful stepping stone.
If you are not worried about AGI, when do you think it's time to start worrying? Once it's there and ready to be deployed with the intellectual power of a billion humans?
Agreed with your assessment of LLM “thinking”, etc.
My argument has long been that we think in sentence-sized pieces, not one-word-at-a-time; and that this is essential to the nature of thought.
Language mirrors thought in many ways, but does not provide a model of it.
Dare I say the freshest, most unique story around is right here!
God has come to Earth; with a podcast, an agenda… and a personality.
The Godcast: A work of speculative fiction that blends spirituality, technology, science, geopolitics and pop culture as humanity confronts its biggest existential threats.
https://godcast.substack.com/p/the-godcast-1-s1-e1-part-1?r=18ajrt&utm_medium=ios
The ARC prize developed by François Chollet addresses the ability for AI to approach AGI.
Maybe coding will turn into AI operatives overseen by a much more experienced manager?
It looks like DeepSeek has recently upset the AI apple cart in terms of cost:
https://arcprize.org/results/deepseek-v4-flash-0731
https://news.ycombinator.com/item?id=49214008
There are some things that I can thankfully still justify my job with. A program manager can certainly feed code into an llm and have it add features and refactor existing code. He'll get into trouble when it goes off the rails and tires to produce solutions that aren't relevant for what you're trying to accomplish. A risk of trusting an ai to refactor is of course that it will chop out lines of code that were actually important while saving things that don't help or are vestigial.
I'm not really worried about writing the thing now as much as writing a thing that can convince me that I'm writing the correct thing. So the unit test layer is increasingly necessary. I also am moving towards having a daemon in your containers monitoring the state and being able to notify the user that things have gone off the rails if a configuration is incorrect. So I'm increasingly black boxing everything I touch.
The other thing I’d point out about “know it when I see it” is that I genuinely struggle to describe Milan’s cathedral in a way to surely dispel any other candidate. But that doesn’t mean I don’t recognize it and don’t confuse other cathedrals for it.
My ability to convert neural connections to words isn’t really the measure of my knowledge even if it’s the only way for someone else to measure it.
The problem with coding agents is orchestrations. So, to really have a comparison, you need to first answer what, say, IntelliJ would look like if you spent $500B and 50M man hours on it.
I’m not entirely convinced it would not just have a button which generated a perfect bespoke e-commerce platform after you put some key info like the project’s name into a text field.
Because that’s the scale of the inputs we’re talking about.
I’m attempting some things I might not have considered otherwise, pre-AI…but I find I’m offloading boring jobs on it (looking for recommendations for apps to do some filmmaking thing, looking up how to use this or that app once I’ve gotten it.). I haven’t seen it perform any flashy feats of insight…maybe I’m afraid to ask, though.=p