Some thoughts from inside the AI bubble

Useful? Yes. Sustainable? Unclear.

AI can be useful: that much has been clear for a while. After my employer made the choice to “lean in” on AI usage, we’ve now seen a lot more of the exact ways it can be useful, particularly for software development. At a pure technology level, it’s clear that many forms of software development can be transformed using AI. At work, I have now seen examples of software developers doing very impressive things. They’ve taken on various tasks whose scale, scope, or tedium mean that they could never have been accomplished without the use of AI, or at least not in the time frame that was required.

At the same time, we’ve also learned a lot about the fact that AI is not magic. Or rather, it’s pretty much the same kind of magic that “old-fashioned” software is. One of the well-known characteristics of software is that you have to cast the spell exactly right, or else things don’t go well. As a follow-up to my previous item about the AI bubble, here are some more recent thoughts about AI-related economics, employment, and expertise.

Economics

What’s still unclear is how much difference AI makes economically. The ability to achieve remarkable speedups of certain parts of the software development process will not be interesting if that speedup takes 10 to 100 times the amount of money it takes to do it without AI. Or, rather, we will organize ourselves quite differently if AI ultimately proves to be a super-expensive way of making certain tasks happen super-fast.

Right now, the implicit assumption is that AI will be roughly the same cost but faster, or much faster and only slightly more expensive. For the AI-consuming organizations, right now, that assumption is roughly true. However, the organizations that are providing the AI are losing tremendous amounts of money, at a level that’s clearly not sustainable.

Some people make casual references to other famously money-losing businesses that ultimately turned profitable, like Amazon or Uber. Unfortunately, I think that those comparisons are not well thought out. When I read more careful analyses, it seems as though the AI companies are both profoundly unprofitable and have no plausible route to profitability.

It’s also possible to compare AI to the air travel industry, which has collectively been a money-losing proposition for most of its existence. Warren Buffett made the comment that, “if a farsighted capitalist had been present at Kitty Hawk, he would have done his successors a huge favor by shooting Orville down.” Notwithstanding the competitive reasons that the airline industry loses money, it does seem clear that individual airlines have been profitable intermittently. Except for Nvidia, no-one is currently making any money on AI.

The revenue numbers for AI companies are impressive in isolation, both for their absolute size and their spectacular growth rates. But when compared to the amount that AI companies are spending, those revenue figures are tiny. This leads to a strange business problem: the immediate need for funding is so urgent, and the management of growth is so critical, that those extremely short-term concerns completely dominate. Who has time to be concerned about a path to profitability when you need to raise billions of dollars somehow just to keep going? If the house is on fire, who can afford to spend time worrying about whether it’s also perched on a cliff being eaten away by erosion? And yet, ultimately, AI must be economically sensible… or the money will run out.

Employment

People worry a lot about losing their jobs to AI, and we seem to be seeing that happening… but not, perhaps, in the way that people might have expected. Few, if any, job losses take the form of someone being let go because “AI can do their job.” Instead, it seems that there are two similar phenomena happening, both of which relate more to economic flexibility and ordinary business behavior than to straight substitution of AI for humans.

The first phenomenon is that hiring, particularly of junior personnel, is slowed down or paused entirely. This is not strange in itself: it’s how businesses often respond in a time of increased uncertainty and volatility. Hiring in general, and of young employees in particular, requires a degree of optimism about business conditions. When management teams genuinely don’t know how the AI transition will affect their business, it’s natural to exercise more caution in hiring.

The second phenomenon is that employees who are judged to be of relatively low value are laid off to free up money for some combination of buying AI services and hiring people with more relevant skills. This is also not strange in itself: it’s how businesses respond to broad shifts in technology.

Of course, if you are a person who’s laid off, it’s somewhat academic as to whether it was because AI could do your job or because your salary was needed to pay for AI. It hardly makes any difference. Nevertheless, it’s worth underscoring that human expertise is still crucial for the nontrivial deployment of AI.

Expertise

If you just want the kind of AI summary that now happens when you ask for a Google search result, that’s not very hard to do and it’s correspondingly not very valuable. In contrast, if you want a nontrivial reworking of a large body of software to achieve some kind of regulatory compliance, that turns out to require considerable human expertise: both for getting AI to do the task, and for validating that the task was done correctly.

As I’ve previously noted, the core behavior of large language models (LLMs) is to hallucinate something plausible. The concise summary (both technically and colloquially) is to say that LLMs are bullshitters. People in the AI business obscure this reality by only labeling erroneous outputs as hallucinations, but there is no meaningful difference between the way in which “good” answers are generated and the way in which hallucinations are generated. We, the human users, make that distinction based on our knowledge of reality and of the task at hand.

Unfortunately, this structure means that an AI is not very good at identifying situations in which it is failing. It is possible to check one LLM’s answers by using an entirely different LLM, but that’s both expensive and still may not hit the level of reliability required. Instead, humans are needed to do the checking, and those humans need to be able to distinguish between what’s merely plausible and what’s correct. (There can also be subtler issues, like a tendency for LLMs to say more than is really needed.) Although LLMs have improved so that cringingly bad mistakes are less common, that’s a double-edged sword: a subtle error, or an error in a detail, may be enough to cause a serious problem but may well be hard to detect.

Accordingly, there is more of a role for a certain kind of senior-level skeptical expertise in the presence of AI, simply because AI is potentially constructing so much more stuff per unit time than was true of human developers with the same task. Our software development organizations are shifting over time to have many fewer junior programmers cranking out code while still having the same number (or perhaps more) senior programmers to supervise and review what’s produced.

In the somewhat longer term, it’s not clear how we will grow and maintain this store of senior-level expertise. Historically, senior programmers worked their way up to it over the course of a career, starting with being junior programmers. If the reduced level of hiring of junior programmers is a permanent feature, it’s not clear where future senior programmers with the right kinds of expertise come from.