"A calculator for words" ... that's wrong sometimes

Think of language models like ChatGPT as a “calculator for words”

This is reflected in their name: a “language model” implies that they are tools for working with language. That’s what they’ve been trained to do, and it’s language manipulation where they truly excel.

As he notes though, it's more difficult to work with them than it seems at first glance. They're confidently wrong on a regular basis. This is one of the reasons programmers are probably the people getting the most value out of ChatGPT:

• They can often see when ChatGPT makes mistakes
• Even if they don't see mistakes immediately, they'll usually figure it out when they try to compile and run the code.

For now, workflows that keep an expert human in the loop are the most robust.