AVFTCN 020 - The One Article About AI and ChatGPT You Need To Read
Finally, a good analogy that make sense
It was 4:45am. I’d just let our 15-year-old miniature poodle out in our backyard and put the water for tea on. A few minutes later she was back inside and I was sitting on our couch scanning Mastodon using the Ice Cubes app on my iPhone.
And there it was … a post referencing an article titled:
“ChatGPT Is a Blurry JPEG of the Web”
BOOM! Go read that article by Ted Chiang. Now! Go ahead… this email will still be here when you’re done!
IF YOU HAVE NO IDEA WHY YOU ARE RECEIVING THIS EMAIL, you probably signed up way back at the beginning of this newsletter in the 2000s. Please feel free to just hit the UNSUBSCRIBE link at the bottom of the message! No hard feelings.. I get it. You haven’t heard from me in 3 years. Life changes.
Maybe it’s that I’ve done a lot of work with audio, images, text - and compression of all of those - but this article just resonated with my brain.
The comparison of ChatGPT and other “large language models” (LLMs) to lossy and lossless compression models is not necessarily an exact fit.
But it works. At least for me.
We’ve all seen this. You take a small GIF or JPEG image and try to enlarge it. Everything gets all pixelated and poor quality. As you expand the image, your computer is making up the missing pixels. It’s looking at the surrounding pixels and estimating what might be in between. (Referred to as “interpolation” or “image scaling”.)
You can see the difference, too, if you compare a raster graphic (GIF, JPEG, PNG) such as a photograph to a vector graphic (SVG, EPS, PDF) created in a graphic design program. The raster graphic is a set of pixels capturing the image and gets pixelated as it is scaled up. The vector graphic is a set of geometric shapes and instructions, and can scale indefinitely. Lossy compression versus lossless.
As Ted Chiang writes:
Think of ChatGPT as a blurry jpeg of all the text on the Web. It retains much of the information on the Web, in the same way that a jpeg retains much of the information of a higher-resolution image, but, if you’re looking for an exact sequence of bits, you won’t find it; all you will ever get is an approximation. But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, it’s usually acceptable. You’re still looking at a blurry jpeg, but the blurriness occurs in a way that doesn’t make the picture as a whole look less sharp.
The “blurriness”, or the creation of “compression artifacts”, is a good way of thinking about the times when ChatGPT gets things wrong.
To be fair, ChatGPT is amazing in what it can do. I’ve been amazed at how accurate it can be when I’ve asked some questions. I can see the potential there.
But other times, it is clearly lacking. Back in December on a whim, I asked ChatGPT:
> Tell the biblical Christmas story in iambic pentameter
I was surprised at holes in the resulting story. So I regenerated the response. And regenerated it again, and again, and again. You can read the five different versions it created. As I wrote:
Herod, Egypt, and Nazareth only show up in version 1.
The shepards only appear in 2, 3, and 4.
The wise men are only in 2 and 3.
Gold, frankincense and myrrh only appear in 2.
Angels only appear in 3 and 4.
Version 5 has almost no details but instead focuses on the religious meaning.
None of them mentioned why Joseph and Mary where traveling (to register for a census in his home town for tax purposes for the Roman government).
If you know the Christmas story as written in the gospels of the Christian Bible, you’ll get a sense just from my bullets here that ChatGPT is giving incomplete pictures.
Using Ted Chiang’s analogy, these are demonstrating “compression artifacts”. They are the text equivalent of pixelated images. They are taking text from its LLM and trying to change and scale it to fit the desired output format.
Just to see if the “iambic pentameter” had any effect, I later went back and tried the experiment again with just “Tell the biblical Christmas story”. The results were better. Angels, wise men, and shephards appeared in all of them and there was even the mention of the source being the gospels of Matthew and Luke. One version even included part of the “why” (the census).
But there were definite holes and discrepancies. Some detailed segments of one version were completely left out of other versions. The stories were slighly different.
If you relied on ChatGPT, you would not get the complete story.
You get a pixelated approximation of the story.
Maybe that’s good enough for whatever you are doing. But if people are relying on the “truth” and accuracy of the results of ChatGPT… well…
… all they are getting is a blurry version of the “truth”.
Thanks to Ted Chiang for writing this article. It’s a useful analogy for thinking about where we are - and perhaps where we need to go.
Where To Find Me These Days
I joined Twitter on October 24, 2006. I was user number 10,312. It was an amazing time as we all collectively discovered that we liked this short-form messaging… and we wired everything up to it.
Fast forward to 2023 and, well, I’m not a big fan of the direction the new management has gone - and the (mostly self-induced) chaos and drama is all a bit much.
These days you can find me the most on Mastodon at https://mastodon.social/@danyork (or search in your client for “@danyork@mastodon.social”). I’ve been there since 2016, but with the ongoing #TwitterMigration the “Fediverse” finally has enough people and organizations to be a thriving and vibrant place to be.
I’m not leaving Twitter… yet … but I find I don’t really need to go there any more, and so I don’t. If you haven’t tried you Mastodon yet, you can go to https://joinmastodon.org/ to get started. Feel free to drop me an email if you have questions or would like some tips about getting started.
That’s all folks… maybe it won’t be another three years before you see another newsletter… 🤣
P.S. If I do send out more newsletters, they may start coming from a different service. Automattic recently started promoting a newsletter service based on WordPress, and I may explore that for the future.