Right now, AI is a popular topic, and I mean it. AI has changed everything about our world, mostly for the worse, if we’re being honest. At times, it feels like AI can do everything, if your definition of “everything” is cheating on your math homework.
But do you know how this technology actually works? I mean, you know a computer can’t actually read or write, so how does ChatGPT work?
It all starts with a magical process called “tokenization.” Tokens are small chunks of data that correspond to words that the computer uses to process requests. Then the AI attaches additional data to these tokens, which correlate to a definition, allowing it to “read” your messages. In the case of special characters like a space, the token will correspond to instructions. Tokenization is the process of turning words into tokens that the AI can use.
By converting the words into numbers, the AI can easily see the patterns. For example, the word “thinking” would be two tokens, one for “think” and the other for the “-ing” suffix. Now, let’s take the word “working.” It would have a different token for the “work” part, but the same token for the “-ing” suffix. It also allows it to differentiate between definitions. For example, a word like “lie” with two different definitions gets two different tokens, one for each meaning.
As the AI is fed more training data, it is able to produce tokens faster and better than before. This is what separates the big-ticket AI websites, like ChatGPT, from the smaller stuff. However, at the end of the day, AI is still just predicting what words go next based on probability, which is why it still frequently gets things wrong or makes things up.
No matter how sophisticated AI gets, the basic process of breaking words into smaller chunks will always remain. It seems so simple, you may find yourself thinking, “How didn’t we figure this out earlier?” The simple answer is that we did. We’ve had this process figured out for quite some time, actually.
Enter ELIZA, the first ever chatbot released in 1966. ELIZA was made by Joseph Weizenbaum at MIT University. Joseph introduced it to the world as “a computer program for the study of natural language communication between man and machine” in his paper on the program, and it seems to do just that.
ELIZA was designed to be a therapist, specifically one in the Rogerian school of thought. This mainly entails mirroring the client’s words back at them in the form of questions, which is exactly what ELIZA does. In fact, ELIZA is even simpler than the simplest of AIs today.
ELIZA breaks the sentence down into chunks and scans those chunks for keywords. Those keywords are then ranked according to ELIZA’s code. Once ELIZA has identified the most important keyword, it will follow the transformation rules associated with that keyword to transform the user’s sentence into a question.
If you started to think that the keywords technique could be similar to modern AI’s tokens, you’d be right. Very right, in fact. This general process of breaking user input into chunks, jumbling it up, adding some spice, and spitting it back at the user has been working since 1966, and it’s not leaving any time soon.
Whether you like it or not, AI is here to stay. As AI encroaches even further into our lives – which were functioning just fine without it – it’s more important than ever to have a basic understanding of how it works, no matter if you’re using AI or not.
A web recreation of ELIZA.
