https://img.inleo.io/DQmXZV1Uquge5XeqdkzgpcxWAsY2jcug2THYe1jJEewjzPT/ai-generated-8705809_1280.webp
I've have been fascinated with artificial intelligence and machine learning for some time now, long before ChatGPT and the likes got popular. The last few years though have been amazing. Each day something new comes out that just pushes us further into the future.
Artificial intelligence is not as smart as you think
While ChatGPT is amazing, it's really just a glorified auto correct engine. Most of what is being released today is a transformer, aka auto correct.
How ChatGPT works at it's core is has been trained on an incredible amount of text and is then asked to predict the next token. A token is how large language work with text, not the typical words or characters we work off. Think of a token as a group of characters that act as a "meaningful unit" of data.
A lot of times this is individual words, but some parts of words are important as well, for example the word unforgivable, "un" has a significant meaning in the word, and contains a lot of data regarding the context of the input. Another example is the opposite, instead of breaking a word into multiple tokens, a token can be multiple words, such as "New York". The meaning of New and York are considerably different than "New York" together.
Even the same words can be different tokens, for example "The" and "the" have very different meanings as one is the beginning of a sentence or in rare cases part of a name, and the other is likely an article.
ANI vs AGI and ASI
You may not have heard of ANI, but you have most likely heard of AGI or it's more common names "General AI" or "Artificial General Intelligence". I always describe the difference like this. If I taught a computer to play chess, and all it can do well is play chess, this is ANI or Narrow Artificial Intelligence. If I can teach a computer to play Chess and it is able to adapt and play Monopoly, this is a general intelligence. Now if I taught it Chess and it is then able to cure cancer, this is ASI or Artificial Super Intelligence.
A lot of people are saying we are close to or have developed Artificial General Intelligence. I see this talked about a lot and I can see why people think this. I don't believe anything I have seen so far is anywhere near close to AGI. To us, sure it looks that way, but what is actually being done behind the scenes is nowhere near AGI.
Large language models (LLM, ChatGPT, etc) as I said are just auto complete engines trained on so much data it almost can feel like it knows everything or at least can glean it from the training data.
Every day I see something amazing with new models, I see something equally stupid. For example, I was playing with Facebook's new Llama 3 model, and asked it a simple question.
While this in itself isn't a big deal to me, as it is likely not much effort is put into training the model with meta data about itself. Further tests are much more interesting.
When I ask the same question three times, I get three completely different answers with very different (and inaccurate) logic. The question I asked is:
If Sally's brother's each have three sisters, and Sally has four brothers, how many sisters does Sally have? Explain step by step.
This is a very popular test for new models, and in fact it is so popular many models are being trained on it specifically and will sometimes regurgitate an answer from the training data rather than "solve it".
The last part of the prompt "Explain step by step." is very important. This serves two purposes, first it allows us to see how it arrived at its answer. It also will frequently help the model provide a more accurate answer. As you can see, it only did the first.
I can provide examples all day where AI has blown me away with it's answer, and a the same time made me laugh my ass off at how stupid it is. Training, fine tuning, prompting, alignment (aka censorship), and system message all play a big part in how well a model works. Each model can respond very differently to the same prompt, the popular model Claude is well known to require a different approach to prompting for good answers.
AI is just a tool, and sometimes it can be a tool. How you wield and use that tool can dramatically determine your results.
Why you should vote me as witness
