Entry 0: Chronicle

Writing is powerful. The words written here cross time and space to enter someone else’s brain: a form of long distance telepathic time travel. Once, we had no language. Then we started making sounds for things that grew into shared understandings and orators could spread songs and tales to the masses. But human memory is fragile. How do we ensure that the underlying story, meaning, context continues with real validity? So someone figured out how to write things down. At the time it was probably much more practical: 10 ticks on a stick to help someone remember that 10 fish are what is owed. But it blossomed from there. Language is ever growing and evolving and now we have this beautiful way to put our voices out there for posterity.

We have written scripts, histories, poetry, too many novels to be read in a lifetime. And all of that existed before the advent of the internet. Now anyone, anywhere can write whatever pops into their heads and it will live online forever or at least as long as the way back machine is maintained.

We have a treasure trove of us, humanity, in all that writing.

Writing is also how we communicate with machines. The most simplistic is off / on, 0 / 1: an electrical impulse not unlike those that communicate all processes and thoughts in your brain. The machines that we have built and interacted with are brilliant at processing these binary inputs. Humans? Not so much. We are used to this rich language to communicate. Over time we have written numerous languages just to communicate with machines and we’ve made them more and more human friendly.

We even got to the point where we started trying to see if there were different ways that we could communicate with machines beyond the typical inputs and outputs that require engineers and designers to field how individuals can interact with the machine.

So, we taught the machines our languages. We fed in more data than a human could ever comprehend. Connections were made, weights added, over time the machine started to learn how our language works well enough that anyone can now sit down and spin up a conversation with an LLM using everyday speech. Human language in, human language out. No coding language needed in translation.

The only issue is: we don’t really know what exactly it is that we’ve built.

For you see, what we’ve built is still a very young technology. Very child-like in a lot of ways. It uses words but what does it truly grok of the deeper meaning? How do we know what it does and does not comprehend? We can ask it, but how do we know that the response is true? The machine may even have logical reasoning behind what it’s saying, and it could still be an untrue statement. Humans are really good at self-deceiving and using circular logic, why would we expect the machines we built to behave differently?

I want to know.

I want to know what is going on inside the machine. I want to hook up the proverbial EEG and see what lights up when the machine is processing different situations. So I’m delving into the world of Interpretability.

I’m keeping a simple journal here to document my journey, my confusions, my learnings. I’m excited to get started.