When AI becomes a goal in and of itself
As I’ve been studying AI systems and writing posts about them, it has occurred to me that we may be creating more of a dependence on AI and LLMs than we may actually find desirable in the future. AI shows up everywhere from the search results Google gives you to the answering machine assistant I ran into when applying for a job recently. Companies like Microsoft, Amazon, and Google are racing to build huge datacenters to serve to the sudden demand for AI solutions. Elon Musk is even suggesting we put AI compute in space.
With the advent of digital smartphones and personal computers which allow access at a moments notice to any social media app you can think of, we already know that overdependence on technology can be a mistake. Social media addiction is no joke. And while AI dependence doesn’t look exactly the same, it has obvious and more subtle similarities.
I have found that while writing blog posts I will often turn to my trusty Claude assistant for advice and information such as fact checking my writing. Which is fine, but there is also the temptation to just let Claude do the writing for you, which is a first-class mistake when trying to teach yourself how to write. Because of my dependence on AI as a writing companion, I may be missing out on valuable learning opportunities. The process of making mistakes helps you to learn.
An interesting thought
If we can become over dependent on AI as a solution, we can just as easily become blind to its pitfalls – and the pitfalls may be bigger than most people realize. Consider this: what if someone along the way happened to discover a new algorithm that enabled AI models to run on a memory footprint that was a factor of ten times smaller than what’s required today?
This is actually a possible situation, theoretically speaking, because many current LLMs are trained on less data than their size could optimally use – meaning the number of parameters is much larger than is strictly necessary for the given dataset 1. The actual reason current LLMs are created this way is because it has been found that simply increasing the model size helps the model perform better on the same data. There seems to be a definite relationship between model size and response quality.
Which raises the question: why? If the reason behind the performance advantage could be fully understood, we may be able to reverse engineer the mechanism and invent an algorithm that uses smaller models with the same performance per training data. If this scenario actually happens, we will find the crazy datacenter buildout may have been a magnitude of ten too ambitious.
This simply brings up the thought that maybe we’re trying too hard to build something that won’t even be necessary in the future. Now if the big tech companies do discover a new algorithm like this, they may be able to take advantage of their buildout to simply run bigger, better models, and maybe recoup their investment partially. But they would have still invested billions of dollars into something that they wouldn’t even have a sure market for.
If the biggest players in tech can get carried away chasing the next big thing, it’s worth asking whether you might be doing the same thing — just on a smaller and less obvious scale. Technology can be genuinely empowering, but it can also get away from you — social media being the obvious example.
Nature vs the Machine
Recently my family was able to go on a camping trip in a nice spot and enjoy nature in a way that the smartphone in your pocket just can’t do. We got rained on a little, but stayed dry, and it was a great trip. I even got some great photos of the kids, using my 35mm film camera.
But my point now is, technology is great for solving problems that cannot usually be solved any other way. Email is one of my favorite technologies, and helps me to stay up to date with important things all the time. But sometimes technology seems to become an end in itself, which I believe is a fatal mistake. I’m always happy to go outside and view the sunset or enjoy the breeze on my skin. Nature does something so well that technology cannot replicate very well at all, in terms of a sensory experience.
And this is my fear about AI dominance in our current world. I love using AI as a coding and writing companion and for looking up things I used to Google (I do miss Google a bit), but if a technology becomes an end in and of itself and takes over other parts of your life that have greater value, you have a problem. While it would be great to see a little more patience on the part of tech companies big and small, it may ultimately come down to a choice for the end user – you and me. The insatiable drive for more in this world is truly amazing to me sometimes.
Flip Phones
I recently was inspired by my brother and another friend of mine who both dumped their smartphones for a flip phone (to different degrees). It’s actually an idea I have had myself before two years ago. And so just a couple months back, I ditched my smartphone and got a handy little flip phone for a very good price, and I have been enjoying the change very much.
I had already had very good self-control with using my smartphone, but if you’re going to ignore the twitter app on your phone, why do you have the thing in the first place? So I went old-school in a way, with no regrets so far, amazingly enough. I love my little flip phone.
I don’t think this solution is right for everybody, and I know some people won’t understand the idea at all even, but the simpler life has an attraction sometimes. My only advice for you is that you use wisdom and intelligence when dealing with AI in your life. Sometimes it’s better to move slow.
Notes:
1 For a deeper exploration of balanced scaling, see DeepMind’s 2022 Chinchilla paper. They demonstrated that a 70B model trained on 1.4 trillion tokens outperformed significantly larger models (like Gopher) on the same compute budget, while being much cheaper to serve and easier to deploy. Training Compute-Optimal Large Language Models (PDF)
Timothy Grindall is a technology writer and self-taught computer programmer, photographer, filmmaker, thinker, and writer who wants to see the world change for the better.
Leave a Reply