I see people like Demis Hassabis, who for years have been saying that accelerating progress in the world through AI has been his goal. Today, he’s finally able to do that — he got a Nobel Prize for AlphaFold, he’s the CEO of DeepMind, and he’s still in his prime.
What Am I Building Toward?
When I’m in the prime of my career in 10 years, …
(elaborate here…)
def foo():
...Where Things Are Headed
To figure that out, I think I need to extrapolate what people are saying about what’s going to be ready in 10 years.
Automated AI Researchers by 2027
AI 2027 predicts that by the year 2027 we’re going to have automated AI researchers — which is similar to “a country of geniuses in a data center,” which is what Dario Amodei poses in Coming Soon. Honestly, I see where the industry is going, and I see that when we have a verifiable pass — like speeding up an algorithm — we can use AI to explore solution spaces and do better than humans. This has been proved time and time again.
(elaborate here…)
The Alignment Boundary: Where AI Should Stop
But does this extend to non-verifiable domains? I don’t really think so — because this is the essence of the alignment problem.
How do you define something where the value of the function is indescribable? How can you define human values and the things that make humans happy without reward hacking? What’s to stop an AI from becoming Prime Intellect and swapping our reality with something that’s a simulacrum — something that is not real, but only real because we all choose to believe that it’s real? There are people who can decide to put on dopamine grips and live out their entire existence as a zombie. I truly don’t believe that this is an aligned future.
So there is a boundary to what AI should do, and what AI should be, and what role AI should play in our lives. And I don’t know if we will ever define a human reward function — especially when reward signals can sometimes take years. How do we take today’s architecture and apply it to super tasks, like building societies? I don’t see that.
I think that’s something that a lot of people in the industry are glossing over: that there are these bigger tasks with non-verifiable rewards that cannot be replaced.
(elaborate here…)
Doing My Own Thinking
Maybe I need to do some of that thinking for myself.
(elaborate here…)
What We Are Best Equipped To Do
There’s a theme that keeps coming up in my life when I design a different one: promote human connection. Promote the things that we are best equipped to do for each other and for society.
Is that empathy? I don’t know if we are the most empathetic creatures. I’m struggling with that. What is the thing that we are meant to be doing?
What I do know:
- Connection is something that can be deeply fulfilling for humans in our lives
- I believe it’s the responsibility of society to create flourishing for the humans that live in it
- One of the best ways that empathy can be created is through connection and community
So in a way, we can’t have empathy — and we can’t have a functioning society — without community.
(elaborate here…)
What Living Should Be About
I’m at a community spring market right now. Being present and connected. Listening to the sound of waves and wind chimes.
It’s so grounding — this is what living should be about.
Not feeling dread. Not feeling fear instilled by AI companies threatening to put us into a permanent underclass if we don’t figure out how to use AI to its fullest potential.
(elaborate here…)