Taylor Sorensen

Taylor Sorensen

research @ humans& PhD from UW

On joining humans&

13 minute read

Published:

Frankly, my joining may have come as a surprise to many people who know me. If you’d asked me a year ago, my $p(\textrm{joining a startup})$ would have been quite low. I really enjoy how high-agency, high-flexibility academic research can be, and love contributing to open science! I derived a lot of satisfaction from seeing my ideas have some small impact on the world, and was leaning towards staying in academia for the foreseeable future.

So why did I join humans&?

Two words: the mission. When I heard the pitch and the goals of the company, I was totally sold. The way I look at the world is often different from many other people in the AI space, and everything they said absolutely resonated with me. I thought to myself: Oh shoot - I HAVE to do this, huh? To me, it seemed like a literal once-in-a-lifetime opportunity to continue many of the threads that I’d been exploring in my PhD and turn it into real-world impact. Not to mention, I’d continue to have high-agency being on a small team and be surrounded by some of my literal research heroes.

Everyone at humans& brings their own flavor on the mission and their own particular experiences / perspectives they care about that drew them to the company. With that, I’d like to share a few beliefs I have that guide my thinking about humans + AI right now. (Also - while my motivations and way of approaching things may be different from and do not necessarily represent the views of my coworkers or my company, these principles were a big part of why I decided to join humans&!)

Claim 1: People are often different from each other. That’s a feature, not a bug, of humanity. Our AI systems should be explicitly built to support that.

We as humans are often so different from one another - we have varied life experiences, varied cultures, varied values, varied religions, varied politics, etc. Difference can often be the source of division - but more often, I believe it is a source of our collective strength:

  • Scientists have different intuitions and explore diverse hypotheses, but through the messy, imperfect, and wonderful process of science we stumble our way closer to the truth.
  • In a democracy, many people have differing goals about what the world should look like and how to get there. I believe democracies thrive when arguments can be freely made, policies can be attempted, and people can freely change their minds under a shared vision of respect for law and for each other.
  • Wisdom of the crowds, whether through prediction markets or Reddit, gives us one of the best tools for calibrating our epistemic uncertainty over possible worlds.

In contrast, most of the ways that we currently train AI systems make the simplifying assumption that there is always a well-specified, correct answer (see: the focus on math+coding over the last couple of years) and are therefore ill-equipped for the messiness of our real-world collective processes.

Instead, we need to build systems which are explicitly designed to be able to ingest and support a range of perspectives, enhancing and augmenting our (sometimes deteriorating) institutions for collective decision-making. For example, properly-trained AI has the potential to act as an uninterested third party to guide a discussion, reach consensus, or bridge divides. AI has the potential to aggregate complex information across thousands of people in ways that would be difficult for any one individual. Used properly, I also think that AI could help us to communicate effectively across divides or even learn more about those who are most different from ourselves. In organizations and communities, AI has the potential to surface expertise and enhance collaboration.

Claim 2: Modern AI is incredibly good at optimization. If we aren’t incredibly careful about what we optimize for, we are likely to be in a bad place. We should optimize for human flourishing.

It turns out that modern LLMs and RL are incredibly good at optimizing whatever objective we throw at them - usually at the detriment of whatever things we care about that we forgot to include in our problem specification (see: the alignment problem; Goodhart’s law). This problem isn’t unique to AI, but also to many of our other systems - optimizing social media for engagement (the economically rational thing to do for an ads-based platform) often leads to polarization or anxiety and just siphons away our valuable time. (According to one study, teens spend an average of 4.8 hours a day on social media.) Additionally, naively optimizing chatbots for human preferences can lead to sycophancy or “AI psychosis”.

Instead of optimizing for myopic, hackable objectives such as preferences after a single chat turn or time spent on a service, we need to optimize our AI systems on longer-term, more people-centric metrics that we would actually choose upon reflection. Does an AI system actually help us to achieve our stated goals? Do we feel good after interacting with a system for a long time? Or days later? Does our technology help us to have meaningful connections with other people? It is much harder to optimize AI systems for these objectives - but not impossible (see Full Stack Alignment for one compelling vision!). We live with the technology we choose to create, and I would like to live in a world where we choose to optimize AI for helping humans live good lives.

Claim 3: AI systems should enhance and augment human agency whenever possible, instead of displacing it.

The natural conclusion to the current direction of research are AI systems that can do tasks for longer and longer without any human supervision (see METR evaluations).

This may work fine and even be desirable for tasks that are perfectly well-specified or quantifiable. However, sometimes we don’t know what we actually want going into a project - it requires iteration, and thinking carefully about what the right thing to do is. Additionally, I believe that people get incredible meaning from having agency in their lives, and I think a world where people are in control and have agency for their lives is a far better one to live in.

I think this point is related to AI slop - content generated with little thought or effort on the part of the creator is unlikely to be meaningful or enjoyable for people. There’s a very big difference in my head between a 2,000 word blog post that was generated from a 2 sentence prompt, and a 2,000 word blog post that was generated based on a 20 minute stream-of-consciousness voice recording. AI can do amazing things to synthesize, transform, and translate our thoughts - but I believe something important is lost when we outsource our thinking itself.

In addition, I also think that LLMs are actually an incredibly under-explored amazing technology for increasing our agency! If properly tuned, they could serve as the ultimate socratic questioners, helping us to explore our beliefs and strengthen our reasoning. Instead of training our chatbots to churn out perfect 500-word high school essays, we could train LLMs to be powerful, ever-patient tutors that know exactly which concepts are just in reach for a student and help guide them to actual understanding.

Claim 4: Creativity often comes from people’s unique contexts and approaches. We as a society will be more creative by combining people’s strengths and contexts with AI’s strengths.

This is the claim that I’m least confident about - but I think there’s something to it. I believe that many forms of creativity are probably some mix of high-entropy exploration and making new connections between preexisting concepts. Claude Shannon, the father of information theory, attributed the inspiration for much of his research to the mere fact that he was exposed to both symbolic logic and electrical engineering: “It just happened that no one else was familiar with both those fields at the same time”. This resonates with my lived experience - my most successful research didn’t require many grand intellectual leaps, but mostly came from my having thought a lot about problems in two very different areas and combining them in new ways. I believe that this phenomenon is part of the reason there are so many simultaneous discoveries in science - the requisite contextual “building blocks” are often sitting there already, and it’s just up to somebody to have the context and the entropy (for lack of a better word) to put them together. (see: independent discovery of Calculus by Newton and Leibniz, the theory of evolution of species by Darwin and Wallace, etc.)

In some sense, LLMs seem perfectly suited for creativity under this theory - they’ve been exposed to more context than any person, essentially the entire body of human knowledge (the internet)! For some reason though, the “country of geniuses in a datacenter” has yet to manifest. Maybe this will be fixed in one more generation of scaling up compute and data 10x. But to me it seems that something is still missing from the current recipe. It may be that our RL training is overly mode-seeking, and we stamp out enough entropy to squash any huge leaps in scientific discovery. It may be that training on all of the internet is insufficient, and it’s essential to have only the correct two requisite building blocks in context at a time to make the creative leap. It may be that pretraining fits too strongly to the existing distribution of human knowledge.

In contrast, each person has their own life history and context, and humans have been nothing if not creative through the millennia. For now, I expect that the most creative leaps will still involve humans in some way. That being said, I do expect AI to be increasingly useful in creative endeavors - as a brainstorming partner, an extension of knowledge, a quick executor of ideas, and more! (That is, as long as we avoid the risk of homogenizing everyone’s context by everybody using the same AI systems in similar ways.) I’m delighted by music and art and scientific leaps, and want to see more of them, and believe that AI (deliberately built) can help to enable all of those things.

So, with all of that running through my head, I joined humans& in November. It’s been an absolute whirlwind so far, but so delightful!

Some initial observations from these first couple of months:

  • One thing I really appreciate about the mission and the people is that it’s so positive! It’s so easy in AI to focus on building without thinking about the consequences, or to focus mainly on forestalling the worst possible consequences. While I’m so glad that there are many great people working on reducing the risks of the worst harms from AI, I find it so invigorating to be working towards a positive, affirmative vision for AI and humanity.
  • A great mission attracts great people! My colleagues at humans& are some of the finest people I’ve had the pleasure to work with, including old research heroes of mine, old friends of mine, and some newly made friends. They are hard-working, kind, fun, and just a joy to be around.
  • The mission is incredibly ambitious and far-reaching - we basically want to alter the momentum of the entire field + change the world for the better! Who knows where things will take us, but I’ve always been a fan of swinging big and taking big risks for things you believe in. Cautiously optimistic for the future!
  • So much of my PhD research was a combination of seemingly disparate things: the human and the artificial, the philosophical and the concrete, the normative and the technical. I’ve found the tension between these spaces to be a really interesting and productive space to be in, and one that is often underexplored. Humans& is pretty unique because basically everything we do exists between these spaces. It requires dreaming while being practical, and doing fundamental research while designing product. Continuing to exist in this space is thrilling to me.
    • Relatedly, I think to make progress on this mission you need people who understand LLMs deeply, as well as have thought about the human-side of things a lot. This is a relatively small group in the AI community, but we have a very high density of those folks here, which I love.
  • It’s a small team and a very high-agency place. I’ve quite enjoyed being here early and getting to see and be a part of things from the ground up.
  • The research problems and technical details are SO interesting.
  • Hopefully (and most importantly), if we do our mission right, this will hopefully have a positive impact for the world!

If this all resonates with you, please stay in touch! Go humans!

Also, you should check out these great posts by some of my colleagues!