I’m generally quite level-headed when it comes to the future of AI. But you know what? I want to lean into the sometimes-frightening-but-ultimately-uplifting magic of it all for a moment.

Last September, Aaron Levie shared the following on X: “AI coding agents offer the best glimpse into what the future of agents will look like in many other fields of knowledge work.” As the CEO of the software company Box, he has had a front-row seat to watch how AI is disrupting and enabling his industry. And he offers a positive spin: we are slowly building the capabilities to invite other industries to the party. And having people from other industries run experiments born out of their subject matter expertise and enabled by software is a massively good thing.

Since September, new models and features have proven Aaron’s assumption right, at least in terms of building capabilities and connecting industries other than the software and the tech industries to this immense processing power. In addition to the chat interfaces that introduced most people to AI, tools like Claude CoWork and ChatGPT Canvas lowered the barrier for non-engineers by bringing the power of those frontier models into more familiar interfaces to work with and output different file types.

But I would love to know: what experiments are you running?

The Real Differentiator

I’ll spare you the exposition: Silicon Valley doesn’t exactly have a perfect track record when it comes to running experiments that make the world a better place. That said, Silicon Valley has proven that experimentation drives innovation. For instance, when AWS launched 20 years ago, 12,000 developers signed up on the first day, ushering in an era of small teams competing with large incumbent companies due to the simple fact that those teams could scale their software at the speed of pushing a button. Now that AI capabilities are becoming more accessible, the hypothesis-driven mindset is producing the next round of real breakthroughs. This mindset works.

What went wrong - and continues to go wrong - were the constraints. The venture capital model creates pressure to optimize for scale and extraction. Even when VCs invest in companies with product-market fit, the portfolio approach pushes them toward “will this scale to a billion users?” This produces experiments like Juicero: a $120 million WiFi-connected juice press optimized for impressing investors rather than serving a real need. Lasted 16 months. (NOTE: Give “The Power Law” by Sebastian Mallaby a read if you are interested in the history of venture capital as a driver of growth and innovation, really great read)

Organizations grounded in subject matter expertise have different constraints: service and impact. Will this serve the people who need it? Will this create the outcomes we’re trying to achieve? These aren’t limitations. These are structurally better constraints. And the exciting part? Your ability to use software to deliver impact is getting closer and closer to what the tech industry has traditionally exclusively had access to.

Experimenting with Purpose-Driven Constraints

I love helping mission-driven organizations design better experiments to find ways to deliver impact through software. Whether it was guiding product management at Driver’s Seat Cooperative to help gig workers make the most of their time using crowd-sourced data, or connecting curiosities to requirements for a team at ChildFund International building educational tools for rural communities, I have experienced what it’s like to lean into purpose-driven constraints.

As generative AI capabilities continue to expand, organizations have the chance to push their assumptions in ways they didn’t before, and I’m not sure anyone has quite figured out how things will play out. We are all, after all, still experimenting! But there are a few things I do know that will help make use of this emerging technology:

Start with what you need to learn

Most experiments fail because they optimize for “did we ship?” rather than “did we learn?” Before you prototype anything, get clear on what you don’t know. What assumption are you testing? What would change your approach if you discovered something different?

This is where I think exploring AI can really help. If your communities are asking for help, like families wanting more access to after-school resources for their children, AI tooling makes it easier than ever to test different approaches to solving that problem in the time it used to take to research a development firm that might be able to one day build something for you. A non-engineer can now prompt into existence chatbots, text notification systems, and searchable directories, but only if you’re clear on what you’re trying to learn.

Design friction intentionally

When you’re constrained by growth metrics, friction becomes the enemy. Remove all obstacles to engagement. Make everything frictionless. But when you’re constrained by serving people well, friction can be productive, because people are messy. And an obsession to make everything work on the first try doesn’t always meet people where they are.

With AI tooling in the hands of leaders who understand that messiness, they can now quickly experiment on solving the small chunks of larger problems, like testing how to verify housing eligibility without making people feel surveilled, or trying different approaches to onboarding that work for varying literacy levels.

Sometimes doing something quickly isn’t the goal. Sometimes doing something the right way is. This is bounded creativity. Venture-backed companies experiment within constraints set by scale and extraction. Mission-driven organizations experiment within constraints set by service and impact. And constraints set by impact tend to produce experiments that serve people rather than extract from them.

Measure success by impact, not scale

Scale-focused companies ask “will this scale?” Mission-driven leaders get to ask “will this create the outcomes we’re seeking?” Sometimes that means building something that deliberately doesn’t automate everything. Sometimes it means serving 1,000 people well instead of 100,000 people poorly.

This is where your constraints become your advantage. You can experiment on whether a solution creates meaningful change for a specific community, rather than whether it can grow to a billion users. Those are fundamentally different experiments with fundamentally different measures of success.

What it all means

When mission-driven organizations can experiment as quickly as profit-driven companies, different questions become askable. Not “how do we scale this to millions?” but “what would serve these 1,000 people better?” Not “how do we optimize delivery efficiency?” but “how do we preserve dignity?” These aren’t just different questions. They lead to different experiments, different products, different outcomes. The tech industry will keep optimizing for scale and extraction. You get to optimize for something better.

Before You Experiment

AI gave us all access to the same “magic wand.” The difference isn’t the tools. It’s the constraints that shape how you use them. So before you start your next experiment, or embark on any learning journey with AI, ask yourself three questions:

• What do I need to learn in order to make progress? Not what do I need to build. What do I need to understand that I don’t understand now? What assumption am I testing?
• Who do I need to learn from? Not who can build this. Who lives with this problem every day? Whose experience would change how I approach this?
• What do I need to see or observe that confirms I’m making progress? Not how many users signed up. What would meaningful change actually look like for the people you’re trying to serve?

The people closest to real problems have always had the expertise. Now you have tools that caught up to it.

So what are you doing with your new magic wand?