arrow_back All issues
how i ai Issue #4 · 6 min read

How I measure the success of AI use cases

I use this framework to pick and measure what use cases to build with AI, and it works every single time

Vikra Vardhan

Life when credits run out

Last weekend, I tried my hand at making my own notebooks.

I realize “I made this” whenever I write in my notebook. Ngl, it’s an awesome feeling. I even made a reel.

Btw, the AI challenge in our community starts today. I am inviting you to the kickoff call. Details in a couple of seconds.

How I measure the success of my AI use cases

When I was drafting this newsletter, my heading was “How I measure the success of my AI experiments.”

Immediately switched it to “AI use cases” because we are not experimenting anymore.

Companies want AI. Founders think AI is cool. As marketers, our current focus is on solid use cases that bring meaningful business results.

So here is my simple framework to measure the success of AI use cases. I call it Level 1-2.

I treat AI like an employee. Every time I assign a task, I also assign KPIs.

With KPIs, my actions, workflows, and agents aren’t optimizing for output but results.

In previous newsletter editions, I shared how I outsourced community management to AI, and how AI takes care of Caffeineletter’s Twitter account end-to-end, and how it even got us a subscriber.

Quick recap:

Agent Name

Use Case

KPIs

Burr

Repurposes our newsletter content into 50+ tweets and posts on Twitter. I only approve tweets, everything else is handled by the agent.

Primarily, I want burr to increase engagement and impressions. As a second-order effect, email subscriptions.

Pingy

Pings me when a community member’s subscription is about to end. Plus sends a newsletter about what’s happening inside the community to our members.

Increase community engagement and member retention.

Coming back to measuring success… I see every AI use case has two levels. Level 1 and Level 2.

  • In Level 1, AI introduces possibilities. Think save time, produce more output, reduce costs, etc.

  • In Level 2, AI gives you results. Think subscribers, paying members to your products, clients, etc.

From my projects:

Project

Level 1

Level 2

Burr

Save time by repurposing and posting tweets that’d take me a few hours. Plus a possibility of a Twitter channel for our newsletter.

Social media engagement, followers, and newsletter subscribers.

Pingy

Creates alerts and eliminates the need to look at the database. Creates and sends newsletter. Saves time and effort.

Increase in community engagement and member retention.

Our job as an AI-first marketer is to always think, “How can I unlock Level 2?”

In fact, every time I start a new use case, I answer: What’s my Level 1? What’s my Level 2?

I might not achieve Level 2 right away, but I will only move forward if the Level 2 unlock helps the biz.

Being aware of Level 2 also answers “What success looks like with this use case?”

This means you’re always moving towards Level 2 with the upgrades you make to your workflows.

AI Challenge starts today

Inside our community, we host an AI challenge where we pick one marketing problem and solve it with AI in 7 days.

We have our kickoff call today. I will share:

  • Two frameworks I use to choose what AI use cases to work on

  • List of use cases if you’re still exploring

  • What we do in the challenge

Even if you are not part of the AI challenge yet, you can still join this call to decide if you want to join.

I hope the framework was useful. See you at the kickoff. Have a great week ahead!

Love,
Vikra