Marketing Baby

Growth terminology: What’s a smoke test?

Smoke testing in the context of growth marketing is a way of quickly, and with very limited resources, finding out whether a growth idea is worth executing or not.

If you’re trying to gain traction for your software product, there are a ton of growth experiments you could run. Most of the time, growth marketers have no problem coming up with ideas, the challenge is to find out which idea to execute next.

Typically there’s a reliance on engineering that slows things down as well. “We’d love to execute that idea… but engineering is busy and they don’t have time to do this now.”

A smoke test is a method to validate a growth idea without relying on engineering.

Here’s an excerpt from Lean Startup:

Smoke test with its marketing materials. This is an old direct marketing technique in which customers are given the opportunity to preorder a product that has not yet been built. A smoke test measures only one thing: whether customers are interested in trying a product.

DistroDom (aka Dominic Coryell) released an excellent free course on smoke testing that you can go through in less than 1 hour.

Smoke test process

Dominic shared a 6 step process for smoke testing growth ideas:

  1. What is the hypothesis?
    Keep your hypothesis simple: If we do______(growth idea) then we’ll be able to achieve ______ (result) in ______ (time period) by investing ______ (dollar amount).
  2. What is the minimum proof I need?
  3. How can I get that proof with no coding?
  4. How can I get the right traffic?
  5. How can I measure conversion?
  6. How do I analyze and optimize?

=> What did I learn?

Why is this smoke test process so valuable? Because it enables you to let customer demand decide how to grow your business, rather opinions and guesstimates.


Smoke testing the smoke test course

One of the things I like the most about DistroDoms course is actually that he at the end of the course reveals how this free course is a smoke test for a paid course he’s launching on the same topic.

DistroDom’s hypothesis was:

  1. Hypothesis: If I give free content, people will like it enough and buy the course if they need more advanced courses from deep-dive experts.
  2. Minimum proof: If 15% of the people who see the landing page sign up for the free course, and there’s at least 100 signups for free.
  3. Do it with no coding: For this free course, no coding was required, but it would take DistroDom about 1 week to record all these videos for the free course before getting minimum proof.
    He set the landing page up using InstaPage and Canva to design some of the creative assets on that landing page. He also used emojis from EmojiOne.
    All the signups went into Autopilot using a simple 1-click integration for InstaPage. (He said it’s one of his favorite email automation tools, very low priced compared to comparable alternatives. Plans start at $20/month! He did a video screenshare of the Autopilot interface, and it looks really awesome. I might finally get rid of that old Aweber account I still pay for and move things over…)
  4. Right traffic: DistroDom didn’t have time to implement a referral program. So instead he sent people directly to the paid program landing page to see if anyone would buy. Some people did!
    When people sign up for the free course, he’s running their emails through ClearBit to get some context around who it is that’s signed up.
    In terms of getting early traction, he’s got a nice framework for how to get it:
    Network: Easy (low impact)
    Paid: Medium (medium impact)
    Community (blogs, forums, press, buzz, influencers…): Hard (high impact)
  5. Measure conversion: Since this was just an email signup form, it was super easy. Number of visitors & number of signups.
  6. Analyze & optimize: DistroDom hasn’t done that yet, since this is so early in the process.

With every growth experiment, there’s 3 phases:

  1. Smoke test it
  2. Get distribution
  3. Keep or toss it

Random things:

  • There are several useful templates in there that you can use to turn the theory into action.
  • He uses Jeff Bezos’ / Amazon’s press-release method to “announce” new ideas.
  • He often feeds data into these three tools to use them in different ways:
    Stamplay – This is one I haven’t heard of before. It’s a visual API builder, and according to DistroDom “the most robust way to connect data”. Curious to try this out soon. They have a free plan, and then the paid plans range from $24/month to $499/month.


Leave a Comment