Is your idea worth it?
How to quickly test the market for your solution with a dry wallet test
So, you have an idea for a new product offering. And presumably, you think it’s a good one. Here is a surefire way to prove that you’re right (or find out that you’re horribly wrong).
The “ideation” phase of a project — which happens before anything is designed, tested, or developed, and often before the project is funded — is full of crazy-making abstractions and “what if” questions. In this phase, a new product idea is vulnerable to the rightly-placed skepticism of corporate stakeholders or potential investors who want to know:
- Is this a big enough customer problem?
- Will customers pay to solve it?
- And if so, how much will they pay?
Sometimes designers get uncomfortable talking about money. Discussions about designing for monetization often lead to talk of dark patterns or other forms of user trickery. But determining willingness to purchase is none of those things.
Instead, a willingness-to-purchase test, sometimes referred to as a “dry wallet” test, aims to determine if there is a market for your idea, without using any manipulation or coercive tactics. You are simply posing a straightforward, no-nonsense question to your potential customer: “Do you want to buy this product?” And when it comes to testing new ideas, there really is no better currency than, well, currency.
What is a dry wallet test
According to Learning Loop, A dry wallet test is a simulated “purchase now” experience in the form of a simple e-commerce checkout or a pricing page leading to a “We are not ready yet” page, “out of stock” message, a letter of intent, or a similar elegant way of letting the user finish without actually billing them.
In our case, the simulated “purchase now” experience and consequent let-down message were facilitated face-to-face, over Zoom, with the help of our Sales team. Later on we’ll talk about how to determine whether you should facilitate your fake sale in-person (meaning, over Zoom in the Covid era) or via an online, e-commerce channel.
When to run a dry wallet test
If you’re working on a new product or service, it is best to run a dry wallet test after you’ve done some initial customer interviews and user testing. Customer interviews will tell you if there’s a problem to be solved, a dry wallet test will tell you how big the problem is. The size of the problem will help inform the size of your available market.
Another reason to do user testing before your dry wallet test is that you will have some snazzy, validated mock-ups to include on your landing page when it comes time to market your not-yet-real product.
Who you will need to run it
Running a monetization test is a lot like marketing a real product, only with quite a bit more smoke and mirrors. In order to run a test like this, you will need people with the following skill sets:
- Someone with a vision for the product
- Someone to design and deliver an end-to-end purchase experience
- Someone to analyze the results of the test
- Someone to either develop or facilitate the transaction
This could look like any number of roles within your organization. Look for those folks around you who are good at wearing multiple hats. If you don’t have a Marketing or Finance person at your disposal, don’t be discouraged. You can always do some quick competitor research to ballpark appropriate pricing and messaging. As the product team, it is still your responsibility to determine if you even have a product or not.
This is what our small but mighty team looked like:
- 1 Product Manager
- 1 Product Designer (me!)
- 1 Researcher
- 3 Sales Consultants
In-person sales vs e-commerce
The composition of our team reflects our need to test enterprise software, where customers expect a Sales call or live product demo before making a purchase decision. In our case, the “live product demo” involved some images from Figma in a Google Slide deck. This sort of high-touch transaction won’t work for every customer type, especially if the product being sold is a low-cost commodity.
However, the in-person dry wallet test works great when:
- You are selling enterprise software (or an otherwise expensive product)
- You need to test something fast, and you want to avoid the complexities of billing systems
- Or, you want an artificially high barrier to entry to prove that this is a significant customer pain point (i.e. “10 customer out of 100 bought our productivity app over the phone… in 2020!”)
How to design the experience
If you are planning on conducting an in-person dry wallet test, you will still need to market the product online. Our funnel looked something like this:
It is best to create a single landing page with one main value proposition. It can be tempting to want to A/B test different messages at this point, but remember that there will always be time to refine the messaging once you’ve proven that your idea is viable. For now, focus on brevity and clarity — laser focus on the problem you’re solving.
How to measure success
Measuring success can be tricky in any endeavor, but it is especially tricky with a new product. Our team went through the struggle of how we would define success. We ended up landing on a two-fold definition.
We will be successful if:
- We validate that our product idea interests customers
- We validate that our customers would pay at least N amount for the product
To measure this, we decided to use landing page conversion rate as a proxy for interest (i.e., the number of form submissions / the total unique viewers of the landing page), and we would use purchase rate as a measure of willingness to purchase (i.e., the number of customers who agreed to have their credit card charged / the total number of customers who received a product demo).
As for numerical targets, half of the team wanted to stick to modest numbers that aligned with industry standards, while the other half of the team wanted to set the bar high, aiming to beat those averages by more than 10x. Sticking to historical data might be the safest way to define success in terms of aligning with probable outcomes, but I learned that it’s not the best way to define the success of a new product. We wanted something truly differentiated rather than incremental.
We compromised on a conversion rate goal of 5x the average and kept the purchase rate goal modest. We ended up beating both goals by nearly double.
How to let ’em down easy
One moment of the test that caused my people-pleasing insides to churn was the moment after a closed sale, when our Sales Consultants had to inform our newly acquired customer that the product they had just decided to buy was not available for purchase.
Much like a breakup, it is best to deliver the bad news in person — particularly if you are conducting an in-person test. After they had agreed to have their credit card on file charged, we wanted to let them know quickly and definitively that their card would not in fact be charged because the product was not available for purchase yet.
Now, I won’t promise that your customers won’t get mad. Some will. But I was surprised to find that the overwhelming majority of customers were OK with being told that the product was not available for purchase at this time, as long as we could keep them updated as we progressed in development. We also left our die-hard fans with a parting gift — an interim, prototype solution in the form of a Google Sheet. Whenever you can, always aim to give your customers something of value in exchange for their time.
A dry wallet test is a scrappy way to validate your assumptions up front before getting too far down the road. Plus, with the right team, they’re a lot of fun.
***
Here is a list of tools that helped us design our end-to-end experience:
