I tried running a design sprint by myself… Here’s what worked, and what didn’t

What happened

I was in Gordon Ramsay’s kitchen and he was telling me how to cook a chicken supreme with roasted vegetables (supreme is a fancy way to say breast…I didn’t know either).

He told me to turn on the oven and salt the chicken breast.

“What temperature?” I asked…

He kept babbling on about the chicken, like he didn’t hear my question…

Because he didn’t. He was on an app on my phone. The Masterclass app, more specifically, which I had been using to learn how to improve my cooking skills during this quarantine.

1. I paused the video.
2. Read the description. No oven temperature there.
3. I looked around the lesson page, nothing.
4. I went to exit the lesson, it took me all the way back to the Masterclass home page. Agh!
5. I searched Ramsay, clicked on the course. It was the wrong course.
6. I searched Ramsay again, clicked on the right course.
7. I found the pdf workbook. Downloaded it.
8. Scrolled to page 16. 425 Degrees.

This has happened several times since then, often when I’m in the middle of a recipe when a 5 minute wander through the interface means the difference between a perfectly cooked or burnt ribeye.

I love the content in Masterclass, I recommend it to all my friends.

I don’t love certain aspects of the interface.

Sprint

Around this time I was reading Sprint by Jake Knapp. In it he defines his process of walking teams through, defining, and solving big problems in just five days.

What if I attempted to run a sprint by myself in order to define and solve the problems I was experiencing with the Masterclass interface?

Doubts

Sprints are made for teams. They are designed to pull the knowledge out of all the stakeholders involved. Would it work if I did one by myself?

I’m also a new designer. While I’ve had 2 week deadlines for projects where I’ve had to sprint, I’ve never done a sprint in the exact method proposed by the book. Could I facilitate a sprint with no experience?

Finally, assuming that I can do it myself with no experience, is 5 days enough time?

Pre Sprint

Attempt 1: impersonate stakeholders

How I felt impersonating a CEO.

This failed. Don’t do this.

I thought about who would be in the room during a feature design sprint at the Masterclass headquarters.

Maybe the CEO/Visionary, the head of product, a customer service expert, an engineer, Marketing director?

Then I thought about what these people would want/push for in a design sprint. I even went as far as researching who was actually in these roles on LinkedIn and seeing if they had written any articles about their experience.

This is not only creepy, it is also ineffective.

I am highly biased, and I can’t pretend to see things through the eyes and experiences of other people. I’m a former teacher, so for me the learning experience is super important and I will push for it until I am blue in the face. This might not be the priority of a marketing director.

Attempt 2: Study the product, Study the company

This worked much better.

I scoured the internet for every relevant article I could find on the company.

I needed to know the: vision, mission, plans for the future, what features they tried in the past, what worked, what didn’t, and most importantly, what the customers were saying.

This took a week…. So much for a 5 day sprint.

Starting the Sprint

Setting up the office

This took all morning on Monday much longer than I planned for and not a very solid start to the sprint.

I did not have a whiteboard in my home office (I have since painted one on the closet), so I resorted to printer paper, painters tape, and sticky notes… Lots and lots of sticky notes.

My computer also has a complicated relationship with my printer so after an hour and a half reading software docs on my model of Canon Inkjet I decided to move on with handwriting.

Score: Sprint: 1 Jesse: 0.

Monday (afternoon)

Here is a link to the official sprint process outlined in the book that I will be referring to throughout this article.

Since I didn’t need to introduce myself to myself or explain what a sprint was, I started with setting a long term goal and listing sprint questions.

Long Term Goal

Time Suggested: 45 min

Took Me: 2 hours

Crap! What’s my long term goal?! How do I make a long term goal? This is so much pressure in the first step of the sprint!

I reversed it, and started with questions.

How could I fail?

Easily. I have no idea what I’m doing. I’m the captain of the boat and I don’t know where it’s going. How could I not fail?

I took a deep breath, and went back to the goal.

What is the vision of the company? What does their future look like? I had the answer to those questions in the research.

Masterclass wants to be the bridge between education and entertainment. They want their classes to be timeless, with content that can be relevant for the next 100 years. They want to produce high quality content featuring masters who are the best in the world at what they do. The future of the platform may include more languages, short form classes, and augmented reality.

Equally as important, what is Masterclass not trying to do? It is not trying to be a how to guide, or a video tutorial.

“If you want to learn how to use your DSLR camera, this is not the place for you,” MasterClass co-founder and CEO David Rogier tells me. “That’s not what you should learn from Annie Leibovitz.”

This focus on quality, engagement and timelessness allowed me to narrow in on a goal.

The courses are great and are currently the focus of the platform. I didn’t want to change that. However, right now, the user’s ability to engage in the courses through the mobile platform are limited.

My goal was to allow for more user engagement and satisfaction. I wanted the platform to be as easy to use as possible and I wanted users to be so engaged in the classes that they recommended Masterclass to their friends.

Goal:

The quality of our platform matches the quality of our classes.

Questions

Time Suggested: 30 min

Took Me: 30 min

I approached the questions using the guide provided in the sprint book and a couple pages of notebook paper. The questions they recommend are:

1. What questions do we hope to answer by the end of the sprint

2. To meet our long term goal, what has to be true?

3. If our project fails, what might have caused that?

Another question I added is: What assumptions am I making about users/the platform/ in general?

For each of these four questions I took a piece of notebook paper and wrote down all the questions or statements I could think of related to assumptions, questions to have answered, etc.

A picture of my “whiteboard” at the end of the day on Monday

Then I circled the questions I felt were the most important, edited and rewrote them. I landed on:

1. Do enrichment materials increase user satisfaction?
2. How do users want to engage in classes?
3. How do we make it easy for users to engage?

Map

Time Suggested: 1 hr 30 min

Took Me: 15 minutes

The map, which was supposed to take an hour and a half took 15 minutes… max.

I had a pretty good idea based on my research, customer reviews, and my own experience that the focus should probably be right after the customer started a course.

Tuesday (Day 7 of my 5 Day Sprint)

Tuesday morning I was coming from behind but feeling excited about the progress I was making. Even though I fumbled through Monday making a ton of mistakes, I was enjoying the process.

Expert Sessions and HMW Notes

Time Suggested: 2 hours

Took Me: 2 hours

I was the only expert in the room, and I was far from an expert. So my experts were the users.

I went through the user review notes I compiled from the week before. I spent 10 hours going through user review notes from 8 different websites, the most helpful of which was Trustpiolit.com.

From these I created How Might We notes based on the negative and/or constructive feedback.

The sprint was focused on user engagement so I also had to understand how Masterclass was attempting to engage users currently.

I knew that users were finishing the courses at 5–8 times higher than the average MOOC completion rate of 4%. What I didn’t know was how often users were engaging with the enrichment materials and how they were using them.

Access to this information would have improved the final prototype, and in hindsight I could have done user interviews or a contextual inquiry looking into how users engage with enrichment materials. But I didn’t think of it at the time… and frankly, there’s not enough time in a sprint to run two sets of user testing.

That might be one drawback of the sprint process or something that companies need to keep in mind when choosing who to include in the room. I know the book mentions having a customer expert, but I don’t think they harp on it enough.

In my opinion, access to in depth user research and someone who can explain the findings is critical in running a successful sprint.

In order to understand how Masterclass was attempting to solve the question of user engagement I researched the workbooks for six of the classes and read through the community forum.

From these I took HMW notes in the areas I felt could be improved, remembering the feedback I learned from users the previous day.

Organize and Vote (where I cheated)

Time Suggested: 30 min

Took Me: 40 min

I think this part of the sprint is very important to do with another person. So I grabbed my roommate, explained the platform, vision, project, and goal, and we started organizing the notes into categories.

We ended up with five categories: customer retention, forum, app function, content, and activities.

Next we did a silent sticky vote.

In a normal sprint, the whole team goes around with two sticky notes each (four stickies for the decision maker) and they vote by placing stickies on their preferred HMW notes.

Our process was different. A normal sprint has 5–7 people and a decision maker each with two to four votes. Since there were only two of us, I gave my roommate 5 votes and myself 7. From there I took off all the stickies with no votes.

This was my favorite part of the sprint. Probably because of how collaborative it was. But it was also really efficient. Coming to a decision is usually a lengthy process, and I often worry that my biases push me to focus on solutions that make sense for me, but not for my users. Sticky voting allowed me to check my bias against that of another person and the categorization and targeting took half the time it’s taken me in the past.

Pick a Target

Time Suggested: Not sure (I think it’s supposed to be pretty quick)

Took Me: 40 min

After voting, we had around 10 HMW notes that we deemed important.

To narrow it down further, I took a play out of this AJ&Smart video. I made a graph of effect vs effort where the bottom left quadrant was low effort and little effect and the top right was high effort and highly effective.

I placed each sticky note on the graph. From there I got rid of the solutions that would have little effect. I grouped similar notes together and rewrote the HMW question to better fit the group.

I ended up with two HMW goals for the first app prototype:

1. HMW make it easier to locate enrichment materials?
2. HMW make the in-class video features easier to use?

The community forum was an area of discontent for many users. There was not a lot of engagement and it was not even accessible from the mobile app. This could have easily been the focus of my sprint, but it was not.

Redesigning the community forum may have been highly effective, but there was also a high likelihood that it would be a dud. With more time and more research, I may have chosen to focus the sprint on engaging the community by making the community forum accessible through the app and creating activities that users could engage in.

The reason I chose to put minimal effort into the community forum is because I agree with David A Chang’s analysis that it may be more effective for student engagement and the acquisition of new students if students were encouraged to engage on social media rather than the community forum.

I also did not do much to create new materials and assignments.

I think Masterclass could really benefit from doing a social media outreach where it shows how students are applying the knowledge they learn. The example given by Chang is Steph Curry challenging his Masterclass students to make 5 three pointers in a row.

After looking through several of the classes’ workbooks, I realized that many of these challenges and assignments were already created. They were just hard to get to and even harder to read on a mobile screen. The challenges were also directing users to use the community forum instead of social media.

I decided to focus on in-class navigation so that the challenges, assignments, and enrichment materials that already existed could be more easily accessed.

Wednesday

One of the things I learned about doing the sprint process alone is that some of the steps don’t apply to you. Things like Divide or Swarm where you assign sections of the map for different people to sketch don’t make much sense alone.

The other realization I had on Wednesday was that the amount you can get done in 5 days alone using this process is significantly less than the amount you would be able to do with the help of a team.

Things like lightning demos, where each person looks into different solutions to your HMW problems from your company and others across different industries, take longer because you have to look into each of the solutions yourself.

Lightning Demos

Time Suggested: 2 hr 30 min

Took Me: 3 hours + an afternoon where I didn’t record

My HMW problems were related to video features and navigation.

I wanted to make sure that the video interface was as clean as possible. I know from experience and by watching users how easy it is to click the wrong thing in a video and end up somewhere you don’t want to be.

I also learned from the research that people pause the videos in MOOCs to read what is on the screen. If there are too many buttons, it makes it hard to see what is behind the pause screen.

I took screen shots of all the major video platforms I could think of, both on my phone and on my Fire TV, and circled the features I liked. Then I compared them to the Masterclass video screen.

One of the things that stood out was that in Masterclass you can skip ahead to the next class but you cannot go back to the previous class. In the lesson page of the app, it also does not show the previous classes, just what’s up next.

There is no room for user error when it comes to accidentally skipping ahead.

What this means is that if I want to hit the settings but accidentally hit skip, I have to go through at least a 5 step process to get back to the lesson I was previously watching.

I used this same process to compare Masterclass with other apps that provide content to users.

At the end, instead of presenting my findings to myself, I looked through all my notes and made a summary page. The whole thing took pretty much all day.

Thursday (where I veered off)

Tuesday afternoon through Friday in the sprint process are dedicated to sketching, creating a user flow, voting, prototyping, and testing. A lot of time is dedicated to voting, which is important when you have a team full of ideas.

In my case, voting didn’t make sense. My process from crazy 8’s to final paper lofi includes marking up the sketches with a red pen and the vigor of the teacher I once was. I’ve got circles, comments, etches in the margins and I come out of it with a fairly good idea of how the final screen will look.

But I’m getting ahead of myself…

The Four-Step Sketch

Time Suggested: 2 hr 30 min

Took Me: 3 hr 45 min

The four steps are:

1. 20 minutes to walk around taking notes
2. 20 minutes to jot down ideas
3. Crazy 8’s (a personal mimesis for someone who draws as slowly as I do)
4. A final solution sketch.

I didn’t have a room full of whiteboards and ideas, I had a closet full of sticky notes, a notebook full of exercises, and a 21 page google doc full of everything else.

It was too much stuff and very hard to keep organized. I have since painted whiteboards on my closet and I am sure it will be a much better way to run a sprint.

However, one of the benefits of the way I worked was that my google doc had summaries after every major task I completed. So in these 20 minutes (full disclosure it was more like 35) I went through the notes and took notes on my notes… 🤔

I used the time dedicated to ideas to run a Moscow method list of features. Moscow stands for must have, should have, could have, and won’t have. By the end of it, I had a good idea of features that were going to make it on the page.

Crazy 8’s… Crazy 8’s drive me crazy. One minute is not enough time for me to draw a full page layout. And by the time I get to 8 sketches, the 6th, 7th, and 8th look pretty much identical.

So I do crazy 6’s. Two minutes per sketch, six times. My sketches look much better. The layout and features are more understandable and there is still enough pressure to move quickly that I don’t get to waste time trying to draw Malcolm Gladwell’s hair in a thumbnail… which I have done.

Tell me you don’t want to draw that squiggly hair.

I spent an extra 10 or so minutes using another pen to zealously mark up sketches, as described above.

I repeated this process a second time for the next screen/interaction I was changing, and then I moved on to the final lofi paper sketch.

Friday, Monday and Tuesday

I skipped the entire next part of the process. No voting, no combining, no dividing, no fake brand names.. I don’t really understand the brand name thing yet, I think I have to re-read that section.

I also skipped the storyboard.

The storyboard to me looked like something in between a user flow and a journey map, which I had established long before I started sketching. I knew what interaction was going to lead to what page and how to get back.

I think the difference was that I was prototyping through the eyes of a UX Designer.

My prototypes were full pages of the interface with a user flow already in place. I think the sketching exercises in a group sprint are more designed for building on feature ideas or showing a single interaction.

This probably would have led to deeper thinking and more creative solutions than the method I used.

I will explore their method of sketching further in my next sprint.

Prototyping

Time Suggested: 2 hrs 30 min

Took Me: 8 hours…..

I took much longer to prototype than suggested. This was on purpose.

I wanted a high fidelity clickable prototype that as closely mimicked the actual app as possible. I would not have done this in a normal sprint, but in a normal sprint I would probably have brand components and in this one I had to recreate everything.

I also ran into some errors in the final high fidelity prototype. These were my errors, and they probably would have been caught earlier if I was working in a team or more experienced.

I failed to create one of the interactions crucial to my user flow. Luckily, there was a screen very similar to the page I needed to create in the Masterclass interface. So, I recreated it, added the component I needed, and attached it to the prototype.

Here are the full prototypes: Fullscreen Video , Lesson Page Vertical

Testing

Time Suggested: All day Friday

Took Me: One month….😬

One of the drawbacks of doing a sprint by yourself is normally the testing is coordinated by someone on the team who is not the facilitator. I was the facilitator, the organizer of testing, and the team, all in one. So I did not schedule testing.

This was not just because I was being lazy, I frankly didn’t know how much time this process would take and when I would have a working prototype. If I had scheduled it for Friday, the users would be testing based off of a paper sketch… just like back in design school.

So I waited.

After finishing a high fidelity prototype, getting caught up in another project, and helping my dad through knee surgery, it took me about a month to run user testing.

Specifically, I wanted to check the recognizability of the icon I chose for the lesson challenges and recipe cards in the full screen video. The icon is usually used for information, and I did not know if my users would associate that with information about the class or information about the quality of the video, playback speed, etc.

The next thing I was testing for was the ease of navigation in the lesson page. I wanted users to click on more in order to find all the resources relating the lesson and the course.

I targeted Masterclass users specifically because I wanted feedback on changes in the interface. I have a couple friends who use the product so I reached out to them first.

Though it is frowned upon in the UX community, when necessary, I did remote testing where I had them record their screen and answer three simple questions through a video. This helped me get more feedback than I normally would have when scheduling a time was a hassle, either because of conflicting schedules or the fact that they live on the other side of the world.

Every time I do user testing I’m reminded of how important this process is.

I did the research and I am a user, so I thought I had a good idea of how the user would navigate through the product. I was wrong.

The first feedback came from my friend who is a developer and former chef. He has a lot of experience watching youtube for recipes and techniques in the kitchen. I had him try to find a recipe card while using the app full screen.

His biggest complaint was that he had to pause the video in order to access the card. He’d rather have it in the description while the video was playing because that’s where he was used to seeing it in youtube and because he didn’t want to have to stop the video.

This was baffling to me. I couldn’t imagine not wanting to stop the video to find the recipe…. But that’s usually because I’m trying to pay attention while frantically keeping up with the chef.

I got a lot of other good feedback from users that I would not have thought of.

The biggest lesson I learned was that the prototype needs to be as close to fully functioning as possible to get real results from the test. What I mean by that is you can’t just design the screens that you are using for the user flow, you have to include all the buttons the user could push while they are in the app.

An easy way to do this in a preexisting app is to just take screenshots and make invisible boxes over the components you need to be clickable.

Lessons

So, let’s look again at the doubts I had when starting the sprint:

Can you run a sprint by yourself?

Yea… kind of. But not as effectively

This may get some push back. “I’m an experienced designer, I run sprints by myself all the time, you’re just inexperienced and you don’t know what you’re talking about!”

This is all true. But the fact is, good design is never done in isolation.

Technically if we’re testing users we’re not designing by ourselves. But even if we’re focusing solely on the first 4 days of the sprint before we test users, they would be done more effectively in a team environment.

One of the biggest obstacles to good design, especially when we are experienced users of the product, is our own biases. It’s easy to think that you know the product and it’s pitfalls, and therefore you are a good candidate to redesign it to make it better.

But you’re making it better for yourself, maybe some other people who use it like you do, not for the general population.

I remember the first time I was hit in the face with this realization. I was working through my third project in design school where we were redesigning the mobile experience of Google Docs. Our idea was that we could redesign the Docs app to make it a better experience for taking notes, because that’s what we assumed people were using a mobile word processor for.

What we were surprised to find was that many users, especially from younger generations, were using the Docs app on their phone to do word processing… writing essays, papers, stories, etc.

We couldn’t imagine writing a whole essay on a phone… but we didn’t account for the fact that some users are faster at writing on their phones than on a keyboard (many Gen Zer’s) or that people didn’t have access to a laptop and needed to publish directly from a mobile device (journalists, low income students, etc.).

So we shifted our direction and made a word processing app that could support the formatting necessary for a fully fledged document, while also including note taking features that made it easy to organize and format notes on the fly. Here is a full write up.

Can you run a sprint with no experience?

You can try.

I take that back. Absolutely you can. In fact, sprints are not made for Product Designers. They’re designed to get a team of people together who know about certain aspects of the product’s life cycle in order to make the most effective product possible.

However, as a new facilitator I made a lot of mistakes. Most of them were my fault, I didn’t understand the necessity behind some of the methods, and I probably could have reread sections of the book to have a more effective sprint.

Is five days enough time?

Not when you don’t work for the company.

I think honestly the biggest consumer of time during this process was the research stage where I had to update myself on all that was Masterclass. Understand the direction of the company and the feedback from its users was critical to establishing a goal for the sprint.

With that knowledge, I think it could be done in five days if you work diligently because some processes take far longer in a group than individually. Others, as seen above, take much longer alone.

En Fin

I loved doing this sprint. I learned so much about the process and I had a good time. Even though it took far longer than I expected, I felt like it streamlined the decision making processes that so often slowed my progress in the past.

I’m excited to try a new sprint, maybe with a team this time 😂.

Let me know what you think about the sprint process, things you’ve learned, experiences had etc.

Thanks for reading!