Estimating design time: why some designs take longer than others
My VP of Product Management was frustrated. She didn’t understand why sometimes a design would take us a couple of days, and why sometimes it could take many people several months to complete. So I developed a model to explain it.
On the first day that I started at the company, I drew a simple diagram on my whiteboard to describe the design process to one of the PMs I was meeting. The diagram had four words: Discovery, Design, Critique, and Validation. I drew arrows down between the four parts of the process, and then arrows on the side, going from Critique back to Design and from Validation back to Design, to demonstrate how feedback was the mechanism we used to iterate and improve our designs.
I explained that this was nothing new or magical, it was a pretty standard process. Over the next 18 months, I drew that diagram a lot, and even made a slide that defined what I meant by each of the words:
Over time, my narrative included statements like, “And all of this comes before the design gets to development. I want to be as stingy as possible with our development resources, because it’s a lot cheaper to change a drawing than to code it up, QA it, document it, and then change it.”
Estimating design time
A year and a half into this role, we had a process; we were meeting regularly with PMs and Engineering managers to estimate how long it would take to design each new feature. Our estimates were really accurate. So accurate that over time, Engineering started basing their estimates on ours. For example, if it would take a month to design, it would take 2 months to build.
But the heuristics we were using to create those UX estimates were in our heads, based on our experience. Sometimes our thought process made it onto the whiteboard, as we tried to understand: how complicated a thing might be, if we had a pattern for it already, and how many pages might be impacted. So when the PM leader asked me, “Why is it that some designs take 2 days, and others take 2 months or 2 quarters?” I had a place to start, if not an immediate answer.
The Situational Model
As I thought about answering the question, it occurred to me that there were four types of situations we found ourselves in: regular roadmap, firefight, pivot, and surge.
Regular roadmap: This is the situation that was easiest to estimate and manage. Work was planned one or more quarters out, the scope of the work tended to be larger, but we could see it coming, had time to work through requirements with PM and Dev, and we could do all the UX work in the quarter before the feature needed to be developed.
Firefight: Imagine a hot bug fix. This kind of work was very time-constrained, but because it was already in production, we usually had a lot of context and the UX work in these situations tended to be smaller.
Surge: This situation can happen at startups any time, and at larger companies in the last quarter of the year. It’s the notion that if we had some amount of funding, say $100k or $1M, what could the organization do with it in a quarter? Development and UX tended to start at the same time, but because it was time constrained to a quarter, the work would be medium-sized.
Pivot: In this situation, an entire product team would stop working on product ‘x’ on one date, say December 31st, usually in response to a shift in product strategy. Then on January 1st, they would start working on an entirely new product. These were large projects, and (like the Surge) we’d find development and UX starting at the same time. Not ideal, because we’d be on the critical path from day one. Hurry up or UX is holding up development.
Our preference, if it’s not obvious, was regular roadmap — it gave us time to understand the requirements, estimate and staff the work properly, and then ensure we had a validated solution before engineering started coding.
One thing to call out here is the relationship between the stages of UX. Projects that required more discovery and design required more critique and validation. It seems intuitive, but it wasn’t, which is why I created the diagram above and this table below.
And while the above table showed that there was a relationship, it didn’t demonstrate what factors impacted design time, or the impact of the interaction between those factors.
Factors impacting design time
So what things were we mentally calculating as we estimated design work?
As I mentioned before, when we needed the whiteboard to talk through an estimate with our stakeholders, the things we’d consider were what we already knew — or didn’t — and then how we could see the design manifesting: how many pages would be impacted and how complicated might it be?
I created a simple model to demonstrate that several factors impacted how long it took to complete the UX process: the technology we had to design for, the user personas, how many screens would be impacted, and how complex the interaction might be.
Technology: We had a web platform that was the default way of surfacing new features, but we also had integrations with a dozen or so other products. Whether we were designing for a known technology or a new technology had an impact on the time it would take for discovery and design, which would then impact the time it would take to critique and validate.
Users: Likewise, was this new feature for a user we already had deep experience with, or was it for a new user? If it was a new user, it’d take time to understand their expectations, workflows, and preferences. For example, developers wanted to work in their IDE without logging into our web UI, and DevOps engineers wanted technology that they could plug into their existing tool chain.
UI impact: Would this feature impact one screen or 20 screens? More screens mean more mockups and more critique.
Complexity: Even if a feature only impacted one screen, making the feature functional and usable might be really complicated. In this case, we knew where the feature would live in the UI, but it wasn’t easy to figure out how it would manifest to the user.
To demonstrate the relationships between these factors, I had another slide where populated the two grids with the names of projects and features that fit into each quadrant, so the PM leader had concrete examples of what I meant. I won’t show it here, but I recommend that you do the same, if you decide to use this model.
Tying it to numbers
Since we were talking about time, and in particular the scale of time it took to design a thing, I assigned numbers to each quadrant, 1, 3, or 9, so we could do the math.
And once we did the calculation, we could see the difference in scale of a design, based on multiplication:
The numbers were not meant to represent UX estimates, just the factors that went into our estimates, and how the factors impacted one another.
Results
The diagram comparing UX to development really resonated with the folks who were in engineering. And the mathematical model really landed with our quant-focused PMs and leaders. We had a shared understanding now, by making our estimation process more explicit.
Postscript
I have been sharing this presentation again recently. And it’s highlighted some other variables that weren’t problems at my previous company. If you want to use or adapt this model, I think it’s important that I call them out.
Roadmap: Not every team has visibility into what will be developed next quarter, much less the quarter after that. Without a roadmap to guide planning, everything falls into the pivot or firefight category. For example, if your team is changing product priorities many times a quarter, you either don’t have a roadmap or the team is ignoring it. I can imagine the absence or presence of a roadmap being on one axis of another 2x2 matrix or represented as a wildcard value.
Requirements: When you have a roadmap, PMs have time to write requirements, the UX and Development teams have time to review, discuss, and add to them. Requirements ensure that everyone has a shared understanding of what needs to be created. Without requirements, design is just guessing. I’ve seen this manifest as endless rounds of high-fidelity mockups that PMs and executives “react to”, sending the designers back to the drawing board as many times as it takes to figure out what the user outcomes and experience should be. This is by far the most expensive way to design, and is often labeled as “agile”. But I’ve seen Agile processes work, and the process I just described isn’t it. I can imagine the absence or presence of requirements being on the other axis of the 2x2 matrix with roadmap or its own wildcard value.
Overall, I’ve found that the model has been helpful with peers and with stakeholders, so I’m curious to know how this resonates with you, your challenges, and your experience.
