Demystifying Agile: Rants of a Madman – Story Points
First of all I want to raise a glass of Champagne to toast the folks who came up with the concept of “Story Points”. You know that way of estimating work without actually saying anything useful or making any commitments to your business customers? Freaking geniuses they are! The best practical joke ever played on the business continues to this day and spreads like wildfire as more Agilists pull Story Points out of their Tower of Agile Babble as they convert more and more teams to the ways of Agile. Don’t get me wrong, I like Agile, but story points? You’ve got to be kidding me, right?
I had a strange series of dreams last night. They went as follows…
I was at an airport waiting at the gate for my flight. From the speaker embedded in the ceiling came the garbled announcement “We are sorry, but the departure of Flight 7734 to Bakersfield has been delayed. Expect a new departure time in 4 story points.” Of course, it was going to be delayed, but wait… What the heck did they say? I went to the agent at the gate and joined the mob of travelers who wanted answers to listen in on the following dialog:
Traveler – “When will flight be leaving?”
Agent – “As we announced in 4 story points.”
Traveler – “When?”
Agent – “As I said, in 4 story points.”
Traveler – “What is a ‘story point’?”
Agent – “Why silly customer, that is a unit of measure we use here at Agile Express to plan and communicate how long we think something will take.”
Traveler – “Okay… So how long will it take?”
Agent – “4, that’s F-O-U-R story points.” (eye roll as she looks at another agent next to her)
Traveler – “So how long is a story point? I mean in minutes. Like…in real time?”
Agent – “Well we don’t actually know. It depends.”
Traveler – “Depends?!? Depends on what?”
Agent – (big sigh of frustration) “Lots of things. You see the mechanic is checking out why the pilot is seeing a particular fault light in the cockpit. Depending on the light, it could mean that a different system is involved. Depending on the system it could impact how long it actually takes to fix. Since this isn’t a precise science, but an art we use story points to estimate how long we think it will take. As to not frustrate our passengers we don’t make any commitments. However, for visibility, we share the story points we think it’ll take in order to make people feel better.”
Traveler – “And?”
Agent – “Our best guess is 4 story points.”
Agent 2 – “Well, that is unless Bob picks up the story card to make the repair. At that point then it’s anybody’s guess as Bob is slow as molasses and takes lots of breaks.”
Traveler – “So in human time, not story points, when do you expect the plane to actually board and depart?”
Agent – (very frustrated now) “Don’t you understand? I cannot and will not tell you! You’ll get it when you get it. Here is a chart that shows how many story points we’ve made it through thus far. Draw a trend line and see where you think it’ll be completed. Now go away. You’re bothering me with these stupid questions.”
Loudspeaker – (30 minutes after initial announcement) “We are proud to announce the on-time departure of Flight 7734… Please proceed to…”
I woke up. It was surreal. Story points? On time departure after a delay? Oh, just a dream. I quickly fell back asleep. This time I was at the same airport, but at a different gate waiting on a different flight for my connection. It was the same airline though given the uniforms the gate agents were wearing. “We are sorry to report that Flight 710-77345 to Houston is delayed. We should be boarding again in 4 story points.” Crazy… I just had this dream… Well a similar dream, but in the previous segment at least I remembered, albeit fuzzy, that 4 story points somehow came out to be about thirty minutes… Or so I thought. In this segment of my dream more than an hour and fifteen minutes wasted away before I gave up and went up to the desk at the gate.
Dream Mark (some great looking actor 20 years my junior playing the role of me) – “So when does this flight depart?”
Agent – “Oh, like we said, in 4 story points.”
Mark – “Huh? Last time that took 30 minutes. It’s been well over an hour since your announcement. How’s that so?”
Agent – “Silly man. Story points don’t actually represent repeatable units of time. They are arbitrary. That and you are at a different gate. At this gate we actually use story points differently here than at other gates. It works better for this particular flight crew.”
Mark – “Say what? How can that be?”
Agent – “You just don’t get it. Are you that stupid? A story point is simply an arbitrary unit of measure for estimating how long we think it will take to finish something. Because it is arbitrary and we are Agile Airlines any gate at this airport can use story points to mean whatever they want them to mean. Now go away, you bother me… That, and you are wearing no clothes except your underwear; nobody else ever point that out to you in a dream before or did you think nobody noticed?”
My head was spinning… The dream shifted and the rest of the night was filled with similar conversations: Cable Co quoting me in how many story points they would come to my house to fix my cable box; the pizza delivery guy saying that his guys are running behind by about 10 story points; the same movie playing at two different theatres listing the duration of the movies at different story points; the guy framing the addition to my house quoting how many story points his framers would need before the roof could be added… ARRRRRRGGGGGGHHHHHH! After a restless night of what could be described only as frustration dreams I woke up, got ready for work, and drove in to the office only to join a meeting where a team was trying to tell me how many story points a particular thing Bob was working on would take…
So, did the dialog in these dreams sound familiar? I get that estimating how long it will take to do things is something that both developers and development managers have hated from the days of the first punch card. I also get that when it comes to the development of software there is often more art than science involved. I appreciate that it makes many teams uncomfortable to actually make commitments when they are wrong so often. For rough estimates required for sizing exercises, teams have also been able to successfully use arbitrary buckets like Small, Medium, Large, SuperSize. But to do the same to estimate work that should fall within the length of an iteration? I mean really guys–we are talking about days, not weeks or months here folks. I don’t get how making up a random unit of measure for time, which obfuscates the ability for two intelligent human beings to share information in a meaningful way, helps to make any of us what we need to be: Better Estimators…
That’s right. Instead of working on being better estimators at tasks that should take between half a day and three to four weeks (at the max) we have decided to hide behind some arbitrary unit of measure, which can mean anything to any team. Not only that but we insist that the mere mortals, our customers, are expected to learn our form of a made up Klingon sub dialect and play along with this game of 3 Card Monte when it comes to estimates. It also makes it very hard for the business to buy in to how better Agile is when this piece of Agile Babble makes no sense to them…
If you want to earn the respect and credibility of your business customers, don’t try to teach them this as you yourself are trying to learn Agile. How can you teach something to a business customer when you don’t even know it yourself? Especially when it is something that we probably shouldn’t even need to teach anybody in the first place. Hiding behind the fact that estimating “is hard” and by tricking customers to speak Agile Babble as the answer constantly shifts out from underneath you just isn’t ethical. This practice would never cut it in any other industry where tasks for the next two to four weeks need to be estimated, so why does software think it can get a free pass?
Here’s my challenge to you: Go in to your Agile tools and configure them such that a single Story Point equals an Earth day; a unit of measure that has been used by all of mankind worldwide since the first human took notice of the daily cycle made as the sun rises, sets, and rises again every day. You can use a decimal for a unit of time shorter than a day… Communicate with your business partners in terms of how long you estimate a given piece of work would take, your forecast as to the remaining work, and then track your actuals in terms of how long it really took. Only by practicing with a real unit of measure that everybody knows can you actually become better estimators over time.
Please don’t let this fad continue to expand and even spill over in to other industries… If it does we might as well throw our watches, clocks, and calendars (yes our apps that measure time as well) in to the trash… That and we’ll have to purchase a lot of meds…
About this series: I love Agile; I also hate Agile. I love how it can free teams to truly delight customers while delivering high quality products on time. I hate how Agile zealots can use the Tower of Agile Babble to confuse the heck out of teams trying Agile on for size. My goal is to help new teams actually embrace and become Agile without having to learn all of the pomp and circumstance in one big fat swallow.
Mark Lawler 
Point taken. Very valid arguments indeed. It is very frustrating to not know ~when something can be completed in order to see the product in progress and give “feedback”, an often forgotten and usually very misunderstood part of the process.
I have been on a few Agile teams that use “story point” measurements as a relative cost weighting with respect to other user stories, an “internal” measurement to mean the degree of difficulty relative to other user stories. Typically, user stories with higher costs would take longer, have more dependencies and be more critical to the overall deliverable (aka have the most risk, etc).
These user stories should only have this relative cost weighting for as long as they remain in the product backlog of things todo, so the customer has a means to organize and prioritize the workload to be delivered. Thus, this measurement is more appropriately used as a form a scheduling, not estimating.
You really cannot determine the length a time something will take until you have established some other metrics such as, but not limited to knowledge, experience and skillset of the team, the team size, the amount of interruptions/distractions, customer availability to be involved in the process, frequence of changing requirements, and eventual velocity that can only be determined by doing actual work.
Clearly, it is best to start small, with simple tasks that can be clearly defined and estimated and help the team build momentum (which carries you further than velocity… think freight train vs. a bird moving into a 60 mph wind). Tackle the low handing fruit and build on that, until the more difficult, ambiguous tasks become clearer (usually be way of analysis and hardening the customers wants into needs) and then they will be easier to slap a time estimate on them.
We used 2, 4, and 8 hour tasks, where 1 or more tasks represented a complete user story (requirement/product feature). A task typically should not take longer than a day otherwise it is not granular enough and should be broken down so a couple engineers can work on the problem concurrently.
However, to not garner some kind of heartbeat to your Sprint can leave estimates much to be desired as they turn out more SWAG. The point is, you want accurate, good estimates and while practice makes perfect (we will get better with time, maybe, maybe not, not if you keep doing the same thing expecting a different result), you want a bit more guarantee than that.
In terms of the all important “feedback”, expectations with customers should be set up front so that when they see the product, they are not seeing the “finished” product. They are getting to see the product in order to make refinements earlier than they would typically get to see the product, period. This much should go without saying.
Disclaimer: I am not an Agile product owner, coach or master. I am a software solutions developer with much experience through practice.
Cheers!
By the way, I really enjoy your posts. I love your elevator speech and think it is spot on.
Thanks! Thanks also for the thoughtful comments. I think we all bring our best practices and ideas to bear and I appreciate you sharing yours to this post.
Mark I have to say you have really thrown points under the bus here! Points are an important skill for an agile team, and, more than that they can help a team focus on delivering value instead of on delivering inaccurate predictions that they feel unaccountable for. I have seen teams that know how to use points to their advantage deliver better information to the business, faster.
First, points serve a purpose, and that purpose is not for communicating expectations and predictions to the business. So if you have teams trying to communicate business expections using points, they are completely missing the boat. That’s nuts, and on that I am sure we agree.
Points should be used by teams to speed up their planning, and keep planning at an appropriate level of abstraction, so that teams do not devolve into relentless hours of planning discussions that do not improve predictions.
Here is the methodology I recommend:
1. Team estimates the backlog, using points.
2. Team performs a “sprint outlay” excercise (credit Mark Goschie)
3. The sprint outlay shows an estimated number of sprints that it will take to deliver the effort, including extra sprints to accommodate stabilization and release activities.
4. The team communicates the sprint outlay to the business as a baseline schedule. Sprints have beginning and ending dates, so this schedule has a date that a human understands.
5. The project is tracked at a sprint by sprint level, at each sprint review, changes to the outlay are made extremely visible to the business.
So what is a sprint outlay? Draw a series of boxes on your white board, time running left to right, and position stories and backog items into discreet sprints. You can do this based on predicted velocity, but more important, the team looks at each sprint as a whole, discusses the work, and gives the sprint a serious reality check. Can we really do all of these stories in that sprint? Could we do one more? Which story is most likely to slip out of this sprint? I have seen many teams successfully produce sprint outlays for 3-6 month projects, in a few days. Adding some backlog elaboration, grooming, architecture, and estimating in front of this, strong teams can produce relevant project plans in under two weeks.
Hmm. So why is this better than estimating in days? That’s simple. When you estimate in days, the team spends (wastes) more time in detailed task planning, planning things that are months away, that they actually know little about, and to represent risk and unknowns, the estimates grow and become irrelevant. Then someone gant charts the data (or makes a crude velocity calculation) and if they are good, they will factor in everything they know about the people on the team and their quirks. You end up with a schedule that shows a degree of precision that you certainly don’t have. But worse, you end up with a team that feels less ownership over the schedule commitment, since they only estimated the tasks. Whoever added up all of those numbers, well, I guess they are the ones accountable for the prediction, not the team.
My experience, repeatedly, is that projects planned this way are not as successful as those planned more efficiently with points and a team-owned sprint outlay.
Best,
Adam
Pre-planning your future sprints diminishes the flexibility that Agile was designed to provide.
This depends on whether you make commitments to your customers or ship what you have and hope they buy it. For software like web tools, MS Office, mass-market software, you ship what you have. For ERP and other mission-critical software, you make commitments to your customers. You have to pre-plan to some degree in that case. The trick is to only do the right amount, and to not become trapped by the plans.
First let’s give credit here – this has to be the FUNNIEST post ever on Agile (not that there’s much competition). The previous replies from experienced Agilists make a lot sense but I must say I’ve also seen story points misused as Mark describes. I recall the first time I heard story points discussed as a new-ish PM. I asked “what does a story point represent” and got the response “about a day.” So I asked “why not just call it a ‘day’?” and got a reponse equivalent to “because it’s a story point.” I was reminded – as has happened to me often in my career – of the moment in the film Spinal Tap when What’s-his-name says “but this one goes to 11.” This classic scene is on YouTube if you haven’t seen it. Mark’s series is as much lamenting Agile malpractice as it is explaining its value when skillfully applied.
Is a software developer more like a worker in a light bulb factory or more like Thomas Edison inventing the light bulb? Some software developers are “trained” to churn out solutions that have been well-defined years ago. Other software developers strive to take a fresh look at problems and innovate. Factory workers know exactly which filament to insert and how long it will take.Thomas Edison didn’t know exactly when he would discover the best filament for his bulb.
A business that embraces innovation will reap rewards greater than a business that stands in the same place.
I like to say that hours (or days) are always wrong.
In my opinion, points are never to be used as a measure of time. They are to be used to assign a relative size to a User Story. Size does not tell you how long it will take. However, points are used to calculate the velocity of a team. This measure can tell you approximately when you will deliver a User Story. Each team can have its own velocity and typically assigns points to their own work.
There has always been confusion by comparing points to time (or hours). Time is useful when estimating a very small task (time to get the plane ready for departure). Velocity is extremely useful to estimate how many similar User Stories can be delivered in a longer period of time (5 points per take-off ; proven velocity = 20 points per sprint; 1 sprint = 1 day: Result= 20 take-offs per day).
The average velocity allows for very precise predictability over a period of time. Since many unpredictable factors cannot be included in the estimation using time, average velocity naturally includes them.
In conclusion, points are very effective to calculate when I will deliver in the long term. Hours or days are very effective to track tasks using a burndown chart in a very short term iteration.