Using Use Case Points

Early on we decided to record our functional requirements as use cases. Estimating how much effort was required to complete the project based on the use cases collected turned out to be a much more challenging problem. In creating our estimates I turned to use case points estimation(pdf), a technique developed by Michael Cohn of Mountain Goat Software.

The basic premise of use case points is that by counting the number of transactions in a use case and then applying some general rules of thumb to the number (possibly taking into account adjustment factors based on quality attributes and environmental factors) you can get a rough order of magnitude estimate for understanding how big a use case is. This is same basic premise used for all estimation (count, optionally adjust, project based on data) McConnell talks about in his excellent estimation handbook, Software Estimation: Demystifying the Black Art.

Counting transactions is easy and straightforward. For the SQUARE Tool, we calculated approximately 255 unadjusted use case points and 244 adjusted use case points. [A complete breakdown of use cases and their corresponding point estimations are available on our archived project website.] The use case point estimates gave us a rough idea of how big and complex each use case would be compared to other use cases. The tricky part for us was projecting effort and schedule based on the use case point number. Being a new team, we didn't have historical data. To further complicate matters, we were conducting this estimate in parallel with our architectural design work, much later in the life of the project than Cohn's paper implies this technique should be used.

Not having a means of projection I turned to some interesting sources. Keep in mind that the total effort allocated to the studio project is only about 5,300 person hours (over a 16 month period) and time must be split among all stakeholders including faculty (e.g. end of semester presentations). At the time these estimates were created about 4,000 person hours of work remained in the project.

Assuming 20 - 28 hours per point means we would need between 4,800 and 6,800 person hours of effort to complete the project.

Converting use case points to function points to Java lines of code (result is approximately 8,000 LOC) and then running these numbers through COCOMO II (default settings) gives an estimate of 3,700 - 5,900 person hours of effort to complete the project.

Surely MSE Students work faster than the default COCOMO II assumptions. Given that MSE teams typically produce code at a rate of 4.48 LOC/hour, the Square Root team will need only 1,819 person hours to complete the project.

According to estimates, none of which corroborate with one another, the project will take between 2,000 and 7,000 person hours of effort! So we'll either finish under time or blow the whole project. Not very useful.

To overcome the variation in our estimates and hopefully come up with something a little more useful, we conducted a Wide-band Delphi estimation session, sampling a set of use cases to determine an approximate value for a use case point. Following the session, we determined that use case points for our team were worth between 8 and 12 hours. This gives us an estimated range of 1,800 and 2,300 person hours of effort, a much more manageable range and certainly (hopefully) more realistic.

We used the average use case point value of 10 hours for the purposes of planning. Tracking the team's progress over time using earned value, it became clear that we should have chosen the lower, 8 hour point value.

Conclusions

Use case point estimation worked out OK for the team. Realistically, any reasonable proxy would have done. We wasted a lot of time trying to find reliable industry sources for point values when the most accurate estimation of all was a simple Wide-band Delphi estimate done by the team.

The most important thing about the estimates was that, for the first time it gave us the ability to see beyond the next iteration and allowed us to project through to the end of the summer. That we were able to produce these estimates, in my mind, marked the end of the "Period of Uncertainty." From this day forward we had a plan, knew how long it would take, and could definitively determine where within that plan we currently were and whether we were where we needed to be to successfully complete the project.

Use case points were unsatisfying because use cases were generally unsatisfying as a means for recording requirements. While the nature of use case points would have allowed us to create an estimate earlier, the Wide-band Delphi session was so successful only because we had enough detail to have meaningful conversations about what was in each use case. Had we attempted this too early, the estimate would naturally of been less accurate (though perhaps still useful if you could figure out a way to track architectural progress within the created estimates).