Designing an Estimation Process
In many organisations (especially larger commercial or government) it is common for the project governance processes to require an estimate of project costs before funds can be released and the project launched. Therefore, it is necessary to develop an estimate that can be used as the basis of a go/no-go decision.
In the example discussed below, I was asked to create a process for developing an estimate that satisfied the above requirements. Rather than simply creating process flow and swim-lane diagrams, I tried to consider how the process could be adapted and matured over time as the organisation changed over time. In doing so I found the many agile/lean principles were being subtly introduced.
Strategy
As is common with most things, your first design isn’t going to be perfect and it will require progressive enhancement. With this in mind, my approach was to start simple, develop the most minimal process and then provide support in the form of training, infrastructure (tools, templates etc). Realising that my first iteration of the process would be a tactical solution allowed me to move past trying to make it perfect and enabled me to focus on the bigger picture of how the organisation could adapt and mature the process.
Establishing the Current Issues
To begin with I reviewed the state of the current estimation practices from a number of different perspectives including developers, project managers, business analysts and solution architects. I have included the results together with comments below as the issues are probably common to many larger organisations.
| Issue | Comment |
|---|---|
| Current process is undocumented, poorly understood and not consistently followed | New process will require documentation and key staff will require training. Documentation may also serve as induction material |
| Lack of ownership during estimation | Process needs to assign a person to act as the estimate lead who is responsible for managing the estimate and coordinating all technical inputs |
| Importance of estimate is not properly understood | Project governance and purpose of different estimates needs to be properly understood by all involved |
| Insufficient detail within the proposal documents to properly estimate effort | Improve the process for requirements capture and quality of understanding |
| An adversarial environment is present that is fostered by poor communication and lack of understanding around accountability | Ensure process facilitates collaboration between all parties and that the estimate and added contingency remains transparent |
| No process metrics enabling measured or improvement | Implement a standard file/directory structure for estimate management and a register for recording past estimates and metrics |
| Confidence intervals required by project management methodology and governance are unrealistic | Develop common expectations around estimation complexity and risk using training materials. Ensure business priority for requirements is captured so that high value features can be estimated more thoroughly |
Key Training and Process Concepts
Based on the identified issues, I designed the training materials to focus on a number of key concepts. Additionally, I structured the process documentation inline with the following methodology for process design.
- Design
Purpose
Context
Documentation - Metrics
Derivation
Uses - Ownership
Identity
Activities
Authority - Infrastructure
Tools
Templates - Roles
Skills
Knowledge
Behaviour
This approach is based on work by Dr. Michael Hammer and is simple, but ensures that the major process areas are considered.
What is an Estimate?
An estimate is an informed assessment of an uncertain event. Informed means that there is an identified basis for the estimate, and uncertain recognizes that multiple outcomes are possible.
Why Estimate?
There are many reasons why estimates are created, but here are three important ones.
- Estimates enable assessment of value. An estimate enables the customer to determine if the cost of the deliverable matches the expected value.
- Estimates enable prioritisation of work. Once the estimated size of a task is known its value can be determined and therefore priority assigned.
- Estimates assist planning. The process of preparing an estimate begins the process of planning a project and identifying task, skills, constraints, etc. This process adds value during project implementation.
Who is Accountable?
It is common for project costs and/or timescales to be exceeded and for the person who developed the estimate being held to blame. To discourage this type of behaviour I tried to create a culture where everyone involved is held responsible for developing an estimate. In the event that the estimate is wrong, the whole business suffers, therefore it is in everyone’s interest to get the estimate as accurate as possible based on the given requirements.
Redesigned Process
1. Design – Overview
As mentioned earlier, I created a minimal and generic process because the majority of estimation tasks are different and this approach would provide the necessary flexibility and business agility. The process encourages collaboration and communication by incorporating 3 separate meetings and a number of progress discussions.
The process starts with a discovery meeting that occurs once a request for estimate is received together with a definition of solution requirements. This meeting is intended to set expectations around project scope, maturity, estimate timing and estimate uncertainty. Coming out the meeting is a decision verifying that the maturity of the solution matches the expectations for an estimate in terms or timing and uncertainty.
Moving forward, an estimate leader is assigned who has the responsibility of coordinating the creation of the estimate by drawing together the relevant technical inputs. As the time taken to gather the necessary data may span weeks (especially if key staff needed to be scheduled), it is suggest that the estimate leader provides progress advice to the customer/project manager to ensure that the process does not become a black-box.
Once the majority of technical tasks are identified and estimated a peer-review meeting is arranged to provide an independent review. Typically, experienced technical staff would review the estimate ensuring that lessons from past projects are incorporated and the estimate can be properly justified – especially when compared to previous similar projects.
The handover meeting is intended to discuss the final estimate with the customer and project manager and review task breakdown, estimate, confidence level and assumptions. Once accepted the estimate is closed and the documentation and register updated.
2. Metrics
Without metrics, a process cannot be properly measured and evaluated. The metrics proposed were intended to encourage practices that would address the identified issues.
| Metric | Purpose | How Measured | Target |
|---|---|---|---|
| Estimate completion time | Document time taken to complete estimates | Estimate register records start and end dates | Completion time to be agreed during discovery meeting. Target is to achieve that to within 10% of agreed time. |
| Change from previous estimate for same project | Document variation between estimates for the same project scope | Estimate Tracker Spreadsheet records each successive estimate for a project | Less than 50% of total project costs |
| Comparison with final costs | Measure the accuracy of estimation practices | Project effort extracted from time-tracking system | Less than 20% of actual project costs |
3. Ownership
A single person (and their role) was nominated as the process owner. This helps facilitate process change as they are able to approve process improvements.
4. Infrastructure
The infrastructure for this process consisted of templates that assisted the preparation of an estimate. The templates help ensure that the process can be consistently followed. Additionally, a file structure was established to support organisation of the documents supporting estimates as shown below.
5. People
The responsibilities for the roles of Project Manager, Solution Architect, Business Analyst, Application Architect and Estimate Leader were clearly articulated in the training documentation.
Improving Estimates
While there are numerous references in estimation, I tried to extract some key principles that would be easily adopted as part of the process. These included:
- Leveraging past experience on similar projects
- Capturing better quality requirements
- Utilizing multiple estimation methods
- Prioritise requirements to ensure estimation concentrates on high priority
- Scheduling resources rather than relying on just effort estimate
Task Breakdown
I always recommend using as many different methods as possible to arrive at an estimate but starting with an initial task breakdown is a good starting point and helps with planning the project. The following are some guidelines on task breakdown.
Decompose tasks into activities that can be estimated.
- Tasks smaller than ½ day is probably too small
- Tasks larger than 1 week are probably too large
Link technical tasks to functional requirements where possible – or to a general/common group. This helps when a functional requirement is removed the effort to change the estimate is minimised. For every task consider the following:
| Requirement | Task | Priority | Skill | Effort | Uncertainty | Comments |
|---|---|---|---|---|---|---|
| Management | Project meetings | High | Team Leader | 8 | 10% | Based on 1hr weekly meeting for 10 weeks for all team members |
| User Registration | Form design | High | Designer | 16 | 50% | |
| User Registration | User authentication | High | Developer | 40 | 50% | Design must be completed before development can be finalised |
Everyone has different preferences on which tools are best for task planning but I generally use a spreadsheet as tools vary between organisations and spreadsheets offer good flexibility. In some cases I have setup template spreadsheets but rolling your own is pretty straightforward.
Adding Uncertainty
Typically uncertainty is added as a percentage of the estimated effort for the task. The effect of applying the uncertainty is that upper and lower limits of the estimate are created. These effectively provide a range that the actual effort/costs should fall within.
It is important to remember that the upper limit should be used for reserving budget, while the un-adjusted estimate should be used for scheduling tasks and resources. This is because schedule typically drives cost so planning effort against the unadjusted effort gives the most realistic requirement of resources. Planning against the upper value of effort would tend to keep resources working against a task longer than expected and therefore tend to increase costs.
Adding Contingency
Generally an estimate is based upon the most realistic scenario, and then contingency is included to account for uncertainty (known unknowns NOT unknown unknowns). It is important to be transparent about where contingency is being added so that (a) contingency it is not added in more than one place, and (b) contingency is not assumed to be included when it is has not been included. The need for contingency arises from things like staff skills and experience, project familiarisation, staff availability, task management, meetings, progress reporting, etc.
Summary
Estimation is a deceptively complex activity and I have tried to simplify and incorporate into a process that can be easily adopted by a large organisation. However, it is important to remember that processes have a level of maturity which often includes deficiencies, gaps or missing decision points. Therefore, for a process to be successful, the participating people need to collaborate and use the process as a guideline. Over time a process matures and the tools/artefacts improve and the tacit knowledge penetrates into the business, hence attitude and support is critical to the success.
Categorised as: Agile, Software