Agile Metrics - What Management Wants to Know!

A lot is made in the Agile world especially in XP and Scrum that executive level reporting should be focused on reporting that top priority stories are getting developed and can be shipped based on the product owner’s demands. And in theory this is a valid argument, however if you are taking several iterations to deploy/release the software, or your company has not fully gone Agile and it is only being used as a engineering process then you may need another way to measure.

Also as senior managers (and yes I consider scrum masters in this category) we also need to measure different factors to see if there are opportunities to improve. As a manager of scrum masters running multiple project teams all in different phases of development, I have been searching on a Pareto analysis that can measure different factors of agile teams to show our progress and overall value to the organization.

This can fly in the face of most agile practices but I am brainstorming and utlizing a formula to show story completion value (SCV) to any organization. Calculate the following for each month.

• Only measure a story when it has passed all User Acceptance Criteria and the team agrees it is done.
• Capture the priority (P) of the story. (P1=100 points, P2=75 points, P3=50 points, P4=25 points)
(Key Measure - Not enough emphasis on higher priority stories)
• Capture the number of iterations (I) that it took the team to achieve done & pass UAC for the story.
(Key Measure - Average number of iterations larger than 1
• Number of Story Points (D) (Difficulty)
(Key Measure - Stories are being estimated incorrectly, or stories are defined at the wrong granularity.)
• Number of Tasks(S) (Size)
(Key Measure - shows the amount of work needed to complete the story.)

SCV = (P/I)*(D+S)/100

For example a Priority 1 story is completed in 1 iteration achieving 100 points.
It had a small degree of difficulty - story points = 8 points
It also had a few tasks - 4 tasks

SCV = 24, if the same story was completed in 2 iterations it would have a SCV = 12

• This means that it’s value to the company was lower because it took twice as long to complete it and thus used the resources longer.

You calculate the SCV for each story that was completed in the month and trend the number for the entire organization. Once data is collected you can then notice trends and corrective action you can take within the agile project teams and review it during your scrum master brainstorming events.

For instance if your average iteration completion for a month is 2.1 then the scrum masters should work with the product owners to break down stories further, or brainstorm with the team on why it takes more than an iteration to complete a story. It could be that the UAC cannot be completed for some reason.

Other things to look for is a team that is creating too many tasks for a story which may indicated that the story is epic, or that a team is estimating the story points for every story the same or all of them extra large which may indicate that they need so guidance on estimation.

Remember the goal here is not to measure individuals or try to change the velocity of the team but to analyze some data points that can be represented to other departments, stakeholders and to understand if any corrective measures need to be taken to remove obstacles for the teams.