Test Team Changes Needed to Support Agile ... Part 1

This is my first post on Agile Testing topics but will be an ongoing project as I start recording my thoughts on one of my favorite topics which is software testing. I will be cross-linking with Fusion Testing Blog as I pull together the threads of my thoughts on Test Team changes and Test Technique changes.

In this blog I will be working on the Test Team and Management direction changes that need to occur.

The first thing to focus on when implementing Agile into an engineering organization is helping the test team understand that they still have a major role to play in Agile teams. This is not an easy task as most published works support the developer testing principle and do not mention software test engineers roles in Agile.

As if testers do not have enough of an identify crisis and inferiority complex, Agile books and implementation is once again devaluing the contributions that test engineers can have on engineering teams. One thing for sure is that if your company has any legacy products there will have to be a long transition period for the Agile teams to catch up on technical debt and unit and integration testing before any team can truly do away with testing positions.

I agree that the best principle in Agile teams is having a group of individuals who are strong in coding, testing, documenting but the reality of the situation is that you have a group of experts coming together on the Agile teams that do not have the cross functional expertise to sign up for any task whether it be documentation, coding or testing.

So how do you convince your test engineers that there is still a role for them in Agile teams. Here are some options:

1. Support their role as testing experts and encourage them to bring up testing challenges for each story.
2. Help them to learn how to identify testable or non-testable UAC, this will provide immediate value to the product owner and the team.
3. Train the team in different methods of automation testing tools such as Fitnesse, Selinum or automation testing techniques like Unit, Integration or Performance Testing.
4. Get your test engineers to learn about Test Driven Development and how to move the testing upstream, and then get them involved in promoting the values to their Agile teams.
5. Implement a Lightweight Test Approach like Fusion Testing to reduce the documentation but to increase the testing quality and visibility.

More to come ....

Setting a good inter-Iteration Cadence

Today we had some interesting discussions in our organization about departmental bottlenecks and the fact that even though we are fully immersed in Scrum that code is still delivered near the end of an iteration making it difficult or impossible for the test team to complete the testing of stories before the iteration end.

One issue our product owner brought up was that we are bumping up against the team's average velocity and only accepting that many points in an iteration, and while this in theory is the correct idea it doesn't necessarily take into account a good work flow for utilizing resources or setting a good cadence for having smaller effort stories upfront, medium effort stories delivered in the middle and larger stories delivered at the end.

Here are some ideas that I came up with:

• In addition to the top priority stories equalling your team's velocity, discuss several lower priority small estimate stories (5 or less points)
• Determine the best order for stories to be worked on, not only based on priority but by breaking the iteration into 3 phases. (1st period, 2nd period and 3rd period) I like hockey so I use this analogy.
• Now discuss stories based on what can be delivered in the 1st period the goal is to find a story that development can be handed off in a day, testing can be completed the next day and any reworked completed in the next day. Also you can identify any of the larger stories pieces that can be delivered and testing started within that first period.
• Next discuss  stories that can be delivered in the 2nd period, items that will take longer than 3 days to fully code, or pieces of larger stories that take longer than 3 days to code that testing can start on during the 2nd period
• Lastly discuss the 3rd period items that need to be completed and ready for potentially shippable deliverables at the end of the iteration. This could include discussing what stories would be good candidates to pull into the iteration if time and velocity permitted.
• Other ideas to ensure a good cadence within an iteration is to plan on having a story or defects roll over, so basically accept more points within an iteration knowing that you would not get all of the work (probably not the testing or documenting) and that those tasks would carry over and provide a better cadence for the entire team.
• As I discussed with the QA team and the product owner organizing your iterations into periods should provide a better organization and even out the peaks and valleys for any teams that are still maturing through the specialized phase of Agile maturation.
• Ultimately as a self-organizing highly responsible team, they all should be less concerned about having testing items for testers, or no coding for developers and they should look at the priorities and determine how to complete the top business value with all resources on the team and with any team member taking the remaining open tasks and finding a way to complete them.
• Another approach is promoting static testing early and often and making sure that quality as important to the team as delivery, this way more test driven development approachs can be implemented and will reduce the rework and increase the quality. This can be done by the test members by creating test checklists, doing code or implementation reviews and testing in non-official environments to provide information quickly to the team.

Interestingly I interviewed a person for scrum master today who said that in the real world coding and testing are serial actions and require different iterations or elongated iterations that have the classical dev to test hand-offs. I was really not for this idea, and have actually tried it and my experience with it is it segregated the teams and allowed them to have different priorities. While this is an approach that can be implemented and can solve some of the cadence problems I worry about the precendent that this sets through out an organization, now it takes a longer iteration or at least 2 iterations to have potentially shippable software. I really think that this becomes a problem or that this becomes the solution when people do not want to undertake the tough changes that need to occur on an organization, employee responsibility and philosophy level to truly adopt and Agile process.

Lastly, I am truly irked by the continued statement of QA or Test bottlenecks and have been making the argument that these items cannot exist in an Agile team. You should have a fixed set of resource on a team that fluxuates very infrequently (this is how it is where I work), those resource each have different expertise, however any person on the team can try to do anything required including writing docs and testing. So any items that cannot be completed or added because their is insufficent testing cycles with a set number of resources then becomes the guage for the team on how much work can they truly start and finish within an iteration. If you look at it as what can developers accomplish, and then can QA test everything  they coded and the answer is no, then the premise is that QA is a roadblock to getting items completed.  This could or could not be true but it really comes down to determining those factors.

• Is there a good cadence for story delivery or is coding taking up 8 days of the iteration leaving testing to the final 2 days.
• Are UAC's poorly defined or changing on a consistent basis causing the team to be unsure of what needs to be validated or changing the previously defined size of the story
• Are new priorities (support of a product) getting in the way of completing coding or testing
• Are stories coding being finished but no integration or unit testing happening thus have a high defect find rate blocking test completion.
• Are more than just testers reviewing tests, writing automation and focused on quality or is that still left up to the test resources
• Are testers asked to do items outside of testing stories, such as doing performance or support of previously developed stories, that is not accounted for in the team's tasking for the iteration
• Are developer unwilling or unable to allow testers to test in sandboxes
• Are developers unwilling or unable to hand off code in increments that can be tested
• Are their UAC blocking defects that block the continued testing for a story

All of these items can contribute to what looks like a QA bottleneck but in reality just point to different obstacles, or improvement areas for the team to focus on and fix.

I will argue that there cannot be a department bottleneck in a fixed resource Agile team, just a lack of planning, understanding velocity and utilization of every resource on the team working towards common goals and priorities.

Department Designations in Defining Tasks

So lately I have been battling the departments with breaking down the barriers and the historical definition of employees roles. Agile teams promote not only having different expertise on teams but allowing individuals to continue their career growth by taking on new challenges.

So a testing expert could pick up a coding task, a coding expert can pick up a documentation task or a documentation expert could pick up a testing task. If the team and the product owner are supportive, even in a team that has silo'd knowledge they can begin to break that down and begin to cross train each other and remove barriers to getting specific tasks completed.

I consider everything in a agile team development whether it be the code, the tests, the documentation, the training, the marketing material all of these items need to be developed, reviewed, and published and all of these go into having a potentially shipping product at the end of the iteration.

So in this light, the teams that I work with have gotten into the habit of labeling their tasks in the following manner:

• QA - test the system
• TP - document the system
• develop/write the code for the system

My argument to them is that QA is a department and as such this type of task designation associates this as not a team task but a dept task that does not or should not have any team member sign-up for the task. The only task items that do not have a department designations to them are the coding tasks.

Instead of designating departments I would like to see the tasks defined by what the action is, or even more preferably is to utilize the idea that task in Agile is a development task. Here is how I would define the tasks:

• develop the test, automation and execution to validate the system
• develop the documentation to enable a user to utlize the system
• develop the code to meet the UAC of the system

In this manner the task make no designation of who should do them, during the iteration planning meeting I would like to see the team discuss who is the best person to implement the task, and while the majority of the time that will fall to the department expert the team makes no assumptions that this will be the case and they sign up for tasks to make the team the most efficient and self organizing.

Agile Metrics - What Management Wants to Know!

A lot is made in the Agile world especially in XP and Scrum that executive level reporting should be focused on reporting that top priority stories are getting developed and can be shipped based on the product owner’s demands. And in theory this is a valid argument, however if you are taking several iterations to deploy/release the software, or your company has not fully gone Agile and it is only being used as a engineering process then you may need another way to measure.

Also as senior managers (and yes I consider scrum masters in this category) we also need to measure different factors to see if there are opportunities to improve. As a manager of scrum masters running multiple project teams all in different phases of development, I have been searching on a Pareto analysis that can measure different factors of agile teams to show our progress and overall value to the organization.

This can fly in the face of most agile practices but I am brainstorming and utlizing a formula to show story completion value (SCV) to any organization. Calculate the following for each month.

• Only measure a story when it has passed all User Acceptance Criteria and the team agrees it is done.
• Capture the priority (P) of the story. (P1=100 points, P2=75 points, P3=50 points, P4=25 points)
(Key Measure - Not enough emphasis on higher priority stories)
• Capture the number of iterations (I) that it took the team to achieve done & pass UAC for the story.
(Key Measure - Average number of iterations larger than 1
• Number of Story Points (D) (Difficulty)
(Key Measure - Stories are being estimated incorrectly, or stories are defined at the wrong granularity.)
• Number of Tasks(S) (Size)
(Key Measure - shows the amount of work needed to complete the story.)

SCV = (P/I)*(D+S)/100

For example a Priority 1 story is completed in 1 iteration achieving 100 points.
It had a small degree of difficulty - story points = 8 points
It also had a few tasks - 4 tasks

SCV = 24, if the same story was completed in 2 iterations it would have a SCV = 12

• This means that it’s value to the company was lower because it took twice as long to complete it and thus used the resources longer.

You calculate the SCV for each story that was completed in the month and trend the number for the entire organization. Once data is collected you can then notice trends and corrective action you can take within the agile project teams and review it during your scrum master brainstorming events.

For instance if your average iteration completion for a month is 2.1 then the scrum masters should work with the product owners to break down stories further, or brainstorm with the team on why it takes more than an iteration to complete a story. It could be that the UAC cannot be completed for some reason.

Other things to look for is a team that is creating too many tasks for a story which may indicated that the story is epic, or that a team is estimating the story points for every story the same or all of them extra large which may indicate that they need so guidance on estimation.

Remember the goal here is not to measure individuals or try to change the velocity of the team but to analyze some data points that can be represented to other departments, stakeholders and to understand if any corrective measures need to be taken to remove obstacles for the teams.