Auditing Agile Projects
Using the The Big Agile Toolkit, you deliver ALL your projects to time and to budget. Therefore the audit will not need to address issues of alignment
to time and budget. In the instances where significant changes have taken place to the project plans, budget and scope the audit will need to
forensically address the planned time, planned budget and the planned scope of delivery. A review of the project using a workshop to uncover the
successes, failures and changes (in the format of the SCRUM retrospective) will develop and explore the issues surrounding these changes. The impact
of these changes and their effect on the planned time, planned budget and the planned scope of delivery will be explored and documented. Now, before
we get underway with the audit of agile projects, let us clarify some terms.
An audit is an inspection of records and the activity that these records evidence. They are conducted in financial, government and many other types of organisations all the time.
An agile audit is an audit carried out using agile techniques. The objective is to run an individual audit or an audit section in an agile fashion. This is to allow the audit or the
auditing section to respond swiftly, deliver audit agility and deliver tighter audit outcomes using less resources. The agile audit approach considers audit risk and seeks to concentrate on
very tight audit scope, on the organisational high risks only and on the right (timely and fit for purpose) level of audit output.
An audit of an agile project is an examination of a project that is operating or purports to be operating in an agile manner. This form of audit will traditionally perform a health check,
examine performance of and alignment to the agile process or may seek to uncover specific project problems either associated with or not associated with the agile delivery method.
We look at the last of these examples: auditing agile projects. Auditing is not a word normally associated with agile projects. We review and coach agile projects rather than strictly audit
them. It is a case of fitting semantics to the environment. However, we deal with lots of big organisations and they like to use the word audit. It is recognised as a review process of sorts.
Bear with the title, if you can for now, but consider the environment in which this review (or audit) takes place. In a long standing SCRUM environment, the word Retrospective will have
more positive response than the term audit. With this important proviso established, let us get underway.
Traditionally, an audit team reviews documentation, organisational output and activity from the past.
The audit considers financial accounts, personnel records, performance statistics and so on.
Auditing Agile Projects - Audit Data
When we conduct an audit of an agile project, we are interested in data at four levels.
- Development. This is an audit carried out examine the development process used on the project, to ensure it is planned using agile planning, to ensure it is using continuous
integration, to see that changes are communicated across the team or teams, that environments are appropriate and available in a timely fashion. Look also at rework following redesign and
after bug fixing. Look at the process to get a customer or business change into an assessment. How fast is it? What are the barriers and the points where the project fails to perform.
Auditing Agile Development is a book in its own right and some have been written already. We will stop here, for now.
Design. This is an audit carried out primarily to ensure there IS a design process being used on the project, to ensure there is not programming hacking without design
considerations, to ensure the design is shared, that it is performed in an agile fashion, that change is being performed, that change is being welcomed and encouraged where necessary and
that the daily meetings record some element of the design changes sufficient for audit. This could be simply that a named specific change was made to the design on a particular day and
that this took an extra number of recorded hours to resolve, finalise and restore.
Management. This is an audit carried out examine the management of the project, to ensure it is being delivered using an agreed agile approach, to see if commitments are being
examined, daily meetings taking place, assessments being performed, teams engaged, that the manager is regularly examining the team, that coaching is taking place, that all the
management stakeholders are in place, communicating their renewed commitment and that the team is delivering and delivering in a high performance manner.
Process. This is an audit carried out examine the alignment of the project to an agile approach and to understand if the commitments made at the outset are being maintained. It is
also used to examine how well the agile approach is improving the performance of the project delivery environment and therefore the organisation. We debate this further in the section on
As with all audits, we use data from previous events, from past experiences on the project and via feedback from the team members. Auditing Agile projects has only one significant difference. It
looks at past data, sure. However, because a successful agile project is running at high performance, circumstances change swiftly and documentation gets out of date. So we try to
concentrate effort on examining the current activities: the now events. Now we must look at the approaches to auditing agile projects.
Auditing Agile Projects - Objectives
There are a number of approaches and the decision on the type of approach to adopt, in our experience, depends on what your objectives are. Your objective, in all cases, is to have an open
discussion without hindrance and without outside influence on any of the participants. There must be an openness on the part of the organisation to commit to the audit process and to share
views openly without feeling hampered by dogma or other outside influences. We like to call it a Drains Up Review. The analogy it tries to portray is that that your organisation is
prepared to open its processes up for detailed examination and is so prepared to have all its grimiest secrets revealed that even the drains can be opened to see what fell through. You seek
total openness and this can sometimes demand the involvement of a senior executive within your organisation to instigate the message and communication it clearly and candidly.
Regardless of your audit pre planning work, if you feel any level of organisational or management comeback, oppression, teasing or at worst persecution of the participants will take place,
you may need to be an element of anonymity for participants. If this happens, and it does in some rather edgy audits, it is incumbent upon the audit team to deal with it. First of all, seek
anonymity for the participants, advice and formal communication from management. be prepared to evidence any comebacks to your reporting line, to the customer if you are in a supplier customer
arrangement. Finally, be prepared to share frank discussions and evidence with those who partake positively, and negatively, during the audit.
Auditing Agile Projects - Approaches
In all instance also, we need to have open questioning and active listening performed throughout. There are sections on these specific topics elsewhere in the Toolkit. Now, assuming we have a
motivated audit audience, we need to determine our approach. There are four, well at least four, approaches to choose from.
One to One Audit Interview. This is really self explanatory. You arrange and conduct one to one interviews with as many audit candidates
as you can. Once more, it depends on how much pre planning you can perform before the audit to examine and target the area you need to understand and interview first. In your
audit of agile projects you can still introduce agile techniques by concentrating on the priority risks and the priority groups.
Group Audit Interview and Assessment. This is a combined audit interview and monitoring assessment with small domain based groups. It involves selecting a particular group or groups
that you feel need further examination as part of the audit. This is where the topics of auditing agile projects and agile coaching merge slightly. In this instance, you will have managed
to target a specific part of the project
or the organisation that demands further scrutiny. First of all take the group, or parts of the group if it is large, through a small review. Again, ask open questions such as "How is it
going?". Go round the room, facilitate the discussion and gradually open the audience and the debate. Record the outcomes and share the results (consider anonymity of these results) with
your reporting line or customer.
Audience Audit. This is an open audit carried out with all the team members, all management of the project, key business stakeholders and is designed to be an open debate with all
stakeholders in the form of an agile retrospective to examine successes and failures, elicit feedback and suggestions for change in the agile process and behaviours.
The important element here is to ensure you can bring up all the issues that the team needs to debate. You must be certain that you can at least have frank and open discussion without
outside influence. You cannot, of course, pre-empt the outcomes of the discussions and whether these will be supportive, questioning or outright controversial.
They are often great lively
debates to take part in and to manage. However, as the openness of the discussion favours the more vocal participants, it demands the intervention of the facilitator to promote the voice of
the non vocal or less vocal participants. This is a great method to gain feedback on the approach certainly. However, this is also a superb way to have an open forum in which to gauge
reaction to the overall agile process and begin to examine opportunities and difficulties in expanding adoption of the process and the next projects to utilise an agile approach.
Combination of the above. This is likely to involve a more comprehensive audit but invariably it is more costly to perform. If you elect this approach, our suggestion is to perform
an audience audit first of all. Examine the results with the audit team and decide if selected groups can be taken for further audit. Consider if key individuals need to be personally
interviewed if they had an impact on agile outcomes and if they need to be part of the post audit repair process.
Auditing Agile Projects - Plan of Repair
In all audit instances, we need to deliver some form of output to evidence the audit and communicate the next steps. This should take the form of a plan of action. Use the four levels of
audit data as a starting point: Development, Design, Management and Process. Select each level in turn and explain the positives and negatives. Pay particular attention to the commitments and
alignment, or misalignment, of the project to the agile process. Take each misalignment
and negative in turn and describe the repair. Detail how the agile team and coach will pick up
each failure and turn it into an aligned and working agile process.