Organisations which have executed an Agile Transformation are often disappointed with the results. They typically adopt the Scrum Framework and create cross-functional teams with all the skills required to do the job.
Agile coaches and Scrum Masters outline the three pillars of Scrum, Transparency, Inspection and Adaption, and help everyone understand the various Scrum events, such as the Sprint, the Daily Scrum, Sprint Planning, Sprint Review and Sprint Retrospective.
The teams are mentored on the importance of continuous improvement and it is explained to them that each Scrum ceremony is an opportunity to Inspect and Adapt.
Despite all of these efforts, the delivery of business value seems to take much longer than expected and the organisation is not reaping the rewards or promises of the Agile Transformation.
The Scrum + CI/CD + Test Automation Combination
It is important to point out that Scrum is a management framework. It does not outline software engineering practices. A software organisation needs to combine the Scrum framework with software engineering best practices in order to achieve high performance.
This leads to a critical point regarding Agile Software Development. The team can only Inspect and Adapt in the Daily Scrum if they receive feedback regarding the previous day’s work. This is where Continuous Integration and Continuous Delivery, CI/CD, and DevOps come into the picture.
As outlined in The DevOps Handbook, DevOps is based upon three principles:
- The principles of Flow
- The principles of Feedback
- The principles of Continual Learning
The basic idea of CI/CD is to automate test cases as part of User Story development and include these test cases in the CI/CD pipeline. This enables fast and reliable delivery of all future software updates by executing fully automated testing of all updates. In other words, we fully automate the path to production and remove the need for manual approval and bureaucracy. This greatly improves the flow of business value from the development team to production and also reduces the feedback time.
No Continuous Integration or Continuous Delivery
The diagram below shows what happens within an organisation lacking CI/CD. This organisation is inherently anti-Agile as it places zero trust in the people who do the work.
There is extensive bureaucracy and approval required at every stage of the software delivery cycle.
Over time the code delivery process takes longer and longer as extra layers of approval are added. This labyrinth of bureaucracy is stifling to the creativity of development teams.
Eventually, organisations face the stark realisation that engaging in the delivery process is only worth the effort when delivering a large update. Tragically, this is the exact opposite of the regular delivery of small software updates which Agile promotes.
Typically organisations that work in this way struggle to retain the best employees and are extremely slow to deliver value to customers, even for small software changes.
CI but no CD
The next example outlines an organisation which has implemented a CI pipeline, but still suffers from the black hole of Manual testing and approval processes.
I state black hole as it is a good analogy for what happens to software delivery. Any business value/software entering the black hole will be very lucky to escape and achieve delivery in the end product or service.
Ultimately, this organisation still suffers severely from the lack of a Continuous Delivery pipeline and finds it very difficult to deliver value to Customers.
Full CI/CD Pipeline
The final example shows a simplified CI/CD pipeline which facilitates:
- the fast flow of business value/software from the developer to the end product,
- the fast feedback from automated testing performed in the pipeline to the developer,
- a culture of learning, as fast flow and fast feedback provide the environment to support experimentation.
Ideally, the complete pipeline would be as short as possible in terms of execution time. A pipeline which takes longer than 24 hours will negatively impact on team performance as the opportunity to Inspect and Adapt will be delayed.
Any issues due to the new software will result in a visible failure somewhere along the pipeline and automatically prevents delivery of the software. This allows the developer to perform rapid code changes and get super fast feedback.
Testing should be shifted left as much as possible within the pipeline. This means that the testing should occur as early as technically possible. This is done for two reasons. Firstly, it enables the fastest possible feedback in case of a failure. Secondly, the execution time of test cases generally increases along the pipeline from left to right and the Deployments used become larger. For that reason, these scarce resources should only be used if all earlier stages of testing have passed.
But we already do all that!
At the time of writing this blog, the DevOps movement, which includes CI/CD and Test automation, has become standard across the software industry.
This has led to a new and strange phenomenon by which organisations often genuinely believe they have a full CI/CD pipeline, but in reality, they have only implemented the CI pipeline with no CD or only partial CD.
One product component may have a full CI/CD pipeline while another component only has a CI pipeline. As you can probably guess, the product component with a CI/CD pipeline is typically of a higher quality and supports fast delivery of new features, bug fixes and code refactoring, while the product component with only a CI pipeline typically has quality issues and is difficult to update.
Summary – why continuous integration is important for agile
Agile is built on the expectation that developers can deliver small incremental updates to a product or service. In practice, this will only be achieved if an organisation has put the time and effort into developing and maintaining a CI/CD Pipeline for their products and services.
Anything less will result in extensive delays and a lack of Agility.
By James Langan
Agile Practices Manager | Ammeon
- The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations. Authors: Gene Kim, Patrick Debois, John Willis, Jez Humble.
Discover the Benefits of Agile
Our large team of Agile experts have a wealth of experience guiding critical transformation projects in some of the most challenging sectors. We will provide you with a unique insight into how your people, processes and technology can work together for positive change.