Play book for PratiCing DevTestOps
I. Introduction
Quality is not an act, it’s a habit. ~ Aristotle
II. Continuous Testing & DevOps
Previously, there was a popular misconception about DevOps, that bringing development and operations teams together would result in greater efficiency, eliminating the very need for testing. However, it was quickly realized that this was not the case. On the contrary, DevOps transformed the role of testing from that of a gatekeeper to a guardian of quality across the lifecycle. This transition eliminates testing silos and instead embeds it in each stage of the DevOps process to deliver much needed quality at speed.
Testing in DevOps, shifts left towards Development, by helping the Dev team embrace testing to run quality checks during the development process itself, often before they start coding. At the same time, Testing in DevOps also shifts right towards Operations, allowing for incidents in production to be detected faster and before they adversely affect the end-user experience.
a. Pre-Build QA
The cost of fixing error grows exponentially as it proceeds undetected through SDLC. Pre-build QA helps in detecting issues early on and prevent the cost of software errors. Especially from DevOps perspective. It paves the way for early testing bringing the required agility in the QA cycles and speeding up the time to market
b. Post-Build QA
Post-Build Quality Assurance includes a round of detailed build hardening and build verification testing, before the build is deployed. The build deployment is followed by a quick round Configuration and Smoke testing, to get a rapid feedback on the readiness of that build, for elaborate testing cycles in terms of its availability and accessibility.
The idea is to raise quick alarms with the development (or) environment teams in case of any showstoppers for testing (or) to give a green signal to QA teams to proceed with subsequent test executions. This practice potentially saves significant effort waste that testing teams otherwise would have spent on the poor-quality builds and deployments, just to re-execute them again with a new build.
Additionally, this is also the stage where test data is provisioned into the test environment. Along with the environment stability, test data is another critical factor for successful test execution with higher pass rate and minimal false negatives.
c. Functional Assurance
Testing in DevOps essentially means testing early and testing often. Thus, it is implicit that functional automation must go beyond the traditional UI based tests, to leverage other architectural components such as data and services. This speeds up test execution and helps with shift left as now you can validate these components as soon as they are developed, rather than waiting for the Development team to deliver a working UI.
d. Experience Assurance
In DevOps, it is crucial to shift non-functional testing to left to provide faster feedbacks on potential performance bottlenecks, security vulnerabilities and accessibility issues. The pipeline moves through different stages of testing and generates reports on non-functional defects, addresses test incidents and confirms the overall state of experience delivered by the application.
e. End to End Execution
The idea is to demonstrate how the continuous pipelines will be executed in real time progressing the code through various quality gates and ensuring that every aspect of a code change is tested and confirmed before it is certified for pre-production environments.
