DevTestOps

Continuous Testing is the key to your DevOps

Continuous testing is a key element in successfully implementation of DevOps pipeline. Continuous testing, which is often called shift-left testing, is an approach to software and system testing in which testing is performed earlier in the software lifecycle, with the goal of increasing quality, shortening long test cycles and reducing the possibility of software defects making their way into production code. This concept of continuous testing can be achieved through progressive test automation as well.

One way that DevOps teams do Continuous Testing is by shifting integration testing to the left of its usual position in the delivery pipeline so that it occurs as close as possible to the build process and this shift left happens through progressive test automation scripts, which means your testing stages start in parallel to development stage. TDD and BDD are good example of doing progressing testing.

The goal of most DevOps projects is to automate as many manual processes in the software development process as possible. Some roadblocks in a DevOps pipeline that lead to slow deployment include error-prone manual processes such as handoffs from a development group to a QA group, including ones that require signatures or bureaucratic approval. These kinds of handoffs mean there is a lack of shared ownership of the end product, which is contrary to the basic Agile testing and development methodology that says all members of a cross-functional Agile team are equally responsible for the quality of the product or the success of the project. Because of this, testing on an agile project is done by the whole team, not just designated testers or quality assurance professionals, including team members whose primary expertise may be in programming, business analysis, database or system operation.

Continuous Testing is one of several continuous activities that should take place simultaneously in a DevOps pipeline, including

• Continuous Build.
• Continuous Integration (CI).
• Continuous Delivery (CD).
• Continuous Deployment.

Continuous build or build automation is the first stage in implementing a Continuous Delivery DevOps pipeline. If your developers are practicing test-driven development (TDD), they’ll write unit tests for each piece of code they write, even before the code itself is written. An important part of the agile methodology, TDD helps developers think through the desired behavior of each unit of software they’re building, including inputs, outputs, and error conditions. New features implemented by developers are then checked into a central code base prior to the software build, which compiles the source code into binary code.

Continuous Integration is a practice where members of a software development team use a version control system and integrate their work frequently to the same location, such as a master branch. Each change is built and verified by means of tests and other verifications in order to detect any integration errors as quickly as possible; which may include your BDD scripts. With build automation, the software build happens automatically, using tools such as Gradle or Maven, rather than when a developer manually invokes the complier.

In the last stage of a Continuous Delivery pipeline, once an application passes all the required tests, it’s then released into production. For all intents and purposes, this means releasing every good build to users. A fully automated CD pipeline requires the ability to deploy and release any version of a software application to any environment.

Another aspect which can be added in Continuous Testing is to pull Non-Functional testing within your CI/CD pipeline scope. This practice can help achieve proper Shift-Left. ie. performing performance, security, accessibility testing within your pipeline scope.

DevTestOps = DevOps + Continuous Testing

Patrick Dubois, who coined the term DevOps, said in a recent interview (How Dev vs. Ops Became DevOps”) that he was inspired to promote increased collaboration between developers and operations because he felt, as a developer, “if you’re really passionate about what you’re building…you do feel responsible about how your application is doing in production.” In the past decade, DevOps has lead to a cultural shift in the way many organizations deliver software applications.

These days, however, most organizations have realized that DevOps without continuous testing leads to the quicker deployment of buggy software into the production environment, which is why the term DevTestOps is gaining currency as a way to describe the culture of collaboration needed among all of the teams involved in software development (developers, operations, quality assurance, business analysts, management, etc.) in order to speed up the delivery of high-quality software.

Building a successful Continuous Delivery pipeline means reducing the cost, time, and risk of delivering software changes by allowing for more incremental updates to applications in production. As your CD pipeline becomes automated, the obvious next step is to incorporate more quality control steps into the pipeline process, either through test-first approaches — such as test-driven development (TDD), acceptance test-driven development (ATDD), and behavior-driven development (BDD) — or by continually running automated scripts to test the new features.

As your software releases increase in frequency, the suite of automated test scripts — including those that do the performance, load, spike and soak tests described above — can be stored and re-used in future release cycles. These scripts can also be incorporated into test automation frameworks and used to reassure project stakeholders that thorough and adequate testing has been successfully completed.

Tools & Frameworks which help in Continuous Testing

• Cucumber
• JBehave
• FirNesse
• TestLeft
• SpecFlow
• BeanSpec
• Concordion
• JDave

References: https://www.getzephyr.com/insights/devops-implementation-steps-continuous-successful-deployment