Continue the Conversation. Reach Stephanie or Mike at groups@coveros.com or call 929.777.8102.
Stephanie Fender: Hey everyone, my name is Stephanie Fender, and I work for the Coveros team. I’ve worked for Coveros and TechWell for over ten years. I’m a client advocate for on-site training and consulting engagements.
Mike Sowers: Hey everyone I’m Mike Sowers, and I’ve been with Coveros for 6 years, and I lead our training business. I do some training myself and consulting, and I also help with the programming of our conferences. Great to have you with us today.
Where to Start with Automation
SF: So I hear from a lot of clients that are new to automation or their company is new to automation. Do you have any suggestions for these newbies?
MS: Yeah, a few things, Stephanie. First, I always like to say that automation is not a silver bullet – it’s necessary, but it’s not sufficient. And then second, like any great project that we want to implement, we need to understand where we are today and then decide where we’d like to be and then how we’re going to get there. So things like conducting a gap analysis, identifying the key pain points, and key opportunities, and then establishing a vision and a roadmap to get there are important. And then third, like any great thing that we build, we really need some kind of an architecture, like a picture, that speaks to how different automation capabilities work together to support all the workflows.
Refining Your Automation
SF: So what about the folks that are further along in their automation journey that need help with refinement. I hear questions about that a lot. Do you have any suggestions for those folks?
MS: Great question. You know a key tenet of agile and DevOps is this whole idea of continuous – continuous development, continuous testing, continuous delivery, continuous deployment, but also continuous learning. So for those teams and organizations that are more mature and experienced with automation, simply asking the “what’s next” question can help.
For example, if a team has mastered the automation of unit testing, maybe the next opportunity is to drive further left and apply some static analysis – things like code scans or security scans, open source library usage scans, or complexity analysis, and so forth. Or if a team is consistently and effectively deploying changes from their local development environment into their DIT – their developer integration and test environment – then maybe the next step is to extend their pipeline into the systems integration and test environment, or the SIT environment.
I think the key is conducting regular assessments of your capability and your tooling and your processes, identifying the next opportunities, and putting a plan in place to continually close those gaps. As you know, we’ve worked with several organizations – we’re working with an organization right now which is a financial services company – which is following this approach. We’ve helped them do an assessment of their current state to help them identify gaps, help them to lay out a roadmap, and then we’re helping them to close those gaps. Now we’re assisting them with some hands-on implementation of their CI and CD pipeline.
Who Should Be Involved in Automation?
SF: So paint the community a picture, Mike, of who should be involved with automation on a team.
MS: Well of course, the right answer is any key stakeholder, Stephanie. But the most important thing is to identify who the key stakeholders are. Team members vary based on the category of automation, as I mentioned before. If the focus is automating the CI/CD pipeline, then roles such as architects, developers, release engineers, change or configuration management team members, DevOps engineers, operations team members, infrastructure engineers, security team members, compliance. Lots of roles there that we want to think about as key stakeholders.
From a test automation perspective, roles such as the test architect, the test designer, or test execution engineer. Of course, the test automator, developers themselves, again change management, configuration management, and then even non-functional testing specialists, such as performance testers, usability engineers, and so forth. And I always like to remind teams that automation is not testing – automation really supports testing. So key stakeholders really want to be involved up front to decide what the important testing is to do, and then to have automation support that.
Open-Source or Commercial Tools?
SF: I also get tons of questions about tools. Tools, tools, tools. So this is a two-part question: When should the client choose open source versus commercial, and also, do we help create set up their product lines?
MS: Yeah, I think you hit the nail on the head with those two questions. Indeed there are thousands of tool choices these days, both open source and commercial. Both of those are good choices or even a mix of those two are good choices. Open source comes with some benefits, challenges, and constraints, as do the commercial tool sets. For me the key is the team, the organization, the enterprise’s requirements – and so we must treat the selection and implementation of tools just like a well-run project. This means that we’ve got to first define our tool requirements, and then evaluate the available tools that are available to us in the marketplace.
Usually when we work with clients, we recommend a multi-step approach:
- Conduct a gap analysis.
- Specify tool requirements.
- Survey the market of available open source and commercial tools.
- Attend or engage in demos of tools.
- Pilot the tool on your projects, using your processes.
And that last point is one I can’t amplify or emphasize enough. It’s really important to bring a tool in-house and try it out on your projects and your processes because that’s when you find the deficiencies. Then simply, next, select the tool that best fits your requirements and then continually refine that tool and the methodology that supports it. You know as I mentioned earlier, I think having a picture or an architecture or a vision of how all these tools fit together is also important and useful.
How Have You Seen Automation Evolve?
SF: The last question for you, Mike. Let’s take a trip down memory lane – in all your years of coaching and consulting, how have you seen automation evolve?
MS: Did you just ask how old I was? 🙂 Actually my first automation project was testing the reliability of a point of sale keyboard. In it in that project I used an air-driven fixture that was programmed with punch cards, believe it or not. So that gives us some insight to how long I’ve been in the software engineering business.
But seriously from a testing perspective, we initially tried to automate using record playback, and we did that mostly for regression tests. Then we discovered that we could create automation using scripts and functions. These were really blocks of automation functionality and reuse them. And then next we discovered that it was beneficial to separate the data from the scripts, thus we could reuse the scripts with different data sets so we can do different types of testing but use the same test infrastructure.
Next, we moved to something that was called action or keyword-driven testing, which essentially allowed our scripts to be referenced with simple keywords or a language of the business workflow. Things like “check balance” or “log in” are examples of keywords. Then we moved to coatless or point and click, or drag and drop, automation. And now of course, we’re seeing the application of a AI in test automation.
From a from a code movement perspective, you know automating the movement of code from the developers and local environment through to production, that is now rapidly being adopted as we extend agile and DevOps. So here too we’ve started with some less sophisticated tools, and now existing tools are getting better and better, and even new tools continue to emerge. So tools like Docker, Jenkins, Chef, Kubernetes, Ancible – maybe you’ve heard some of these names – help us wire together the movement of code.
Of course again, it’s important to assess where we are, identify the opportunities, create some type of implementation roadmap, and then understand how our existing or our new tools are going to integrate with our tool architecture. And then just continually learn and continually build out our pipeline.
Automation Learning and Coaching Opportunities
SF: Well those are all my questions for you today, Mike. Thank you so much. I wanted to let the community know that we’ve got a few upcoming Live Virtual classes that focus on automation. The first is the fundamentals of automation in April, and then an agile test automation course in May. If you prefer an instructor-led, on-site course at your company, we can also offer those courses to you in that form too.
MS: Great to be with you today, Stephanie, and thanks everyone out there for being with us and listening in on our blog. I just wanted to lift up our conferences. We have six conferences a year – three of those are focused on software testing and test automation, methodology, and leadership, and another three are focused on agile and DevOps – so look for those on our TechWell.com web site. And as always, we’re here to assist you with both mentoring and coaching, as well as consulting, if you need us.
