Quality is about perspective and lies in the core values, of people, of companies. It has grown from a concept of additional overhead to and imminent pillar in the development process. With thousands of dollars being invested and costs rising, it challenges how the code and test coverage needs to be carried out. The answer lies in optimal automation. Open source tools like Selenium have been the go-to tool for automated testing for years and they are only becoming more popular. These tools can be used with, and improve the performance, of Agile methodologies and Devops to bring about a faster release while ensuring product quality. More than that, because Selenium is free, it dramatically reduces costs while simultaneously granting access to a community of experts that can support the process. It is clear that as the need for automation and smart analytics continues to grow, new tools and platforms will arise that build upon the current foundation. These developments can produce major benefits to the quality of the software but also improve the overall function of a business. By automating the testing process, developers are free to focus on writing code which will improve their productivity and likely improve their code. And because this testing can be done simultaneously and continuously, it means the software is market ready in a shorter period of time. Time is indeed money and more time saved means more money saved. Rapid deployment of new or updated software also allows businesses to keep a competitive edge, no matter the industry
Using a free and an open source framework like Selenium requires specific scripting expertise, hence they may not always be the best option. Developing intelligence into such a framework requires a lot of man-hours making it quite expensive in the long run. This is where the use of Machine Learning for automation testing comes in where the goal is to dynamically write new test cases based on user interactions by data-mining their logs and their behavior on the application/service. Thus Machine Learning in “Test Automation” can help prevent some of the following cases:
- Easy modification and maintenance of Test cases
- Saving on Manual Labor of writing test cases
- Sometimes Test cases are brittle, so when something goes wrong, a framework is most likely to either drop the testing at that point or to skip some steps, which may result in a wrong/failed result.
- Tests are not validated until and unless that test is run. So, if a script is written to check for an “OK” button, then we wouldn’t know about its existence until we run the test.
The machine can help recover tests on the fly by applying fuzzy matching, which means if an object gets modified or removed, then the program and the script must be able to find the closest object to the one it was looking for and then continue the test. For example, if a web service has the options, “small, medium, large” at first, and the script was written according to that, and if another choice, i.e. “extra-large,” is added, then the script must be able to adapt to that and anticipate that change so that the test run can continue running without fail.
In case of Regression testing, a suite of test cases needs to be developed. When we write test cases, we test how the software is supposed to behave theoretically, and there’s no real data with us, so some of the test cases might never be used in real life and some that missed the test cases might be the most important ones. That is why data-mining the logs and letting the machine write test cases according to those logs automatically saves a lot of man-hours and helps in practical testing. Services like HockeyApp and TestFlight are providing automated mobile app testing as a service.
As for GUI tests, there are some research papers out there that talk about deep learning and reinforcement learning for automation of the test. The systems that were being tested were first data-mined to get the meaningful clicks, texts, and button pushes on the GUI interface, which generated a good amount of training data. That data was then used to perform tests on the software for a few hours. The best part was that there was no need for models or test cases to be written and the bugs were being found as the time passed by, but some of the cases were not being tested which can be due to lack of training data. The reinforcement approach improved the testing as they were running through multiple iterations.
Many companies are investing heavily in Deep Learning and related algorithms to achieve results more quickly. Moving from a mobile-first world to an AI first world. We know that for testing a certain product, there can never be enough of the right test cases and that is why developers and testers are encouraged to write more and more test cases in order to make their product more stable.
How AI can help automated testing Software teams are under constant pressure to deliver better quality products in ever-shorter timeframes. To do that, testing has shifted both left and right, and the automation of tests has become critical. However, in the meantime, traditional test automation has become a bottleneck. Over the past several years, we’ve told testers you have to become more technical, you have to learn how to code. They’ve now become Selenium coders, and that’s really not the best use of their time. If you’re an enterprise, you want to take those expensive resources and have them developing products, not just test cases. Rather than having engineers write scripts, there are now solutions that can automatically generate them. When we talk about test automation today, for the most part, we are really talking about the automated ‘execution’ of tests. We’re not we’re not talking about the automated ’creation’ of tests,” said Joachim Hershmann, research director at Gartner.
One approach is to provide an autonomous testing solution with a test case written in a natural language and it will autonomously create the test scripts, test cases, and test data. The autonomous nature of the system frees up testers to do other things such as explore new technologies, advocate for the customer or line of business, as well as be more influential and strategic, said Voke’s Lanowitz.
Web and mobile app developers have options that help ensure the user experience is as expected.
The benefit of machine learning is pattern identification. In the case of automated testing, that means, given the right training, it is able to distinguish between a failed test and a past test, although there are other interesting possibilities.
“It could be used to understand which tests should be run based on a change that was made or risk of going into production with a particular release,” said Gartner’s Murphy. “This is where the more that people use cloud-based tools that allow them to run analytics across anonymized data, you can start looking for patterns and trends to help people to understand what to focus on, what to do. We’re in the early phase of this.”
It’s a mistake to underestimate the dynamic nature of machine learning, because it’s a continuous process as opposed to an event. Common goals are to teach the system something new and improve the accuracy of outcomes, both of which are based on data. For example, to understand what a test failure looks like, the system has to understand what a test pass looks like. Every time a test is run, new data is generated. Every time new code is generated, new data is generated. The reason some vendors are able to provide users with fast results is because the system is not just using the user’s data, it’s comparing what the user provided with massive amounts of relevant, aggregated data.
Three or four years ago, Google said that their code base then was like 100 million lines and it’s well past that now. Every day, that code base is growing linearly and so is their test code base, so that means that test execution is growing exponentially and at some point it’s no longer affordable,” said Gartner’s Murphy. “They built tools to determine which tests need to be fixed or thrown out, which tests are of no value anymore, what tests should be run based on what changes have been checked into [a] build. These things are what organizations have to look at and now you’re seeing other companies other than Google do this.
People have to be ready for this mode of more autonomous, more artificially intelligent, guided solutions within their organization to be ready to embrace it. If you’re currently an automated test engineer, this is an opportunity to increase your skill set because what you’re going to be doing is not just using a tool, you’re going to be training that software that fits within the scope of what you want to do in your own organization.
Intelligent Test Automation Tools AI and machine-assisted automated testing tools are relatively new. The only way to understand exactly what they do and how their capabilities can benefit your organization is to try them. Following are five of the early contenders:
Applitools Eyes is an automated visual AI testing platform targeted at test automation engineers, DevOps and front-end developers who want to ensure their mobile, web and native apps look right, feel right and deliver the intended user experience.
AutonomIQ is an autonomous platform that automates the entire testing life cycle from test case creation to impact analysis. It accelerates the generation of test cases, data and scripts. It also self-corrects tests assets automatically to avoid false positives and script issues.
Functionize is an autonomous cloud testing platform that accelerates test creation and executes thousands of tests in minutes. It also enables autonomous test maintenance.
Mabl is machine learning-driven test automation for web apps that simplifies the creation of automated tests. It also identifies regressions and automatically maintains tests.
Parasoft SOAtest API testing is not a new product. However, the latest release introduces AI to convert manual UI tests into automated, script-less API tests.