Today, software development and quality teams need to automate aspects of non-functional testing areas like security and performance as part of their continuous testing—a key component of continuous integration and continuous delivery practices.
Failing to automate these testing functions will put you behind your competition and make it difficult to scale along with increasing testing demands resulting from the exponential growth of mobile and the Internet of Things (IoT). If you haven’t added automated performance and security tests into your continuous integration paradigm, especially in regards to mobile, read on so I can convince you to get started.
Mobile dominates our times. Analyst Benedict Evans’ fantastic presentation “Mobile Is Eating The World“ articulates how mobile has changed the world. Like speech, the written word, television, and the Internet were before it—mobile is the new way and today’s “…centre for investment and innovation.” Mobile won and at this point it’s just a given that you need to make optimizing your technical delivery for mobile your highest priority.
The mass automation of testing and deployment makes up the core of continuous delivery. Engineers write more tests up front as test-driven development (TDD) and behavior-driven development (BDD) enable speed while maintaining, if not raising, the quality bar. Continuous delivery works, and you need to understand its benefits.
By automating testing, you build a self-diagnosing system. The software will tell you immediately if you broke it. Engineers get instantaneous feedback while they’re writing the code instead of weeks or months later. Automating testing and placing it earlier in the build process results in faster and cheaper bug fixes, period.
Automate more than functional testing
Most normal automation strategies have not historically included non-functional test areas. Performance and security testing typically occurred later in the release cycle due to their complexities and the overhead involved. You needed to hire specialists, buy specialized tools, and sometimes hire outside vendors to do it for you.
Worse yet, because tests occur later in the app development cycle, fixing the inevitable bugs that arise are more difficult and expensive. Legacy testing workflows create delays between the availability of test results and when engineers last worked on their code. I hear from my peers and people I talk to at conferences that this is still the predominant scenario.
Organizations typically have unit, integration, and system tests wired up, along with some amount of functional tests. What has hindered the adoption of non-functional tests in continuous integration environments was a lack of mature tools. Fortunately, a number of tools have recently come online that make the automation of security and performance testing possible. Yay technology!
Automate mobile app security testing to avoid technical debt
Over the last couple decades, security testing always challenged my teams. We needed a specialist familiar with the security landscape for our given field and capable of articulating test plans to cover the risks. We performed most security testing manually, beyond limited coverage provided by static code analysis tools. When I managed quality at FIS, the nation’s largest financial services provider, we had well over 3,000 mobile banking apps in market used by more than 35 million people. We hired specialists, procured static analysis tools, and contracted with outside specialists to assess our apps.
Fast-forward to today—mobile security testing tools now exist that directly hook-up to continuous integration systems. These tools assess apps on real devices and provide accurate baseline test results on demand. Working with a recent client, we added automated performance and security “baseline” tests that ran in parallel with other continuous integration-based tests. We didn’t configure those tests to run at every check-in, but we did run security and performance tests daily (a 100 percent improvement over what I produced with my prior team). In the end, we achieved continuous testing of security and performance. That’s HUGE!
If the tools, methods, and successful use cases exist, and you ignore them, you may be contributing to avoidable technical debt for your organization. Better living through robotics is what I say!
The Internet of Things (IoT) increases testing workload
A report published by Cisco and DHL predicts that by 2020 more than 50 billion devices will be connected to the Internet. The report also forecasts eight trillion dollars in value generated by the IoT. In a separate study, Cisco forecasts that by 2019 global IP traffic will exceed 124 zettabytes annually (one zettabyte equals one trillion gigabytes) compared to 3.4 zettabytes per year in 2014.
We already see some of the effects of the IoT today—demands for more data storage, higher throughput, and faster processing speeds, not to mention increased pressure on APIs and services. These demands will only grow with more devices coming online.
One major IoT security concern involves preventing potential stress fractures due to increased load versus targeted attacks on specific IoT devices. The real IoT kicker is the coming flood of connected devices and what that means for your apps: more network connections, more services activity, more processing, more devices to manage, more data to manipulate and store, more traffic, and so on and so forth. A significant increase in testing workload is coming.
Prepare for tomorrow, today
Automate as much as you can now to put as much of your current delivery flow as possible on auto-pilot. Taking that step will afford you the time and awareness to focus on net-new friction points driven by continued mobile growth, continuous delivery’s impact on speed to market, and new scalability pressures created by the IoT.
Yes, getting your testing process right, even with a legacy approach, is a challenge. Today, however, you need to both get it right and speed it up. A large number of enterprises all around you are successfully operating continuous integration, continuous delivery and continuous testing initiatives. You need automation to maintain your position in the market, increase team efficiency, and deliver your apps to market quickly. The best part is that even the mere pursuit of continuous delivery and continuous integration will MAKE YOU BETTER (and I speak from experience)! Testing earlier and continually is cheaper and will soon become the de facto standard. Get those robots built and let them work for you!
