- November 20, 2019
- Posted by: Sagar
- Category: Blogs
What’s the cost of poor software quality?
- The total cost of poor-quality software was $2.84 trillion for the United States in 2018.
- Research from Cambridge University’s Judge Business School showed organizations spending more than $300 billion debugging software.
- A 2015 Parasoft research shows that the stock price for companies “experiencing a news-making software failure fell about 4% on average in 2015 and 5.7% when they suffered multiple failures” costing $2.7 billion in lost market capitalization.
There’s no doubt that poor quality software will hit home hard.
Given that software quality can have such a massive financial impact, building a comprehensive software testing strategy must be a business priority. And a comprehensive strategy includes a sharper focus on non-functional testing as well.
The ABCs of Non-Functional Testing
So, what is non-functional testing? What are non-functional testing types? While buying a car, for example, you would check how smoothly the car runs, the comfort of the driving experience, do the internal functions operate as intended, the airbags deploy, etc. Non-functional testing would consider if the car performance is optimal, that it can handle the stress to go from 0 to 40Kmph fast enough, the security features kick in when needed, etc.
Much in the same way, in software testing, functional testing calls for testing the applications against the business requirements and verifying that the software is fit for release. Non-functional testing checks the non-functional aspects of the software such as performance, usability, reliability, etc. It is explicitly designed to cover the aspects of testing that functional testing does not address. Non-functional testing validates the performance of the application, is focused on customer expectations, and concentrates on performance specifications and its influencers.
So, what are the five kinds of non-functional testing that every application must go through?
Performance Testing
In a performance crazy world, it is mandatory to ensure that your application performance is never under par. For this, you need to be sure that your application is working smoothly and perfectly under expected workloads. Performance testing helps you assess if your application is capable of shouldering its performance-related tasks. It tests the application under different load conditions and checks the response times. It helps eliminate all performance bottlenecks from the application and ensures that the application meets client expectations when put to the test in the real world. Performance testing also measures the average response time of the application and identifies the areas and issues that impact the application performance.
Load and Stress Testing
Load testing and stress testing are two very important testing areas to focus on to drive application performance. Load and volume testing test the application usage scenarios under various amounts of load. This test does not aim to break the system. It’s all about testing the application with the actual workload.
Stress tests determine how the application responds when it is pulled to capacity. In the context of apps, the Black Friday sale volume is a good example of how loads and stress can spike periodically. One of the objectives of stress tests is to determine the limit at which the system, software or hardware breaks. It also shows if the application effectively demonstrates error management under such extreme conditions. For example, if you copy 5GB data from a website and paste it on a notepad the notepad is ‘stressed’ and may give the error message “Not Responded”.
Usability Testing
How well will end-users be able to use your application? Is it easy to navigate? What is the learning curve for the end-user? Usability testing answers these and many more such questions. Usability testing aims to identify any and all usability related problems and gather all qualitative and quantitative data to assess the likely satisfaction levels of the user.
Satisfaction is hard to measure, and this makes usability testing hard. However, it can be evaluated in light of the skills needed to learn how to navigate and use the application, the time needed to get used to the software, the measure of the likely productivity increase, and an objective assessment of the user’s attitude towards the software. Usability testing is especially important when you test GUI and it also evaluates error diagnosis and reportage while the software is in operation.
Reliability Testing
Applications today must be failure-free and reliable. And that must be a key testing focus too. Reliability testing checks that the software application can operate failure-free for a specified period of time and in a specific environment.
The objectives of reliability testing are to:
- Identify the perpetual structure of repetitive failures
- Identify the number of failures occurring in a specific time frame
- Discover the cause of these failures
To measure the efficiency of your reliability testing you will usually measure the mean time between failures, the mean time to failure, and the mean time to fix the failure.
Compatibility Testing
Given the sheer breadth of relationships an application has to navigate it makes sense to test the compatibility factors that influence this relationship. For this, we need to conduct compatibility testing to check if your application can capably run on different operating systems, browsers, hardware, network environments, and devices. These tests are also done to ensure that all application influencers such as internet bandwidth, OS versions, etc. do not drive failure.
You need multiple sets of real test environments with different configurations of hardware, software, networks, database servers, and operating systems for compatibility testing. This ensures suitable operation with all possible configurations.
Of course, these tests are just the tip of the iceberg of all non-functional tests. To release fool-proof and robust applications you will have to build in another battery of non-functional tests to run. Compliance, security, database, fail-over testing, interoperability, portability testing, etc. all find room on this laundry list.
We will deep dive into all these other forms of non-functional testing, non-functional testing types, etc., in our future blogs. Stay tuned for more!