To ensure success in any operation or project, a set of guidelines must be decided for the team to refer to. Similarly, when it comes to software testing, quality assurance teams must choose from a set of software testing methodologies to meet their testing goals and objectives while ensuring the project succeeds.

What is a Software Testing Methodology?

A software testing methodology is a set of procedures, guidelines, and standards to ensure that software products meet the expected quality and performance standards. It is a framework for organizing, planning, and executing the different types of software testing activities.

The importance of software testing methodologies lies in their ability to provide a structured approach to software testing, ensuring that testing is comprehensive, efficient, and effective. They help minimize the risk of defects and errors in the software by identifying them early in the development lifecycle, thereby reducing the cost and time required for defect resolution. 

Additionally, a well-defined software testing methodology helps establish clear communication and collaboration between the development team, testing team, and other stakeholders, promoting a quality and continuous improvement culture.

The Major Types of Software Testing Methodologies

Depending on the situation and requirements, an organization can implement any methodology for software testing. Here are three standard methodologies one can choose from.

The Waterfall model

A linear and sequential approach, the Waterfall model divides the testing process into distinct phases, each of which must be completed before proceeding to the next phase. It provides a structured, step-by-step approach that ensures that all essential aspects of testing are considered and addressed. The model also provides clear phases and deliverables, making managing the testing process and tracking progress easier.

This model follows the processes in this sequence:

1. Requirements Gathering: The first testing phase involves gathering requirements and defining objectives and acceptance criteria. This helps ensure that the testing process is aligned with the project’s overall goals, which is also one of the best practices in test automation.
2. Planning: In this phase, the testing strategy is developed, including the types of tests to be performed, the testing environment, and the resources required. A test plan is also developed, which outlines the scope of the testing, the testing schedule, and the responsibilities of the testing team.
3. Design: the test cases and scenarios are designed in this phase. This includes defining the test data, the load profiles, and the expected performance results. The testing environment is also configured, and the test tools are selected.
4. Implementation: the tests are executed, and results are recorded. The testing environment is monitored to ensure the test conditions are consistent and the results are accurate.
5. Analysis: test results are analyzed to determine if the system meets the objectives. The results are compared against the acceptance criteria, and any performance bottlenecks or issues are identified.
6. Reporting: includes the test results, the analysis, and any recommendations for performance improvements. The results are reviewed with the stakeholders, and any necessary changes are made.
7. Maintenance: test results are maintained and updated to remain relevant and accurate. This includes updating the test data, load profiles, and expected results as needed.

Pros and Cons of the Waterfall Model

Pros:

• Well-defined phases and deliverables can lead to better planning and control
• Provides a clear understanding of project requirements and scope
• Enables better documentation and traceability
• Works well for small, well-defined projects with minimal changes
• Provides a stable foundation for testing

Cons:

• Limited flexibility to accommodate changes in requirements or priorities
• Lack of customer involvement and feedback
• Late detection and resolution of defects may lead to delays and increased costs
• Tends to focus on following the plan rather than adapting to changing circumstances
• Can result in over-reliance on documentation and a lack of focus on the final product

The Agile model

Unlike the Waterfall Model, the Agile model is flexible and iterative software development. It allows for continuous development and delivery of working software through short iterations, called sprints.

With the Agile Model, testers can get frequent feedback and iteration, enabling the team to quickly make changes and improve the system’s performance. This approach sits well with projects that need frequent and rapid changes due to fluctuations in requirements, such as software performance testing. It is most commonly adopted by testing teams that must work closely with the development team.

The key features of the Agile Model for testing include

1. Continuous Testing: testing is performed throughout the development process rather than at the end. This allows for early detection and resolution of performance issues and reduces the risk of late-stage surprises.
2. Collaboration: there is a heavy emphasis on collaboration between the development and testing teams, enabling them to work together to optimize performance. This includes regular meetings, called stand-ups, where the team reviews progress and identifies any issues that need to be addressed.
3. Adaptability: since the Agile model is designed to be flexible and adaptable, requirements and priorities change as needed. This helps ensure the testing process aligns with the project’s goals.
4. Incremental Delivery: delivery of working software in short iterations is enabled, allowing the testing team to see progress and provide feedback regularly. 

Pros and Cons of the Agile Model

Pros:

• Emphasis on customer satisfaction through continuous delivery of working software
• Flexibility to accommodate changes in requirements or priorities
• Promotes teamwork and collaboration among team members
• Allows for early detection and resolution of defects
• Provides regular opportunities for feedback and improvement

Cons:

• Lack of documentation can lead to confusion and misunderstandings
• Requires active participation and involvement from all team members
• May result in incomplete or inadequate testing if not properly managed
• Relies heavily on communication and may lead to misinterpretation if communication is poor
• Can be challenging to implement in larger organizations

The DevOps model

DevOps aims to improve software delivery’s speed, quality, and reliability. As such, the DevOps Model emphasizes collaboration and communication between the development, operations, and testing teams. 

In this model, testing is integrated into the overall software development process and is performed continuously throughout the development lifecycle. This assists in the early detection and resolution of performance issues, reducing the risk of late-stage surprises and improving the overall quality of the software.

The key features of the DevOps Model for testing include

1. Continuous Integration: testing is done continuously as code changes are made, especially in parallel testing.
2. Automation: this model emphasizes automation, including the automation of testing. This enables the testing team to run tests quickly and repeatedly and to identify and resolve issues rapidly.
3. Collaboration: collaboration and constant communication are core components of the DevOps model. This help integrates testing into the software development lifecycle and optimizes performance.
4. Continuous Delivery: since the teams are working closely, faster and performance-optimized software delivery is enabled with this model. The operations team can thus easily ensure that the testing objectives are met.

Pros and Cons of the DevOps Model

Pros:

• Continuous integration and delivery lead to faster delivery of software
• Emphasis on automation and collaboration leads to increased efficiency
• Early detection and resolution of defects through continuous testing
• Improved communication and collaboration among teams
• Enables faster response to changing requirements or priorities

Cons:

• Can be challenging to implement in traditional organizations with siloed teams
• Requires a significant investment in tools and infrastructure
• May result in increased workload and pressure on team members
• May lead to over-reliance on automation and a lack of focus on the final product
• Requires a cultural shift towards increased collaboration and continuous improvement

How to Choose the Right Software Testing Methodology?

Selecting the most suitable type of software testing methodology rests on factors such as the nature and timeline of the project as well as the requirements of the client. Moreover, it can also depend on whether the software testers wish to wait for a working model of the system or if they want to input early in the development lifecycle. 

For this, it is better to consult a technology solutions provider with expertise in QA, such as VentureDive’s QA services. Besides having in-depth and working knowledge of all aspects of quality assurance, our teams can readily work with different types of testing, making us your reliable partner for QA consulting.

FAQs Related to Software Testing Methodologies

You might also like…

Did you know, the different types of software testing market size in 2021 was over $40 Billion? And we are expected to reach $70 Billion by 2030, which is massive growth in a decade. These numbers not just signify money made during this time, but the potential of the entire industry, which is part of a bigger picture known as software development.

Software testing is a crucial element within the software quality assurance domain. Each application development phase has its own set of testing modules and protocols that they must follow, ensuring smooth workflow and automation throughout. Regardless, a tester’s journey is never a short one, with various types of testing lined up as soon as the development phase reaches its end. 

These different types of software testing have their own set of features and advantages that benefit the product. From manual to automation testing, from functional to non-functional or agile testing, the list is long and extensive, giving us a scope of how rigorous and effective these testing processes are, not to mention time-consuming. But to deliver a bug-free and elite code, it has to go through all these testing phases. 

Let’s take a look at the types of testing that exist within quality assurance, their features, benefits, and their usage throughout a software development lifecycle.  

Manual VS Automated Testing 

Manual Testing

An old-school yet extremely crucial element of end-to-end testing, manual testing is where a tester conducts in-person testing by clicking and interacting through the app and its various APIs. They create a separate environment on various devices to test out the essential features of the application. While this may seem an appropriate way with a real-time human element involved, this can be extremely costly, time-consuming, and slightly flawed as well. Anything involving human interaction can be prone to human error, or typos that can alter the results based on the elements or steps they missed out. 

Automated Testing

While manual testing was all about humans, automated testing is the exact opposite in terms of testing techniques. Automated tests are performed by a machine running on a test script that was written in advance, based on a complex manner. These scripts can vary from complexities to the actions they want the machine to execute. From a single action to a series of actions within the UI that could lead to the same results. While it may seem extremely easy, with a machine conducting all the tests, it all comes down to how well the testing script was written. 

While anything automation has taken over the entire QA process in these times, it’s still integral to visit back to the roots and rely on manual testing from time to time, also known as exploratory testing, but we’ll talk more about it below.

The different types of tests

1. Unit Tests

One of the most regularly used, effective, but basic testing methods, Unit testing is used on a separate unit and component to test out its features and corrections. Usually conducted during the application development phase, to find any defects, it’s cheaper to automate and run on a continuous integration server.

2. Integration Tests

As stated within the name, this type of test provides us with results on whether the components involved within the software are working well together or not. Integration tests are expensive and need to be run continuously on multiple grounds and levels to ensure each component is well integrated and in sync with another one. 

3. Functional Tests

Functional testing works around testing the business requirements of any application without pondering over the entire functioning of the system. It pays close attention to all the components whether they are functioning well and producing their desired results individually or not.

4. End-to-end Tests

One of the most important testing phases is quality assurance. End-to-end testing replicates a human behavior of conducting the entire functions within the software, from start to finish. Its main aim is to verify whether basic and complex tasks within the software such as clicking on the right tab or getting the right options in the menu are functioning well or not. 

While this is one of the most important types of testing, it sure is expensive and requires constant attention when automated. To save cost and inconveniences, it is mostly recommended to conduct fewer end-to-end tests and rely more on other similar, but smaller testing types to get similar results on an individual level.

5. Acceptance Testing

is Also known as the User Acceptance Test (UAT), the final phase of testing where the client, business, or the customer themselves take control over the app. Their testing is conducted in real-time, based on business scenarios and goals they want to achieve success via the application. The final outcome from acceptance testing usually deciphers whether the system is ready to go live or still needs alterations.

• Alpha Testing

The final round of acceptance testing before the application is released to the customers.

• Beta Testing

Conducted by the users and customers themselves, the complete application is released to a special set of users who have signed up for this service on the app store. The app is released specifically for these users who conduct a thorough examination of the application in real-time and send in their feedback. The feedback is then considered and incorporated within the app before the official release. 

• Operational acceptance testing (OAT)

To ensure that the system administrator is working effectively to cater to a user query in a real-time environment, OAT is conducted. 

6. Performance Testing

To test how an application performs under various scenarios and workloads, Performance Testing is the go-to testing type that you would target for this. It gives an overall idea of how well your application is performing in terms of responsiveness, speed, reliability, and more. To test these out, tools like Loader.IO, JMeter, and LoadRunner are used throughout the process.

This test can help you determine the capacity your application can work, the load it can handle, or the number of customers per minute it can handle. Whether the data that goes through the system is reaching accurately or not. Performance testing is the best possible way to measure any future roadblocks or bottlenecks that may occur, measure stability during peak hours, etc.

Performance testing on its own is divided into further branches of testing like, 

• Load Testing
• Stress Testing
• Scalability Testing
• Volume testing (flood testing)
• Endurance Testing (Soak Testing)

7. Smoke Testing

If you want to find out whether your application is functioning well overall before it’s deployed, or is adjusting well in a new environment after deployment, a Smoke test is your answer. Without going deep into the system, smoke tests do a quick, in-expensive run-through of your software assuring that the system is well intact and all the major functionality is running smoothly.

While the smoke test in itself is inexpensive, it does help you decipher whether a more expensive test is to be conducted any further or not. Usually, a smoke test takes place right after it is deployed into a new environment, to see if there are any discrepancies or not.

8. Security Testing

Laying under non-functional testing, security testing is one of the most essential testing types that is conducted by a special team with expertise and background in cybersecurity. They conduct a clean sweep of the entire interface and software of the app to check on any internal or external threats it may be vulnerable to. The testing covers the system for any viruses, and hacks, and also takes a deep look into the authentification and authorization processes if any. An overall protective shield is also applied to prevent any attacks as well.

• Penetration Testing

Pen testing is an authorized self-inflicted cyberattack on the application to declare any weak aspects or data leaks within the complete system. This form of ethical hacking is performed by professionals who then submit the final report to the organization.

9. Usability Testing

While testing is a pretty rigid and robotic task on its own, what is it that tests out the overall “user experience” of the application? Usability testing. To see whether the app is working well, a human perspective is instilled in this testing method to test the user-friendliness of the application overall. 

• Exploratory testing

An informal testing method whose aim is to find any flaws within the existing application through their knowledge of the business domains. 

• Cross-browser testing

It is essential to find the look, feel, and performance of an application on multiple browsers, operating systems, mobile devices, and platforms. This is an essential feature for a well-ensured user experience, as not every user owns the same device, but they deserve the best experience regardless. A browser stack is a good tool to carry out this type of testing.

• Accessibility Testing

Leaving no stone unturned here, accessibility testing is focused on making the app effective and user-friendly for the disabled. Features like font size, color saturation, sound, etc are taken a good look at to avoid any inconvenience. 

10. Compatibility Testing

Finding compatibility is not just a human factor anymore, even applications and systems require a compatibility test that deciphers whether the system is working well on various platforms and devices or not. It is important to figure out whether the system is in sync with the web servers, hardware, and network environment or not. It ensures whether it’s good enough to work around various configurations, databases, and browsers, or not.

Other Types of Testing

The world of testing doesn’t end here, with various complexities around us in technological forms, now fixing their roots in AI and metaverse, this factor within software quality assurance has evolved as well. To keep itself relevant the field of QA has further extended its roles and created new factors within testing that not only cover the basics but have the technical abilities to delve deeper. 

Some of these non-conventional testing methods are as follows: 

• Ad-hoc Testing
• Back-end Testing
• Browser Compatibility Testing
• Backward Compatibility Testing
• Black Box Testing
• Boundary Value Testing
• Branch Testing
• Comparison Testing
• Equivalence Partitioning
• Example Testing
• Graphical User Interface (GUI) Testing
• Incremental Integration Testing
• Install/Uninstall Testing
• Mutation Testing
• Negative Testing
• Recovery Testing
• Regression Testing
• Risk-Based Testing (RBT)
• Static Testing
• Vulnerability Testing

Final Thoughts

Software quality assurance testing is necessary, and more importantly, it’s an integral part of the software development timeline. Without all these testing branches and features the application will come across various developmental and scalability crises that will result in the overall app and business performance as well.

FAQs Related to Different Types of Testing

You might also like…

Businesses around the world succeed based on the quality of services they provide. The easier it is for customers to navigate and use the developed products & solutions, the happier they are with you as a service provider. Quality assurance plays a major role in making your products bug-free and intuitive. This means, once you deliver a pristine product to your customers, there would be a lesser number of do-overs or revisions, hence, saving time and resources for your business. 

Let me share an example of how a bad-quality product affects business, as well as, human lives:

NASA’s shuttle, Challenger, exploded just after minutes of taking off, resulting in the death of seven astronauts who were on board that space shuttle. The mechanical cause of this failure was the Solid Rocket Boosters ’O’ rings. An extensive report by Jeff Forest, from Metropolitan State College, points to a flawed Group Decision Support System which misinterpreted the risks and failed to communicate concerns surrounding Quality Assurance. The whole project was shut down at the time.

In light of this incident, we can say that Quality Assurance of hardware, software, or any product of any kind, plays a very important role in business growth. Lack of it causes a waste of effort, time, and money. Before knowing how Quality Assurance is profitable for any business, let’s first have a little background of the term ‘Quality Assurance Testing’, and explore the whats, whys, and hows of it.

What is quality assurance testing?

Quality Assurance is about making sure that your software conforms to the requirements set by the client, and it works without any defects and failures. QAs/testers should run the program by keeping in mind every possible scenario to detect any defects, ensuring a successful run of the program. Now, what do I mean when I say defect-free software? For that, we have to define the term “defect”.

Defect/Bug: A defect is an anomaly in the product, something that deviates from the expected behavior. For example, you want to log in to the website on your browser. The login button is supposed to direct you to the login page but instead, it takes you to the help page. This unexpected behavior is called a defect or a bug as it is not conforming to the requirement.

What does a tester do and why do you need them?

A tester is someone who assures the quality of the product by running the program for every possible scenario while testing.

Basically, this is what a QA tester does:

To ensure the quality of software under development, there is a series of processes followed to deliver a product — It’s called Software Testing Life Cycle (STLC).

STLC includes the following stages:

1. Gathering requirements
   Requirement gathering is the first and foremost step in the STLC. In large-scale projects, it is vital for the project team, i.e., project managers, developers, designers, and QAs, to know exactly what the client is expecting from them. The bigger the project, the more extensive the requirements. Requirements are gathered from the clients and their stakeholders in the form of a written document or prototype, generally. 
   
   Time is money. With poor quality requirements, projects either fail, are completed late, or go over-budget. This is a reason enough for any business to pull off their projects from a software company, hence costing you money.
   
2. Test planning
   Test plans are written to define the testing strategy. Test planning is done by the Test Manager or the Test Lead. It tells us how and when the testing activities will be carried out. It is done to plan the time, effort, and cost consumption of testing for the entire project.
   
   Test planning not only gives us a strategy for testing activities but also helps us estimate the budget of the project. Poor test planning leads to failure in reaching the milestones. In a fixed-cost project, it can cost you your time, effort, and money.
   
3. Test case development
   After test planning, test case development is the next step. In this phase, the testers create test cases that cover every possible scenario in a very detailed manner. Test cases are developed to ensure even the edge cases won’t cause problems in the workings of the software.
   
   Test case development, if done right, can save you money, because, in this particular step, you are making sure the software does not break at any step. Hence, this assures that no further effort of the developers is required and the product is credible. A credible delivery brings in business, which brings in money.
   
4. Test environment setup
   Setting up the test environment is a vital part of the STLC. Basically, the test environment decides the conditions on which the software is tested. This is an independent activity and can be started parallel to the test case development. In the process of setting up the testing environment, the test team is not involved in it. Based on the company’s policy, it could be that a developer or customer creates the testing environment.
   
5. Test execution
   In this phase, the testing team starts executing all the test cases in a particular test environment that was set up by the development team or the customers. Each test case is marked ‘passed’ or ‘failed’, depending on its failure or success. Some test cases can be blocked for execution due to some system constraints, and hence, marked ‘blocked’. If the test case is blocked, the relevant bug is reported.
   
6. Bug reporting
   When a test case fails, the reason is that it is not working as expected. Thus, a bug/defect/issue is logged using a preferred tool and planned in the test plan.
   

After going through all the phases of the STLC, the final test execution summary report with exit criteria is prepared to show the stakeholders whether the software is ready to be launched or not. So, testing the software by following the STLC will save you money as it takes care of your estimates of the effort, time and money you are putting into it.

When to start testing?

It is always preferred to start testing alongside development. An early start to testing reduces the cost and time to rework and produce error-free software that is delivered to the client. However, in the SDLC, testing can be started from the requirements gathering phase and continued until the deployment of the software. 

SDLC and STLC go side by side in a project. The diagram below can better explain how they are being done simultaneously while the software is being developed.

What impact does SQA have on your business?

In Software Quality Assurance, a defect is the stitch which resolved on time saves you time, money and effort. Allocating some of your resources to QA Testing can actually reduce your total development costs while also stimulating more growth.

SQA brings the following benefits to the business:

1. Quality product
   Quality is everything for a customer. It is achieved through proper testing of a product. If you want your product to survive in the industry, that will be possible when it has gone through testing.
   
2. Reduce developmental cost
   Finding defects in the earlier stages of development can save you a lot of effort. In that way, you do not continue building upon faulty code. A faulty piece of code can be longer than the one written upon fixing a defect. Reducing lines of code reduces the developmental cost.
   
3. Save time & cost in fixed budget projects
   In a fixed cost project, finding bugs and fixing them in the early stage of the project proves to be beneficial because when the deadline is here, there is no room for defects and efforts to be made. There will be a spill-over.
   
4. Customer satisfaction
   A happy customer is not a myth when a quality product is delivered. It helps you gain a larger number of customers. If you are a product-based company, when you deliver the software and customers use it, it will bring more customers and more customers to bring more money and business. When clients receive defect-free software, there will be positive feedback and positive feedback will bring a large number of projects to the business.
   
5. Less promotion, more recommendations
   A customer would love to use a good quality product. A product won’t need much promotion if a happy customer starts recommending it. So, the cost of marketing is saved as word of mouth will be the source of it.
   
6. Profitable
   Surely, the idea of hiring QAs does not seem profitable. It makes you think that you are spending more money. However, testing a product before selling it in the market will only increase the number of buyers. Nothing throws a software company out of business than an unsatisfied customer who regrets buying a product that does not work. It’s a long-term gain.
   
7. Client confidence
   Retaining your client-base so they bring in more projects is only achieved if the very first software you deliver is defect-free. The quality of the product leaves your client satisfied and they will always confide in your ability to deliver a quality product every time.
   
8. Saves corporate-level data hazards
   If you are deploying your software in an enterprise-level company that involves high-level security of data, without proper QA of the software, it may result in a defect that results in leakage of sensitive data. It is a hazard that’s difficult to overcome on this level of deployment. All of this could be avoided by testing the software beforehand.

All in all, the benefits of quality assurance are significant. Your software and your services gain customers, recognition and market value.

Debunking the myths

Over time you might have heard some myths about Quality Assurance and how it is not necessary or how it adds to the cost and time of the project. However, as of now, these myths have no meaning because almost every software development company hires not only developers but QAs for the projects too. QA reduces costs instead of adding to them. It is surveyed that a product delivered without testing costs a business more than hiring a team for it. A non-tested product is usually buggy, and may not cover the initially set requirements or direction.

Another myth is that QA is only done by the Testers. This is a big no. It is not only the responsibility of the QA but the developer as well. Developers have to give a working code to QA for testing and it cannot happen unless the developer tests it first.

Quality assurance is vital throughout a project’s life cycle. It always adds value to your business and saves you money by making sure that after a product is delivered, it is defect-free and doesn’t cause any problems to the client. Quality assurance and quality control, both fall under quality management and are essential to the STLC process. Testing and development should be done in parallel to avoid the defects found there. Quality delivery builds consumer-client trust, enhances customer satisfaction, and reduces developmental costs, which, in turn, adds money to your business.

---

Quality Assurance

You might also like…

After weeks of hard work, countless meetings, and a successful project delivery later, we’ve decided to pull back the curtains on exactly what it takes to assure the quality of a high-availability healthcare app.

Back in April 2020, when the first wave of the novel Coronavirus was at its peak, VentureDive reached out to the Government of Pakistan to help the country combat the virus through technology. As part of the ‘Digital Pakistan’ initiative, spearheaded by Tania Aidrus, an ex-Google executive, we collaborated with her team to build and launch the COVID-19 telehealth portal. It is a website specially designed to combat the crisis of the pandemic. It allows Pakistani doctors and all healthcare professionals to register on it and volunteer to remotely help the patients who might have COVID-19 symptoms.

Doctors who want to give back during Ramzan: consider volunteering your time by signing up to provide #Covid_19 consultations at https://t.co/6Joc1VFVRM

Deep gratitude to @venturedive, @ptcl, @eocean, @infobip & @ntc for making this happen.@zfrmrza @fslsltn @UdarOfficial

— Tania Aidrus (@taidrus) April 25, 2020

“VentureDive team, I cannot thank you enough on behalf of the entire team for leaning in to help! There has been SO much interest in the platform and what’s interesting is how much interest we are seeing in other verticals. For example, today we had a call with the Law Ministry who are very eager to do something similar to sign up volunteer lawyers to provide free guidance to victims of domestic abuse. I hope this is just the beginning of our working relationship – excited to have started off on doing something that I hope can help thousands of Pakistanis during this time.” — Chief Digital Officer, Digital Pakistan

Healthcare is a very sensitive subject and it was a technology to be used by millions of people across Pakistan. Therefore, it demanded the highest quality, with zero downtime, zero bugs, and intuitive user journeys.  In this blog, we’ve highlighted our experience of testing a portal that was to be used by healthcare professionals to reach out to patients via our technology.

What follows is a tale that tells the challenges we faced during the three-week-long project, and how we resolved them to successfully deliver a web and a mobile application.

The functional, security & scalability challenges of testing the telehealth portal

Before the project kicked off, the quality assurance team at VentureDive gathered the application requirements and shared them with the experts dedicated to working on this. A thorough documentation and sample mockups helped the QA team to begin working on the test plan, test design, and test cases during the development phase. We conducted daily stand-ups so the development & testing teams could stay synced and brainstorm on maneuvering through this project smoothly and in time.  We faced six major challenges during our course:

Time management

The main challenge was racing against time to meet the client’s expectations while ensuring the security of the sensitive healthcare data, and zero glitches within the app. This meant that the QA team had to keep track of every requirement and reporting templates for testing updates that helped the development team fix defects and bugs timely, prior to delivering any milestone to the client.

3rd party integrations

The application was to be integrated with third-party software such as WhatsApp chatbot for doctor-patient communication & telecom operators to enable anonymous calling mechanism and receiving SMS OTP. These were essential for fetching data from official sources and making sure both our applications remained in sync with the whole system we were creating.

Mobile responsiveness

The telehealth portal was supposed to be a hybrid mobile application, which meant that the testing team had to test it across various mobile devices and operating systems to make sure that it was responsive and compatible.

Security

Cyber attacks and threats are a real-world problem today with thousands of networks and websites being compromised each day. To help identify, classify, and address security risks, we performed vulnerability assessment and penetration testing activity including server VA, API penetration testing, and web application penetration testing to identify possible routes an attacker could use to break the system.

System performance

Monitoring the performance of the application was an integral part of building the portal, since we anticipated a large number of users, including doctors and patients. The QA team planned to automate the scripts on JMeter to determine how the system performs in terms of responsiveness and stability under heavy load and a huge volume of data.

Standards

A big challenge for the QA team was to keep the testing practices as standardized as possible even with a short time to spare. VentureDive believes in maintaining the quality of the deliverables as our utmost priority, regardless of the length, complexity, or intent of the project.

Adopting a smart testing strategy for successful project delivery

The process of Alpha testing was done remotely. The QA team collaborated and focused all their efforts towards detect any major defects in data security. We carried out usability, performance and security testing for private and sensitive information in a healthcare setup.

It was pertinent for the QA team to also analyze business criticality, plan around testing efforts in minimum time, make the application usable for thousands of users and ensure that testing was compliant with the Open Web Application Security Project (OWASP) standards.

The test strategy called for having separate environments for development, staging, and production. We performed the following steps in the given order:

Functional testing

Keeping in mind the criticality & nature of health-related projects, requirements had to be precise and the validation had to be perfect. We performed static analysis on requirements followed by actual test execution to meet the requirements and clients’ expectations. Data flow integrity and business rules were repeatedly tested via automated suites in our regression cycles

Tools:

• Postman for API automation & integration testing
• TestRail for test cases and test cycle reporting
• PostgreSQL for data validations
  

Cross browser testing

Browser compatibility was mainly focused on Google Chrome version 80+ on windows. Extended smoke and regression cycle was performed on Firefox and Safari for Windows and Mac respectively.

Tools:

• crossbrowsertesting.com
• Browserstack.com

Responsiveness

Understanding the market trends of portable devices, we analyzed the data of the target audience and performed UI/UX testing on mobile & other portable devices. The application was tested on 6 different Android and iOS devices with different screen sizes and resolutions having different OS versions.

Security testing

Thorough security testing was performed on infrastructure, API, and application level, keeping the top 10 OWASP standards in mind. 

Tools:

• Burpsuite
• ZAP 
• KALI Linux operating system

The QA team identified the following vulnerabilities during the security testing activity:

• Broken access control
• Broken session management
• Disclosure of internal directories
• Unrestricted file upload
• Missing server validation
• Sensitive data exposure
• Brute Force 
• No rate limit
• WAF & ACL implementation

Our goal was to immediately address these issues and recommend further best practices that should be followed as pre-emptive measures against any potential cyber-attacks.

Performance testing

The system undergoing the test was required to have a load-balanced infrastructure supporting thousands of interactions between the patients and the doctors. The flow includes the signup process including uploading of images, populating, and fetching patient data lists and assignments of users one to one. 

Tools:

• Jmeter
• Blazemeter

We analysed all the results, and generated an extensive report using SmartMeter, which was later shared with the stakeholders. The primary issues identified were load balancing, CPU utilisation, and WAF configurations. These were addressed and recommended configurations were made for resolution.

Project delivery

A standard process was put in place to validate the requirements and meet the client’s expectations. After complete and thorough testing, we demonstrated and delivered the project to the client successfully.

Wrap up

Working on the telehealth portal as part of the ‘Digital Pakistan’ initiative was a short, knowledge-packed, and completely amazing journey that helped us learn and implement advanced quality assurance methodologies for a secure application. We adopted agile software quality practices to align software quality with product requirements and accelerate the software lifecycle. In addition, the continuous feedback we received from the project managers helped minimize retesting for verification and validation. Our iterative approach and short sprints enabled us to deliver quality products within a set deadline, successfully.

Thanks, team for all the amazing support. As I mentioned on slack, thanks to your hard work, we have 3000 doctors signed up and 1000 who submitted their documents. I had an amazing experience working with you all and truly admire your work ethic and efficiency. We couldn’t have done it without you. We will keep you updated on the stats and the launch event! — Project Coordinator, Digital Pakistan

Here’s where having a process-driven, and designed around ‘best-in-class’ software technologies delivery model helped us greatly.  It enabled robust scalability while maintaining cost-efficiency within strict quality control measures.

Thanks for stepping up to swiftly contribute towards our nation in these challenging times. It’s been a privilege to watch such a well-oiled team in action. — CEO, Digital Pakistan

---

HealthcareQuality assurance & testing

You might also like…