BENCHMARKING SOFTWARE TEST AUTOMATION FRAMEWORKS: A COMPREHENSIVE STUDY ON EFECTIVENESS AND MAINTAINABILITY
Keywords:
Test Automation Frameworks, Software Testing, Continuous Integration and Deployment (CI/CD), Selenium and Appium, AI in Software TestingAbstract
Test automation systems are crucial in contemporary software development since they provide higher accuracy, efficiency, and maintainability in testing. Here we provide an extensive analysis of the several test automation systems, including their distinct strengths, shortcomings, and relevance in many settings. The study also considers the major challenges of test automation, such as dynamic handling of web elements, complex structures of applications, and CI/CD pipeline integration. Also covered are some best practices for preserving maintainable framework development including modularity, correct coding organization, and frequent maintenance. The case studies provide actual evidence of the advantages of frameworks for dealing with Selenium in web testing, Appium for mobile apps, and hybrid systems. By means of a comparative study, one can determine the most suitable frameworks for use cases, hence suggesting the suitable automation solutions for companies. Furthermore, discussed was the future extent of test automation and the part artificial intelligence and machine learning would play in streamlining the testing process. Thus, this work is a useful manual for maximizing test automation methods to preserve long-term maintainability and software quality.
References
Dustin, E., Rashka, J., & Paul, J. (1999). Automated software testing: introduction, management, and performance. Addison-Wesley Professional.
Luo, L. (2001). Software testing techniques. Institute for software research international Carnegie mellon university Pittsburgh, PA, 15232(1-19), 19.
Kong, P., Li, L., Gao, J., Liu, K., Bissyandé, T. F., & Klein, J. (2018). Automated testing of android apps: A systematic literature review. IEEE Transactions on Reliability, 68(1), 45-66.
Liu, K., Wang, S., Koyuncu, A., Kim, K., Bissyandé, T. F., Kim, D., ... & Traon, Y. L. (2020, June). On the efficiency of test suite-based program repair: A systematic assessment of 16 automated repair systems for java programs. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (pp. 615-627).
Zhou, X., Peng, X., Xie, T., Sun, J., Ji, C., Li, W., & Ding, D. (2018). Fault analysis and debugging of microservice systems: Industrial survey, benchmark system, and empirical study. IEEE Transactions on Software Engineering, 47(2), 243-260.
Baggen, R., Correia, J. P., Schill, K., & Visser, J. (2012). Standardized code quality benchmarking for improving software maintainability. Software Quality Journal, 20(2), 287-307.
Bian, Z. (2020). Test automation process assessment.
Zein, S., Salleh, N., & Grundy, J. (2016). A systematic mapping study of mobile application testing techniques. Journal of Systems and Software, 117, 334-356.
Garousi, V., & Zhi, J. (2013). A survey of software testing practices in Canada. Journal of Systems and Software, 86(5), 1354-1376.
Velmurugan, R. S., & Dhingra, T. (2015). Maintenance strategy selection and its impact in maintenance function: A conceptual framework. International Journal of Operations & Production Management, 35(12), 1622-1661.
Laaziri, M., Benmoussa, K., Khoulji, S., & Kerkeb, M. L. (2019). A Comparative study of PHP frameworks performance. Procedia Manufacturing, 32, 864-871.
Zhu, J., He, S., Liu, J., He, P., Xie, Q., Zheng, Z., & Lyu, M. R. (2019, May). Tools and benchmarks for automated log parsing. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) (pp. 121-130). IEEE.
Athanasiou, D., Nugroho, A., Visser, J., & Zaidman, A. (2014). Test code quality and its relation to issue handling performance. IEEE Transactions on Software Engineering, 40(11), 1100-1125.
Marquez, A. C., & Gupta, J. N. (2006). Contemporary maintenance management: process, framework and supporting pillars. Omega, 34(3), 313-326.
Benton, S., Li, X., Lou, Y., & Zhang, L. (2020, December). On the effectiveness of unified debugging: An extensive study on 16 program repair systems. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering (pp. 907-918).
Papadakis, M., & Malevris, N. (2010, November). Automatic mutation test case generation via dynamic symbolic execution. In 2010 IEEE 21st International Symposium on Software Reliability Engineering (pp. 121-130). IEEE.
Boehm, B. W., Brown, J. R., & Lipow, M. (1976). Quantitative evaluation of software quality. Software Engineering: Barry W. Boehm's Lifetime Contributions to Software Development, Management, and Research, 25.
April, A., & Abran, A. (2008). Software maintenance management: evaluation and continuous improvement (Vol. 67). John Wiley & Sons.
Márquez, A. C. (2007). The maintenance management framework: models and methods for complex systems maintenance. Springer Science & Business Media.
Tahir, T., Rasool, G., & Gencel, C. (2016). A systematic literature review on software measurement programs. Information and Software Technology, 73, 101-121.
Su, T., Fan, L., Chen, S., Liu, Y., Xu, L., Pu, G., & Su, Z. (2020). Why my app crashes? understanding and benchmarking framework-specific exceptions of android apps. IEEE Transactions on Software Engineering, 48(4), 1115-1137.
Al‐Turki, U. (2011). A framework for strategic planning in maintenance. Journal of Quality in Maintenance Engineering, 17(2), 150-162.
Planning, S. (2002). The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and Technology, 1(2002).
Brown, A. (2001). Towards availability and maintainability benchmarks: A case study of software raid systems. Computer Science Division, University of California.
importance-of-testing-in-sdlc. (n.d.). https://u-tor.com/topic/importance-of-testing-in-sdlc.
Vlinkinfo. (n.d.). Role of Automated Testing in software product Development | VLink. VLink. https://vlinkinfo.com/blog/role-of-automated-testing-for-software-product-development/
Wickramasinghe, S. (n.d.). Test Automation Frameworks: The Ultimate Guide. BMC Blogs. https://www.bmc.com/blogs/test-automation-frameworks/
Gupta, Y. (2018, August 30). Six major components of a test automation framework. Software Testing Genius. https://www.softwaretestinggenius.com/six-major-components-of-a-test-automation-framework/
Admin. (2019, June 11). What is Automation Testing? Types, Tools and Benefits | W3Sofetch. Software Testing Blog | Software Testing Company and QA Services | W3Softech. https://w3softech.com/blog/automation-testing-types-tools-benefits/
Islam, M. N., & Quadri, S. M. K. (2020). Framework for automation of cloud-application testing using selenium (facts). Adv Sci Technol Eng Syst J, 5(1), 226-232.
TheKnowledgeAcademy. (n.d.). Top challenges in automation testing: Explore all. https://www.theknowledgeacademy.com/blog/challenges-in-automation-testing/
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Well Testing Journal
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
This license requires that re-users give credit to the creator. It allows re-users to distribute, remix, adapt, and build upon the material in any medium or format, for noncommercial purposes only.
