SOFTWARE QUALITY ASSURANCE IN AGILE DEVELOPMENT: STATE OF THE ART REVIEW

Abstract-The basis of this project is to give a significant review on software quality acquired through agile development by focusing on the articles related to agile testing and further comparing software testing techniques in different agile development methodologies like scrum and extreme programming. The pilot study on the research questions identified based on the findings and observations on the challenges faced in agile testing will determine that agile testing practices have a broad acceptance by the way it complements the overall software development process.

1. INTRODUCTION

Agile is a self-explanatory word meaning ability to move easily and quickly. Agile in software development does have the essence of its definition. Agile methods are widely implemented in organizations due to its ability to adapt changes and it carries the ability to handle the software development process with less difficulties having its agenda cleared to make understand the team on what are the client expectations and how to deliver what is been expected.

The most popular agile methodologies techniques widely being accepted in the industry are: Scrum, Extreme Programming, Crystal Method and Fast Driven Development [8]. Irrespective of the agile methods, all are characterized by customer satisfaction, fast response to changes, and release in less time [1]. Agile believes to deliver software in chunks but in iterative manner which makes it even more convenient in adapting new requirement request from client. It also provides a low-cost maintenance software as it helps to identify the issues as well as prevent them even if the software is not been fully developed.

Agile testing not only means testing an agile application but testing it with a plan to learn about it [4].

This diagram illustrates how agile testing works. In agile, there is not a separate testing phase, but testing is considered as an ongoing continuous activity. Thus, the requirements are also been tested which also reduces the chances of having loops holes in the requirements.

Based on the articles analysed, this paper discusses on agile testing practices, weighs the benefits and challenges involved in it, having ultimate focus on providing a review on software quality achieved during agile testing and finding that unfocused area in existing agile testing issues. This will lead to research question and a pilot study having an objective to find answers or an approach towards filling this gap in existing study. Finally, stating the conclusion and future work involved in agile testing.

2. SOFTWARE TESTING IN DIFFERENT AGILE METHODOLOGIES

Articles reviewed focused mostly on Scrum and Extreme Programming (XP) methods. However, the behaviour of software testing in different agile environments remains the same in all agile environments as they all follow ‘requirements change-fault detection-increased performance-iterative & incremental delivery-flexibility of design & quality improvement’ [8]. Below are the observations on Scrum and XP methods from the articles reviewed:

• Scrum: In Scrum the quality attributes are analysed based on the functional requirements. [1]. Scrum is a framework for managing project that is applicable to aggressive deadline projects having complex requirements and a degree of uniqueness [8]. It is beneficial for the projects that requires lot of automation testing. Testers keep testing in coordination with developers [7].
• XP: It is widely used in the organizations [8]. It focuses on practices like test driven development, pair programming, release in short iterations, improving the design of existing code and performing acceptance test on the user stories developed during the sprint. To ensure the quality of the software, automated tests are carried out to ensure the quality of the result [11]. The defect rate is considerable low if testing in XP is properly done.

3. BENEFITS OF AGILE TESTING

1. Testing in any agile development method that is being implemented on any size of projects allow the quality assurance team to communicate on regular basis for any priority and severity defects being caught during the sprint which shrinks the communication gap within the development team.
2. Agility testing has the ability to handle the unstable requirement changes and thus makes it easy to maintain the testing documentation frequently.
3. Agile testing not only allows team to identify failures but also to prevent one.
4. Agile testing is fast and flexible than any other testing methodologies as it makes use of automation tools for testing.
5. Customer satisfaction is achieved as adhering to the customer requirements is the ultimate goal of agile testing.
6. With the use of smoke testing and regression testing, the health of the project can be identified in slice of time.
7. Agile testing saves time and money [9].

4. CHALLENGES IN AGILE TESTING

Last minute changes in requirements can lead to lot of regression testing on the entire project.

There can be times when not enough information on the user stories is provided by the user story writers. Thus, brainstorming for all possible testing coverage becomes challenging when you have less information on the story.

Agile welcomes changes which leads to continuous testing. Therefore, having good acceptance criteria for testing is required.

Testing non-functional requirements in Scrum becomes challenging as its user story backlogs focus only on functional features [1].

Training is required to understand how testing in agile works [8] as inadequate training can cause failure to understand on how to cop up with frequent requirement changes.

Only experienced developer/tester are capable of making decisions based on change request [9]. Therefore, cannot rely on freshers of agile team members to take decisions in the absence of experienced team members.

6. RESULTS OF ARTICLES REVIEWED

7. CONCLUSION AND FUTURE WORK
8. RESEARCH QUESTION

1. Despite several benefits of agile testing, what are the chances of agile testing getting spilled over in the next iteration of sprints and what can be the major cause for this?

2. There are always possibilities of encountering fault-prone areas in the software. Can adding an extra testing sprint towards the end of the agile development sprint, help delivering a zero-defect software?

9. RESEARCH PLAN

A thorough study needs to be conducted in order to identify the root cause for agile testing getting spilled over and for identifying whether introducing an extra testing sprint towards end of the agile development could help deliver minimal defects in the software.

Feedback needs to be collected through a survey as the research questions identified are the real time scenario problems faced in agile teams which none of the articles reviewed has had mentioned while discussing the demerits of agile testing. Considering, that delivering the project on time is not always achieved and testing getting delayed in most of the sprints could be one of the concerns for this, further study on this research is required.

Conducting a survey with more than 10 years of agile development experience is required. Thus, planned to choose participants like Product/Project Managers, Business Analysts, Software Developers and Software Testers each having 10 or more years of experience working in agile development team.

One of the major concerns during this pilot study is to find such employees who would participate in survey by providing their genuine feedback.

10. PLAN/DESIGN OF STUDY

The plan was to create an online survey and share this link with on social media using Facebook messenger and WhatsApp technology.

The objective of this study is reviewing the quality attributes in a software that is implemented using agile methodologies and to discover the possibility of delivering error-free software and to establish the cause of testing getting delayed in ongoing sprints which leads to few requirements testing to get spilled over which ultimately impacts the delivery of project.

The keywords used during the research were as follows: “agile testing”, “agility testing”, “software quality in agile”, “quality assurance in agile”, “testing in scrum”, “testing in extreme programming”. To come up with interesting materials with the combination of mentioned keywords, I preferred using search engines like, Google, University of Limerick library and ResearchGate.

11. PILOT STUDY

Following are the questions asked in a survey:

Result analysis: Was successful in getting a feedback from highly experienced agile development team members. ~64% participants had 10+ years of experience. Which made me confident to relay on the final outcome of this survey.

Result analysis: Highest number of participants were Managers ~45% who had experience working in Agile Teams which makes it easy for the participants to answer the survey genuinely as the managers are believed to be diverse in terms of knowledge.

Result analysis: It brought no surprise when ~90% of the participants shared that they had experienced maximum testing spill overs in agile environment. This made it thorough that more research is required to understand the root cause of this turmoil.

Result analysis: ~37% of participants agreed that testers underestimate the testing estimation and equally, they lack skills to test the requirements. Whereas, only 25% believed that taking unrealistic count of requirements to be developed and tested in a single sprint could also be one of the causes. However, the result did not give a clear vision to locate the exact cause of testing spill over.

Result analysis: It was interesting to find that almost 81% agreed that test cases creation, test case review and updating them as per the feedback comments does take valuable time but it is essential as this test document created during sprint be accurate as this stage as it acts like a baseline for further testing.

Result analysis: ~81% participants agreed that it’s highly impossible to deliver a bug free product even though agile has its true power over the development. Alternatively, further research on this agenda is required to discover the possibilities of delivering a bug free software.

                    Result analysis: The result gave a level of satisfaction which made me curious to

understand why participants agreed on having an extra testing sprint during agile. Maximum 81% contributed to this agreement which makes it firm to believe however, in-dept research is definitely required to disclose the actual causes of testing spill overs.

Result analysis: Maximum participants ~50% agreed having smoke test on fresh build along with regression testing that would be sufficient to delivery a bug free software. However, these testing require a good sufficient time to cover all impacted modules plus the top to bottom flow of the software. So, if implemented an extra testing sprint it should provide a sufficient timeline for QA to uncover errors in the system.

Result analysis: It was hard to believe that maximum participants would blame testers for testing spill overs. This makes oneself more curious on why only testers could be blamed. This needed to be more accurate for further analysis.

Result analysis: Almost 55% participants believed this survey had real time scenarios covered that leaves the scope for further in dept research.

12. REFERENCES

[1] Sanghoon Jeon, Myungjin Han, Eunseok Lee, Keun Lee, ‘Quality Attribute driven Agile Development’, 2011 IEEE Ninth International Conference on Software Engineering Research, Management and Applications. pp.1-2.

[2] Rafael Di Bernardo, Ricardo Sales Jr., Fernando Castor, Roberta Coelho, Ne´lio Cacho, Se´rgio Soares, ‘Agile Testing of Exceptional Behavior’, 2011 IEEE 25th Brazilian Symposium on Software Engineering, pp.1-2.

[3] Ming Huo, June Verner, Liming Zhu, Muhammad Ali Babar, ‘Software Quality and Agile Methods’, 28th Annual International Computer Software and Applications Conference, 2004 IEEE.

[4] Eliane Figueiredo Collins, Vicente Ferreira de Lucena Jr., ‘Software Test Automation Practices in Agile Development Environment: An Industry Experience Report’, 2012 IEEE AST 2012, Zurich, Switzerland.

[5] Fernando Selleri Silva, Felipe Santana Furtado Soares, Angela Lima Peres, Ivanildo Monteiro de Azevedo, Pietro Pereira Pinto, Silvio Romero de Lemos Meira, ‘A Reference Model for Agile Quality Assurance: Combining Agile Methodologies and Maturity Models’, 2014 IEEE 9th International Conference on the Quality of Information and Communications Technology. pp.1-2.

[6] Farrukh Latif Butt, Shahid Nazir Bhatti, Sohail Sarwar, ‘Optimized Order of Software Testing Techniques in Agile Process – A Systematic Approach’, (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 8, No. 1, 2017. pp.1-2.

[7] Shubhangi A.Deshpande, Anand N. Deshpande, Manisha V. Marathe, G. V. Garje, ‘Improving Software Quality with Agile Testing’, 2010 International Journal of Computer Applications (0975 – 8887) Volume 1 – No. 22. pp.1-2.

[8] Pratibha Singh, Puja Patel, ‘Impact of agile testing over traditional testing’, Jaipur International Journal of Converging Technologies and Management (IJCTM) Volume 1, Issue 2, 2015 ISSN: 2455-7528.

[9] Mohd Hamid, ‘Agile Testing’, International Journal of Scientific and Research Publications, Volume 5, Issue 9, September 2015 ISSN 2250-3153.

[10] Theodore D. Hellmann, Abhishek Sharma, Jennifer Ferreira, Frank Maurer, ‘Agile Testing: Past, Present, and Future’, Conference Paper. August 2012

[11] Bernhard Rumpe, ‘Agile Test-based Modeling’, 2006 International Conference on Software Engineering Research & Practice.