Table of Content:
Integration of Lean management concepts in Agile Software development.
This research project is about applying lean management concepts in the agile software development environment of InfoStreet Inc, which is an IT company dealing in Software development. I work at this company in the role of Quality Analyst with prospective of transitioning into Project manager or Business Analyst roles after completing my master’s degree. This research will help me with understanding in-depth details about all the management aspects of the software development life cycle. As part of continuous improvement for this company, I will look at the current process InfoStreet, applying lean principles to the agile software development process.
The motivation of this project is due to 5 years of experience in the agile software development environment. Concepts applied and included in this report will include Lean principles, quality management as well as, project management which I have learned at CSUN, to eliminate the waste and optimize the whole software development life cycle. The main objectives of this research case study will be:
So here, I will going to apply the principles of Lean manufacturing systems from MSE-507, concepts from Leadership of Engineering Professionals And High Tech Firms MSE-608B, Seminar in Quality management concepts – MSE-618 and Project management concepts from SOM-466.
This research study examines how the lean ideas behind the Toyota production system and other Quality management concepts can be applied to agile software project management. By consolidating the concepts of Lean with Agile ideology of Software Development, software organizations like InfoStreet can manage their complex projects and optimize the whole software development life cycle. [Alan Shalloway, 2009] A combination of Lean-Agile approach will definitely enhance both business value of software and productivity of Software developer, tester, project manager, Architect, client manager etc.
Agile technique includes the ability to create and respond to change in order to succeed in an uncertain and turbulent environment. Agile technique demonstrate immense potential for developing more effective, higher-quality software. [Jochen Krebs, 2009] Agile Software Development is basically a lightweight software engineering framework that promotes iterative development throughout the life-cycle of the project, close collaboration between the development team and business side, constant communication, and tightly-knit teams. [Understanding Agile Methodology, 2008] And the basis of lean is the continuous elimination of waste. This requires a focus on the work flow through the system to ensure that material, (service in case of software) is produced only when it is needed and in the quantity required.
The Toyota production systems lean concept which can be used in Agile software development are, Kanban which has three elements – visualize the workflow, limit the Work in Progress (WIP), and measure and optimize the flow, Another concept of TPS on comprehensive suite of automated unit and functional tests and continuous integration could help us apply “Jidoka” to software development. Whenever there is a defect introduced, the continuous integration server would stop building and provide feedback early to the team; no defects released. [Abbas Moshref Razavi & Rodina Ahmad, 2016] Scaling flexibility, business management involvement and waste reduction were found as challenges, whilst setting up teams, self-organization and empowerment appeared easier to achieve.
At InfoStreet, Agile methodology of software development is used and each project is broken up into several ‘Iterations’.
All above Iterations also known as scrum, ranges between 2 to 8 weeks. At the end of each iteration, a working product is delivered for feedback. So, we can say that here in this Agile approach the project is broken up into several releases. Project manager and solution Architects decides the basic core features that are required in the product and decide which of these features can be developed in the first iteration. Any remaining features that cannot be delivered in the first iteration will be taken up in the next iteration or subsequent iterations, based on priority. At the end of the first iterations, the team delivers a working version of the software with the features that were finalized for that iteration. [A. Cockburn, 2006]
There are generally 5-6 iterations (scrums) and at the end of each iteration the customer is delivered with a working version of the software that is incrementally enhanced and updated with the features that were shortlisted for that iteration and each follows its own workflow. During an iteration, feedback from the customer play an important role to ensure that the features meet their needs. So, the process flow here is more of a loop and not a linear process. [Principles of Lean Software Development for Agile Methodology, 2014]
Figure 1. The iteration cycle of an Agile project
[Agile Methodology, Examples, When to Use It, Advantages and Disadvantages, 2017]
It is important to start the highest priority and most difficult features in the first two iterations, as it can take multiple iterations to complete features. Leaving the hardest features for last increases the possibility that they will not be completed by the end of the release. For mission control operations, the customer conducts a certification of the software for operational readiness after we deliver the release. Note that the last iteration of a release focuses on bugs, usability, and testing, not feature additions. [J. Highsmith, 2002]
A value stream mapping is technique of analysing the value added and non-value added items in the whole process, so that waste can be identify. By analysing the past data and information from my project manager, I have created the below value stream map for the whole software development life cycle of the particular module which is implemented for the HugesNet (our client) that last for 44 days with the team size of: Developer – 3, Project manager – 1, Architect-1, Software tester – 2. Each resource worked for 8 hrs/day. (The timing for each iteration will be different which depend on the module size and its complexity)
Figure 3, Value Stream of Agile flow.
With this value stream, we can see that the most time-consuming actions are requirement gathering that is because the clients are not clear with the requirement, it took 117 hrs to complete the whole requirement gathering and finalizing process and in the rest of the actions we spend most of our time waiting for someone or something else, as the wait time between the iteration are very high, which is the waste. And with initial iteration we got the module which was 50% defective and it took 3 iteration of testing to take it to less than 2 to 3 % which is again a waste.
This problem is the biggest problem InfoStreet is facing right now, it occurs when functionality for a particular project is complete, but the stakeholders put off reviewing the work, and then later find issues. It forces the programmer into extra work and have to return to the requirements, mindset, and flow of the assignment after believing that it had already been completed. As a result we miss the deadlines, or release projects with unstable functionalities, leads to production bugs and to fix these production bugs we have need to give the patches which cause further delays. [M. Poppendieck & T. Poppendieck 2003] and holistically delays results in the loss of workforce and resources which can we translated to monetary value, customer satisfaction and trust level may drops that can results in loss of future project contracts. Also delays in single iteration can start a chain of delays in all other iterations.
Here at InfoStreet there are multiple software development projects going on simultaneously, and these projects are knowledge work which require a lot of concentration and thinking, so if people doing these tasks keep on switching from one to the next constantly none of the tasks actually progress and the result is that no work gets done. For example, developer that is developing new features but time to time, unexpectedly, needs to react to hot fixes for found bugs and jump off from developing the new feature, which results in inefficiency. [Will Kelly, Sep 2015] Another example is both Automation and manual testing are performed parallel, and here single Engineer is working on both automation and manual tasks, so it is really difficult to switch between them, as automation tasks involve complex coding and requires continues concentration.
InfoStreet have several clients HughesNet, Topica Plus, Ocean Motors etc. for which we create the software solution, these clients sometimes unclear with the requirements, because of which we are uncertain about the technology to used in the designing and integration of the product. And that leads to loss of time and makes difficult to budget, as a small change in the technology, can changes the whole budget of the project. For example, couple of months before we got the project from HughesNet to automate their web Portal using Selenium Web-driver (it’s an automation tool), but they are not sure about if they want to just focus on the Front end (User Interface) automation or they want to include the backend automation as well, which leads to uncertainty and makes it hard to move forward with the project.
Christopher Lindquist (15 Nov. 2005) said “As many as 71% of software projects that fail do so because of poor requirements management, making it the single biggest reason for project failure”
Here at Infostreet our client’s keeps changing their mind after giving the final requirement because either a competitive landscape shifts, or a new and better technology goes live, which results in rework, recoding and retesting. Documents like User stories, test scripts need to be redesigned and it makes it worse, if the automation of the software is already started, because then we need to make change in the automation coding as well [M. Ikonen & P. Abrahamsson, 2010]
Defect occurs when the software or the system fails to meet the requirements. Defects cause rework, decrease customers’ satisfaction and cost time just to name a few. [Riku Suomela,22 Oct 2015 ] Defects cause extra work at bad times, forcing the programmer to stop with what he or she is currently working on, which cause lower quality on both the bug fix and the stopped work. [Szul, Michael, Jan 2016]
Partially Done Work are Untested Code, Undeployed Code and Undocumented Code, it is not possible to track the progress of this work, because it is very difficult to see the output from partially coded work. Software work-in-process (WIP) is all activity subsequent to business case approval but preceding deployment to production. [Drew Stephens, 31 January 2016] This includes requirements documents, design and functional specifications, project plans, source code, test scripts, and the time required to create these artefacts. For the long project which last for a year or more, WIP keeps on building over the time till it is is deployed to production, which involves high level of risk, in terms of software investment because if the project never delivered or ends prematurely then no business value is generated and the WIP is lost. [P. Abrahamsson, 2010] This happened to Infostreet once with the SkyStreet Product, it was in house product, which failed to build the customer base and this project was ended prematurely.
This waste is caused when testing efforts or the code that has been developed, required rework, because of defects or change in requirement. One of the most common rework at InfoStreet is the Duplicate defects created by different Quality Assurance Engineer and sometimes just before the final delivery of the project, some minor changes have been made which requires the testing of whole application again which is another kind of rework.
This table shows the alignment of the waste with the tools used to resolve it.
| Tools | Waste |
| Kanban | Defects, Partial work done, Changing, Requirements |
| People oriented development | Delays, Task Switching |
| Test-driven development | Delays, Defects, Rework |
| Pair Programming | Delays, Defects |
| Constant Feedback – Inspect and Adapt | Changing Requirements, Unclear Requirements, Rework |
| Engage and Empower everyone | Delays, Task Switching |
| Retrospective | Changing Requirements, Unclear Requirements, Task Switching, defects |
| Root Cause Analysis | Defects, Unclear Requirement |
| Test Automation | Defects, Delays, Rework |
Kanban systems provide a good framework for organizations getting started with lean principles. Kanban board is a visual representation of the work stream, where each work item is represented visually. [M. Cusumano, 2007] Project managers and stakeholders can easily access information on the status of deliverables throughout the project -without being overwhelmed by details. Here at InfoStreet, we are using the tool called Skydesktop, where we created a task for every work item and assign working hours on that, and this task is visible to everyone in the team, so that visibility and the progress of each task can be easily accessed, which helps everyone in the team to track partially done task and also the status of any Defects. Kanban systems specifically focus on the flow of value; in fact, flow and bottlenecks. And this is used to keep the track of changing requirement as well, like whenever project manager gets updates from any client about the requirement change, he created a task in Skydesktop with critical priority with some pop-up alert and assign it everyone in the team, so that communication is quick and everyone is aware of the changes.
In Agile Software development “People factor” has been highly emphasized [A. Cockburn, 2006]. And in Lean thinking respect people is an essentials characteristic [J. K. Liker, 2004], which ranges from more philosophical to more practical aspects such as humanity, respecting culture and customs, enhancing quality of life and enhancing individual creativity and teamwork. In the specific case of InfoStreet, team work is strongly promoted. In fact, the whole segment is organized around Scrum/Kanban teams with a common team goal. Because software development relies on knowledgeable and creative people, teams’ self-organization and empowerment is also strongly promoted. We allow everyone to make switching task decision on their own rather than forcing them to do a particular task, which gives them the sense of responsibility and when team members have responsibility, they can make decisions faster and speed up the development process. [Abbas Moshref Razavi, 2016]
Test-driven development is an software development technique which combines test first development technique where we write the test code and script before writing the any production code, so that code can be write to fulfil the test and refactoring. In other words, testing and coding goes hands in hands. So, to implement this at InfoStreet, instead of writing functional code first and then your testing code as an afterthought, we write Junit test cases first, for every module of code, so that backend testing of that code module can be performed, before verifying the front end output of that module. If the developer knows about the test scenarios, then they will likely to write code which addresses each of the scenarios. [Introduction to Test Driven Development, 2013]
Another concept which we are using here at InfoStreet is Extreme Programming which involves method stubbing, which involves writing method in different classes and writing unit tests for each test conditions before writing the actual code. The developer will then writes the code to fulfil and pass the tests. Both these techniques come from Extreme Programming and both focus on reducing the code waste, unnecessary features, movement, and transition wastes. [M. Poppendieck & M. A. Cusumano, 2012] It is done by two techniques that decrease partially done work: continuous validation, and continuous integration. Continues validation removes extra features and defects, continuous integration makes shorter delays. [C. J. Torrecilla-salinas, J. Sedeño, M. J. Escalona & M. Mejia, 2016] [G. Barabino, D. Grechi & D. Tigano, 2014]
Here at InfoStreet, we highly emphasized on Pair Programming, where minds of two developers is applied on the certain critical task to avoid the quality issues or defects, as defects are more time consuming than the small discussion between the developers about the logics to be used in critical module of the programming. Benefits of Pair Programming is improved quality, because every person think differently, which helps in catching issues before they actually occur. It results in enhanced productivity as someone have experience with the same code and they can suggest a better solutions [M. Poppendieck & M. A. Cusumano, 2012]
Empowering people, encouraging teamwork, and moving decision-making to the lowest possible level are fundamental to any lean implementation. [O. Cawley, X. Wang & I. Richardson, 2012] Instead of placing software development in a separate department called “IT,” software development tends to be thought of as product development and located in line business units. Responsibility for discovering, creating, and delivering value falls to a team that encompasses the complete value stream. [J J. Sutherland & D. Ph, 2009] So here at InfoStreet a value stream team that includes software development will almost certainly include additional functions; in fact, it will probably look like a miniature version of a business, it includes people who understand customers, designers, developers, testers, operations and support.
Here at InfoStreet development are done in small increments, which involves a close collaboration between the team and the the Product Owner with the constant two way feedback loop. And this feedback is very important in inspecting and adapting the product regularly to make sure the correct quality level is maintained and bugs fixed on time. [Kelly Waters, Sept 2010]. For constant feedback from client, we have a client specific manager, who is continues communication with the client to track their need and any change in the requirement, and he is in continues communication with our technical team, so that the feedback can be implemented.
Here at InfoStreet we called it a retrospective meeting which is done after each release to discuss what went well, what didn’t, and what could be done differently in the next iteration or release. [J. Sedeño, M. J. Escalona & M. Mejías, 2015]. And to keep the track of everyone’s suggestion, we created a task in sky desktop and everyone adds their suggestions and viewpoints which are discussed, finalized and implemented and with agile development, these retrospectives enable the team to make small improvements regularly, and tackle changes in manageable small sized pieces that can be actioned immediately.
RCA is a mechanism of analysing the defects, to identify its cause. Here at InfoStreet, we use RCA with the defects reported in both production and test environment. We brainstorm, read and dig the defect to identify whether the defect was missed by tester, developer or was due to incomplete requirement or designs issue. Doing the RCA accurately while helps in avoiding the defects in the future releases. If we find, that a defect was due to design miss, we review the design documents and take appropriate measures. Similarly, if we find that a defect was due to lack of testing, we review our test scripts and update it accordingly to enhance the coverage. Also if we determines the defects are due to requirement miss, then we improve the requirement gathering phase by introducing more reviews sessions.
InfoStreet currently highly concentrates on increasing the test automation coverage. Test automation is to test the application with the help of program, which human intervention, so that the tedious testing task can be done quickly. For this InfoStreet is using Selenium web-driver, on with we are writing the framework on TestNG using Java, though it’s a continuous improvement process but it already started giving the results. Automation testing will help to speed up the testing process and also the defects can be found quickly.
A key Lean element is “optimize the whole.” as if you try to fix one part of the software development process, you could end up breaking something else thus making the overall process worse. [Gilda Bonanno, 2012] So each part of the lifecycle – coding, testing, analysis, and deployment, should be synchronized and here at InfoStreet, we have to ensure that all of these processes are aligned so that we are make the best use of our limited resources and it is a difficult task as in knowledge work, resources are intangible and even harder to manage. They’re the skills, knowledge, time, and energy of your employees. [Hamlet D’Arcy, 2011] And to fully utilize the potential of all our employees, we have a strong infrastructure to communicate, share, and align around common goals.
When a customer reports any defect in our software or web application, we do an incidental root-cause analysis to address the code quality or configuration problem, as I have mentioned above. We deploy tools, introduce new processes, measure constantly and yet – a few months later, sometimes we encounter a similar problem, so this is a continues improvement process which keeps improving with regular feedback [Tanmay Vora, Mar 2015]
Writing production code is something lot more than just making the unit tests pass. The more important thing here is to know what customer needs and for that which test to write. So it is highly promoted that developer and business analyst should work closely. Analysts document the behaviour of the system in one format which we called a user stories, developers re-document it in another format functional tests, and QA re-encodes it in another format of test plans, which can be used by either developer, tester or project manager [Régis Medina, 2013]
Putting all of this together with the better optimized workflow, it can be extremely beneficial for our organisation in terms of improved efficiency, better products, which can directly translate to increased revenue and can ultimately make organisation more competitive.[Noni Cabrera,26 Jan. 2015]
As InfoStreet is growing and size of the team is expanding, now have globally widespread teams, agile methodologies efficiency is reducing as, one of the limitations of Agile technique is that it does not perform effectively on big Projects which have large team size and widespread teams, So to overcome this limitation, Scaled Agile Framework which also known as SAFe is a proven, publicly available framework for applying lean and agile practices at enterprise scale. This framework helps large team in an organisation to meet the strategic goals by providing a centralized strategy. SAFe has three levels that centralize the strategic themes of an organization [Mike Hube, 12 May 2015]:
At this level, we have large business initiatives also called as business Epics which required some approvals from the higher management before teams start working on it. Here the cost benefits of such kind of initiatives are also calculated. The main events of this levels are to Review intake requests and Ensure the enterprise architecture has sufficient runway to support the new initiatives. The Business initiates are stored at the Portfolio Backlog and when after getting approved for implementation, they are divided into features and feed to the Program level
At this level, we looks into the Project management, release management aspects of the features and consider the multiple team to divide the work. Here the analysis is performed to dividing the features into smaller pieces which is then called as the User Stories. The prioritization of each feature is performed very carefully by the project management team
Infostreet uses the algorithm called “Weighted Shortest Job First” to define the sequencing of features, this algorithm consider the factors like time, cost of delay, business values and other factor for making the decision.
At this level, user stories are implemented, this is the actual development phase where we are running Agile development sprints. Here we are using techniques like Test driven development, paired programing to ensure the quality and scalability. And after each sprint, developer provided the build to the tester, who gives feedback to the developer and raises the defects which gets fixed and tester is provided with the new build in the next iteration. We generally have 3-4 iteration cycle, before the actual product is delivered.[Izenbridge, 20 Jan. 2016]
Infostreet had just started implementing this Scaled Agile Framework for the TopicaPlus (InfoStreet’s Client) team, which is still in initial planning phases and after discussion with my project manager I come up with these that centralize the strategic themes of an organization.
Software companies select those elements from Agile and Lean that suit well in a software development context, creating their own interpretation of Lean Software Development. InfoStreet considers elements that works better for them, like combination of elements from Agile Software development – to achieve flexibility, Lean thinking – to scale Agile and make software development processes more efficient, and good practices of software engineering – to make coding process cleaner. The “softer” side of software development has been highlighted [A. Cockburn, and J. Highsmith, 2001]. So, it is interesting to observe that technical and human aspects were balanced. Thus, InfoStreet recognizes the importance of the social aspect of software development, but without forgetting engineering practices, similarly important. [C. Ebert, P. Abrahamsson A, & N. Oza, 2012] For example, from a technical perspective practices such as continuous integration, pair programming concurrent development and testing, test automation and static code analysis which is Test driven development as we discussed earlier were considered as essential during the focus groups.
We found the Kanban is more and more used and helps to visualize and manage work in progress, and WIP limits are important element for achieving flow in a software development context [P. Middleton, 2001] [A. Flaxel, A. Cookson, 2005]. As as mentioned previously continuous integration and test automation appeared as keys to support flow by frequent and smaller builts. [P. Middleton, D. Joyce, 2012] Also, the concepts of transparency and streamlined communication, emphasized in previous studies [B. Staats, D. Brunner &D. Upton, 2011] were found essential. What is different in the case of InfoStreet is its strong reliance on Skydesktop software for supporting transparency. InfoStreet uses Skydesktop software not just for tracking the status of the product (features and user stories) but also for making visible everything in its product development cycle without restrictions in sharing information, from development tasks to business goals and strategy.
Based on the results of the assessment, some lesson can be learned. On the one hand, we learned that setting up Agile and Kanban teams appears as a relatively simple task. Aspects such as self-organization and focusing on customer value got also good scores in the assessment. [M. Mehta, D. Anderson & D. Raffo, 2008] However, involving business management and achieving company level agility remains still as challenges, which challenges also to achieve short feedback loops and prevents flow. Thus, Lean thinking supports in scaling Agile from development teams to software development units.
The objective of this study was to investigate how, lean principles can be applied to agile software development to minimize the scrum issues and improved the scrum process certainty, productivity, and stability. Also, Lean thinking opens a wide variety of opportunities for advance research. The challenges recognized when moving towards a combination of Lean and Agile software development particularly advantage required further study.
As the study was conducted on a particular company, it is possible that my observations cannot be fully generalized to specific setting. However, Lean Software Development is a relatively unexplored phenomenon. Since InfoStreet is transforming from Agile software model to Lean agile software development model, it brings a new insights and guides organizations pursuing a similar endeavor. It may be also questioned whether the Lean implementation of InfoStreet is fully conformant with Lean thinking. My study does not focus on epistemological concerns. However, I convinced that InfoStreet is continuously trying to transform itself to a Lean organization.
As future work, elements identified as important, as well as strengths and challenges, are topics of interest to be studied in greater detail. We need to conduct similar studies in other companies as well, to enable analytical generalization and extend results to cases with common characteristics. [Pilar Rodriguez, 2016]
Reference:
You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.
Read moreEach paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.
Read moreThanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.
Read moreYour email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.
Read moreBy sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.
Read more