Agile software development

Agile software development is a conceptual framework for undertaking software engineering projects. There are a number of agile software development methods, such as those espoused by The Agile Alliance.

Most agile methods attempt to minimize risk by developing software in short timeboxes, called iterations, which typically last one to four weeks. Each iteration is like a miniature software project of its own, and includes all of the tasks necessary to release the mini-increment of new functionality: planning, requirements analysis, design, coding, testing, and documentation. While an iteration may not add enough functionality to warrant releasing the product, an agile software project intends to be capable of releasing new software at the end of every iteration. At the end of each iteration, the team reevaluates project priorities.

Agile methods emphasize realtime communication, preferably face-to-face, over written documents. Most agile teams are located in a bullpen and include all the people necessary to finish software. At a minimum, this includes programmers and their "customers" (customers are the people who define the product; they may be product managers, business analysts, or actual customers). The bullpen may also include testers, interaction designers, technical writers, and managers.

Agile methods also emphasize working software as the primary measure of progress. Combined with the preference for face-to-face communication, agile methods produce very little written documentation relative to other methods. This has resulted in criticism of agile methods as being undisciplined.

History

The modern definition of agile software development evolved in the mid 1990s as part of a reaction against "heavyweight" methods, as typified by a heavily regulated, regimented, micro-managed use of the waterfall development model. The processes originating from this use of the waterfall model were seen as bureaucratic, slow, demeaning, and contradicted the ways that software engineers actually perform effective work.

A case can be made that agile and iterative development methods are a return to development practice seen early in the history of software development [1].

Initially, agile methods were called "lightweight methods." In 2001, prominent members of the community met at Snowbird and adopted the name "agile methods." Later, some of these people formed The Agile Alliance[2], a non-profit organization that promotes agile development.

Early agile methods?acreated prior to 2000?ainclude Scrum (in management) (1986), Crystal Clear, Extreme Programming (1996), Adaptive Software Development, Feature Driven Development, and DSDM (1995).

Extreme Programming, while it may not have been the first agile method, inarguably established the popularity of agile methods. Extreme Programming was created by Kent Beck in 1996 as a way to rescue the struggling Chrysler Comprehensive Compensation (C3) project. While that project was eventually canceled, the methodology was refined by Ron Jeffries' full-time XP coaching, public discussion on Ward Cunningham's Portland Pattern Repository wiki and further work by Beck, including a book in 1999.^[1] Elements of Extreme Programming appear to be based on Scrum and Ward Cunningham's Episodes pattern language.

Principles behind agile methods ?a The Agile Manifesto

Agile methods are a family of development processes, not a single approach to software development. In 2001, 17 prominent figures ^[2] in the field of agile development (then called "light-weight methodologies") came together at the Snowbird ski resort in Utah to discuss ways of creating software in a lighter, faster, more people-centric way. They created the Agile Manifesto, widely regarded as the canonical definition of agile development, and accompanying agile principles.

Some of the principles behind the Agile Manifesto^[3] are:

• Customer satisfaction by rapid, continuous delivery of useful software
• Working software is delivered frequently (weeks rather than months)
• Working software is the principal measure of progress.
• Even late changes in requirements are welcomed.
• Close, daily, cooperation between business people and developers
• Face-to-face conversation is the best form of communication.
• Projects are built around motivated individuals, who should be trusted
• Continuous attention to technical excellence and good design.
• Simplicity
• Self-organizing teams
• Regular adaptation to changing circumstances

The publishing of the manifesto spawned a movement in the software industry known as agile software development.

In 2005, Alistair Cockburn and Jim Highsmith gathered another group of people ?a management experts, this time ?a and wrote an addendum, known as the PM Declaration of Interdependence.

Comparison with other methods

Agile methods are sometimes characterized as being at the opposite end of the spectrum from "plan-driven" or "disciplined" methodologies. This distinction is misleading, as it implies that agile methods are "unplanned" or "undisciplined." A more accurate distinction is to say that methods exist on a continuum from "adaptive" to "predictive".^[4] Agile methods exist on the "adaptive" side of this continuum.

Adaptive methods focus on adapting quickly to changing realities. When the needs of a project change, an adaptive team changes as well. An adaptive team will have difficulty describing exactly what will happen in the future. The further away a date is, the more vague an adaptive method will be about what will happen on that date. An adaptive team can report exactly what tasks are being done next week, but only which features are planned for next month. When asked about a release six months from now, an adaptive team may only be able to report the mission statement for the release, or a statement of expected value vs. cost.

Predictive methods, in contrast, focus on planning the future in detail. A predictive team can report exactly what features and tasks are planned for the entire length of the development process. Predictive teams have difficulty changing direction. The plan is typically optimized for the original destination and changing direction can cause completed work to be thrown away and done over differently. Predictive teams will often institute a change control board to ensure that only the most valuable changes are considered.

Agile methods have much in common with the "Rapid Application Development" techniques from the 1980's as espoused by James Martin and others (see RAD ).

Contrasted with iterative development

Most agile methods share iterative development's emphasis on building releasable software in short time periods. Agile methods differ from iterative methods in that their time period is measured in weeks rather than months and work is performed in a highly collaborative manner. Most agile methods also differ by treating their time period as a strict timebox.

Contrasted with the waterfall model

Agile development has less in common with the waterfall model. In some eyes the waterfall is discredited, but as of 2004, this model is still in common use.^[5] The waterfall model is the most predictive of the methodologies, stepping through requirements capture, analysis, design, coding, and testing in a strict, pre-planned sequence. Progress is generally measured in terms of deliverable artifacts?arequirement specifications, design documents, test plans, code reviews and the like. The waterfall model can result in a substantial integration and testing effort toward the end of the cycle, a time period typically extending from several months to several years. The size and difficulty of this integration and testing effort is one cause of waterfall project failure.^{[citation needed]} Agile methods, in contrast, produce completely developed and tested features (but a very small subset of the whole) every few weeks or months. The emphasis is on obtaining the smallest workable piece of functionality to deliver business value early, and continually improving it/adding further functionality throughout the life of the project.

Some agile teams use the waterfall model on a small scale, repeating the entire waterfall cycle in every iteration.^{[citation needed]} Other teams, most notably Extreme Programming teams, work on activities simultaneously.

Contrasted with "cowboy coding"

Cowboy coding is the absence of a defined method: team members do whatever they feel is right. Agile development's frequent reevaluation of plans, emphasis on face-to-face communication, and relatively sparse use of documents sometimes causes people to confuse it with cowboy coding. Agile teams, however, do follow defined (and often very disciplined and rigorous) processes, distinguishing agile approaches from cowboy coding.

As with all methodologies, the skill and experience of the user(s) define the degree of success and/or abuse of such activity. The more rigid controls and/or checks and balances systematically embedded within a process offer stronger levels of accountability of the user(s). It is the degradation of well-intended procedures that lead to activities often defined as Cowboy coding.

Suitability of agile methods

Although agile methods differ in their practices, they share a number of common characteristics, including iterative development, and a focus on interaction, communication, and the reduction of resource-intensive intermediate artifacts. (Cohen et al., 2004)^[6] The suitability of agile methods in general, can be examined from multiple perspectives. From a product perspective, agile methods are more suitable when requirements are emergent and rapidly changing; they are less suitable for systems that have high criticality, reliability and safety requirements, although there is no complete consensus on this point (Cohen et al., 2004).^[6] From an organizational perspective, the suitability can be assessed by examining three key dimensions of an organization: culture, people, and communication. In relation to these areas a number of key success factors have been identified (Cohen et al., 2004)^[6]:

• The culture of the organization must be supportive of negotiation
• People must be trusted
• Fewer but more competent people
• Organizations must live with the decisions developers make
• Organizations need to have an environment that facilitates rapid communication between team members

The most important factor is probably project size (Cohen et al., 2004).^[6] As size grows, face-to-face communication becomes more difficult. Therefore, agile methods are more suitable for projects with small teams, with fewer than 20 to 40 people.

In order to determine the suitability of agile methods individually, a more sophisticated analysis is required. The DSDM method, for example, provides a so-called ??suitability-filter?ˉ for this purpose. Also, the Crystal family of methods provides criteria on how to select the method for a given project. The selection is based on project size, criticality and priority. However, other agile methods do not provide such explicit instruments to assess their suitability for a project.

Some agile methods, like DSDM and Feature Driven Development (FDD), are claimed to be suitable for any agile software development project, regardless of situational characteristics (Abrahamsonn et al., 2003).^[7]

A comparison of agile methods will reveal that they support different phases of a software development life-cycle to varying degrees. This individual characteristic of agile methods can be used as a selection criteria for selecting candidate agile methods.

Agile development has been widely documented (see Experience Reports, below, as well as Beck^[1] pg. 157, and Boehm and Turner^[8] pg. 55-57) as working well for small (<10 developers) co-located teams. Agile development is expected to be particularly suitable for teams facing unpredictable or rapidly changing requirements.

Agile development's applicability to the following scenarios is open to question:

• Large scale development efforts (>20 developers), though scaling strategies have been described.^[9]
• Distributed development efforts (non-co-located teams). Strategies have been described in Bridging the Distance^[10]and Using an Agile Software Process with Offshore Development^[11]
• Mission- and life-critical efforts
• Command-and-control company cultures

Barry Boehm and Richard Turner suggest that risk analysis be used to choose between adaptive ("agile") and predictive ("plan-driven") methods.^[8] The authors suggest that each side of the continuum has its own home ground:

Agile home ground:

• Low criticality
• Senior developers
• Requirements change very often
• Small number of developers
• Culture that thrives on chaos

Plan-driven home ground:

• High criticality
• Junior developers
• Low requirements change
• Large number of developers
• Culture that demands order

Adaptation of agile methods

A method should be flexible enough to allow adjustments during project execution. There are three key issues related to the topic of adaptation of agile methods: the suitability of agile methods (in general and in particular), method tailoring, and finally, project management support.

Agile methods and method tailoring

In the literature, different terms refer to the notion of method adaptation, including ??method tailoring?ˉ, ??method fragment adaptation?ˉ and ??situational method engineering?ˉ. Method tailoring is defined as:

A process or capability in which human agents through responsive changes in, and dynamic interplays between contexts, intentions , and method fragments determine a system development approach for a specific project situation.^[12]

Potentially, almost all agile methods are suitable for method tailoring. Even the DSDM method is being used for this purpose and has been successfully tailored in a CMM context.^[7] Situation-appropriateness can be considered as a distinguishing characteristic between agile methods and traditional software development methods, with the latter being relatively much more rigid and prescriptive. The practical implication is that agile methods allow project teams to adapt working practices according to the needs of individual projects. Practices are concrete activities and products which are part of a method framework. At a more extreme level, the philosophy behind the method, consisting of a number of principles, could be adapted (Aydin, 2004).^[12]

In the case of XP the need for method adaptation is made explicit. One of the fundamental ideas of XP is that there is no process that fits every project as such, but rather practises should be tailored to the needs of individual projects. There are also no experience reports in which all the XP practises have been adopted. Instead, a partial adoption of XP practices, as suggested by Beck (Kent Beck), has been reported on several occasions.^[13]

A distinction can be made between static method adaptation and dynamic method adaptation.^[14] The key assumption behind static method adaptation is that the project context is given at the start of a project and remains fixed during project execution. The result is a static definition of the project context. Given such a definition, route maps can be used in order to determine which structured method fragments should be used for that particular project, based on a predefined sets of criteria. Dynamic method adaptation, in contrast, assumes that projects are situated in an emergent context. An emergent context implies that a project has to deal with emergent factors that affect relevant conditions but are not predictable. This also means that a project context is not fixed, but changing during project execution. In such a case prescriptive route maps are not appropriate. The practical implication of dynamic method adaptation is that project managers often have to modify structured fragments or even innovate new fragments, during the execution of a project (Aydin et al, 2005).^[14]

Agile methods and project management

Agile methods differ to a large degree in the way they cover project management. Some methods are supplemented with guidelines on project management, but there is generally not comprehensive support.^[7]

PRINCE2 has been suggested as a suitable, complementary project management system.^[15]

Agile methods

Some of the well-known agile software development methods:

• Extreme Programming (XP)
• Scrum
• Agile Modeling
• Adaptive Software Development (ASD)
• Crystal Clear and Other Crystal Methodologies
• Dynamic Systems Development Method (DSDM)
• Feature Driven Development
• Lean software development
• Agile Unified Process (AUP)
• Dialogue-Driven Development aka d3

Other approaches:

• Agile Documentation
• ICONIX
• Microsoft Solutions Framework (MSF)
• Agile Data Method
• Database refactoring

Examples of the use of agile principles in non-software activities:

• Lean manufacturing

Criticism

Agile development is sometimes criticized as cowboy coding. Extreme Programming's initial buzz and controversial tenets, such as pair programming and continuous design, have attracted particular criticism, such as McBreen^[16] and Boehm and Turner.^[8] However, much of the criticism was believed by Agile practitioners as misunderstandings about agile development.^[17]

In particular, Extreme Programming is reviewed and critiqued by Matt Stephens' Extreme Programming Refactored.^[18]

Criticisms include:

• lack of structure and necessary documentation
• only works with senior-level developers
• incorporates insufficient software design
• requires too much cultural change to adopt
• can lead to more difficult contractual negotiations

The criticisms regarding insufficient software design and lack of documentation are addressed by the Agile Modeling method which can easily be tailored into agile processes such as XP.

Experience reports

Agile development has been the subject of several conferences. Some of these conferences have had academic backing and included peer-reviewed papers, including a peer-reviewed experience report track. The experience reports share industry experiences with agile software development.

As of 2006, experience reports have been or will be presented at the following conferences:

• XP (2000, 2001, 2002, 2003, 2004, 2005, 2006)
• XP Universe (2001)
• XP/Agile Universe (2002,2003, 2004)
• Agile Development Conference 2003, 2004) (peer-reviewed; proceedings published by IEEE?)
• Agile (2005, 2006) (peer-reviewed; proceedings published by IEEE)