A software development methodology is a framework for planning, controlling and processing the transfer of technical information.

The software development industry is a relatively young and therefore the methodologies employed to design, build and maintain software solutions have also evolved and morphed over the past sixty years. The first published framework specifically designed for building software was in the 1960’s and it was called “System Development Life Cycle” (SDLC). It was created for building and maintaining large information systems that dealt with heavy number crunching for big businesses.

Development Methodologies and Frameworks

There are a number of different development methodologies and frameworks that exist today. They can either be either rigid systems that are implemented in certain steps within the production lifecycle or, encapsulate the full day to day work cycle. They can also be flexible frameworks that companies can utilise to create customised steps for a specific team or organisations.

All of the development frameworks have both strengths and weaknesses, so it’s important to know what kind of project you are working on, and what are the inherent technical, project and team limitations that you will have to deal with in order to understand which methodology is best suited for you and your project. Remember that the development methodology is a tool used to achieve the goal of building the best software application possible, it is not the objective, so don’t get too carried away with all the hype and marketing crap.

The overall goal is to create a repeatable and predictable process that both improves productivity and raises the quality of the product.

Mixing and Matching Development Models

It is a common practise to mix and match different software development models and create a customised hybrid methodology that is best for your project. Regardless of the framework that you decide upon, documentation is essential to the success of the project and is usually written in parallel with the development. The overall success of a project can be derived from how accurately the project developed in relation to the initial project plan, or in my case the “pixel accurate click sequence” of screenshots presentation.

The Evolution of Software Development Methodologies

I will now briefly go through the evolution of software development methodologies and frameworks chronologically and provide some insight to the pro’s and con’s of each technique.

1. Waterfall – 1970 – Waterfall is a rigid linear development method, where the software designer creates a product “blue print” that includes every stage of development for the product. It is designed to focus on a specific set of goals for each development stage. Once a specific phase has been completed then the next stage is built upon the previous one. The main focus of this technique is efficiency and adhering to a rigid timetable. The first problem with the waterfall technique is that your clients may not know exactly what they want, and they will only understand when they see the finished product. Waterfalling does not take into account inevitable changes and rapid advances in technology that may occur throughout the building process, meaning that the product will be out-of-date by the time it is completed. The end results will probably also fall short of customer expectations and changes and new features are often difficult to implement because of the rigid nature of the blueprint methodology. I find the waterfall method is ineffective and time consuming without coming close to producing the expected results, as well as it being a hardware building technique that has been adapted to the software industry.
2. Joint Application Development (JAD) – 1970 – JAD is a methodology that includes all of the relevant stakeholders in the design process, end-users, marketing, sales and the development teams. The stakeholders focus on the “business problem” and together they create a collaborative product solution over the course of a series of meetings. The underlying principle of this methodology is that by including all of the interested parties in the process the product design is more focused and represents a collaborative effort of both end users and developers. The main advantage to using the JAD method is that it decreases the time and costs associated with the requirement definition stage of development. It brings industry experts together and increases knowledge transfer. It is easy to understand how to effectively use this technique. The disadvantage of this technique is that it requires detailed preparation, highly skilled and experienced management and can potentially waste a lot of time. Secondly, when conflicting points of view are provided by junior employees, the company management hierarchy can often be detrimental and intimidate other junior team members. This can cause a stumbling block to obtaining true and meaningful product requirements and finally JAD does not take into account the less vocal participants opinions, ideas and thoughts.
3. Spiral – 1986 – The spiral lifecycle model is an extension of the waterfall methodology. The product functional/server architects start the initial product prototype design and try to keep it basic, focusing on solving a simple set of problems with minimum risk, then slowly expanding and spiraling out to include more functionality. Then, after testing the product is deployed. As the name suggests, the product design spirals out to include more feature and functions as the project progresses. The basic methodology follows four basic steps. First, the product functional architect writes a detailed functional specification. Second, a preliminary design is then created for the application. Third, a basic prototype is built and the pro’s and con’s are evaluated and then finally, the second prototype is built and the process is repeated. As with the waterfall methodology, the spiral model is also rigid and difficult to change the product trajectory after developments has commenced. It also requires highly skilled, experienced and product segment knowledge as well as experienced management and development teams. It can also be time consuming as well as relying on a rigid pre-designed product blueprint.
4. Lean Development – 1988 – The main focus of lean development is to drastically reduce the amount time and effort involved in building a software solution by two thirds. This means building a lean simple product in a third of the time it would usually take to build a product with the same customer goals. The main principles in lean development are as follows; Customer satisfaction, provide value for money, active customer participation, each stage is a team effort, build a domain solution, everything is changeable, complete the tasks quickly, 80% solution today instead of 100% tomorrow and always maintain minimalism. The main problem with using the lean methodology is what I refer to as “feature greed”, which refers to the inability of stakeholders to build simple features without adding more and more functionality. Or, demanding that features that exist in competitor solutions is a basic requirement (there is no true method to verify this assumption) such as customer usage. Some vendors are glad for their functionality to be mimicked and copied, as they can then leverage this information to prove that they are the true market leaders. Furthermore, large companies can also pack useless features into a design in order to drain competitors resources. Did someone say “Star Wars” project to the Soviet Union.
5. Rapid Application Development (RAD) – 1991 – RAD initially evolved as an alternative to the traditional waterfall methodology. RAD software development put less emphasis on planning and more emphasis on process. Adversely to the waterfall methodology, which implements highly defined and detailed specifications to be authored before starting the development phase. The RAD approach focuses on adaptability and adjusting the requirements after knowledge has been gained as the project progresses. Prototypes are often used in addition to the design specification. RAD produces a good quality application, minimises risks and improves the ability to deliver on time and on budget. the problem with RAD is it requires a completely new and different approach to application building (people are almost always averse to change) there is less control of the production process and projects built according to this methodology often suffer from poor design and the application often lacks scalability.
6. Dynamic Systems Development Model (DSDM) – 1994 – DSDM is the second generation of the RAD methodology. It is an extremely well documented methodology and provides some of the best supported training compared with any other development technique or methodology. It relies heavily on user participation, enables teams to make their own decisions, is designed to deploy frequent product deliveries, reversible functionality during development, integrated testing throughout the development cycle and collaboration between all stakeholders.
7. Crystal Method – 1995 – The Crystal Method is the “people method”. It’s main focus is on the people, community, skills, talent and communication. This method assumes that it is the people and not the process which determines the quality of a product. The word “crystal” refers to the many facets of a gemstone. Each facet represents a surface with an underlying set of values and principles. Each facet represents a specific set of elements such as techniques, roles, tools, and standards. This method relies heavily on the “use case” as the basis for the organisation framework.   
8. Scrum – 1995 – Scrum is an agile method for project management. It’s goal is to speed up production and streamline and simplify process heavy methodologies. The scrum method is characterised by the following attributes; a dynamic prioritised backlog, the completion of tasks in sprints, a daily scrum meeting (where daily tasks are discussed) and obstacles are raised, quick planning sessions and a retrospective where the previous sprint is discussed. The scrum is run by a “scrum master”, whose main job is to remove obstacles and help the team deliver their sprint goals. The scrum master is “not the leader of the team” but acts as a productivity enhancer for the team and any destabilising influences.
9. Rational Unified Process (RUP) – 1996 – RUP is an iterative approach that takes into account the inevitable and ever changing project requirements, continuous integration and continuous product delivery which speeds up product delivery by releasing the product in small iterations. Reduces the risk by releasing small application packets and focuses on the reuse of common parts. Early testing and a learn as you advance system for developers (the development process is improved and streamlines along the way). An assessment is performed at the end of each iteration and looks at the status of the project from a product perspective and also analyses what should be improved in the organisation and the process.
10. Feature Driven Development (FDD) Since 1997 – FDD states that a rigid system is required when attempting to build large and scalable projects. A detailed and well defined process works best. Each process step must be clearly understood by all team members. The process is a tool and not the objective. Short iterations of a feature driven life cycle work the best.
11. Extreme Programming (XP) -1999 – The main goal of extreme programming is to reduce the cost of changing software requirements. The core principles of XP are early test driven development that is closely tied to a customer reference point. Tasks are performed by implementing “pair programming” and continuously integrated into the product. Design improvements are constantly added, changed and improved. The product is delivered in small releases and the design is kept very simple. There is collective code ownership and coding standards and conventions are rigorously maintained.
12. Agile Software Development – 2005 – There are a number of agile software development methodologies out there, such as Crystal Methods, Dynamic Systems Development Model (DSDM), and Scrum. The agile method minimises risk by developing software in short iterations, which last one to four weeks. One of the main goals when designing a feature is to create a series of processes with the shortest path between two points. Each iteration includes designing the functionality, planning, requirements analysis, design, coding, testing, and documentation. Agile focuses on realtime face-to-face communication over written documents. Agile methods produce very little written documentation relative to other methods.

Remaining Focused on the Main Goal

The goal of software development methodologies is to ease the process of development. For decades software development organisations have struggled to find repeatable and predictable processes that improve productivity and quality. Some try to systematise or formalise the task of designing software and others implement management techniques to designing software. Many software projects do not meet their expectations in terms of functionality, cost, or delivery schedule because is the inherent complexity of software construction.

“My definition of an expert in any field is a person who knows enough about what’s really going on to be scared.” – P. J. Plauger

The advancement in technology has increased exceptionally over the course of the previous century. Technologies are being produced at lightening speed and their inherent technological advances double almost every three years. This rate of improvement is also moving faster and faster. A university student that studies a specific technology is theoretically out-of-date by the time they complete their studies, for example. Every day a new vendor creates a new technology in an attempt to win the consumer war against another company. This consumerism is driving the rate of advancement faster and faster each year.

“Technological progress is like an axe in the hands of a pathological criminal.” Albert Einstein

So, Why Do I Care….

I have worked on a range of different technologies and helped convert them into software implementations with many development teams. These teams all employed a variety of techniques, from the oldest tried and tested formal technique to the cutting edge hybrid mashups. The biggest stumbling block that I have encountered is ego and politics and not specifically this type of methodology or that development technique.

No matter what technique or techniques you choose to employ in your endeavour to productise technology, it is done with people. Even the smartest individuals in the world require a range of talents in order to produce their dreams and cover their personal blind spots. Being “symbiotic organisms” that must communicate with each other to achieve our goals it is essential that we finds ways to not only interact, but also create repeatable mechanisms that we can improve upon. That is why, all this technical “mumbo jumbo” is important.

My Personal Preference

I personally like to write “thin user stories” heavy based on visual guides. I create “pixel accurate” click sequences for a specific features using visually detailed UI/UX screenshot decks. I believe that nothing can replace knowledge and experience so I prefer working with “pair programming” developers that know how to share information, rather than personal ownership of code.

Resist input from participants whose only knowledge input is copying functionality from competitor solutions, copying is not leading. Verifying “actual usage” of a specific function or feature is almost impossible. Try to expand your products functionality by conservatively solving problems that derive from actual product usage and not perceived usage or playing or demoing the product, as this is a lie.

To Conclude

Finally, I think that the best results can be achieved in four to five week sprints and I prefer continuous integration and continuous delivery because it creates a calmer and less stressed work environment and gets the goods to the customer, fast. I also believe that frontal face to face communication is the absolute best method for sharing and transferring information and not formal meetings. I see no advantage in involving large groups of people. I find that those individuals that do like to create lots of meetings and informal brainstorming sessions, simply lack confidence and waste time by creating doubt and confusion to take you eye off the ball. Too many cooks spoil the broth, so keep your interactions lean, preferably one-on-one meet-ups focused on the task at hand.

So basically, I like to mix it up drawing from a range of development techniques and methodologies. I try not to get too attached to a specific methodology and remind myself to have fun and be passionate about my work.

After all, we are here to build the future and make things better.