Most of the Software companies large, medium, small, or startup usually face issues in their software development projects and its delivery. The issues can vary from lack of documentation, lack of following the process, lack of process governance, lack of the integration and collaboration between the teams, lack of requirements traceability, lack of technology management, …etc.

We have discussed in a previous post the trends of software projects and that large software projects on the average run 66% over budget and 33% over schedule; as many as 17% of projects go so badly that they can threaten the existence of the company.

Therefore, some methods and techniques started to exist to tackle the software process issues to suggest different improvements and identify issues and inefficiencies in the process. These methods became a standard which the companies can follow to improve their software process. Moreover, each method established its ecosystem, from providing the training and certificates for the method to provide consultancy to help companies to improve based on actual practices.

In this article, It will be good to ask yourself if the software process improvement is a peril to have or a promise for a better change for the organization and to have a superior advantage in the market. For answering this, we will discuss in this article what is SPI?, what is the SPI process steps? what are the different methods?, the motivators and demotivators of SPI projects, what are the common success factors for SPI project implementation?

What is SPI?

Software Process Improvement (SPI) methodology is defined as a sequence of tasks, tools, and techniques to plan and implement improvement activities to achieve specific goals such as increasing development speed, achieving higher product quality or reducing costs.

This definition is combined from [1][2]. SPI can be considered as process re-engineering or change management project to detect the software development lifecycle inefficiencies and resolve them to have a better process. This process should be mapped and aligned with organizational goals and change drivers to have real value to the organization.

SPI mainly consists of 4 cyclic steps as shown in the figure below, while these steps can be broken down into more steps according to the method and techniques used. While in most cases the process will contain these steps.

Current Situation Evaluation

This step is the initial phase of the process and it is mainly to assess the current situation of the software process by eliciting the requirements from the stakeholders, analyzing the current artifacts and deliverables, and identifying the inefficiencies from the software process. The elicitation can be conducted through different techniques. For example, individual interviews, group interview, use-case scenarios, and observations.

The key considerations in this step to identify organization goals and ask the solution-oriented questions. Moreover, identifying the measurement using the GQM (Goal – Question – Metric) technique that will help in measuring the current status and measuring the effectiveness of the improvement process.

Improvement Planning

After analyzing the current situation and the improvement goals, the findings should be categorized and prioritized according to which one is the most important or have the most severity. We should observe what is the new target level of improvements should look like.

Moreover, in this step, the gap between the current level and the target level should be planned in terms of a set of activities to reach that target. These activities should be prioritized with the alignment of the involved stakeholders and the organization goals, for example, if the project is using the CMMI model, the target could be reaching maturity level 4 and the company at level 3, in that case, the plan should be focused on the process areas and their activities which is related to that level of improvement with the alignment of the organization goal.

Improvement Implementation

In this step, the planned activities are executed and it puts the improvements into practice and spreads it across the organization, what can be effective at the 2nd, 3rd, and 4th step that planning and implementation could be an iterative way, for example, implementing improvement for improving requirements first, then implementing the reduction for testing process time, and so forth. This iterative way of implementation will help the organization to realize the early benefits from the SPI program early or even adopt the plan if there is no real impact measured from the improvement.

Improvement Evaluation

What is cannot be measured cannot be improved, that’s why in this step, the impact measurement is applied compared with the GQM. The before improvement measures, after the improvement measures, and the target improvement measure. Measurement, in general, permits an organization to compare the rate of actual change against its planned change and allocate resources based on the gaps between actual and expected progress.

Why are Companies Seeking SPI – The motivators?

There are a lot of motivators from different perspectives for companies, management perspectives, sales perspectives, employee perspectives, and others. I will mention here the most common motivators for SPI:

Standardization and Process consistency

To have a standard and practical process for software development mapped to organization goals and strategy.

Cost Reduction

To improve projects cost by enhancing the process and eliminate issues, redundancies, and deficiencies.

Competitive Edge

Being certified in CMMI for example, can put the company in higher competitive edge and make it gain more sales due to the evidence of existing mature software process based on standard method.

Meeting targets and reduce time to market

Meeting organization goals, projects delivery, quality standards, valuable products, professional documentation are outputs from SPI.

Improve customers satisfaction

Project delivery on time and based on the specification with high quality will improve customers satisfaction and improve the sales process.

Job satisfaction, Responsibilities, and Resource Management

Employees get job satisfaction from producing a good quality product and knowing what to do without workload and the time consumed to resolve conflicts or to eliminate issue due to an immature process.

Automation and Autonomy

Introducing tools to automate things and improve quality and ensure consistency. Moreover, enabling different employees to play different roles in the project.

Proven outcome

There is a lot of evidence for the value of SPI projects which are successfully implemented.

Despite these motivators, companies may be afraid to go through SPI project because a lot of factors and some companies may already have sufficient process and have a good revenue and it is successful, is it a necessity for that kind of companies to pursue SPI project!

I think not, if you already do not have a pain you do not have too, so, what about the other companies?

What are the Demotivators for SPI?

Similar to the motivators, demotivators can be taken from different perspectives, and here are the common demotivators for SPI and they are very correlated:

Time pressure

Due to the nature of the companies to deliver the projects on time, they faced a lot of time pressure which make it harder for them to dedicate time to the SPI project. While I see this as weakness and actually a driver for SPI.

SPI took a long time and it is a costly process while It is necessary if you have issues as discussed before.

Budget Constraints

As we just mentioned SPI is a costly process, because it needs time and dedicated resources, and not only that but also skilled resources especially in SPI. And you may need SPI consultant and train the resources and orient them on SPI initiative.

Inadequate metrics

Most of the small companies do not have metrics to measure and compare their progress or improvement which make it sometimes impossible to identify measure the improvements of the SPI.

Lack of Management Commitment

It is mainly because the management cannot understand the benefit from SPI and they do not fully support doing this change as well as the other factors like lack of resources, budget, time, …etc.

Staff turnover

Sometimes, the company has a high staff turnover which can be an issue to impose the SPI culture change and this can lead to endless SPI.

Micro Organization

Some organization are very small and have very few resources. The SPI will be too big for that kind of companies.

Bad Experience and lack of evidence for direct benefits

Some organizations may face a bad implementation for SPI that made them do not want to be involved in the same issue once again or maybe from another organization who tried to go through the SPI project. Another demotivator that they did not have the proper orientation of SPI and the direct benefits in practice.

What are the different SPI methods?

Similar to the SDLC, SPI has a lot of methods and you can as well define your own method if it is effective or combine between more than one if you do not have any preferences or organization need to adopt a specific method.

SPI methods mainly categorized into two categories: inductive (bottom-up approach) and prescriptive (top-down approach) and most of them are cyclic with four main steps. The table below contains examples of the methods in each category.

Inductive (bottom-up approach)	Prescriptive (top-down approach)
Quality Improvement Paradigm (QIP) Improvement framework utilizing lightweight assessment and improvement planning (iFLAP)	Capability Maturity Model Integration (CMMI) Software Process Improvement and Capability dEtermination (SPICE)

Also, there are lots of other methods and techniques, for example, OWPL, PRISMS, SPIIMM, MESOPYME, …etc.

What are the common success factors for SPI project?

The SPI project is like any project which can have challenges which can make it failed. It has some special characteristics as it involves change management and re-engineering practices. There are huge reported failed SPI projects due to many reasons. It is important to understand what are the success factors for the SPI project to consider them during the implementation.

You may be heard a lot of companies want to be certified CMMI and seek different target levels without actually having any mature software process at hand. There are a lot of research papers discussed the failure of the SPI project especially in small and medium companies who have been involved in SPI project by getting CMMI certificate as a goal.

I tried here to categorize the SPI success factors according to the SPI process steps:

Current Situation Evaluation	Improvement Planning	Improvement Implementation	Improvement Evaluation
Stakeholders Involvement Top Management Commitment Clear & relevant SPI goals Staff involvement Encouraging communication and collaboration Business Orientation Information sharing	Stakeholders Involvement Top Management Commitment Measurable Objectives Staff involvement Creating process action teams Encouraging communication and collaboration Focussing on long-term benefits Information sharing	Stakeholders Involvement Top Management Commitment Staff involvement Staff time and resource Creating process action teams Encouraging communication and collaboration Focussing on long-term benefits Lack of Motivations Lack of Short-Term Goals Information sharing	Stakeholders Involvement Top Management Commitment Encouraging communication and collaboration Clear & relevant SPI goals Information sharing

Moreover, there are a lot of common factors that can also important for successful SPI project, for example, lack of SPI expert consultant and training for resources, not creating the motivated environment for change for better, not having a pilot project to test the SPI, and wrong selection of SPI method.

It was mentioned before that the SPI project is a change management project which is not dealing only with the process, it affects people and the organizational culture. As you may have noticed that management commitment and leadership support are considered the most common factors across the SPI process, losing that commitment and support will demotivate the people involved in the SPI project and it will fail.

One of my favorite books published by Harvard business review regarding the change management, they have mentioned important steps to enable the change, one of these steps to create the sense of urgency for the change, without the engagement of the people in the organization to enable that change, it will fail. Not only the engagement but their involvement to be a part of that change and understand why it is important and how they will benefit from that.

I will write another article about SPI methods and their advantages and disadvantages, and what are the criteria to select a suitable method for the organization and project context.

Conclusion

Definitely, to answer the question if the SPI is a Necessity or a Luxury – a Risk or a Reward, you should assess your organization process maturity, observe the gaps and inefficiencies, Do you have delays in the projects delivery, unqualified product, unsatisfied customers, the revenues became lower, overloaded staff, duplicated efforts? or everything is going as expected and planned.

In the first case, I think it is a necessary move to start the SPI project, it may be risky if you did not manage that well, and consider the success factors we talked about, if you managed it and make the staff involved and the leadership committed and involved, for sure you will see its reward and the direct benefit from it.

References

[1] Winkler, D., O’Connor, R., and Messnarz, R. (2011). Systems, software and services process improvement. Berlin: Springer, pp.97-108.

[2] IGI Global. (n.d.). [online] Available at https://www.igi-global.com/dictionary/software-development-process-standards-for-very-small-companies/41542 [Accessed 17 Jun. 2018].

[3] Baddoo, N. and Hall, T. (2002). Motivators of Software Process Improvement: an analysis of practitioners’ views. Journal of Systems and Software, 62(2), pp.85-96.

[4] Rainer, A. and Hall, T. (2002). Key success factors for implementing software process improvement: a maturity-based analysis. Journal of Systems and Software, 62(2), pp.71-84.

[5] Staples, M. and Niaz, M. (2010). Two Case Studies on Small Enterprise Motivation and Readiness for CMMI. [online] Available at https://ts.data61.csiro.au/publications/nicta_full_text/2191.pdf [Accessed 19 Jun. 2018].

---

Also published on Medium.