When to perform Software security analysis and tests?

Most of the software security practitioners would agree that the common practice of postponing security analysis and tests after the software implementation phase and even after it has been deployed (i.e., during its acceptance phase), makes it extremely difficult to address in a cost-effective, timely manner any vulnerabilities and weaknesses discovered during the analysis and testing process.

Figure [1] illustrates the relation between cost and time in security testing process which may be doubled or tripled due to lack of this testing coverage during its proper time.

Figure [1][i] Security testing cost vs. time – The cost of fixing software bugs

Source: OSSTMM – Open Source Security Testing Methodology Manual

Security testing involves in software developing lifecycle to ensure the implementation of security requirements. It is worth mentioning that Security testing is not a phase only in SDLC but it involves also in many system components and processes as illustrated in figure [2] below.

Therefore, each component of the system has different methodologies and techniques to assure the security, while our focus here on the software development lifecycle. The Figure [3] and Figure[4] below illustrate the security testing existence at SDLC

[ii]Figure [4] illustrates the software security touch-points (a set of best practices) and shows how software practitioners can apply the touch-points to the various software artifacts produced during software development.

These best practices first appeared as a set in 2004 in IEEE Security & Privacy magazine. Since then, they have been adopted (and in some cases adapted) by the U.S. government in the National Cyber Security Task Force report, by Cigital, by the U.S. Department of Homeland Security, and by Ernst and Young.

So here in the below table, a range of security reviews, analysis, and tests can be mapped to the different software lifecycle phases starting with the requirements phase:

Life Cycle Phase	Reviews/tests
Requirements	Security review of requirements and abuse/misuse cases
Architecture/Product Design	Architectural risk analysis (including external reviews)
Detailed Design	Security review of the design. Development of test plans, including security tests.
Coding/Unit Testing	Code review (static and dynamic analysis), white box testing
Assembly/Integration Testing	Black box testing (fault injection, fuzz testing)
System Testing	Black box testing, vulnerability scanning
Distribution/Deployment	Penetration testing (by software testing expert), vulnerability scanning, impact analysis of patches
Maintenance/support	(Feedback loop into previous phases), impact analysis of patches and updates

Security testing in software test plan

The security test plan should be included in the overall software test plan, and should define:

1. Security test cases or scenarios (based on misuse and abuse cases)
2. Test data, including attack patterns
3. Test oracle
4. Test tools (white box and black box, static and dynamic)
5. Analysis to be performed to interpret, correlate, and synthesize the results of the various tests and outputs from the various tools.

The security test plan should acknowledge that the security assumptions that were valid when the software’s requirements were specified; will probably have changed by the time the software is deployed. The threat environment in which the software will actually operate is unlikely to have remained static. New threats and attack patterns are continually emerging. Also, emerging has new versions of non-developmental components and patches to those components. All these changes have the potential to invalidate at least some of the security assumptions under which the original requirements were specified.

[cite]

[i] http://www.agitar.com/solutions/why_unit_testing.html

[ii] http://www.swsec.com/resources/touchpoints/

---

Also published on Medium.