What is white box testing?

White box testing and analysis, by contrast with “black box” testing and analysis, that are mainly performed on the source code. Also known as glass box, structural, clear box, and open box testing.

White box analysis and tests include:

Static Analysis

It is known as “code review,” static analysis analyses source code before it is compiled, to detect coding errors, insecure coding constructs, and other indicators of security vulnerabilities or weaknesses that are detectable at the source code level. Static analyses can be manual or automated. In a manual analysis, the reviewer inspects the source code without the assistance of tools.

In an automated analysis, a tool (or tools) is used to scan the code to locate specific “problem” patterns (text strings) defined to it by the analyst via programming or configuration, which the tool then highlights or flags. This enables the reviewer to narrow the focus of his/her manual code inspection to those areas of the code in which the patterns highlighted or flagged in the scanner’s output appear.

Direct Code Analysis

Direct code analysis extends the static analysis by using tools that focus not on finding individual errors but on verifying the code’s overall conformance to a set of predefined properties, which can include security properties such as noninterference and separability, persistent_BNDC, non-inference, forward-correct ability, and non-deductibility of outputs.

Property-Based Testing

The purpose of property-based testing is to establish formal validation results through testing. To validate that a program satisfies a property, the property must hold whenever the program is executed. Property-based testing assumes that the specified property captures everything of interest in the program and assumes that the completeness of testing can be measured structurally in terms of source code.

The testing only validates the specified property, using the property’s specification to guide the dynamic analysis of the program. Information derived from the specification determines which points in the program need to be tested and whether each test execution is correct.

A metric known as Iterative Contexts Coverage uses these test execution points to determine when testing is complete. Checking the correctness of each execution together with a description of all the relevant executions results in the validation of the program with respect to the property being tested, thus validating that the final product is free of any flaws specific to that property.

Source Code Fault Injection

A form of dynamic analysis in, which the source code is “instrumented” by inserting changes, then compiling and executing the instrumented code to observe the changes in state and behavior, which emerge when the instrumented portions of code are executed. In this way, the tester can determine and even quantify how the software reacts when it is forced into anomalous states, such as those triggered by intentional faults.

Moreover, This technique has proved particularly useful for detecting the incorrect use of pointers and arrays, and the presence of dangerous calls and race conditions. Fault injection is a complex testing process and thus tends to be limited to code that requires very high assurance.

Fault Propagation Analysis

This involves two techniques for fault injection of source code: extended propagation analysis and interface propagation analysis. The objective is not only to observe individual state changes that result from a given fault but to trace how those state changes propagate throughout a fault tree that has been generated from the program’s source code. Extended propagation analysis entails injecting a fault into the fault tree and then tracing how the fault propagates through the tree.

The tester then extrapolates outward to predict the impact a particular fault may have on the behavior of the software module or component, and ultimately the system, as a whole. In interface propagation analysis, the tester perturbs the states that propagate via the interfaces between the module or component and its environment.

To do this, the tester injects anomalies into the data feeds between the two levels of components and then watches to see how the resulting faults propagate and whether any new anomalies result. Interface propagation analysis enables the tester to determine how a failure in one component may affect its neighboring components.

Pedigree Analysis

While not a security testing technique in itself, the detection of pedigree indicators in open source code can be helpful in drawing attention to the presence of components that have known vulnerabilities, pinpointing them as high-risk targets in need of additional testing.

This is a fairly new area of code analysis that was sparked by concerns regarding open source licensing and intellectual property violations.

Dynamic Analysis of Source Code

Dynamic analysis involves both the source code and the binary executable generated from the source code. The compiled executable is run and “fed” a set of sample inputs while the reviewer monitors and analyzes the data (variables) the program produces as a result.

With a better understanding of how the program behaves, the analyst can use a binary-to-source map to trace the location in the source code that corresponds to each point of execution in the running program, and more effectively locate faults, failures, and vulnerabilities.

There are two types of Dynamic Analysis:

1. Coverage concept analysis
2. Frequency spectrum analysis.

Coverage concept analysis attempts to produce “dynamic control flow invariants” for a set of executions, which can be compared with statically derived invariants in order to identify desirable changes to the test suite that will enable it to produce better test results.

Frequency spectrum analysis counts the number of executions of each path through each function during a single run of the program. The reviewer can then compare and contrast these separate program parts in terms of higher versus lower frequency, the similarity of frequencies, or specific frequencies.

• The first analysis reveals any interactions between different parts of the program.
• The second analysis reveals any dependencies between the program parts.
• The third analysis allows the developer to look for specific patterns in the program’s execution, such as uncaught exceptions, assert failures, dynamic memory errors, and security problems.

A number of dynamic analysis tools have been built to elicit or verify system-specific properties in the source code, including call sequences and data invariants.

References

• Assuring Software Security Through Testing, White, Black and Somewhere in Between by Mano Paul https://www.isc2.org/uploadedFiles/(ISC)2_Public_Content/Certification_Programs/CSSLP/Software%20Security%20Through%20Testing.pdf
• http://www.agitar.com/solutions/why_unit_testing.html
• http://www.swsec.com/resources/touchpoints/
• State-of-the-Art Report (SOAR) July 31, 2007 – Information Assurance Technology Analysis Center (IATAC) Data and Analysis Center for Software (DACS)
• Gu Tian-yang, Shi Yin-sheng, and Fang You-yuan “Research on Software Security Testing” – World Academy of Science, Engineering, and Technology 69 2010
• https://www.owasp.org/index.php/OWASP_Testing_Guide_v3_Table_of_Contents