Shared Flashcard Set

Details

2J - SES 622 - Module 10 - Verification and Validation
N/A
26
Engineering
Graduate
05/03/2012

Additional Engineering Flashcards

 


 

Cards

Term
What is the purpose of verification?
Definition
The prove requirements (i.e., shalls).
Term
Summarize verification.
Definition
Did we make the system right. I.e., was the system made to spec.
Term
Summarize validation.
Definition
Did we make the right system. I.e., does the system meet the mission need.
Term
Is V&V (V and V) internal or external to system development and operation?
Definition
Internal
Term
Is T&E (T and E) internal or external to system development and operation?
Definition
External.
Term
What is IEEE Std. 1012-2004?
Definition
The IEEE Standard for Software Verification and Validation. It includes the processes to determine whether the development products of a given activity conform to the requirements of that activity and whether the software satisfies its intended use and user needs.
Term
What are the three types of "independence" is IV&V testing?
Definition
1. Technical Independence
2. Managerial Independence
3. Financial Independence
Term
What is the meaning of Technical Independence, why is it useful, and how is it employed?
Definition
1. Utilizes personnel who are not involved in the development of the software
2. “Fresh viewpoint” valuable in detecting subtle errors overlooked by those too close to the solution
3. The IV&V effort uses or develops its own set of analysis and test tools separate from the developer’s tools
Term
What is the meaning of Managerial Independence and how is it employed?
Definition
1. Responsibility for the IV&V effort be vested in an organization separate from the development and program management organizations
2. IV&V selects the things to be analyzed and tested, defines the schedule for its activities, and selects the specific technical issues and problems upon which to act
Term
What is the meaning of Financial Independence and how is it employed?
Definition
1. Control of the IV&V budget is vested in an organization independent of the development organization
2. Prevents situations where the IV&V effort cannot be completed because funds have been diverted or adverse financial pressures or influences have been exerted.
Term
Basic Verificaiton and Validation Activities
Definition
1. Identify Verification and Validation Targets
2. Define Verification and Validation Approach
3. Perform Verification
4. Perform Validation
5. Provide Verification and Validation Results
Term
What is the typical main entrance criteria for Validation testing?
Definition
The successful completion of verification testing.
Term
What are leading indicators?
Definition
Security assessments that seem to indicate due diligence in security planning.
Term
What are concurrent indicators?
Definition
Technical target metrics that show plans were properly executed.
Term
What are lagging indicators?
Definition
Indicators that indicate after-the-fact that a security breach took place. E.g., viewing logs weekly to find intrusions that already happened.
Term
What are the types of validity?
Definition
1. Face validity
2. Content validity
3. Criterion validity
4. Construct validity
Term
What is face validity?
Definition
A Layman’s judgment given basic knowledge of a situation and the set of security controls in place.
Term
What does TTOA stand for?
Definition
Technical Target of Assessment
Term
What is content validity?
Definition
Summary: Does the test cover everything it needs to cover?

From web: The items on the test represent the entire range of possible items the test should cover. Individual test questions may be drawn from a large pool of items that cover a broad range of topics.

In some instances where a test measures a trait that is difficult to define, an expert judge may rate each item’s relevance. Because each judge is basing their rating on opinion, two independent judges rate the test separately. Items that are rated as strongly relevant by both judges will be included in the final test.

From charts: Measures weather something represents all facets of a given construct.

E.g., if the construct is a "secure state" and the security of a given TTOA is dependent of the values in a set of variable configurations that make the system difficult to penetrate, then a content validity text would be to compare the variables in the configuration files to a set of values for those variables that have been previously determined to thwart perpetrators. The systems whose configuration files contain the expected content would be validated to warrant a label of “secure,” given this definition.
Term
What is Criterion Validity?
Definition
Smmary: Does possession of a particular characteristic (e.g., passign a test or having some other property) correlate with the property being assessed?

Summary 2: Are the test results correlated to the desired state?

If the system meets a specific test and it can be shown that there is a correlation between that test and the desired state (e.g., security), then the system it said to have criterion validity for that state.

E.g., a comparison of operational test results and organizational practices to a set of standards that have been established to achieve security within the system of interest. An example of this type of test is a regulatory audit. This type of test would be considered to have this type of validity if there was a correlation between the ability to pass audit and the ability to thwart perpetrators who exploit vulnerabilities.
Term
What is construct validity?
Definition
Refers to whether a scale measures or correlates with the theorized psychological scientific construct (e.g., "fluid intelligence") that it purports to measure. In other words, it is the extent to which what was to be measured was actually measured.

Or in more simple terms, are we actually measuring what we think we are measuring?
Term
Basic Validation strategy
Definition
1. Create validation plan
2. Ensure that operator and facilities are ready for validation.
3. Conduct validation
4. If necessary, isolate part of system that is causing problems.
5. Re-validate by component and whole system
6. Analyze, record and report validation
7. Build required assurance cases.
Term
What is TTP?
Definition
The average interval between when a target is first aware of the existence of a new threat and when it successfully deflects it. This measure depends mainly on the speed and effectiveness of a target's response capability.
Term
What is TTA?
Definition
The median lifetime of malicious activity emanating from a specific source. This is useful to measure in situations where attackers must constantly create and abandon original points to evade detection. The shorter this median lifetime, the heavier is the burden on the attacker to continuously change its location to evade detection.
Term
What can you tell me about TTP/TTA?
Definition
The extent the ratio is minimized, the defenders are successfully thwarting attacks. To the extent it increases, the attackers are more successful.
Term
Example validation tests for a centralized identification and authentication function
Definition
1. an active login in any electronic component of the system is randomly sampled and compared with a centralized Identification and Authentication function to ensure that it is represented and a random sample of identities is made from the centralized repository and compared with access that it has recorded.
2. insert an unauthorized login into a system electronic component and observe to ensure that a security incident was detected and reported, and the incident response process resulting in its removal.
3. Parse all logins out of audit logs and verify the correspond to active identities
Supporting users have an ad free experience!