CWE-1426 - Improper Validation of Generative AI Output
CWE-1426
- Abstraction:
- Base
- Structure:
- Simple
- Status:
- Incomplete
- Weakness Name
Improper Validation of Generative AI Output
- Description
The product invokes a generative AI/ML component whose behaviors and outputs cannot be directly controlled, but the product does not validate or insufficiently validates the outputs to ensure that they align with the intended security, content, or privacy policy.
- Common Consequences
Scope: Integrity
Impact: Execute Unauthorized Code or Commands, Varies by Context
Notes: In an agent-oriented setting, output could be used to cause unpredictable agent invocation, i.e., to control or influence agents that might be invoked from the output. The impact varies depending on the access that is granted to the tools, such as creating a database or writing files.
- Related Weaknesses
- Release Date:
- 2024-07-16
- Latest Modification Date:
- 2024-07-16