CWE-1434: Insecure Setting of Generative AI/ML Model Inference Parameters
CWE版本: 4.18
更新日期: 2025-09-09
弱点描述
The product has a component that relies on a generative AI/ML model configured with inference parameters that produce an unacceptably high rate of erroneous or unexpected outputs.
常见后果
影响范围: Integrity Other
技术影响: Varies by Context Unexpected State
说明: The product can generate inaccurate, misleading, or nonsensical information.
影响范围: Other
技术影响: Alter Execution Logic Unexpected State Varies by Context
说明: If outputs are used in critical decision-making processes, errors could be propagated to other systems or components.
潜在缓解措施
阶段: Implementation System Configuration Operation
描述: Develop and adhere to robust parameter tuning processes that include extensive testing and validation.
阶段: Implementation System Configuration Operation
描述: Implement feedback mechanisms to continuously assess and adjust model performance.
阶段: Documentation
描述: Provide comprehensive documentation and guidelines for parameter settings to ensure consistent and accurate model behavior.
检测方法
方法: Automated Dynamic Analysis
Manipulate inference parameters and perform comparative evaluation to assess the impact of selected values. Build a suite of systems using targeted tools that detect problems such as prompt injection (CWE-1427) and other problems. Consider statistically measuring token distribution to see if it is consistent with expected results.
有效性: Moderate
方法: Manual Dynamic Analysis
Manipulate inference parameters and perform comparative evaluation to assess the impact of selected values. Build a suite of systems using targeted tools that detect problems such as prompt injection (CWE-1427) and other problems. Consider statistically measuring token distribution to see if it is consistent with expected results.
有效性: Moderate
引入模式
| 阶段 | 说明 |
|---|---|
| Build and Compilation | During model training, hyperparameters may be set without adequate validation or understanding of their impact. |
| Installation | During deployment, model parameters may be adjusted to optimize performance without comprehensive testing. |
| Patching and Maintenance | Updates or modifications may be made to the model that alter its behavior without thorough re-evaluation. |