In plain language
What this advisory means
The Amazon SageMaker Python SDK used to protect data sent to cloud storage had a serious problem: it stored a secret security key in clear, readable form accessible through certain account functions. This flaw lets someone with permission to view pipeline details steal the key, fake data payloads, and run unauthorized code in another user's processing environment within the same AWS account.
Technical explanation
How the issue affects the environment
The SageMaker Python SDK's pipeline component employs an HMAC key to ensure the integrity of serialized function payloads stored in Amazon S3. However, the HMAC secret key was stored in plaintext within pipeline definitions and exposed via the DescribePipeline API. A user with permissions to call DescribePipeline can retrieve this key, create cloud-pickled payloads with legitimate HMAC signatures, overwrite S3 objects, and execute arbitrary code in another user's pipeline execution context in the same AWS account. This vulnerability affected the HMAC configurations in SageMaker Python SDK versions 2 and 3. The issue was resolved in versions 3.11.0 and 2.256.0 by replacing symmetric HMAC with asymmetric ECDSA signing and removing the stored key, respectively.
Operational impact
Why teams should care
An attacker with certain permissions inside an AWS account could compromise pipeline integrity by injecting malicious payloads, leading to unauthorized code execution within other users' pipeline processes. This undermines data integrity, confidentiality, and can lead to data corruption or broader compromise of computational workflows within the same AWS environment.
Immediate action
Upgrade to Amazon SageMaker Python SDK version 3.11.0 or 2.256.0. Then, update existing pipeline definitions created via the @step decorator by invoking pipeline.upsert() to replace stored symmetric HMAC keys with asymmetric public keys. Users of the @remote decorator only need to upgrade the SDK, as no persisted key material exists. This eliminates exposure of secret keys and secures pipeline execution contexts.
Affected and fixed releases
Temporary risk reduction
Until upgrading, restrict the sagemaker:DescribePipeline IAM permission to trusted users only. Also, configure dedicated per-user s3_root_uri paths for @remote and @step decorator workloads to isolate serialized payloads and reduce risk of unauthorized access or overwriting.
Evidence and validation checklist
- SageMaker Python SDK uses an HMAC key for protecting serialized payloads in S3.
- HMAC secret key stored in cleartext inside pipeline definitions.
- The key is accessible via DescribePipeline API to users with appropriate permissions.
- An attacker can extract the key, create signed payloads, overwrite S3 objects, and execute code in other users' pipeline execution contexts.
- Issue affects SDK versions v2 and v3 HMAC configurations.
- Fixed in SDK versions 3.11.0 (v3) and 2.256.0 (v2) with asymmetric signing and key removal respectively.
- AWS recommends upgrading and updating pipeline definitions to secure existing pipelines.
Authoritative reference
AWS Security Bulletins
QCS detected and normalized this record from the official source. Vendor guidance remains authoritative.