# IAM for Data Pipelines: Least Privilege in Practice

> Roles vs users, how a Glue job or Lambda gets permissions, writing a least-privilege policy, and keeping secrets out of code.

- Source: https://akshaythakare.com/notes/aws/iam-for-data-pipelines
- Topic: AWS
- Level: intermediate
- Reading time: 2 min
- Tags: aws, iam, security, interview
- Updated: 2026-10-11

## Roles, not keys

Pipelines should never use long-lived access keys. Each service assumes a **role**:

| Component | Assumes | Gets |
| --- | --- | --- |
| Glue job | Glue job role | Read raw prefix, write curated prefix |
| Lambda | Lambda execution role | Read manifest, write one DynamoDB table |
| Step Functions | State machine role | Start those Glue jobs and invoke that Lambda |

## A least-privilege policy

```json
{
  "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow", "Action": ["s3:GetObject"], "Resource": "arn:aws:s3:::lake/raw/*" },
    { "Effect": "Allow", "Action": ["s3:PutObject"], "Resource": "arn:aws:s3:::lake/curated/*" },
    { "Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": "arn:aws:s3:::lake",
      "Condition": { "StringLike": { "s3:prefix": ["raw/*", "curated/*"] } } }
  ]
}
```

- Scope **actions** (`GetObject`, not `s3:*`).
- Scope **resources** to prefixes, not the whole bucket.
- `ListBucket` applies to the bucket ARN, limited by a prefix condition.

## Secrets

Database passwords and API keys belong in **Secrets Manager** (or SSM Parameter Store), read at runtime by the role. They should never be in code, job parameters or environment files committed to git.

## How to debug AccessDenied

1. Which role made the call? Check CloudTrail.
2. Is the action and the exact resource ARN allowed?
3. Is there an explicit `Deny` (bucket policy, SCP, KMS key policy)?
4. For encrypted buckets, does the role also have `kms:Decrypt`?

> **Tip: Interview framing**
>
> "Each component gets its own role, scoped to the prefixes and actions it needs; secrets come from Secrets Manager at runtime."