🔥Practical Guide to AWS S3 CLI

1. List Buckets

$ aws s3 ls
2023-11-04 14:39:50 my-ml-bucket
2023-11-04 14:40:23 my-training-data

---

2. Create a Bucket

$ aws s3 mb s3://ml-model-bucket --region us-east-1
make_bucket: ml-model-bucket

---

3. Upload a File

$ aws s3 cp model.pkl s3://ml-model-bucket/
upload: ./model.pkl to s3://ml-model-bucket/model.pkl

---

4. Upload File with Metadata

$ aws s3 cp data.csv s3://ml-model-bucket/ \
  --metadata '{"x-amz-meta-model-version":"1.0","x-amz-meta-created-by":"john"}'

---

5. Add Tags

$ aws s3api put-object-tagging \
  --bucket ml-model-bucket \
  --key model.pkl \
  --tagging '{"TagSet": [{"Key": "environment", "Value": "production"},{"Key": "type", "Value": "ml-model"}]}'

---

6. List Objects with Prefix

$ aws s3 ls s3://ml-model-bucket/models/
2023-11-04 15:30:22     234566 model_v1.pkl
2023-11-04 15:31:45     245677 model_v2.pkl

---

7. Recursive Copy

$ aws s3 cp ./training_data s3://ml-model-bucket/data/ --recursive
upload: training_data/batch1.csv to s3://ml-model-bucket/data/batch1.csv
upload: training_data/batch2.csv to s3://ml-model-bucket/data/batch2.csv

---

8. Lifecycle Policy (JSON)

{
    "Rules": [
        {
            "ID": "MoveToGlacier",
            "Prefix": "archived/",
            "Status": "Enabled",
            "Transitions": [
                {
                    "Days": 90,
                    "StorageClass": "GLACIER"
                }
            ]
        }
    ]
}

---

9. Download an Object

$ aws s3 cp s3://ml-model-bucket/model.pkl ./local_model.pkl
download: s3://ml-model-bucket/model.pkl to ./local_model.pkl

---

10. Check Object Metadata

$ aws s3api head-object --bucket ml-model-bucket --key model.pkl
{
    "Metadata": {
        "x-amz-meta-model-version": "1.0",
        "x-amz-meta-created-by": "john"
    }
}

---

11. Python Example (Boto3)

import boto3

s3 = boto3.client('s3')

# Upload with Metadata
s3.put_object(
    Bucket='ml-model-bucket',
    Key='models/model_v3.pkl',
    Body=open('model.pkl', 'rb'),
    Metadata={
        'model-version': '3.0',
        'accuracy': '0.95'
    }
)

# List Objects
response = s3.list_objects_v2(Bucket='ml-model-bucket', Prefix='models/')
for obj in response['Contents']:
    print(f"Key: {obj['Key']}, Size: {obj['Size']}")

---

12. Delete Bucket with Contents

$ aws s3 rb s3://ml-model-bucket --force
remove_bucket: ml-model-bucket

---

13. Copy Between Buckets

$ aws s3 sync s3://source-bucket/models/ s3://dest-bucket/models/
copy: s3://source-bucket/models/model1.pkl to s3://dest-bucket/models/model1.pkl

---

💡 Important Notes

Storage Class Minimums

• Standard-IA: 128KB min, 30 days min

• One Zone-IA: 128KB min, 30 days min

• Glacier: 40KB min, 90 days min

• Deep Archive: 40KB min, 180 days min

Consistency Model

• PUT (new objects): Read-after-write consistency

• UPDATE/DELETE: Eventually consistent

---

This guide simplifies S3 operations for ML workflows. ✅ Always:

• Use proper IAM permissions

• Consider storage costs

• Clean up resources after use

• Add error handling for production

🎵 Breaking Down S3 Access Control with Code Examples 🔒

---

1. Bucket Policy Examples 🛡️

Public Read Access

{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "PublicRead",
        "Effect": "Allow",
        "Principal": "*",
        "Action": "s3:GetObject",
        "Resource": "arn:aws:s3:::my-ml-bucket/*"
    }]
}

IP Restriction Policy

{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "IPRestrict",
        "Effect": "Deny",
        "Principal": "*",
        "Action": "s3:*",
        "Resource": [
            "arn:aws:s3:::my-ml-bucket",
            "arn:aws:s3:::my-ml-bucket/*"
        ],
        "Condition": {
            "NotIpAddress": {
                "aws:SourceIp": ["192.168.1.0/24"]
            }
        }
    }]
}

Cross-Account Access

{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "CrossAccountAccess",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::ACCOUNT-ID:root"
        },
        "Action": [
            "s3:GetObject",
            "s3:PutObject"
        ],
        "Resource": "arn:aws:s3:::my-ml-bucket/*"
    }]
}

---

2. CLI Commands for Access Control 🚀

Apply Bucket Policy

$ aws s3api put-bucket-policy \
    --bucket my-ml-bucket \
    --policy file://bucket-policy.json

Block Public Access

$ aws s3api put-public-access-block \
    --bucket my-ml-bucket \
    --public-access-block-configuration \
    "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

Check Bucket Policy

$ aws s3api get-bucket-policy --bucket my-ml-bucket

---

3. IAM Policy Examples 🎯

Read-Only S3 Access

{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Action": [
            "s3:GetObject",
            "s3:ListBucket"
        ],
        "Resource": [
            "arn:aws:s3:::my-ml-bucket",
            "arn:aws:s3:::my-ml-bucket/*"
        ]
    }]
}

Full S3 Access with Prefix Restriction

{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Action": "s3:*",
        "Resource": [
            "arn:aws:s3:::my-ml-bucket/models/*"
        ]
    }]
}

---

4. Python Boto3 Examples 🐍

Get and Apply Bucket Policies

import boto3
import json

s3 = boto3.client('s3')

# Get bucket policy
def get_bucket_policy(bucket_name):
    try:
        policy = s3.get_bucket_policy(Bucket=bucket_name)
        return json.loads(policy['Policy'])
    except s3.exceptions.NoSuchBucketPolicy:
        return "No policy exists"

# Apply bucket policy
def apply_bucket_policy(bucket_name, policy):
    s3.put_bucket_policy(
        Bucket=bucket_name,
        Policy=json.dumps(policy)
    )

Block Public Access Programmatically

def block_public_access(bucket_name):
    s3.put_public_access_block(
        Bucket=bucket_name,
        PublicAccessBlockConfiguration={
            'BlockPublicAcls': True,
            'IgnorePublicAcls': True,
            'BlockPublicPolicy': True,
            'RestrictPublicBuckets': True
        }
    )

---

🔑 Important Notes

Access Control Hierarchy (Top-Down Priority):

1. Block Public Access Settings

2. Bucket Policies

3. IAM Policies

4. ACLs (Legacy; avoid using)

Best Practices 🛡️

• 🟢 Least Privilege Principle: Grant only necessary permissions

• 🟢 Avoid ACLs: Prefer policies for access control

• 🟢 Enable Versioning: Protect data from accidental deletions

• 🟢 Block Public Access: Default for new buckets

• 🟢 Audit Permissions Regularly: Detect and fix misconfigurations

---

🧠 Exam Tips

• Understand Resource vs. Identity Policies

• Know Policy Evaluation Logic

• Remember Block Public Access overrides all other settings

• Be familiar with common policy structures

• Troubleshoot access issues effectively

---

🔒 Security is a shared responsibility! Always review and audit your access control policies for a secure S3 environment. 🌟