aws Verified current stable Not installed? Cloud Infrastructure

Aws / Export Logs To S3

Export Logs To S3

Exports logs from a specified CloudWatch log group to S3.

Terminal

aws logs create-export-task --log-group-name <log_group_name> --from <start_time> --to <end_time> --destination <s3_bucket_name>

aws logs create-export-task --log-group-name <log_group_name> --from <start_time> --to <end_time> --destination <s3_bucket_name>

#!/bin/bash

# Export Logs To S3
aws logs create-export-task --log-group-name {{log_group_name}} --from {{start_time}} --to {{end_time}} --destination {{s3_bucket_name}}

import subprocess

# Export Logs To S3
# Make sure to replace <placeholders> with actual values
def run_command():
    cmd = [
        "aws",
        "logs",
        "create-export-task",
        "--log-group-name",
        "<log_group_name>",
        "--from",
        "<start_time>",
        "--to",
        "<end_time>",
        "--destination",
        "<s3_bucket_name>"
    ]
    
    try:
        print(f"Executing: {' '.join(cmd)}")
        subprocess.run(cmd, check=True)
    except subprocess.CalledProcessError as e:
        print(f"Error: {e}")
    except FileNotFoundError:
        print("Error: aws not found. Please install it first.")

if __name__ == "__main__":
    run_command()

When To Use

During compliance audits requiring log data retention in S3.

Pro Tip

Verify the S3 bucket policy; it must allow logs to be written by CloudWatch.

Command Builder

Tune the command before you copy it

Back to syntax

Generated Command

aws logs create-export-task --log-group-name <log_group_name> --from <start_time> --to <end_time> --destination <s3_bucket_name>

Terminal Output

Expected runtime feedback

Simulated preview

Output

$ aws logs create-export-task --log-group-name my-log-group --from 1609459200 --to 1609545600 --destination my-s3-bucket\n\n{\n    "taskId": "12345678-90ab-cdef-1234-567890abcdef"\n}

Anatomy of Output

Understanding the result

ExportTaskId: abcd1234-5678-90ef-ghij-klmnopqrstu Export Task ID

Unique identifier for the export task.

Status: IN_PROGRESS Task Status

Indicates ongoing log export operation.

Destination: s3://example-bucket/logs/ Export Destination

Specifies where the logs are being stored.

Troubleshooting

Common pitfalls

An error occurred (InvalidParameterException) when calling the CreateExportTask operation: The specified start time is invalid.

Solution: Ensure the start time is in milliseconds since epoch.

An error occurred (LimitExceededException) when calling the CreateExportTask operation: The account exceeded the maximum number of export tasks.

Solution: Wait for existing tasks to complete or terminate one.

An error occurred (ResourceNotFoundException) when calling the CreateExportTask operation: The specified log group does not exist.

Solution: Double-check the log group name for accuracy.

Command Breakdown

What each part is doing

aws: Base Command; The executable that performs this operation. Here it runs Aws before the shell applies any redirect operators.
<log_group_name>: log group name; The value supplied for log group name.
<start_time>: start time; The value supplied for start time.
<end_time>: end time; The value supplied for end time.
<s3_bucket_name>: s3 bucket name; The value supplied for s3 bucket name.
--log-group-name: Command Option; Tool-specific option used by this command invocation.
--from: Command Option; Tool-specific option used by this command invocation.
--to: Command Option; Tool-specific option used by this command invocation.
--destination: Command Option; Tool-specific option used by this command invocation.

How To Run

Execution path

Step 1
Run the command: aws logs create-export-task --log-group-name my-log-group --from 1609459200 --to 1609545600 --destination my-s3-bucket
Step 2
Check the status with: aws logs describe-export-tasks --task-id 12345678-90ab-cdef-1234-567890abcdef

Alternative Approaches

Comparable commands in other tools

Alternative cloud infrastructure tools for the same job.

Doctl / Get Logs For App Doctl doctl a l <app_id> Doctl / Get App Details doctl a g Cloud Init / Clean Cloud Init Logs And Artifacts cloud-init clean Nomad / Follow Alloc Logs nomad alloc logs <alloc_id>