aws Verified current stable Not installed? Data Processing

Aws / List Glue Jobs

List Glue Jobs

Lists all AWS Glue jobs in the account.

Terminal

aws glue list-jobs

aws glue list-jobs

#!/bin/bash

# List Glue Jobs
aws glue list-jobs

import subprocess

# List Glue Jobs
# Make sure to replace <placeholders> with actual values
def run_command():
    cmd = [
        "aws",
        "glue",
        "list-jobs"
    ]
    
    try:
        print(f"Executing: {' '.join(cmd)}")
        subprocess.run(cmd, check=True)
    except subprocess.CalledProcessError as e:
        print(f"Error: {e}")
    except FileNotFoundError:
        print("Error: aws not found. Please install it first.")

if __name__ == "__main__":
    run_command()

When To Use

During data workflow management to audit Glue job configurations.

Pro Tip

Utilize --output json to facilitate integration with other automated scripts.

Terminal Output

Expected runtime feedback

Simulated preview

Output

-------------------------
JobName                  | JobRunId                | CreatedOn                
-------------------------
ETL-Job-1                | 1234567890abcdef        | 2023-09-01T12:00:00Z   
DataCleaning-Job        | 0987654321fedcba        | 2023-08-15T09:30:00Z   
-------------------------
Total Jobs: 2

Anatomy of Output

Understanding the result

Job Name State Last Modified Execution Role Column Headers

Denote the different attributes of Glue jobs.

data-cleaning-job                     SUCCEEDED           2022-10-01T08:00:00Z  arn:aws:iam::123456789012:role/AWSGlueServiceRole

Sample Glue Job Record

Indicating the latest state and execution role of the Glue job.

Power User Variants

Optimized versions

aws glue list-jobs --output table

Retrieves a table-formatted output for easier reading.

aws glue list-jobs --region us-east-1

Fetches jobs specifically for the specified AWS region.

Unix Pipeline

Shell combinations

aws glue list-jobs --profile default

Lists jobs under the specified AWS CLI profile.

Troubleshooting

Common pitfalls

An error occurred (AccessDeniedException) when calling the ListJobs operation: User is not authorized to perform glue:ListJobs.

Solution: Ensure that the IAM policy grants the required permissions.

An error occurred (ThrottlingException) when calling the ListJobs operation: Rate limit exceeded.

Solution: Implement retry logic with exponential backoff.

An error occurred (ServiceUnavailable) when calling the ListJobs operation: Glue service is currently unavailable.

Solution: Check AWS service status for outages affecting Glue.

Command Breakdown

What each part is doing

aws: Base Command; The executable that performs this operation. Here it runs Aws before the shell applies any redirect operators.

How To Run

Execution path

Step 1
Run the command: `aws glue list-jobs`
Step 2
Check for job names and details in the output to verify the result.

Alternative Approaches

Comparable commands in other tools

Alternative data processing tools for the same job.

Gdown / Download With Fuzzy Id Extraction gdown --fuzzy <url> Picotool / Convert Elf Bin To Uf2 picotool uf2 convert <path/to/elf_or_bin> <path/to/output> Command / Encode Stdin With Base32 42 Columns <command> | basenc --base32 -w 42 Join / Join Specific Fields File1 File2 join -1 <3> -2 <1> <path/to/file1> <path/to/file2> Join / Join Unpairable Lines File1 join -a <1> <path/to/file1> <path/to/file2>