aws Verified current stable Not installed? Data Processing

Aws / Create Dev Endpoint

Create Dev Endpoint

Creates an AWS Glue development endpoint.

Terminal

aws glue create-dev-endpoint --endpoint-name <name> --role-arn <role_arn_used_by_endpoint>

aws glue create-dev-endpoint --endpoint-name <name> --role-arn <role_arn_used_by_endpoint>

#!/bin/bash

# Create Dev Endpoint
aws glue create-dev-endpoint --endpoint-name {{name}} --role-arn {{role_arn_used_by_endpoint}}

import subprocess

# Create Dev Endpoint
# Make sure to replace <placeholders> with actual values
def run_command():
    cmd = [
        "aws",
        "glue",
        "create-dev-endpoint",
        "--endpoint-name",
        "<name>",
        "--role-arn",
        "<role_arn_used_by_endpoint>"
    ]
    
    try:
        print(f"Executing: {' '.join(cmd)}")
        subprocess.run(cmd, check=True)
    except subprocess.CalledProcessError as e:
        print(f"Error: {e}")
    except FileNotFoundError:
        print("Error: aws not found. Please install it first.")

if __name__ == "__main__":
    run_command()

When To Use

When preparing an environment for data development and ETL job testing.

Pro Tip

Configure security group rules to restrict access to the development endpoint effectively.

Command Builder

Tune the command before you copy it

Back to syntax

Generated Command

aws glue create-dev-endpoint --endpoint-name <name> --role-arn <role_arn_used_by_endpoint>

Terminal Output

Expected runtime feedback

Simulated preview

Output

Successfully created development endpoint:

Endpoint Name   | Endpoint ARN                                       | Status
----------------|---------------------------------------------------|---------
my-dev-endpoint | arn:aws:glue:us-west-2:123456789012:endpoint/my-dev-endpoint | READY

Anatomy of Output

Understanding the result

Creating Development Endpoint: my-dev-endpoint Endpoint Action

Shows that the creation of the endpoint is underway.

IAM Role: arn:aws:iam::123456789012:role/AWSGlueServiceRole Role ARN

Role linked to the Glue development endpoint.

Status: CREATING Current Status

Current progress of the endpoint creation process.

Troubleshooting

Common pitfalls

An error occurred (AccessDeniedException) when calling the CreateDevEndpoint operation: User is not authorized to perform glue:CreateDevEndpoint.

Solution: Verify IAM permissions to ensure required access to create dev endpoints.

An error occurred (ResourceAlreadyExistsException) when calling the CreateDevEndpoint operation: Development endpoint already exists.

Solution: Use a unique name for the new development endpoint.

An error occurred (InvalidInputException) when calling the CreateDevEndpoint operation: The specified role is invalid or does not exist.

Solution: Confirm that the specified IAM role exists and has Glue permissions.

Command Breakdown

What each part is doing

aws: Base Command; The executable that performs this operation. Here it runs Aws before the shell applies any redirect operators.
<name>: name; The value supplied for name.
<role_arn_used_by_endpoint>: role arn used by endpoint; The value supplied for role arn used by endpoint.
--endpoint-name: Command Option; Tool-specific option used by this command invocation.
--role-arn: Command Option; Tool-specific option used by this command invocation.

How To Run

Execution path

Step 1
Run the command: aws glue create-dev-endpoint --endpoint-name my-dev-endpoint --role-arn arn:aws:iam::123456789012:role/MyGlueRole
Step 2
Verify the creation: aws glue get-dev-endpoint --endpoint-name my-dev-endpoint

Alternative Approaches

Comparable commands in other tools

Alternative data processing tools for the same job.

Gdown / Download With Fuzzy Id Extraction gdown --fuzzy <url> Picotool / Convert Elf Bin To Uf2 picotool uf2 convert <path/to/elf_or_bin> <path/to/output> Command / Encode Stdin With Base32 42 Columns <command> | basenc --base32 -w 42 Join / Join Specific Fields File1 File2 join -1 <3> -2 <1> <path/to/file1> <path/to/file2> Join / Join Unpairable Lines File1 join -a <1> <path/to/file1> <path/to/file2>