S3 Trigger Lambda

Description

In this example, you’ll see how to set up an S3 trigger that will invoke a Lambda function in response to a file uploaded into an S3 bucket.

Prerequisites

JDK 17+
Maven 3.8.5+
LocalStack
Docker - for running LocalStack
AWS CLI and awslocal

Starting LocalStack via Docker Compose

version: '3.8'

services:
  localstack:
    image: localstack/localstack
    container_name: localstack_main
    ports:
      - "4566-4599:4566-4599"
    environment:
      - SERVICES=sns,s3,sqs,dynamodb,ses,lambda,logs,stepfunctions
      - DYNAMODB_SHARE_DB=1
      - DEBUG=1
      - DATA_DIR=/tmp/localstack/data
      - DOCKER_HOST=unix:///var/run/docker.sock
    volumes:
      - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

Preparing Lambda Code

Source Code: https://github.com/ZbCiok/zjc-examples/tree/main/aws/aws/lambda/s3-trigger-lambda

Project Structure

.
├── cmds.txt
├── pom.xml
├── README.md
├── s3-notify-config.json
├── sample-file-01.txt
├── sample-file-02.txt
└── src
    └── main
        └── java
            └── example
                └── BucketHandler.java

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.codetinkering</groupId>
    <artifactId>localstack-example</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
        <java.version>8</java.version>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.amazonaws</groupId>
                <artifactId>aws-java-sdk-bom</artifactId>
                <version>1.11.939</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-core</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-events</artifactId>
            <version>2.2.9</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-s3</artifactId>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-dynamodb</artifactId>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.projectlombok/lombok -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.34</version>
            <scope>provided</scope>
        </dependency>

    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.2.2</version>
                <configuration>
                    <createDependencyReducedPom>false</createDependencyReducedPom>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

BucketHandler.java

package example;

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.S3Event;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.S3Object;
import lombok.extern.java.Log;

import java.io.IOException;
import java.io.InputStream;

@Log
public class BucketHandler implements RequestHandler<S3Event, String> {

    public final String AWS_REGION = "us-east-1";
    public final String S3_ENDPOINT = "http://localhost:4566";

    public String handleRequest(S3Event s3Event, Context context) {
        // Pull the event records and get the object content type
        String bucket = s3Event.getRecords().get(0).getS3().getBucket().getName();
        String key = s3Event.getRecords().get(0).getS3().getObject().getKey();
        log.info( ">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bucket  :  " + bucket);
        log.info( ">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Key:  " + key);

        S3Object obj = prepareS3().getObject(new GetObjectRequest(bucket, key));
        try (InputStream stream = obj.getObjectContent()) {
            // TODO: Do something with the file contents here
            stream.transferTo(System.out);
            System.out.println();
        } catch (IOException ioe) {
            //throw ioe;
            ioe.printStackTrace();
        }

        return obj.getObjectMetadata().getContentType();
    }

    private AmazonS3 prepareS3() {
        BasicAWSCredentials credentials = new BasicAWSCredentials("foo", "bar");

        AwsClientBuilder.EndpointConfiguration config =
                new AwsClientBuilder.EndpointConfiguration(S3_ENDPOINT, AWS_REGION);

        AmazonS3ClientBuilder builder = AmazonS3ClientBuilder.standard();
        builder.withEndpointConfiguration(config);
        builder.withPathStyleAccessEnabled(true);
        builder.withCredentials(new AWSStaticCredentialsProvider(credentials));
        return builder.build();
    }
}

Deploy Lambda Function

mvn clean package
Create Function

awslocal lambda create-function \
    --function-name example-lambda \
    --runtime java8 \
    --handler example.BucketHandler \
    --zip-file fileb://localstack-example-1.0-SNAPSHOT.jar \
    --role arn:aws:iam::000000000000:role/lambda-role

zip-file fileb://localstack-example-1.0-SNAPSHOT.jar - generated by mvn clean package

Creating an S3 Bucket

awslocal s3 mb s3://mybucket --endpoint-url http://localhost:4566

Lambda registering events


awslocal s3api put-bucket-notification-configuration --bucket mybucket --notification-configuration file://s3-notify-config.json --endpoint-url http://localhost:4566

s3-notify-config.json

{
	"LambdaFunctionConfigurations": [
		{
			"Id": "1234567890123",
			"LambdaFunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:example-lambda",
			"Events": [
				"s3:ObjectCreated:*"
			]
		}
	]
}

Triggering Lambda


awslocal s3 cp sample-file-01.txt s3://mybucket/sample-file-01.txt --endpoint-url http://localhost:4566

Output: Docker Log

...
localstack_main  | 2024-09-25T14:45:56.053 DEBUG --- [da:$LATEST_0] l.s.l.i.version_manager    : [example-lambda-aeb1b298-15d8-4970-8682-b9b43a811053] START RequestId: aeb1b298-15d8-4970-8682-b9b43a811053 Version: $LATEST
localstack_main  | 2024-09-25T14:45:56.053 DEBUG --- [da:$LATEST_0] l.s.l.i.version_manager    : [example-lambda-aeb1b298-15d8-4970-8682-b9b43a811053] Sep 25, 2024 2:45:55 PM example.BucketHandler handleRequest
localstack_main  | 2024-09-25T14:45:56.054 DEBUG --- [da:$LATEST_0] l.s.l.i.version_manager    : [example-lambda-aeb1b298-15d8-4970-8682-b9b43a811053] INFO: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bucket  :  mybucket
localstack_main  | 2024-09-25T14:45:56.054 DEBUG --- [da:$LATEST_0] l.s.l.i.version_manager    : [example-lambda-aeb1b298-15d8-4970-8682-b9b43a811053] Sep 25, 2024 2:45:55 PM example.BucketHandler handleRequest
localstack_main  | 2024-09-25T14:45:56.054 DEBUG --- [da:$LATEST_0] l.s.l.i.version_manager    : [example-lambda-aeb1b298-15d8-4970-8682-b9b43a811053] INFO: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Key:  sample-file-01.txt
...

See:

INFO: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bucket  :  mybucket
INFO: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Key:  sample-file-01.txt

📄️ Lambda Handler

(example)

📄️ The First Lambda Example

lamdba-handler

📄️ SNS Trigger Lambda

Amazon Simple Notification Service (Amazon SNS) is a web service that makes it easy to set up, operate, and send notifications from the cloud.

📄️ S3 Trigger Lambda

Description

📄️ Lambda Person Request DynamoDB

Description

Description​

Prerequisites​

Starting LocalStack via Docker Compose​

Preparing Lambda Code​

Project Structure​

pom.xml​

BucketHandler.java​

Deploy Lambda Function​

mvn clean package​

Create Function​

zip-file fileb://localstack-example-1.0-SNAPSHOT.jar - generated by mvn clean package​

Creating an S3 Bucket​

Lambda registering events​

s3-notify-config.json​

Triggering Lambda​

Output: Docker Log​

📄️ Lambda Handler

📄️ The First Lambda Example

📄️ SNS Trigger Lambda

📄️ S3 Trigger Lambda

📄️ Lambda Person Request DynamoDB

Description

Prerequisites

Starting LocalStack via Docker Compose

Preparing Lambda Code

Project Structure

pom.xml

BucketHandler.java

Deploy Lambda Function

mvn clean package

Create Function

zip-file fileb://localstack-example-1.0-SNAPSHOT.jar - generated by mvn clean package

Creating an S3 Bucket

Lambda registering events

s3-notify-config.json

Triggering Lambda

Output: Docker Log