AWS Lambda のコンテナイメージ対応を試してみた
先日、AWS Lambda でコンテナイメージがサポートされました。
そこで、コンテナイメージを使用した Lambda の作成と実行を試してみます。
実行環境は以下の通りです。
$ aws --version aws-cli/2.1.28 Python/3.8.8 Linux/4.19.104-microsoft-standard exe/x86_64.ubuntu.20 prompt/off $ sam --version SAM CLI, version 1.19.1
作成
まずは、SAM CLI を使ってサンプルアプリケーション(python 3.8)を作成します。
$ sam init Which template source would you like to use? 1 - AWS Quick Start Templates 2 - Custom Template Location Choice: 1 What package type would you like to use? 1 - Zip (artifact is a zip uploaded to S3) 2 - Image (artifact is an image uploaded to an ECR image repository) Package type: 2 Which base image would you like to use? 1 - amazon/nodejs14.x-base 2 - amazon/nodejs12.x-base 3 - amazon/nodejs10.x-base 4 - amazon/python3.8-base 5 - amazon/python3.7-base 6 - amazon/python3.6-base 7 - amazon/python2.7-base 8 - amazon/ruby2.7-base 9 - amazon/ruby2.5-base 10 - amazon/go1.x-base 11 - amazon/java11-base 12 - amazon/java8.al2-base 13 - amazon/java8-base 14 - amazon/dotnet5.0-base 15 - amazon/dotnetcore3.1-base 16 - amazon/dotnetcore2.1-base Base image: 4 Project name [sam-app]: sample-app Cloning app templates from https://github.com/aws/aws-sam-cli-app-templates ----------------------- Generating application: ----------------------- Name: sample-app Base Image: amazon/python3.8-base Dependency Manager: pip Output Directory: . Next steps can be found in the README file at ./sample-app/README.md
生成されたファイルを眺めてみます。
... Resources: HelloWorldFunction: Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction Properties: PackageType: Image Events: HelloWorld: Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api Properties: Path: /hello Method: get Metadata: Dockerfile: Dockerfile DockerContext: ./hello_world DockerTag: python3.8-v1 Outputs: ...
FROM public.ecr.aws/lambda/python:3.8 COPY app.py requirements.txt ./ RUN python3.8 -m pip install -r requirements.txt -t . # Command can be overwritten by providing a different command in the template directly. CMD ["app.lambda_handler"]
template.yaml には Metadata を使用して Dockerfile をビルドする Context が設定されています。
また、これまで存在した Handler プロパティが無くなっており、
Dockerfile の CMD 命令で指定するようになっています。*1
ビルド
./README.md の手順通りに build します。
$ sam build Building codeuri: . runtime: None metadata: {'Dockerfile': 'Dockerfile', 'DockerContext': './hello_world', 'DockerTag': 'python3.8-v1'} functions: ['HelloWorldFunction'] Building image for HelloWorldFunction function Setting DockerBuildArgs: {} for HelloWorldFunction function Step 1/4 : FROM public.ecr.aws/lambda/python:3.8 ---> 96d8701bf5d1 Step 2/4 : COPY app.py requirements.txt ./ ---> 029a59377f95 Step 3/4 : RUN python3.8 -m pip install -r requirements.txt -t . ---> Running in fa7c16bc7a59 Collecting requests Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB) Collecting idna<3,>=2.5 Downloading idna-2.10-py2.py3-none-any.whl (58 kB) Collecting chardet<5,>=3.0.2 Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB) Collecting urllib3<1.27,>=1.21.1 Downloading urllib3-1.26.3-py2.py3-none-any.whl (137 kB) Collecting certifi>=2017.4.17 Downloading certifi-2020.12.5-py2.py3-none-any.whl (147 kB) Installing collected packages: idna, chardet, urllib3, certifi, requests Successfully installed certifi-2020.12.5 chardet-4.0.0 idna-2.10 requests-2.25.1 urllib3-1.26.3 WARNING: You are using pip version 20.2.1; however, version 21.0.1 is available. You should consider upgrading via the '/var/lang/bin/python3.8 -m pip install --upgrade pip' command. ---> d7e0e2543084 Step 4/4 : CMD ["app.lambda_handler"] ---> Running in 1c0839958a6d ---> 9790df443d64 Successfully built 9790df443d64 Successfully tagged helloworldfunction:python3.8-v1 Build Succeeded Built Artifacts : .aws-sam/build Built Template : .aws-sam/build/template.yaml Commands you can use next ========================= [*] Invoke Function: sam local invoke [*] Deploy: sam deploy --guided $ docker image ls | grep helloworldfunction helloworldfunction python3.8-v1 9790df443d64 48 seconds ago 599MB
コンテナイメージが出来ました。
.aws-sam/build ディレクトリには以下のような template.yaml と build.toml が作成されました。
... Resources: HelloWorldFunction: Type: AWS::Serverless::Function Properties: PackageType: Image Events: HelloWorld: Type: Api Properties: Path: /hello Method: get ImageUri: helloworldfunction:python3.8-v1 Metadata: Dockerfile: Dockerfile DockerContext: ./hello_world DockerTag: python3.8-v1 ...
# This file is auto generated by SAM CLI build command [function_build_definitions] [function_build_definitions.c7334742-ebfc-41b8-8278-371e13dc3e38] packagetype = "Image" functions = ["HelloWorldFunction"] [function_build_definitions.c7334742-ebfc-41b8-8278-371e13dc3e38.metadata] Dockerfile = "Dockerfile" DockerContext = "./hello_world" DockerTag = "python3.8-v1" [layer_build_definitions]
デプロイ
はじめに、コンテナレジストリを作成します。
$ aws ecr create-repository --repository-name sample-app { "repository": { "repositoryArn": "arn:aws:ecr:ap-northeast-1:123456789012:repository/sample-app", "registryId": "123456789012", "repositoryName": "sample-app", "repositoryUri": "123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample-app", "createdAt": "2021-02-28T19:27:57+09:00", "imageTagMutability": "MUTABLE", "imageScanningConfiguration": { "scanOnPush": false }, "encryptionConfiguration": { "encryptionType": "AES256" } } }
続いて、deploy を行います。
$ sam deploy --guided Configuring SAM deploy ====================== Looking for config file [samconfig.toml] : Not found Setting default arguments for 'sam deploy' ========================================= Stack Name [sam-app]: AWS Region [ap-northeast-1]: Image Repository for HelloWorldFunction: 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample-app helloworldfunction:python3.8-v1 to be pushed to 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample-app:helloworldfunction-9790df443d64-python3.8-v1 #Shows you resources changes to be deployed and require a 'Y' to initiate deploy Confirm changes before deploy [y/N]: #SAM needs permission to be able to create roles to connect to the resources in your template Allow SAM CLI IAM role creation [Y/n]: HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y Save arguments to configuration file [Y/n]: SAM configuration file [samconfig.toml]: SAM configuration environment [default]: Looking for resources needed for deployment: Found! Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-31fpkcxpfv0e A different default S3 bucket can be set in samconfig.toml Saved arguments to config file Running 'sam deploy' for future deployments will use the parameters saved above. The above parameters can be changed by modifying samconfig.toml Learn more about samconfig.toml syntax at https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html The push refers to repository [123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample-app] 234939dcf430: Pushed 5394ef41e052: Pushed 20b4eff3dd4d: Pushed 11284767d41d: Pushed d6fa53d6caa6: Pushed b09e76f63d5d: Pushed 0acabcf564c7: Pushed f2342b1247df: Pushed helloworldfunction-9790df443d64-python3.8-v1: digest: sha256:68dc32475a5705c6593497ff35b59a2099b3218956c6307b9123f6373d91fb2b size: 1999 Deploying with following values =============================== Stack name : sam-app Region : ap-northeast-1 Confirm changeset : False Deployment image repository : { "HelloWorldFunction": "123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample-app" } Deployment s3 bucket : aws-sam-cli-managed-default-samclisourcebucket-31fpkcxpfv0e Capabilities : ["CAPABILITY_IAM"] Parameter overrides : {} Signing Profiles : {} Initiating deployment ===================== HelloWorldFunction may not have authorization defined. Uploading to sam-app/0349ea3b59cff300265d848a1ec488a2.template 1169 / 1169 (100.00%) Waiting for changeset to be created.. CloudFormation stack changeset --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Operation LogicalResourceId ResourceType Replacement --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * Modify HelloWorldFunction AWS::Lambda::Function False * Modify ServerlessRestApi AWS::ApiGateway::RestApi False --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Changeset created successfully. arn:aws:cloudformation:ap-northeast-1:123456789012:changeSet/samcli-deploy1614508262/9d2ba819-8fbe-4635-94f8-b7c7ed65d116 2021-02-28 19:31:13 - Waiting for stack create/update to complete CloudFormation events from changeset --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ResourceStatus ResourceType LogicalResourceId ResourceStatusReason --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- UPDATE_IN_PROGRESS AWS::Lambda::Function HelloWorldFunction - UPDATE_COMPLETE AWS::Lambda::Function HelloWorldFunction - UPDATE_COMPLETE_CLEANUP_IN_PROGRESS AWS::CloudFormation::Stack sam-app - UPDATE_COMPLETE AWS::CloudFormation::Stack sam-app - --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CloudFormation outputs from deployed stack ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Outputs ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Key HelloWorldFunctionIamRole Description Implicit IAM Role created for Hello World function Value arn:aws:iam::123456789012:role/sam-app-HelloWorldFunctionRole-4038SRBBOKKK Key HelloWorldApi Description API Gateway endpoint URL for Prod stage for Hello World function Value https://fpo1jvgd1d.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/ Key HelloWorldFunction Description Hello World Lambda Function ARN Value arn:aws:lambda:ap-northeast-1:123456789012:function:sam-app-HelloWorldFunction-3WPICAKIXHD7 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Successfully created/updated stack - sam-app in ap-northeast-1
出力された samconfig.toml に関数にリポジトリが保存されています。
version = 0.1 [default] [default.deploy] [default.deploy.parameters] stack_name = "sam-app" s3_bucket = "aws-sam-cli-managed-default-samclisourcebucket-31fpkcxpfv0e" s3_prefix = "sam-app" region = "ap-northeast-1" capabilities = "CAPABILITY_IAM" image_repositories = ["HelloWorldFunction=123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample-app"]
最終的にデプロイされたCFnテンプレートには、
リポジトリ名を含めた ImageUrl が生成されていました。
... Resources: HelloWorldFunction: Type: AWS::Serverless::Function Properties: PackageType: Image Events: HelloWorld: Type: Api Properties: Path: /hello Method: get ImageUri: 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample-app:helloworldfunction-9790df443d64-python3.8-v1 Metadata: Dockerfile: Dockerfile DockerContext: ./hello_world DockerTag: python3.8-v1 ...
テスト
$ curl -X GET https://fpo1jvgd1d.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/ {"message": "hello world"}
API Gateway からレスポンスが返ることが確認できました。
ローカルテスト
続いて、ローカルでのテストを確認します。
./README.md に書いてある手順でローカルテストを実行します。
$ sam local invoke HelloWorldFunction --event events/event.json Invoking Container created from helloworldfunction:python3.8-v1 Image was not found. Building image.......... Skip pulling image and use local one: helloworldfunction:rapid-1.19.1. START RequestId: f4b3f8d8-fbda-48d7-8ee8-fba315b874aa Version: $LATEST END RequestId: f4b3f8d8-fbda-48d7-8ee8-fba315b874aa REPORT RequestId: f4b3f8d8-fbda-48d7-8ee8-fba315b874aa Init Duration: 0.10 ms Duration: 54.73 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 128 MB {"statusCode": 200, "body": "{\"message\": \"hello world\"}"}
helloworldfunction:rapid-1.19.1
というイメージが作成され、実行されていました。
$ docker image inspect helloworldfunction:python3.8-v1 > /tmp/python3.8-v1 $ docker image inspect helloworldfunction:rapid-1.19.1 > /tmp/rapid-1.19.1 $ diff /tmp/python3.8-v1 /tmp/rapid-1.19.1 ... 5,6c5 < "123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample-app:helloworldfunction-9790df443d64-python3.8-v1", < "helloworldfunction:python3.8-v1" --- > "helloworldfunction:rapid-1.19.1" ... 36,37c33 < "#(nop) ", < "CMD [\"app.lambda_handler\"]" --- > "chmod +x /var/rapid/aws-lambda-rie" ... 42,44c38 < "Entrypoint": [ < "/lambda-entrypoint.sh" < ], --- > "Entrypoint": null, ... 103c97,99 < "sha256:234939dcf430ad1d583a8e5fc9a6a8b6e9bd47d7522e68529e7dc6a3a713b08c" --- > "sha256:234939dcf430ad1d583a8e5fc9a6a8b6e9bd47d7522e68529e7dc6a3a713b08c", > "sha256:29131f8b9548941337f7a3f55f91c42c8620f370aa454a3073f08ff1d8320a31", > "sha256:29131f8b9548941337f7a3f55f91c42c8620f370aa454a3073f08ff1d8320a31" ...
rapid-1.19.1 のコンテナイメージは、
python3.8-v1 の Entrypoint
や CMD
を変更して作成されているようです。
ローカル実行の仕組みは Runtime Interface Emulator (RIE) として実装されているとのこと。
使用しているAWS提供のベースイメージには既に RIE が組み込まれているとのことなので、
README の手順でビルドしたイメージを実行してみました。
$ docker run -p 9000:8080 helloworldfunction:python3.8-v1 time="2021-02-28T11:36:25.564" level=info msg="exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)" time="2021-02-28T11:36:36.87" level=info msg="extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory" time="2021-02-28T11:36:36.87" level=warning msg="Cannot list external agents" error="open /opt/extensions: no such file or directory" START RequestId: 7ef765f5-d0df-4c12-8235-3f7c70de5056 Version: $LATEST END RequestId: 7ef765f5-d0df-4c12-8235-3f7c70de5056 REPORT RequestId: 7ef765f5-d0df-4c12-8235-3f7c70de5056 Init Duration: 0.12 ms Duration: 52.12 ms Billed Duration: 100 ms Memory Size: 3008 MB Max Memory Used: 3008 MB ^Ctime="2021-02-28T11:37:51.112" level=info msg="Received signal" signal=interrupt time="2021-02-28T11:37:51.112" level=info msg="Shutting down..." time="2021-02-28T11:37:51.112" level=warning msg="Reset initiated: SandboxTerminated"
$ curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}' {"statusCode": 200, "body": "{\"message\": \"hello world\"}"}
ローカルで実行したコンテナからレスポンスが返ることが確認できました。
8080 番ポートでListenしてるんですね。
もう一つ、ローカルでの REST API のテストを試します。
$ sam local start-api Mounting HelloWorldFunction at http://127.0.0.1:3000/hello [GET] You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template 2021-02-28 20:40:58 * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit) Invoking Container created from helloworldfunction:python3.8-v1 Building image......... Skip pulling image and use local one: helloworldfunction:rapid-1.19.1. START RequestId: e67691d8-e55e-408a-96e8-253fdbe760cd Version: $LATEST END RequestId: e67691d8-e55e-408a-96e8-253fdbe760cd REPORT RequestId: e67691d8-e55e-408a-96e8-253fdbe760cd Init Duration: 0.11 ms Duration: 54.94 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 128 MB No Content-Type given. Defaulting to 'application/json'. 2021-02-28 20:41:33 127.0.0.1 - - [28/Feb/2021 20:41:33] "GET /hello/ HTTP/1.1" 200 -
$ curl -XGET "http://localhost:3000/hello/" -d '{}' {"message": "hello world"}
ローカル実行した REST API からもレスポンスが返ることが確認できました。
API からリクエストがある度にコンテナが起動されています。
sam local start-api
でAPIを起動した状態でコードを変更した場合は、
sam build
でローカルのイメージを更新することで、
新しいイメージでのリクエストを確認することが出来ます。
まとめ
コンテナ化することで成果物が扱いやすくなり、
バージョン管理もしやすそうに感じました。
ローカルでのテストの手順も簡略化されそうですね。
*1:ImageConfig プロパティで CMD 命令などをオーバーライドできるようにもなっているようです。 AWS::Serverless::Function - AWS Serverless Application Model