AWS Layers and how to install git in your lambda job

aka how to bring more software to the cloud

This other article presents a different (obsolete) way of installing git in your AWS Lambda job, together with some caveats of the process and hopefully some engaging prattle. Make sure to check it out for some tips on how to work with the git CLI from within your functions.

While there is no need in reiterating the whole documentation, I wrote this post as a quick feymanesque walkthrough and reminder of how to use AWS Layers to install git in your lambda jobs. If need be, the following resources are denser and should do a better job at conveying the intricacies of this new feature:

• Cutting Through the Layers: AWS Lambda Layers Explained by Michael Lavers
• Working with AWS Lambda and Lambda Layers in AWS SAM by Eric Johnson
• New for AWS Lambda — Use Any Programming Language and Share Common Components by Danilo Poccia (for AWS Console guide)
• AWS Lambda Layers — Documentation (for AWS CLI)

What are layers?

AWS Layers are a new feature of AWS Lambda Functions which allows you to configure custom runtimes on top of which your lambda job can run. With it, you can install arbitrary dependencies ranging from lower-level OS programs to programming language runtimes, that a Lambda function can make use of while keeping its package size to a minimum.

Section of a λ job

Simply put, layers are zip files that get installed and sit between your function and its initial runtime.

Layers are defined as independent AWS Resources and can be shared across multiple jobs. A job (or more commonly, a function) can reference external, public layers too — thus removing the need of creating our own layers for the simplest use cases.

I will not go into the technical details of how layers get installed and how you can implement your own layer in this article as it is outside its scope; Instead, please check out the resources at the top of the post, if curious.

Why should you care?

This addition of Layers to the AWS ecosystem essentially helps developers to bring software to the cloud in a more convenient manner. A runtime that can be flexibly defined will undoubtedly help in terms of providing a means for faster initial setups for jobs that depend on OS-level configurations. By allowing the re-use of both internal and 3rd party layers, the development process on the AWS stack ought to be a tad more seamless. Other direct benefits of layers may include:

• share code between different jobs within the same project
• write jobs depending on programming languages not natively supported
• deploy large dependencies (think GDAL, Pandoc or Selenium) less frequently

As we will show in this article, installing dependencies like git can be done through a handful of clicks and doesn’t entail an understanding of how lambda environments work in under the hood (as I’ve explored in this article).

At the same time, they do introduce more complexity into the workflow such as security risks and difficulties in predictably testing their behavior locally.

As choices in terms of tech stack and what technology should be used in different scenarios are rarely binary, it’s instrumental to understand the pros and cons of such additions when facing a need for it or when emphasizing on implementation speed and convenience. It’s expected that this will slowly be adopted into more standard tooling (like serverless did) so make sure to keep an eye on subsequent updates.

Install git layer into your lambda job

Remember that you can use public layers for your jobs instead of writing your own ones. While installing third-party dependencies introduces some security risks, it still stands to reason to opt-in for publicly-available resources to minimize the maintenance cost and exploit the collective power of developers. You should, however, always make sure that these resources come from a trustworthy party.

I am recommending this curated list of popular awesome layers where we can also find a layer for git (and SSH) — git-lambda-layer.

TLDR

Without further ado, let’s take a look at the process of adding a layer with the AWS Console. All we have to do is to:

• take the resource identifier arn:aws:lambda:region:553035198032:layer:git:version
• fill in the region and version ; the former one you should match to your job’s hosting region and the latter should be taken from the layer’s README
• go to your job, click on Layers (below the job name), Add a Layer
• in the “Add layer to function” wizard, pick Provide a layer version ARN
• paste in the identifier (from the first step), press Add
• press Save on the main job page
• voila, you have git working within your lambda job

end-to-end process of attaching a git layer to a job

One important caveat to mention — currently, this layer seems not to work with node 10x.

Conclusion

AWS Lambda layers are by design powerful and empower a lot of new of Lambda use cases. The process of adding existing layers is extremely simple. Ultimately, you should always be conscious about the technology choices you make as each one entails different consequences — for example, using public layers can impose security concerns.