Categories
AWS EventBridge Lambda Serverless

API Gateway to EventBridge Integration

Aws recently released the ability to use an API Gateway HTTP route as a source for EventBridge. This is exciting news and allows us to do something pretty cool: remove the lambda proxy from the equation AND greatly increase the number of requests we can handle. Read on for an example and how to implement this using the Serverless Framework.

This is what a typical webhook integration might look like

In the above example, we’re exposing an api route via API Gateway, triggering our lambda function (which does some logic based upon the request body) and finally, firing off a Create or Cancel event to our event bus. This is a pretty common webhook pattern and can work reasonably well. The biggest concern is what happens when the webhook receives a large amount of (burst) traffic, and the lambda reaches a maximum amount (default=1000) of invocations? HINT: it involves either provisioning a massive amount of resources, or not being able to respond to all the requests ie. expensive or not good.

A clean and more scalable solution is to utilize the new API-Gateway-as-a-source-for-EventBridge functionality! Like this:

API Gateway directly to EventBridge

Yay! We’ve got it all figured out now, right? We’ll just use the Serverless Framework to deploy our new stack and call it a day…. But wait, how do we create an API/EventBridge integration in Serverless?

This is where things get a little hairy – Serverless doesn’t currently support creating an API Gateway route without attaching it to a resource (such as a lambda function like below).

Using this simple code, Serverless will auto-create all of the following resources: API, STAGE, ROUTE, LAMBDA and an api to lambda INTEGRATION (I think it also creates a ROLE)

How do we create all those things when we can’t use Serverless to create it for us? We’ve gotta create it from scratch, using the Resources tag:

resources:
  Resources:
    HttpApi:
      Type: 'AWS::ApiGatewayV2::Api'
      Properties:
        Name: http_api_${self:provider.stage}
        ProtocolType: HTTP
    HttpApiStage:
      Type: 'AWS::ApiGatewayV2::Stage'
      Properties:
        ApiId: !Ref HttpApi
        StageName: ${self:provider.stage}
        AutoDeploy: true
    HttpApiRoute:
      Type: 'AWS::ApiGatewayV2::Route'
      Properties:
        ApiId: !Ref HttpApi
        RouteKey: POST /api/webhook
        Target: !Join
          - /
          - - integrations
            - !Ref HttpApiIntegrationEventBridge
      DependsOn:
        - HttpApiIntegrationEventBridge

Ok, so now we’ve created an Api, Stage, and Route for our webhook, but you’ll notice we still need to create an EventBridge, and the EventBridge Integration. Then, we’ll also have to create an EventBridge Role for the API to putEvents on the Bridge. Agh! A lot of custom cloud formation work to do, but here we go – let’s do this!

    CustomEventBus:
      Type: AWS::Events::EventBus
      Properties:
        Name: custom_events_${self:provider.stage}
    HttpApiIntegrationEventBridge:
      Type: 'AWS::ApiGatewayV2::Integration'
      Properties:
        ApiId: !Ref HttpApi
        IntegrationType: AWS_PROXY
        IntegrationSubtype: EventBridge-PutEvents
        CredentialsArn: !GetAtt HttpApiIntegrationEventBridgeRole.Arn
        RequestParameters:
          Time: $request.body.event_time
          Source: 'source.events'
          DetailType: $request.body.event_type
          Detail: $request.body
          EventBusName: arn:aws:events:#{AWS::Region}:#{AWS::AccountId}:event-bus/custom_events_${self:provider.stage}
        PayloadFormatVersion: '1.0'
        TimeoutInMillis: 10000
    HttpApiIntegrationEventBridgeRole:
      Type: AWS::IAM::Role
      Properties:
        AssumeRolePolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Principal:
                Service:
                  - apigateway.amazonaws.com
              Action:
                - 'sts:AssumeRole'
        Policies:
          - PolicyName: !Join
              - '-'
              - - ${self:provider.stage}
                - ${self:service}
                - eventBridge
            PolicyDocument:
              Version: '2012-10-17'
              Statement:
                - Effect: Allow
                  Action: 'events:*'
                  Resource: 'arn:aws:events:#{AWS::Region}:#{AWS::AccountId}:event-bus/custom_events_${self:provider.stage}'
        RoleName: !Join
          - '-'
          - - ${self:service}
            - ${self:provider.stage}
            - !Ref 'AWS::Region'
            - eventBridgeRole

Alright! Now we’ve got everything we need for serverless to create the HTTP Api that will trigger our custom EventBridge. If you look closely, you’ll notice that resource connecting it all together is the HttpApiIntegrationEventBridge ‘AWS::ApiGatewayV2::Integration’. The IntegrationSubtype: EventBridge-PutEvents is where we define the integration type, and the RequestParameters is where we begin our data mapping from the request. Let me explain by showing our lambda:

This creates our lambda function and an eventBridge rule that listens for an event (from our custom event bus) that has a source = “source.events” and a detail-type of “create”

Above in our HttpApiIntegrationEventBridge request parameters we hard coded the source field and extracted the detail-type from the request.body.event_type:

This will “transform” and map the incoming webhook request payload to what our lambda event bridge rule is expecting.

So, if we get an incoming webhook request that has a body.event_type that equals “create”, then it will trigger our createOrder lambda! You can follow the same template for creating the cancelOrder lambda, and give it the detail-type of “cancel” (or whatever the webhook is sending)

I know this is a lot. And I wish it was easier. Serverless has made our lives so easy I don’t think many of us realize how much it’s doing behind the hood to spin up these stacks. It was a great opportunity for me to play around with cloud formation and figure out how it’s creating the custom cloud formation template. I’m going to try and create a serverless plugin that makes this easier. Please let me know if you have any questions or suggestions for making this more readable/coherent!