Setup Lambda Function

AWS Lambda is a great service to deploy SigParser’s email parsing tools. Each email gets its own dedicated RAM and CPU, Lambdas are kept warm for around 5 minutes which means the startup time is decreased per email and they scale really well.

Getting access to the Lambda function deployable

To get access to the Lambda function deployable ZIP file contact SigParser's support team. They'll provide you with a link to download the latest version of the assembly. We don't have a public link available online at this time.

Configuring the Lambda Function

To configure, create a Lambda function.

Name: doesn’t matter.

Runtime: .NET Core 3.1 (C#/PowerShell)

Architecture: x86_64

In Runtime settings section click Edit and set the Handler as one of the following options

Clean Email: JSON

SigParser.EmailParsing.Lambda::SigParser.EmailParsing.Lambda.Function::GetCleanedEmailAsync

Clean Email: MIME:

SigParser.EmailParsing.Lambda::SigParser.EmailParsing.Lambda.Function::GetCleanedMimeAsync

Signature Contact Details: JSON

SigParser.EmailParsing.Lambda::SigParser.EmailParsing.Lambda.Function::GetEmailSignatureV2Async

Signature Contact Details: MIME

SigParser.EmailParsing.Lambda::SigParser.EmailParsing.Lambda.Function::GetEmailSignatureV2MimeAsync

Upload the SigParser.EmailParsing.Utils.Lambda.zip file.

Go to the Configuration tab.

Under Environment Variables add a new entry for SigParserLicenseKey and put your Cryptolens license key as the value

Under General Configuration menu on the left

Set the Memory to 2048MB of RAM. SigParser needs quite a bit of RAM to run all the machine learning systems quickly.

Click Save and then click the Test tab.

Test Examples

Pick the test example based on how you configured the Lambda function.

Clean Email: JSON:

Then use this test email and it should return a JSON result. The first time can be slow but after that it tends to be fast.

{
  "FromEmailAddress": "mary.johnson@fake.com",
  "FromName": "Mary Johsnon",
  "TextBody": null,
  "HtmlBody": "<p>Hi John,<\\/p>\\r\\n\\r\\n<p>Let\\'s get coffee tomorrow.<\\/p>\\r\\n\\r\\n<p>Thanks Mary Johnson<\\/p>On Thu, Aug 29, 2019 at 4:50 PM Taylor Ryan &lt;taylor.ryan@example.com&gt; wrote:<br\\/>Hello there",
  "To": [
    {
      "Name": "Tony Stark",
      "EmailAddress": "tony.stark@avengers.com"
    },
    {
      "Name": "Natasha Romanoff",
      "EmailAddress": "natasha@avengers.com"
    }
  ],
  "Cc": [
    {
      "Name": "Bruce Banner",
      "EmailAddress": "bruce.banner@avengers.com"
    }
  ]
}

Clean Email: MIME

{
  "MimeContents": "MIME-Version: 1.0\r\nReferences: <CABxEEohuqZBoVpsyY4pOFMYixhU2bzfxgs9tRLbUoV2NJMqCJw@mail.gmail.com> \r\n<CAL5Lp9Xyo0mEQ6-c1yAQ+SuKXrT4Xu5y-7BnvnGS4RMjZOBJ=g@mail.gmail.com>\r\nIn-Reply-To: <CAL5Lp9Xyo0mEQ6-c1yAQ+SuKXrT4Xu5y-7BnvnGS4RMjZOBJ=g@mail.gmail.com>\r\nFrom: Chris <c@sigparser.com>\r\nDate: Wed, 9 Jan 2019 08:36:15 -0800\r\nMessage-ID: <CABxEEoizOPyCLkq4+FBGNaw7KC2TJDfTZF5dp8xD9aFjDQoL+Q@mail.gmail.com>\r\nSubject: Re: food for thought\r\nTo: Paul <p@sigparser.com>\r\nContent-Type: multipart\/related; boundary=\"000000000000382db9057f0910d6\"\r\n\r\n--000000000000382db9057f0910d6\r\nContent-Type: multipart\/alternative; boundary=\"000000000000382db0057f0910d5\"\r\n\r\n--000000000000382db0057f0910d5\r\nContent-Type: text\/plain; charset=\"UTF-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nOk.  Just a thought.  Got it.\r\n\r\n--000000000000382db0057f0910d5\r\nContent-Type: text\/html; charset=\"UTF-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n<div><div dir=3D\"auto\">Ok.=C2=A0 Just a thought.=C2=A0 Got it. =C2=A0<\/div>=\r\n<\/div><div><br><div class=3D\"gmail_quote\"><div dir=3D\"ltr\">On Wed, Jan 9, 2="  
}

Signature Contact Details: JSON

{
  "subject": "Re: Great seeing you yesterday",
  "from_address": "john@example.com",
  "from_name": "John Smith",
  "htmlbody": "<body>Hi there, <br/> great seeing you yesterday. <br/>John Smith<br/>Vice President<br/>Mobile 818-334-3433<br> 999 Grand Ave, San Diego, CA, United States<br/>Our mission is to deliver customer satisfaction!</body>",
  "plainbody": "Hi there,\r\ngreat seeing you yesterday.\r\nJohn Smith\r\nVice President\r\nMobile 818-334-3433\r\n999 Grand Ave, San Diego, CA, United States\r\nOur mission is to deliver customer satisfaction!",
  "date": "2017-01-01T00:00:00",
  "to": [
    {
      "name": "Mark Rogers",
      "emailAddress": "mark.rogers@xyz.com"
    }
  ],
  "cc": [
    {
      "name": "Mark Rogers",
      "emailAddress": "mark.rogers@xyz.com"
    }
  ],
  "options": {
    "OutputCleanedEmailHtmlDepth": 0
  },
  "headers": {
    "additionalProp1": "string",
    "additionalProp2": "string",
    "additionalProp3": "string"
  }
}

Signature Contact Details: MIME

{
  "MimeContents": "MIME-Version: 1.0\r\nReferences: <CABxEEohuqZBoVpsyY4pOFMYixhU2bzfxgs9tRLbUoV2NJMqCJw@mail.gmail.com> \r\n<CAL5Lp9Xyo0mEQ6-c1yAQ+SuKXrT4Xu5y-7BnvnGS4RMjZOBJ=g@mail.gmail.com>\r\nIn-Reply-To: <CAL5Lp9Xyo0mEQ6-c1yAQ+SuKXrT4Xu5y-7BnvnGS4RMjZOBJ=g@mail.gmail.com>\r\nFrom: Chris <c@sigparser.com>\r\nDate: Wed, 9 Jan 2019 08:36:15 -0800\r\nMessage-ID: <CABxEEoizOPyCLkq4+FBGNaw7KC2TJDfTZF5dp8xD9aFjDQoL+Q@mail.gmail.com>\r\nSubject: Re: food for thought\r\nTo: Paul <p@sigparser.com>\r\nContent-Type: multipart\/related; boundary=\"000000000000382db9057f0910d6\"\r\n\r\n--000000000000382db9057f0910d6\r\nContent-Type: multipart\/alternative; boundary=\"000000000000382db0057f0910d5\"\r\n\r\n--000000000000382db0057f0910d5\r\nContent-Type: text\/plain; charset=\"UTF-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nOk.  Just a thought.  Got it.\r\n\r\n--000000000000382db0057f0910d5\r\nContent-Type: text\/html; charset=\"UTF-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n<div><div dir=3D\"auto\">Ok.=C2=A0 Just a thought.=C2=A0 Got it. =C2=A0<\/div>=\r\n<\/div><div><br><div class=3D\"gmail_quote\"><div dir=3D\"ltr\">On Wed, Jan 9, 2="  
}

Configure RAM Usage for Lambda

SigParser needs 2048MB of RAM per email to safely execute without running out of RAM when processing emails. The average real human emails needs 962 MB of RAM. The 99th percentile needs 1605MB.

SigParser Email Parser is incredibly CPU intensive. In AWS the more RAM you give a Lambda the more CPU speed AWS gives that Lambda. So having lots of RAM isn’t wasteful since it executes faster.

API Gateway Setup

You can setup API Gateway to consume the Lambda you just setup.

In AWS go to the API Gateway section
Click Create API. Under REST API click Build
1. Choose New API
2. Enter the API Name as SigParser
3. Optionally add a brief description
4. Choose Create API
Choose the root resource (/) under Resources.
1. From the Actions menu choose Create Resource
2. Type parseemail for Resource Name and Resource Path
3. Choose Create Resource
Click on the /parseemail resource
1. In the Actions menu click Create Method
2. Choose POST from the HTTP method drop down list.
3. Click the checkmark to save the settings.
In Method Executionfor /parseemail POST do the following
1. Choose Integration Type = Lambda Function
2. Leave Use Lambda Proxy integration box unchecked
3. Choose the region where you created the Lambda function.
4. Type the name of the Lambda function in the Lambda Function field. It should auto complete if you type part of it.
5. Leave Use default timeout box checked
6. Choose Save
7. Choose OK in the Add Permission to Lambda Function popup.
In the Actions menu select Deploy API
You should now be able to invoke your API using the URL they give you.

You can setup a usage key so that anyone can’t call your API.

On the resource add a requirement that a key is required for authentication.
Then go to the Usage Keys section and define a new Usage Plan.
Setup a new usage key and attach the usage plan.
Try the API by POST to it with the header x-api-key set on the request.

Subscribe to cloud Email Parsing API (ipaas.sigparser.com)

What are the on premise hosting options for the email parsing APIs?

Email Parsing - Self Hosted Web Server Setup

Email Parsing - CLI Executable (Win64 and Linux)

Parse an email with C#