We’re <span>working on</span> integrating Linode and Akamai. <u>Learn more.</u>
Why Linode
Get to Know Us Better
View All Products
Compute
Storage
Databases
Services
Networking
Developer Tools
View All Solutions
Solutions
Pricing
Community
Engage With Us

Create an Email Notification System Using Twilio (IMAP)

Updated , by John Mueller
Select email protocol:
Create a Linode account to try this guide with a $100 credit.
This credit will be applied to any valid services used during your first 60 days.

By default, Linode sends system notifications via email. For example, email notifications are delivered when Linode Compute Instances are rebooted, when they receive hardware maintenance, and when they exceed a CPU usage threshold. You may also want to receive these notifications via text message. This guide shows how to set up a custom script that auto-forwards email notifications to text message.

The auto-forwarding system leverages the API of Twilio, a cloud communications service, along with the IMAP protocol for email. The instructions in this guide focus on how to parse Linode email notifications, but they could be adapted to email from other services as well.

In this Guide

  • In the Forward an Email to Text Message section, a script is created that focuses on the fundamentals of parsing email in Python and how to interact with the Twilio API. This includes:

    • Searching email with the imaplib Python module

    • Fetching an email’s contents with imaplib

    • Parsing the contents with the Python email module

    • Using the messages.create() endpoint of the Twilio API client.

  • In the Auto-Forward Email to Text Message section, the script is updated to run periodically and check for all Linode system notification emails since the last time the script ran.

  • In the Search Email by Subject with Imaplib section, the script is updated to only forward emails that match a keyword in the emails' subject. This allows you to limit forwarding to specific kinds of notifications.

Before You Begin

  1. This guide shows how to set up the email-to-text forwarding system on a Linode instance. A Linode instance is used because it can remain powered on at all times.

    If you want to implement the notification system, create a Linode in the Cloud Manager. The lowest cost Shared CPU instance type is appropriate for this guide. If you already have a Linode instance that you want to set up the notification system on, you can use that instead of a new instance. This guide was tested with Ubuntu 20.04, but should also work with other Linux distributions and versions.

    After you create your Linode, follow our Setting Up and Securing a Compute Instance guide to reduce the threat of a system compromise. Specifically, make sure you Add a Limited User Account to the Linode. The notification system in this guide should be installed under a limited Linux user.

  2. Another guide in our library, How to Use the Linode API with Twilio, shows the prerequisite steps for using the Twilio API. Follow this guide, starting with its Before You Begin section, up to and including the Install the Twilio Python Helper Library section.

    The guide instructs you to install the Twilio API client for Python. When following these instructions, run the commands under the limited Linux user on your Linode instance.

  3. This guide instructs you to create a Python script from within an SSH session on your Linode. You need to install and use a terminal text editor to write the script on your Linode. Common text editors include nano (the easiest option for terminal beginners), emacs, and vim.

  4. Your email service needs to support IMAP, and support for IMAP may need to be manually enabled. For example, Gmail has an option to turn on IMAP access in its settings.

Forward an Email to Text Message

This section provides a code example that connects to and searches an email server with IMAP. The script then shows how to fetch and parse the most recent Linode system notification email. The Linode system notification emails are from a sender named Linode Alert, so the script searches for this string in the FROM field. The parsed information from the email is delivered in a text message via the Twilio API.

The script in this section is updated in the next section to incorporate auto-forwarding behavior.

Import Modules and Initialize Service Credentials

  1. Log into your Linode under your limited Linux user using SSH.

  2. Create a new file named forward-last-email-to-text-message.py with your preferred terminal text editor. For example, when using nano, run:

    nano forward-last-email-to-text-message.py

  3. Copy this snippet into the file:

    File: forward-last-email-to-text-message.py
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

    import os
import sys
import imaplib
import email
from twilio.rest import Client

try:
    twilio_account_sid = os.environ['TWILIO_ACCOUNT_SID']
    twilio_auth_token = os.environ['TWILIO_AUTH_TOKEN']
    twilio_from_phone_number = os.environ['TWILIO_FROM_PHONE_NUMBER']
    twilio_to_phone_number = os.environ['TWILIO_TO_PHONE_NUMBER']
    email_username = os.environ['EMAIL_USERNAME']
    email_password = os.environ['EMAIL_PASSWORD']
    email_server = os.environ['EMAIL_SERVER']
except KeyError:
    print("Please ensure that the following environment variables are set when running the script: ")
    print("TWILIO_ACCOUNT_SID")
    print("TWILIO_AUTH_TOKEN")
    print("TWILIO_FROM_PHONE_NUMBER")
    print("TWILIO_TO_PHONE_NUMBER")
    print("EMAIL_USERNAME")
    print("EMAIL_PASSWORD")
    print("EMAIL_SERVER")
    sys.exit(1)

    This code imports several modules that are used later in the code:

    • The os module, which can be …

Create the Twilio API Python Client

Copy and paste the code from this snippet to the bottom of your script:

File: forward-last-email-to-text-message.py
1
2
3

# copy and paste to bottom of file:

twilio_client = Client(twilio_account_sid, twilio_auth_token)

This line creates a new client object that can interact with the Twilio API.

Log into the IMAP Server with Imaplib

Copy and paste the code from this snippet to the bottom of your script:

File: forward-last-email-to-text-message.py
1
2
3
4

# copy and paste to bottom of file:

mail = imaplib.IMAP4_SSL(email_server)
mail.login(email_username, email_password)

Search Email by Sender with Imaplib

Copy and paste the code from this snippet to the bottom of your script:

File: forward-last-email-to-text-message.py
1
2
3
4
5
6
7
8

# copy and paste to bottom of file:

mail.select('INBOX')
status, email_search_data = mail.search(None, 'FROM', '"Linode Alerts"')

mail_ids = []
for mail_ids_string in email_search_data:
    mail_ids += mail_ids_string.decode("utf-8").split()

You may want to retrieve mail from a mailbox with a specific name, instead of INBOX. For example, if you use Gmail and want to search all of your mail, remove the existing mail.select() function call and insert this new one:

mail.select('"[Gmail]/All Mail"')

  • The first line selects a mailbox that mail should be retrieved from in the subsequent search() …

Fetch Email with Imaplib

Copy and paste the code from this snippet to the bottom of your script:

File: forward-last-email-to-text-message.py
1
2
3
4
5
6
7
8

# copy and paste to bottom of file:

if len(mail_ids) == 0:
    print("No email matching search found.")
    sys.exit(0)

mail_ids.reverse()
status, email_data = mail.fetch(mail_ids[0], '(RFC822)')

  • Lines 3-5 exit the script if no emails matching the search were found.

  • Line 7: The mail IDs …

Parse Email with the Python Email Module

Copy and paste the code from this snippet to the bottom of your script:

File: forward-last-email-to-text-message.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# copy and paste to bottom of file:

response_part = email_data[0]
if isinstance(response_part, tuple):
    parsed_email = email.message_from_bytes(response_part[1])
    email_subject = parsed_email['subject']
    email_body = parsed_email.get_payload()

    message_text = 'New notification from Linode Alerts:\n\n' \
        'Message subject: \n%s\n\n' \
        'Message body: \n%s\n' % \
        (email_subject,
        email_body)

mail.close()
mail.logout()

This section of code parses the email_data variable returned by the mail.fetch() function. The value …

Create and Send a Text Message with Twilio

  1. Copy and paste the code from this snippet to the bottom of your script:

    File: forward-last-email-to-text-message.py
    1
2
3
4
5
6
7
8
9

    # copy and paste to bottom of file:

message = twilio_client.messages.create(
    body = message_text,
    from_ = twilio_from_phone_number,
    to = twilio_to_phone_number
)

print("Twilio message created with ID: %s" % (message.sid))

    The create method tells the Twilio API to create and immediately send a new text message:

    • The text …

  2. After appending the above snippet, save the file and exit your text editor.

    Note
    The code example is now complete. Your script should now look like the code in this file.

Run the Code

  1. Before you run the script, set the environment variables that the script expects in your terminal. In your SSH session with your Linode, run the following commands. After the = symbol in each command, insert the corresponding value:

    export TWILIO_ACCOUNT_SID=
export TWILIO_AUTH_TOKEN=
export TWILIO_FROM_PHONE_NUMBER=
export TWILIO_TO_PHONE_NUMBER=
export EMAIL_USERNAME=
export EMAIL_PASSWORD=
export EMAIL_SERVER=

    For example, the filled-in commands could look like:

    export TWILIO_ACCOUNT_SID=96af3vrYKQG6hrcYCC743mR27XhBzXb8wQ
export TWILIO_AUTH_TOKEN=LD9NWYXZzp3d3k7Mq7ME6L8QJJ8zu73r
export TWILIO_FROM_PHONE_NUMBER=+122233344444
export TWILIO_TO_PHONE_NUMBER=+15556667777
export EMAIL_USERNAME=youremail@yourdomain.com
export EMAIL_PASSWORD=bKfoAoV8Awo8e9CVTFTYKEdo
export EMAIL_SERVER=imap.yourdomain.com

    The values for each variable are as follows:

    VariableValue
    TWILIO_ACCOUNT_SIDThe Twilio account SID located in your Twilio console
    TWILIO_AUTH_TOKENThe Twilio auth token located in your Twilio console. The phone number needs to be entered using E.164 formatting.
    TWILIO_FROM_PHONE_NUMBERThe new number that you selected in the Twilio console when you first signed up
    TWILIO_TO_PHONE_NUMBERYour personal or testing phone number that you signed up to Twilio with. The phone number needs to be entered using E.164 formatting.
    EMAIL_USERNAMEYour email address.
    EMAIL_PASSWORDYour password for your email. Note that some services may require you to create an app-specific password for the IMAP connection. For example, Google requires you to create an app-specific password if you use 2-step verification/2FA on your account.
    EMAIL_SERVERThe server you should connect to. Check with your email service for the correct value. For Gmail, imap.gmail.com is used.

  2. Run the script:

    python3 forward-last-email-to-text-message.py

    If successful, the script generates output like the following:

    Twilio message created with ID: 9FKgk3Vokgx4hVC4937nx2kAraiG7qXDx8

    A few moments later, you should receive a text message similar to:

    Sent from your Twilio trial account - New notification from Linode Alerts:

Message subject:
Linode Events Notification - yourlinodeusername

Message body:
Hello yourlinodeusername! The following activity has recently occurred:

 * example-linode-instance - (100000001) System Shutdown - Completed Tue, 7 Dec 2021 17:35:20 GMT
 * example-linode-instance - (100000002) System Boot - My Ubuntu 20.04 LTS Profile - Completed Tue, 7 Dec 2021 17:35:32 GMT

You can change your notification settings via https://cloud.linode.com/profile.

---

Reduce Deployment Times with Custom Images - https://www.linode.com/products/images/
Durable File Storage with S3-Compatible Object Storage - https://www.linode.com/products/object-storage/

    If you receive an error message when you run the script, review the Troubleshooting section.

  3. If you have not previously received an email from Linode Alerts, you can generate a new one by performing certain actions in the Cloud Manager. For example, if you reboot your Linode, a new Linode Event Notification email is sent after a few minutes. After receiving this email, re-run the script to verify that it works.

Auto-Forward Email to Text Message

The code example from the last section sends the most recent Linode Alert email as a text message when you run the script. This section shows how to automatically deliver a text message whenever a new email is received, without having to manually run the script.

This auto-forwarding system has two parts:

  • The example code is updated to search recent email by date. The script iterates through the matching emails and creates a text message for each of them, instead of just forwarding the single most recent email. Specifically, it fetches the content for any Linode Alert email from the past 60 seconds and forwards it to Twilio.

  • A cron job is created to run the script every minute. This means that every time the script is run, it checks for emails that have been received since the last time the script was run.

Search Email by Date with Imaplib

  1. In your SSH session with your Linode, create a new file named autoforward-email-to-text-message.py with your preferred terminal text editor. For example, when using nano, run:

    nano autoforward-email-to-text-message.py

  2. Copy this snippet into the file. Then, save the file and exit your text editor.

    File: autoforward-email-to-text-message.py
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

    import os
import sys
import imaplib
import email
import datetime
from twilio.rest import Client

try:
    twilio_account_sid = os.environ['TWILIO_ACCOUNT_SID']
    twilio_auth_token = os.environ['TWILIO_AUTH_TOKEN']
    twilio_from_phone_number = os.environ['TWILIO_FROM_PHONE_NUMBER']
    twilio_to_phone_number = os.environ['TWILIO_TO_PHONE_NUMBER']
    email_username = os.environ['EMAIL_USERNAME']
    email_password = os.environ['EMAIL_PASSWORD']
    email_server = os.environ['EMAIL_SERVER']
except KeyError:
    print("Please ensure that the following environment variables are set when running the script: ")
    print("TWILIO_ACCOUNT_SID")
    print("TWILIO_AUTH_TOKEN")
    print("TWILIO_FROM_PHONE_NUMBER")
    print("TWILIO_TO_PHONE_NUMBER")
    print("EMAIL_USERNAME")
    print("EMAIL_PASSWORD")
    print("EMAIL_SERVER")
    sys.exit(1)

twilio_client = Client(twilio_account_sid, twilio_auth_token)

mail = imaplib.IMAP4_SSL(email_server)
mail.login(email_username, email_password)

mail.select('INBOX')
yesterday = datetime.date.today() - datetime.timedelta(1)
status, email_search_data = mail.search(None,
    'FROM', '"Linode Alerts"',
    'SINCE', yesterday.strftime("%d-%b-%Y"))

mail_ids = []
for mail_ids_string in email_search_data:
    mail_ids += mail_ids_string.decode("utf-8").split()

if len(mail_ids) == 0:
    print("No email matching search found.")
    sys.exit(0)

mail_ids.reverse()

def send_message(message_text):
    message = twilio_client.messages.create(
        body = message_text,
        from_ = twilio_from_phone_number,
        to = twilio_to_phone_number
    )

    print("Twilio message created with ID: %s" % (message.sid))

now_timestamp = datetime.datetime.now().timestamp()
EMAIL_AGE_LIMIT_IN_SECONDS = 60

for mail_id in mail_ids:
    status, email_data = mail.fetch(mail_id, '(RFC822)')

    response_part = email_data[0]
    if isinstance(response_part, tuple):
        parsed_email = email.message_from_bytes(response_part[1])
        email_subject = parsed_email['subject']
        email_body = parsed_email.get_payload()

        email_datestring = parsed_email['date']
        email_datetime = email.utils.parsedate_to_datetime(email_datestring)
        email_timestamp = email_datetime.timestamp()

        if now_timestamp - email_timestamp < EMAIL_AGE_LIMIT_IN_SECONDS:
            message_text = 'New notification from Linode Alerts:\n\n' \
                'Message subject: \n%s\n\n' \
                'Message body: \n%s\n' % \
                (email_subject,
                email_body)

            send_message(message_text)

mail.close()
mail.logout()

As in the previous section, you may want to retrieve mail from a mailbox with a specific name, instead of INBOX. For example, if you use Gmail and want to search all of your mail, remove the existing mail.select() function call and insert this new one:

mail.select('"[Gmail]/All Mail"')

The example code is similar to the code from the previous section. The updated lines of code are: …

Set Up a Cron Job

Cron is a Linux tool that runs processes at different time intervals that you specify. Follow these instructions to set up a cron job for the new script:

  1. In your SSH session, start the crontab editor:

    crontab -e

  2. A text file appears in your text editor. This file has some commented-out lines (which begin with # ) that tell you to set up a new scheduled task in the file. Below these comments, copy and paste the following lines:

    1
2
3
4
5
6
7
8
9

    TWILIO_ACCOUNT_SID=
TWILIO_AUTH_TOKEN=
TWILIO_FROM_PHONE_NUMBER=
TWILIO_TO_PHONE_NUMBER=
EMAIL_USERNAME=
EMAIL_PASSWORD=
EMAIL_SERVER=

* * * * * python3 /home/exampleuser/autoforward-email-to-text-message.py

    The first seven lines define your environment variables. The last line represents the scheduled task. The * * * * * at the start of the line represents when the task should run. Specifically, this string says that the task should run every minute.

  3. After the = symbol in each of the first five lines, insert the corresponding value. The values are the same as they were in the previous Run the Code section.

    For example, the filled-in crontab file could look like:

    TWILIO_ACCOUNT_SID=96af3vrYKQG6hrcYCC743mR27XhBzXb8wQ
TWILIO_AUTH_TOKEN=LD9NWYXZzp3d3k7Mq7ME6L8QJJ8zu73r
TWILIO_FROM_PHONE_NUMBER=+122233344444
TWILIO_TO_PHONE_NUMBER=+15556667777
EMAIL_USERNAME=youremail@yourdomain.com
EMAIL_PASSWORD=bKfoAoV8Awo8e9CVTFTYKEdo
EMAIL_SERVER=imap.yourdomain.com

0 14 * * * python3 /home/exampleuser/autoforward-email-to-text-message.py

  4. On the last line, update the file path to the Python script (e.g. /home/exampleuser/autoforward-email-to-text-message.py) so that it matches the path of the file on your server.

  5. Save the crontab file in your text editor and exit the editor. The script now runs at the start of every minute on your server, and the auto-forwarding system is complete.

  6. To test that the auto-forwarding system works, trigger a new Linode Alert email in the Cloud Manager. For example, if you reboot your Linode, a new Linode Event Notification email is sent after a few minutes. After the email arrives in your mailbox, the script is run within the next 60 seconds, and the text message is delivered.

    If you do not receive a text message after triggering a Linode Alert email, try visiting the Troubleshooting section of this guide for help.

Search Email by Subject with Imaplib

Emails sent from Linode Alerts can feature several different kinds of notifications, including:

  • Linode compute instance reboots, new compute instance configurations, and block storage attachments/detachments

  • CPU usage alerts

  • Disk IO rate alerts

  • Outbound and inbound traffic rate alerts

You may only want to receive text messages for certain kinds of notifications. Each kind of Linode notification features specific keywords in the subject of the email. You can search for these keywords to filter your text message notifications.

Follow these steps to only forward CPU usage alerts to text:

  1. In your autoforward-email-to-text-message.py, remove lines 34-36:

    File: autoforward-email-to-text-message.py
    1
2
3
4
5

    # remove the following lines:

# status, email_search_data = mail.search(None,
#     'FROM', '"Linode Alerts"',
#     'SINCE', yesterday.strftime("%d-%b-%Y"))

  2. Insert these new lines of code in the same position as the removed lines:

    File: autoforward-email-to-text-message.py
    1
2
3
4
5
6

    # insert where previous lines were removed:

status, email_search_data = mail.search(None,
    'FROM', '"Linode Alerts"',
    'SINCE', yesterday.strftime("%d-%b-%Y"),
    'SUBJECT', '"CPU Usage"')

This new code adds the SUBJECT IMAP search command to the search criterion. Note that the subject string that is searched for should be wrapped in double and single quotes.

  1. After inserting the above snippet, save the file.

    Note
    Your script should now look like the code in this file.

  2. The updated script is automatically run by the cron job. CPU usage alerts are sent when a Linode on your account exceeds a threshold percentage. The Linodes on your account may or may not currently this threshold, so you may not receive any notifications.

    You can test that the update code works by temporarily lowering the CPU usage alert threshold for one of your Linodes. By default, this value is set to 90%.

Next Steps

The auto-forwarding system is now complete, and it includes email filtering by subject keyword. You can make adjustments to the search criterion to change this filtering behavior. For example, you could search for the string traffic rate to only forward notifications about spikes in your Linodes' networking. You can also tweak the alert threshold values for different resources in the Cloud Manager.

In addition to forwarding emails to text, you may want to forward information from the Linode API to text. The Using the Linode API with Twilio and Monitor your Linode’s Network Transfer Pool with Twilio guides show how to combine the Linode and Twilio APIs.

Twilio’s API offers many other features as well. For example, you can forward notifications to more than one phone number using the Messaging Service resource. Twilio’s quick start guides are helpful when exploring the Twilio API.

Troubleshooting

Several troubleshooting scenarios are outlined in the Troubleshooting section of the How to Use the Linode API with Twilio guide. Review that section for possible solutions.

When troubleshooting email forwarding, remember that you can trigger new Linode system notifications by:

As well, the following possible solution may help:

Incorrect Email Server Credentials

You may see this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/imaplib.py", line 598, in login
    raise self.error(dat[-1])
imaplib.error: b'[AUTHENTICATIONFAILED] Invalid credentials (Failure)'

This indicates that your email password or username are incorrect. Verify that you have set these correctly in environment variables as described in the Run the Code section. Some services may require you to create an app-specific password for the IMAP connection if you use 2-step verification/2FA on your account. (e.g. Gmail)

More Information

You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

This page was originally published on

email

Your Feedback Is Important

Let us know if this guide made it easy to get the answer you needed.

Join the conversation.
Read other comments or post your own below. Comments must be respectful, constructive, and relevant to the topic of the guide. Do not post external links or advertisements. Before posting, consider if your comment would be better addressed by contacting our Support team or asking on our Community Site.