Package: victoria-email
Victoria EmailVictoria Email is a V.I.C.T.O.R.I.A is a plugin that allows you to manipulate emails through an email platformFeaturesLoad testing SMTP endpointsSending single test emails from the CLIReconstructing email from a deconstructed stateRecovering email from Dead Letter Azure Service Bus QueuesUser guidePrerequisitesPython 3.7+PipVictoriaInstallation$ pip install -U victoria_emailOperationConfigurationThis plugin expects a section in your Victoria config file'splugins_configorplugins_config_location. Its key isemail.An easy way to edit your config file if you have VSCode installed is by running:code "$(victoria config path)".A full config example is:email:load_test:mail_send_function_endpoints:-function:"<AzureFunctionendpointURL>",mail_send_function_code:<the code to access the Azure Function>-function:"<AzureFunctionendpointURL>",mail_send_function_code:<the code to access the Azure Function>tenant_ids:[<tenant ID(s) to attach to the email>]timeout:10.0attachments:storage_connection_strings:<encrypted data>mail_toil:service_bus_connection_strings:<cluster-name>:<encrypted data><cluster-name>:<encrypted data># these are the Azure Service Bus Queues we want to check for dead lettersqueues:-"queues"-"you"-"want"-"to"-"check"storage_accounts:<cluster-name>:<encrypted data><cluster-name>:<encrypted data># this is the directory to use when backing up replayed messagesvault_dir:"vault"Theload_testandmail_toilsections are both optional. If you don't need
mail toil or load testing functionality you can leave it out. The plugin will
give you a warning if you're missing either and attempting to use the
functionality.Encrypting data for config valuesIn themail_toilsection of the config, theservice_bus_connection_stringsandstorage_accountsfields expect data that is encrypted. This can be
achieved with the pre-buildvictoria encryptcommand. Details on this
can be found inthe documentationbut for a whistlestop tour:Make sure you've set up yourVictoria cloud encryption backend.Find a service bus you want to use with MailToil in the Azure Portal, and
click on 'shared access policies' on the sidebar. Click on the policy here.
Copy the 'primary connection string' from here.Paste it into the following Victoria command:$ victoria encrypt data <the connection string>The command should output a YAML object containing fieldsdata,iv,key, andversion.
This is the encrypted connection string and can be safely stored in config.
Put this YAML object in a key in yourservice_bus_connection_stringsconfig field.
The key name should be the cluster name, i.e. for uksprod1:mail_toil:service_bus_connection_strings:cluster_name:data:<snip>iv:<snip>key:<snip>version:<snip>Continue to do this for all of the service buses.For storage accounts, follow the same steps, except for step 2. Instead:
Find a storage account you want to use with MailToil in Azure Portal, and
click on 'Access keys' on the sidebar. Copy thekey1connection string from
here.Load testingTheloadtestcommand can be used to load test an SMTP endpoint.It accepts the followingrequiredarguments:-e,--endpoint: The SMTP endpoint to send to, with optional port i.e.smtp.example.com:465if the port is unspecified, i.e.smtp.example.comthen port 25 is used-r,--recipient: The email address to send mail to, i.e.test@example.com-s,--sender: The email address to send mail from, i.e.test@example.comRunning with just the required arguments will send a single testIt also accepts the followingoptionalarguments:-n,--frequency: The number of tests to send per second. Defaults to 1-t,--duration: The number of seconds to run the test for. Defaults to 1-i,--tenant_ids: Custom tenant id. Can be used multiple times. Defaults to randomAll of this information can be found by runningvictoria email loadtest -h.Example of sending a single email:$ victoria email loadtest -e smtp.example.com -s test@example.com -r test@example.comExample of sending 46 mails per second for 60 seconds:$ victoria email loadtest -e smtp.example.com -n 46 -t 60 -s test@example.com -r test@example.comExample of sending using a different port than port 25:$ victoria email loadtest -e smtp.example.com:465 -s test@example.com -r test@example.comExample of sending messages with random data and (recipient, sender):$ victoria email loadtest -e smtp.example.comSending emailThesendcommand can be used to send single emails (based on a YAML manifest
format) to an SMTP endpoint.It accepts one argument, the path to the manifest file to send.The format of the manifest file is as follows:sender:"test@example.com"to:["test@example.com"]smtp_server:smtp.endpoint.comtenant_id:<guid>port:25attach:-"C:/path/to/an/attachment.pdf"Where:sender: The email address sending the email.to: The list of emails to send the email to.smtp_server: The SMTP server to send the email to.tenant_id: The FileTrust tenant ID to send the email with.port: The port on the SMTP server to connect via.attach: The list of files to attach.If that manifest YAML file waslocalhost.yml, then to send it you would run:$ victoria email send localhost.ymlReplaying dead lettersThereplaycommand is used for replaying dead letters.Simply run it with a cluster name and it will scan for dead letters in the queues
and replay them.Example:$ victoria email replay clusterWill replay dead letters using the service bus connection string under keyclusterin the config file'sservice_bus_connection_stringssection.The queues it scans can be modified by editiing the config file'squeuessection.Reconstructing mailThereconstructcommand can be used to reconstruct emails to files, cloud storage(Azure Blob/AWS S3) from transaction IDs.It can also optionally anonymise the email contents if you need to give the email to a developer for them to help identify an issue.Example of reconstructing mail with a transaction ID on local machine:$ victoria email reconstruct cluster -id <guid> -o local -loc output_folderExample of reconstructing mail with a transaction ID on azure blob:$ victoria email reconstruct cluster -id <guid> -o blob -loc output_folder -blob-conn <conn-str-in-single-quotes>Example of reconstructing and anonymising mail on local machine:$ victoria email reconstruct cluster -id <guid> -o local -loc output_folder --anonExample of reconstructing multiple transactions:$ victoria email reconstruct cluster -id <guid1> -id <guid2> -id <guid3> -o local -loc output_folderThe cluster name given corresponds to keys instorage_accountsin the configReplaying mail through the systemTherecovercommand can replay mail through the system right from the beginning.
It does this by using a transaction ID to grab the failed mail, reconstructing it,
and sending it to an SMTP endpoint, as if it were a new mail.This should only be done in the event of an emergency (mostly when SMTP receiver
was not able to put a received email on the message inspection queue). If you
replay a message more than once or replay a message that has already been
transmitted then customers will receive duplicate emails.It accepts a list of transaction IDs to replay in the form of a text file, with
each transaction ID on a separate line:896551ff-520f-4055-8452-36b4de64f0b4
fd18b432-4307-4fbb-a7f6-e149bace0bae
52b4686b-0ac4-4b6a-b3e0-3a8f3db47711
...For example, to replay multiple transactions in filebad-tx.txt(with SMTP Rx
port forwarded on port 25 viakubectl):$ victoria email recover cluster -f bad-tx.txt -o smtp.endpoint.com:25For example, to replay single transaction896551ff-520f-4055-8452-36b4de64f0b4(with SMTP Rx
port forwarded on port 25 viakubectl):$ victoria email recover cluster -id 896551ff-520f-4055-8452-36b4de64f0b4 -o localhost:25The cluster name given here corresponds to a key instorage_accountsin the
config.Developer guidePrerequisitesPython 3.7+PipenvQuick startClone this repoIn the root of the repo, runpipenv sync --devYou're good to go. You can run the plugin during development withpipenv run victoria email