DigDag - Schedule a Job(workflow) multiple times on same day
Tags: digdag, scheduler, productivity, devops
Published: 2020-02-15 18:55:22

Hey folks! So, I had a task of scheduling a Dashboard execution daily at 12.00 pm and 4.00 pm and was supposed to use [DigDag Job Scheduler](https://www.digdag.io/). But, after spending quite some time on [DigDag’s documentation](http://docs.digdag.io/getting_started.html), I couldn’t find anything relevant as per my requirement. Hence, I decided to try a workaround.


>"Instead of scheduling the Dashboard using a single workflow (the *.dig file), I can import/include the workflow into another workflow(s) and schedule it as per my need." 

For example:


Note: The below example is demonstrated on Ubuntu 18.04 (Linux). Please refer DigDag’s documentation for other platform-specific commands.


Lets generate a sample digdag project <em>schedule-demo</em> using command:

<pre>$ digdag init schedule-demo</pre>


The above command will create a workflow <em>schedule-demo.dig</em> in schedule-demo directory/project with contents:

<pre>
timezone: UTC


+setup:

 echo>: start ${session_time}


+disp_current_date:

 echo>: ${moment(session_time).utc().format(‘YYYY-MM-DD HH:mm:ss Z’)}


+repeat:

 for_each>:

 order: [first, second, third]

 animal: [dog, cat]

 _do:

 echo>: ${order} ${animal}

 _parallel: true


+teardown:

 echo>: finish ${session_time}
</pre>

Let's schedule the above workflow for every one and two minutes on same day using file <strong>one_minute.dig</strong> and <strong>two_minutes.dig</strong> workflow.


Create workflows one_minute.dig and two_minutes.dig in the project schedule-demo with below-mentioned contents:



- Workflow one_minute.dig:

<pre>
!include : ‘schedule-demo.dig’

schedule:

 minutes_interval>: 1
</pre>

- Workflow two_minute.dig:

<pre>
!include : ‘schedule-demo.dig’

schedule:

 minutes_interval>: 2
</pre>

Now change your current working directory to <em>schedule-demo</em> using the command:

<pre>$ cd {current_path}/schedule-demo/</pre>

then push the project <em>schedule-demo</em> to digdag server using the command:

<pre>$ digdag push schedule-demo</pre>

(Note: On local Ubuntu setup, to start digdag server execute command <em>‘digdag server’</em> on the terminal)

If you are outside the project directory and want to push project use command:

<pre>$ digdag push schedule-demo --project /{path}/schedule-demo  --revision schedule-demo-v1</pre>

<em>{path} => path to the schedule-demo project</em>


Then, launch a browser (any) and add URL ‘http://localhost:65432/’ and under Workflow tab you will see the result similar to below image (you will have to wait for at least 5–6 minutes to get similar output as of mine):

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/51qnzvdykqvqr9vui3lo.jpeg)

If you observe the image, the output with <em>ID 2</em> and <em>ID 5</em> has 2 minutes difference (refer column 'Last Attempt' ) and the value of Workflow is <strong>two-minute</strong>. You will find similar result for the one-minute workflow (i.e. the execution time difference between every present and next workflow attempt will be 1 minute).


P.S. There could be different ways to achieve the result demonstrated in the above example. This is just one of those :)


If you enjoyed this blog, share it with your friends!


Thank-You for reading!


Sources: 

https://www.digdag.io/
https://docs.digdag.io/getting_started.html
https://docs.digdag.io/workflow_definition.html?highlight=include#include-another-file