Creating External DSLs using ANTLR and C#
Tags: antlr, csharp, dsl
Published: 2020-10-06 05:32:43

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/0v5r06bt2dzn5g6s2f38.png)
  <div class="stackedit__html"><p>In this post I will show you how to write a simple DSL in C# using ANTLR.</p>
<h1 id="what-is-dsl">What is DSL?</h1>
<blockquote>
<p>A domain-specific language is a computer language specialized to a particular application domain. This is in contrast to a general-purpose language, which is broadly applicable across domains.</p>
</blockquote>
<h1 id="what-is-antlr">What is ANTLR?</h1>
<blockquote>
<p>In computer-based language recognition, ANTLR, or ANother Tool for Language Recognition, is a parser generator that uses LL for parsing. ANTLR is the successor to the Purdue Compiler Construction Tool Set, first developed in 1989, and is under active development</p>
</blockquote>
<h1 id="what-are-we-going-to-build">What are we going to build</h1>
<p>In this article, I will show you how to create a DSL that converts a sentence into C# <code>POCO</code> classes that you can serialize and save into the database.</p>
<pre class=" language-bash"><code class="prism  language-bash">book a hotel room <span class="token keyword">for</span> Santosh Singh and 3 guests on 23-SEP-2013 00:12
</code></pre>
<p>We are going to parse the above statement into C# <code>POCO</code> classes by using ANLTR, and the output will be as shown below.</p>
<pre class=" language-bash"><code class="prism  language-bash"><span class="token punctuation">{</span><span class="token string">"Time"</span><span class="token keyword">:</span><span class="token string">"2013-09-23T00:12:00"</span>,<span class="token string">"Person"</span>:<span class="token punctuation">{</span><span class="token string">"FirstName"</span><span class="token keyword">:</span><span class="token string">"Santosh"</span>,<span class="token string">"LastName"</span><span class="token keyword">:</span><span class="token string">"Singh"</span>,<span class="token string">"NumberOfGuests"</span>:3<span class="token punctuation">}</span><span class="token punctuation">}</span>
</code></pre>
<h2 id="prerequisite">Prerequisite</h2>
<ul>
<li>ANTLR</li>
<li>Visual Studio Code with ANTLR extensions<br>
Before writing, grammar let’s first create our <code>POCO</code> classes that we want to populate from our DSL</li>
</ul>
<blockquote>
<p>Booking.cs</p>
</blockquote>
<pre class=" language-csharp"><code class="prism  language-csharp"><span class="token keyword">using</span> System<span class="token punctuation">;</span>

<span class="token keyword">public</span> <span class="token keyword">partial</span> <span class="token keyword">class</span> <span class="token class-name">Booking</span>
<span class="token punctuation">{</span>
    <span class="token keyword">public</span> DateTime Time <span class="token punctuation">{</span> <span class="token keyword">get</span><span class="token punctuation">;</span> <span class="token keyword">set</span><span class="token punctuation">;</span> <span class="token punctuation">}</span>
    <span class="token keyword">public</span> Person Person <span class="token punctuation">{</span> <span class="token keyword">get</span><span class="token punctuation">;</span> <span class="token keyword">set</span><span class="token punctuation">;</span> <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
</code></pre>
<blockquote>
<p>Person.cs</p>
</blockquote>
<pre class=" language-csharp"><code class="prism  language-csharp"><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">Person</span><span class="token punctuation">{</span>

<span class="token keyword">public</span> <span class="token keyword">string</span> FirstName <span class="token punctuation">{</span> <span class="token keyword">get</span><span class="token punctuation">;</span> <span class="token keyword">set</span><span class="token punctuation">;</span> <span class="token punctuation">}</span>
<span class="token keyword">public</span> <span class="token keyword">string</span> LastName <span class="token punctuation">{</span> <span class="token keyword">get</span><span class="token punctuation">;</span> <span class="token keyword">set</span><span class="token punctuation">;</span> <span class="token punctuation">}</span>
<span class="token keyword">public</span> <span class="token keyword">int</span> NumberOfGuests <span class="token punctuation">{</span> <span class="token keyword">get</span><span class="token punctuation">;</span> <span class="token keyword">set</span><span class="token punctuation">;</span> <span class="token punctuation">}</span>
    <span class="token keyword">public</span> <span class="token keyword">override</span> <span class="token keyword">string</span> <span class="token function">ToString</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
    <span class="token punctuation">{</span>
        <span class="token keyword">return</span> $<span class="token string">"{FirstName}-{LastName}-{NumberOfGuests}"</span><span class="token punctuation">;</span>
    <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
</code></pre>
<h2 id="antlr-grammar">ANTLR Grammar</h2>
<p>The first thing we will do that write the grammar file for the above statement. Below is the grammar for our DSL. ANTLR grammar consists of two parts</p>
<ul>
<li>Parser</li>
<li>Lexer</li>
</ul>
<p>In our grammar parser are</p>
<ul>
<li>booking</li>
<li>time</li>
<li>person</li>
</ul>
<p>Booking is the start of the parsing.</p>
<blockquote>
<p>Here I am using Actions and Attribute instead of Visitor. Actions are blocks of text written in the target language and enclosed in curly braces. The recognizer triggers them according to their locations within the grammar. For example, the following rule emits a <code>Booking</code> model after the parser has seen a valid declaration:</p>
</blockquote>
<pre><code>booking
	returns[Booking b]:
	'book' 'a' 'hotel' 'room' 'for' person 'guests' 'on' time {
      $b=new Booking();
      $b.Time=$time.t;
      $b.Person=$person.p;

    };
</code></pre>
<blockquote>
<p>Grammar</p>
</blockquote>
<pre class=" language-bash"><code class="prism  language-bash">grammar Booking<span class="token punctuation">;</span>

@lexer::header <span class="token punctuation">{</span>
    using System<span class="token punctuation">;</span>
<span class="token punctuation">}</span>

booking
	returns<span class="token punctuation">[</span>Booking b<span class="token punctuation">]</span>:
	<span class="token string">'book'</span> <span class="token string">'a'</span> <span class="token string">'hotel'</span> <span class="token string">'room'</span> <span class="token string">'for'</span> person <span class="token string">'guests'</span> <span class="token string">'on'</span> <span class="token function">time</span> <span class="token punctuation">{</span>
      <span class="token variable">$b</span><span class="token operator">=</span>new Booking<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
      <span class="token variable">$b</span>.Time<span class="token operator">=</span><span class="token variable">$time</span>.t<span class="token punctuation">;</span>
      <span class="token variable">$b</span>.Person<span class="token operator">=</span><span class="token variable">$person</span>.p<span class="token punctuation">;</span>

    <span class="token punctuation">}</span><span class="token punctuation">;</span>
<span class="token function">time</span>
	returns<span class="token punctuation">[</span>DateTime t<span class="token punctuation">]</span>:
	d <span class="token operator">=</span> datetime <span class="token punctuation">{</span>
    Console.WriteLine<span class="token punctuation">(</span><span class="token variable">$d</span>.text<span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token variable">$t</span><span class="token operator">=</span>DateTime.Parse<span class="token punctuation">(</span><span class="token variable">$d</span>.text<span class="token punctuation">)</span><span class="token punctuation">;</span>

<span class="token punctuation">}</span><span class="token punctuation">;</span>
person
	returns<span class="token punctuation">[</span>Person p<span class="token punctuation">]</span>:
	f <span class="token operator">=</span> firstName l <span class="token operator">=</span> lastName <span class="token string">'and'</span> n <span class="token operator">=</span> numberOfGuests <span class="token punctuation">{</span>

       <span class="token variable">$p</span><span class="token operator">=</span>new Person<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
       <span class="token variable">$p</span>.FirstName<span class="token operator">=</span><span class="token variable">$f</span>.text<span class="token punctuation">;</span>
       <span class="token variable">$p</span>.LastName<span class="token operator">=</span><span class="token variable">$l</span>.text<span class="token punctuation">;</span>
       <span class="token variable">$p</span>.NumberOfGuests<span class="token operator">=</span>int.Parse<span class="token punctuation">(</span><span class="token variable">$n</span>.text<span class="token punctuation">)</span><span class="token punctuation">;</span>

   <span class="token punctuation">}</span><span class="token punctuation">;</span>

firstName: STRING<span class="token punctuation">;</span>
lastName: STRING<span class="token punctuation">;</span>
numberOfGuests: NUMBER<span class="token punctuation">;</span>
datetime:
	NUMBER NUMBER SEPARATOR MONTH SEPARATOR YEAR NUMBER NUMBER COLON NUMBER NUMBER<span class="token punctuation">;</span>
YEAR: NUMBER NUMBER NUMBER NUMBER<span class="token punctuation">;</span>

NUMBER: <span class="token punctuation">[</span>0-9<span class="token punctuation">]</span><span class="token punctuation">;</span>
MONTH:
	JAN
	<span class="token operator">|</span> FEB
	<span class="token operator">|</span> MAR
	<span class="token operator">|</span> APR
	<span class="token operator">|</span> MAY
	<span class="token operator">|</span> JUN
	<span class="token operator">|</span> JUL
	<span class="token operator">|</span> AUG
	<span class="token operator">|</span> SEP
	<span class="token operator">|</span> OCT
	<span class="token operator">|</span> NOV
	<span class="token operator">|</span> DEC<span class="token punctuation">;</span>

STRING: <span class="token punctuation">[</span>a-zA-Z<span class="token punctuation">]</span><span class="token punctuation">[</span>a-zA-Z<span class="token punctuation">]</span>+<span class="token punctuation">;</span>
JAN: <span class="token punctuation">[</span>Jj<span class="token punctuation">]</span><span class="token punctuation">[</span>Aa<span class="token punctuation">]</span><span class="token punctuation">[</span>Nn<span class="token punctuation">]</span><span class="token punctuation">;</span>
FEB: <span class="token punctuation">[</span>Ff<span class="token punctuation">]</span><span class="token punctuation">[</span>Ee<span class="token punctuation">]</span><span class="token punctuation">[</span>Bb<span class="token punctuation">]</span><span class="token punctuation">;</span>
MAR: <span class="token punctuation">[</span>Mm<span class="token punctuation">]</span><span class="token punctuation">[</span>Aa<span class="token punctuation">]</span><span class="token punctuation">[</span>Rr<span class="token punctuation">]</span><span class="token punctuation">;</span>
APR: <span class="token punctuation">[</span>Aa<span class="token punctuation">]</span><span class="token punctuation">[</span>Pp<span class="token punctuation">]</span><span class="token punctuation">[</span>Rr<span class="token punctuation">]</span><span class="token punctuation">;</span>
MAY: <span class="token punctuation">[</span>Mm<span class="token punctuation">]</span><span class="token punctuation">[</span>Aa<span class="token punctuation">]</span><span class="token punctuation">[</span>Yy<span class="token punctuation">]</span><span class="token punctuation">;</span>
JUN: <span class="token punctuation">[</span>Jj<span class="token punctuation">]</span><span class="token punctuation">[</span>Uu<span class="token punctuation">]</span><span class="token punctuation">[</span>Nn<span class="token punctuation">]</span><span class="token punctuation">;</span>
JUL: <span class="token punctuation">[</span>Jj<span class="token punctuation">]</span><span class="token punctuation">[</span>Uu<span class="token punctuation">]</span><span class="token punctuation">[</span>Ll<span class="token punctuation">]</span><span class="token punctuation">;</span>
AUG: <span class="token punctuation">[</span>Aa<span class="token punctuation">]</span><span class="token punctuation">[</span>Uu<span class="token punctuation">]</span><span class="token punctuation">[</span>Gg<span class="token punctuation">]</span><span class="token punctuation">;</span>
SEP: <span class="token punctuation">[</span>Ss<span class="token punctuation">]</span><span class="token punctuation">[</span>Ee<span class="token punctuation">]</span><span class="token punctuation">[</span>Pp<span class="token punctuation">]</span><span class="token punctuation">;</span>
OCT: <span class="token punctuation">[</span>Oo<span class="token punctuation">]</span><span class="token punctuation">[</span>Cc<span class="token punctuation">]</span><span class="token punctuation">[</span>Tt<span class="token punctuation">]</span><span class="token punctuation">;</span>
NOV: <span class="token punctuation">[</span>Nn<span class="token punctuation">]</span><span class="token punctuation">[</span>Oo<span class="token punctuation">]</span><span class="token punctuation">[</span>Vv<span class="token punctuation">]</span><span class="token punctuation">;</span>
DEC: <span class="token punctuation">[</span>Dd<span class="token punctuation">]</span><span class="token punctuation">[</span>Ee<span class="token punctuation">]</span><span class="token punctuation">[</span>Cc<span class="token punctuation">]</span><span class="token punctuation">;</span>
SEPARATOR: <span class="token string">'-'</span><span class="token punctuation">;</span>
COLON: <span class="token string">':'</span><span class="token punctuation">;</span>

WS: <span class="token punctuation">(</span><span class="token string">' '</span> <span class="token operator">|</span> <span class="token string">'\r'</span> <span class="token operator">|</span> <span class="token string">'\n'</span> <span class="token operator">|</span> <span class="token string">'\t'</span><span class="token punctuation">)</span> -<span class="token operator">&gt;</span> channel<span class="token punctuation">(</span>HIDDEN<span class="token punctuation">)</span><span class="token punctuation">;</span>

</code></pre>
<blockquote>
<p>Parse Tree</p>
</blockquote>
<p><img src="https://1.bp.blogspot.com/-LG6qdRDm3HI/X14J5hSFfnI/AAAAAAAAM2A/abxwkU7NkZ0o3BveOk0WWW7eR6Dc_aqyQCLcBGAsYHQ/s320/parseTree.png" alt=""></p>
<p>Now inside the main method, write the following code.</p>
<blockquote>
<p>Assuming you have generated the parser and lexer code using ANTLR.</p>
</blockquote>
<blockquote>
<p>Program.cs</p>
</blockquote>
<pre class=" language-csharp"><code class="prism  language-csharp"><span class="token keyword">using</span> System<span class="token punctuation">;</span>
<span class="token keyword">using</span> Antlr4<span class="token punctuation">.</span>Runtime<span class="token punctuation">;</span>
<span class="token keyword">using</span> Newtonsoft<span class="token punctuation">.</span>Json<span class="token punctuation">;</span>

<span class="token keyword">namespace</span> antlraction
<span class="token punctuation">{</span>
    <span class="token keyword">class</span> <span class="token class-name">Program</span>
    <span class="token punctuation">{</span>
        <span class="token keyword">static</span> <span class="token keyword">void</span> <span class="token function">Main</span><span class="token punctuation">(</span><span class="token keyword">string</span><span class="token punctuation">[</span><span class="token punctuation">]</span> args<span class="token punctuation">)</span>
        <span class="token punctuation">{</span>
            <span class="token keyword">var</span> input <span class="token operator">=</span> <span class="token string">"book a hotel room  for Santosh Singh and 3 guests on 23-SEP-2013 00:12"</span><span class="token punctuation">;</span>

            <span class="token keyword">var</span> charStream <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">AntlrInputStream</span><span class="token punctuation">(</span>input<span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token keyword">var</span> lexer <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">BookingLexer</span><span class="token punctuation">(</span>charStream<span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token keyword">var</span> tokens <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">CommonTokenStream</span><span class="token punctuation">(</span>lexer<span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token keyword">var</span> parser <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">BookingParser</span><span class="token punctuation">(</span>tokens<span class="token punctuation">)</span><span class="token punctuation">;</span>

            <span class="token keyword">var</span> t <span class="token operator">=</span> parser<span class="token punctuation">.</span><span class="token function">booking</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span>b<span class="token punctuation">;</span>
            System<span class="token punctuation">.</span>Console<span class="token punctuation">.</span><span class="token function">WriteLine</span><span class="token punctuation">(</span>JsonConvert<span class="token punctuation">.</span><span class="token function">SerializeObject</span><span class="token punctuation">(</span>t<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token punctuation">}</span>
    <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
</code></pre>
<p><img src="https://1.bp.blogspot.com/-cfJq1ifj9KU/X13-rrc9iGI/AAAAAAAAM10/WKR0ijonO-A-yehADOD7rQ20-91Aqkm_gCLcBGAsYHQ/s400/out.PNG" alt=""></p>
<pre class=" language-bash"><code class="prism  language-bash"><span class="token punctuation">{</span><span class="token string">"Time"</span><span class="token keyword">:</span><span class="token string">"2013-09-23T00:12:00"</span>,<span class="token string">"Person"</span>:<span class="token punctuation">{</span><span class="token string">"FirstName"</span><span class="token keyword">:</span><span class="token string">"Santosh"</span>,<span class="token string">"LastName"</span><span class="token keyword">:</span><span class="token string">"Singh"</span>,<span class="token string">"NumberOfGuests"</span>:3<span class="token punctuation">}</span><span class="token punctuation">}</span>
</code></pre>
</div>
[My Blog](https://codeguru.co.in)
[ANTLR CSV](https://www.codeguru.co.in/2020/09/how-to-parse-csv-using-antlr-in-c.html)
[Create Compiler In C# Using ANTLR-Crash Course ](https://www.amazon.com/Create-Compiler-Using-ANTLR-Crash-Course-ebook/dp/B09BJ4CRTJ/?_encoding=UTF8&pd_rd_w=wpB9S&pf_rd_p=6e9da02f-f7a3-444f-aea6-9ef09ed8bb89&pf_rd_r=FET7J8TSK07KVT64TNK9&pd_rd_r=da488aa2-6156-4f6b-a7f3-7e573e8df0df&pd_rd_wg=YlRFM&ref_=pd_gw_ci_mcx_mr_hp_d)