Perceived gaps to GPT assisted COS development automation
Tags: ai, chatgpt, tutorial, programming
Published: 2023-11-23 18:56:49

<p>​Keywords: ChatGPT, COS, Lookup Table, IRIS, AI&nbsp;</p>
<p>&nbsp;</p>
<h2>Purpose</h2>
<p><br>Here is another quick note before we move on to GPT-4 assisted automation journey. Below are some "little"&nbsp;helps ChatGPT had already been offering,&nbsp;&nbsp;here and there, during daily works.&nbsp;</p>
<p>And what could be the perceived gaps, risks and traps to LLMs assisted automation,&nbsp;if you happen to explore this path too.&nbsp;I'd also love to hear anyone's use cases and experiences on this front too.&nbsp;</p>
<p>&nbsp;</p>
<h2>Lookup tables</h2>
<p><br>One of the simplest tasks could be Lookup tables.&nbsp; Frequently we have to turn tedious CSV or Excel code tables into some Lookup tables in XML, manually, particularly there are blanks etc randomly distributed in the values across thousand records, and you had to tidy them&nbsp;quite a few times before go-live deadlines.&nbsp;<br><br>Can ChatGPT help a bit? Here is a quick test.<br><br><strong>Prompt</strong>:</p>
<p style="margin-left:40px"><span style="color:#3498db">You are a program to automatically convert a CSV file into a XML lookup table file.<br><br>The input is a comma delimited CSV file with 3 data columns :<br>Blood Science ,XCOM,CANCELLED/REJECTED SAMPLE<br>Blood Science ,ADX ,ADD ON TEST COMMENT<br>Blood Transfusion,DARA,ALERT FOR DARA/CAMELLIA<br>Blood Transfusion, BT1 ,REJECT SAMPLE UNLABELLED<br><br>The output should be a XML file in the form of:<br>&lt;?xml version="1.0"?&gt;<br>&lt;lookupTable&gt;<br>&nbsp; &nbsp; &nbsp;&lt;entry table="HIE.ICE.FilterOBR4" key="XCOM"&gt;Blood Science&lt;/entry&gt;<br>&nbsp; &nbsp; &nbsp;&lt;entry table="HIE.ICE.FilterOBR4" key="ADX "&gt;Blood Science&lt;/entry&gt;<br>&nbsp; &nbsp; &nbsp;&lt;entry table="HIE.ICE.FilterOB</span><span style="color:#3498db">R4" key="DARA"&gt;Blood Transfusion&lt;/entry&gt;<br>&nbsp; &nbsp;&nbsp; &lt;entry table="HIE.ICE.FilterOBR4" key="BT1"&gt;Blood Transfusion&lt;/entry&gt;<br>&lt;/lookupTable&gt;<br>Note: the sequence of the XML attribute in each root should be kept as they are</span></p>

<p><br><strong>ChatGPT</strong>:</p>
<p style="margin-left:40px"><span style="color:#8e44ad">Here's a Python script that reads in the CSV file, processes it, and outputs the resulting XML file:</span></p>

<pre class="codeblock-container" idlang="2" lang="Python" tabsize="4"><code class="language-python hljs"><span class="hljs-keyword">import</span> csv

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">convert_csv_to_xml</span><span class="hljs-params">(input_file, output_file)</span>:</span>
    <span class="hljs-string">"""
    Convert a CSV file to an XML lookup table file.

    Parameters:
    input_file (str): Path to the input CSV file.
    output_file (str): Path to the output XML file.

    Returns:
    int: 0 if the conversion was successful, 1 if there was an error.
    """</span>
    <span class="hljs-keyword">try</span>:
        <span class="hljs-comment"># Create a dictionary to hold the lookup table data</span>
        lookup_table = {}

        <span class="hljs-comment"># Read in the CSV file and process the data</span>
        <span class="hljs-keyword">with</span> open(input_file, <span class="hljs-string">"r"</span>) <span class="hljs-keyword">as</span> csvfile:
            csvreader = csv.reader(csvfile)
            <span class="hljs-keyword">for</span> row <span class="hljs-keyword">in</span> csvreader:
                category = row[<span class="hljs-number">0</span>].strip()
                key = row[<span class="hljs-number">1</span>].strip()
                value = row[<span class="hljs-number">2</span>].strip()
                lookup_table[key] = category

        <span class="hljs-comment"># Generate the XML file</span>
        <span class="hljs-keyword">with</span> open(output_file, <span class="hljs-string">"w"</span>) <span class="hljs-keyword">as</span> xmlfile:
            xmlfile.write(<span class="hljs-string">'&lt;?xml version="1.0"?&gt;\n'</span>)
            xmlfile.write(<span class="hljs-string">'&lt;lookupTable&gt;\n'</span>)
            <span class="hljs-keyword">for</span> key <span class="hljs-keyword">in</span> lookup_table:
                xmlfile.write(<span class="hljs-string">f'&lt;entry table="HIE.ICE.FilterOBR4" key="<span class="hljs-subst">{key}</span>"&gt;<span class="hljs-subst">{lookup_table[key]}</span>&lt;/entry&gt;\n'</span>)
            xmlfile.write(<span class="hljs-string">'&lt;/lookupTable&gt;\n'</span>)

        <span class="hljs-keyword">return</span> <span class="hljs-number">0</span>
    <span class="hljs-keyword">except</span> Exception <span class="hljs-keyword">as</span> e:
        print(<span class="hljs-string">"Error:"</span>, e)
        <span class="hljs-keyword">return</span> <span class="hljs-number">1</span></code></pre>
<p style="margin-left:40px"><span style="color:#8e44ad">This script first reads in the CSV file using the <code>csv</code> module and creates a dictionary to hold the lookup table data. It then processes each row of the CSV file and adds the relevant data to the dictionary.</span></p>
<p style="margin-left:40px"><span style="color:#8e44ad">Finally, the script generates the XML file by iterating through the dictionary and writing the appropriate XML tags and attributes to the output file. Note that the <code>f</code> string syntax is used to dynamically insert the key and category data into the XML tags.</span></p>

<p><br>I ran it briefly in a Jupyter notebook, it checked out. It also stripped out the blanks (was going to be a lot manual tidy-ups). It gave a tool&nbsp;handling&nbsp;lengthy CSVs with messy vlues, without being limited to token size.&nbsp;<br><br>In this specific case, I'd say it increased the productivity about 20+ times over repeated manual approaches handling&nbsp;many messy code tables.<br><br><br>Now that it save me a bit&nbsp;time, so I was wondering whether it can generate&nbsp;a COS sample too to build Util class for future service BAUs etc.&nbsp;<br><br><br>Prompt:&nbsp;&nbsp;</p>
<p style="margin-left:40px"><span style="color:#2980b9">Could you convert the above Python function to intersystems COS classmethod, by using correct COS specific function please</span></p>

<p>ChatGPT:</p>
<p style="margin-left:40px"><span style="color:#8e44ad">Certainly! Here's the converted code as an InterSystems ObjectScript class method:</span></p>

<pre class="codeblock-container" idlang="0" lang="ObjectScript" tabsize="4"><code class="language-cls hljs cos"><span class="hljs-keyword">ClassMethod</span> ConvertCSVToXML(inputFile <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span>, outputFile <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span>) <span class="hljs-keyword">As</span> <span class="hljs-built_in">%Integer</span>
{
    <span class="hljs-keyword">try</span> {
        <span class="hljs-comment">// Create a dictionary to hold the lookup table data</span>
        <span class="hljs-keyword">set</span> lookupTable = {}

        <span class="hljs-comment">// Read in the CSV file and process the data</span>
        <span class="hljs-keyword">set</span> csvReader = <span class="hljs-keyword">##class</span>(<span class="hljs-built_in">%File.CharacterStream</span>).<span class="hljs-built_in">%New</span>()
        <span class="hljs-keyword">do</span> csvReader.LinkToFile(inputFile)
        <span class="hljs-keyword">while</span> (csvReader.AtEnd = <span class="hljs-number">0</span>) {
            <span class="hljs-keyword">set</span> row = csvReader.ReadLine()
            <span class="hljs-keyword">set</span> category = <span class="hljs-built_in">$ZSTRIP</span>(<span class="hljs-built_in">$PIECE</span>(row, <span class="hljs-string">","</span>, <span class="hljs-number">1</span>), <span class="hljs-string">"B"</span>)
            <span class="hljs-keyword">set</span> key = <span class="hljs-built_in">$ZSTRIP</span>(<span class="hljs-built_in">$PIECE</span>(row, <span class="hljs-string">","</span>, <span class="hljs-number">2</span>), <span class="hljs-string">"B"</span>)
            <span class="hljs-keyword">set</span> value = <span class="hljs-built_in">$ZSTRIP</span>(<span class="hljs-built_in">$PIECE</span>(row, <span class="hljs-string">","</span>, <span class="hljs-number">3</span>), <span class="hljs-string">"B"</span>)
            <span class="hljs-keyword">set</span> lookupTable(key) = category
        }
        <span class="hljs-keyword">do</span> csvReader.<span class="hljs-built_in">%Close</span>()

        <span class="hljs-comment">// Generate the XML file</span>
        <span class="hljs-keyword">set</span> xmlWriter = <span class="hljs-keyword">##class</span>(<span class="hljs-built_in">%File.CharacterStream</span>).<span class="hljs-built_in">%New</span>()
        <span class="hljs-keyword">do</span> xmlWriter.LinkToFile(outputFile)
        <span class="hljs-keyword">do</span> xmlWriter.WriteLine(<span class="hljs-string">"&lt;?xml version=""1.0""?&gt;"</span>)
        <span class="hljs-keyword">do</span> xmlWriter.WriteLine(<span class="hljs-string">"&lt;lookupTable&gt;"</span>)
        <span class="hljs-keyword">set</span> keys = lookupTable.<span class="hljs-built_in">%GetKeys</span>()
        <span class="hljs-keyword">while</span> ('keys.<span class="hljs-built_in">%AtEnd</span>) {
            <span class="hljs-keyword">set</span> key = keys.<span class="hljs-built_in">%GetNext</span>()
            <span class="hljs-keyword">set</span> category = lookupTable(key)
            <span class="hljs-keyword">do</span> xmlWriter.WriteLine(<span class="hljs-string">"&lt;entry table=""HIE.ICE.FilterOBR4"" key="""</span>_key_<span class="hljs-string">"""&gt;"</span>_category_<span class="hljs-string">"&lt;/entry&gt;"</span>)
        }
        <span class="hljs-keyword">do</span> xmlWriter.WriteLine(<span class="hljs-string">"&lt;/lookupTable&gt;"</span>)
        <span class="hljs-keyword">do</span> xmlWriter.<span class="hljs-built_in">%Close</span>()

        <span class="hljs-keyword">quit</span> <span class="hljs-number">0</span>
    }
    <span class="hljs-keyword">catch</span> (ex) {
        <span class="hljs-keyword">Write</span> <span class="hljs-string">"Error: "</span>_ex.DisplayString(), !
        <span class="hljs-keyword">quit</span> <span class="hljs-number">1</span>
    }
}
</code></pre>
<p style="margin-left:40px"><span style="color:#8e44ad">In InterSystems ObjectScript, we can use the <code>%File.CharacterStream</code> class to read and write files. We use the <code>LinkToFile()</code> method to open the input and output files, and then use the <code>ReadLine()</code> and <code>WriteLine()</code> methods to read and write data to the files.</span></p>
<p style="margin-left:40px"><span style="color:#8e44ad">To strip leading and trailing spaces from each field, we use the <code>$ZSTRIP()</code> function with the "B" flag.</span></p>
<p style="margin-left:40px"><span style="color:#8e44ad">We also use the <code>%GetKeys()</code> method of the <code>lookupTable</code> object to iterate over the keys in the dictionary.</span></p>
<p style="margin-left:40px"><span style="color:#8e44ad">Finally, the method returns a status code of 0 if the conversion was successful, or 1 if there was an erro</span>r.</p>

<p><br>It didn't give an accurate tool that can check out as straightaway as Python.&nbsp;&nbsp;<span style="color:#8e44ad"><code>%File.CharacterStream</code></span>&nbsp;should be&nbsp; <basefont name="Courier New"> <font color="#008080">%Stream.FileCharacter</font>,&nbsp; stripping blanks should use "&lt;&gt;W", and looping through globals should use $ORDER etc etc.&nbsp;</p>
<p>Prompts can be improved, but I am particularly not good at asking for favors, so I rushed up a few tweaks along the lines.&nbsp;</p>
<p>&nbsp;</p>
<pre class="codeblock-container" idlang="0" lang="ObjectScript" tabsize="4"><code class="language-cls hljs cos"><span class="hljs-keyword">ClassMethod</span> ConvertCSVToXML2(inputFile <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span>, outputFile <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span>) <span class="hljs-keyword">As</span> <span class="hljs-built_in">%Integer</span>
{
  
  <span class="hljs-keyword">s</span> tSC = <span class="hljs-built_in">$$$OK</span>
  <span class="hljs-keyword">try</span> {
		<span class="hljs-keyword">set</span> <span class="hljs-symbol">^lookupTable</span> = <span class="hljs-string">""</span>
        <span class="hljs-comment">// Read in the CSV file and process the data</span>
        <span class="hljs-keyword">set</span> csvReader = <span class="hljs-keyword">##class</span>(<span class="hljs-built_in">%Stream.FileCharacter</span>).<span class="hljs-built_in">%New</span>()
        <span class="hljs-keyword">set</span> tSC =  csvReader.LinkToFile(inputFile)
        <span class="hljs-keyword">while</span> (csvReader.AtEnd = <span class="hljs-number">0</span>) {
            <span class="hljs-keyword">set</span> row = csvReader.ReadLine()
            <span class="hljs-keyword">set</span> category = <span class="hljs-built_in">$ZSTRIP</span>(<span class="hljs-built_in">$PIECE</span>(row, <span class="hljs-string">","</span>, <span class="hljs-number">1</span>), <span class="hljs-string">"&lt;&gt;W"</span>)
            <span class="hljs-keyword">set</span> key = <span class="hljs-built_in">$ZSTRIP</span>(<span class="hljs-built_in">$PIECE</span>(row, <span class="hljs-string">","</span>, <span class="hljs-number">2</span>), <span class="hljs-string">"&lt;&gt;W"</span>)
            <span class="hljs-keyword">set</span> value = <span class="hljs-built_in">$ZSTRIP</span>(<span class="hljs-built_in">$PIECE</span>(row, <span class="hljs-string">","</span>, <span class="hljs-number">3</span>), <span class="hljs-string">"&lt;&gt;W"</span>)
            <span class="hljs-keyword">set</span> <span class="hljs-symbol">^lookupTable</span>(key) = category
        }
        <span class="hljs-comment">//zw ^lookupTable</span>
        <span class="hljs-keyword">do</span> csvReader.<span class="hljs-built_in">%Close</span>()
        
        <span class="hljs-comment">// Generate the XML file</span>
        <span class="hljs-keyword">set</span> xmlWriter = <span class="hljs-keyword">##class</span>(<span class="hljs-built_in">%Stream.FileCharacter</span>).<span class="hljs-built_in">%New</span>()
        <span class="hljs-keyword">set</span> tSC = xmlWriter.LinkToFile(outputFile)
        <span class="hljs-keyword">set</span> tSC = xmlWriter.WriteLine(<span class="hljs-string">"&lt;?xml version=""1.0""?&gt;"</span>)
        <span class="hljs-keyword">set</span> tSC = xmlWriter.WriteLine(<span class="hljs-string">"&lt;lookupTable&gt;"</span>)
        <span class="hljs-keyword">set</span> key = <span class="hljs-built_in">$O</span>(<span class="hljs-symbol">^lookupTable</span>(<span class="hljs-string">""</span>)) <span class="hljs-comment">//lookupTable.%GetKeys()</span>
        <span class="hljs-keyword">while</span> (key '= <span class="hljs-string">""</span>) {
	        <span class="hljs-comment">//w key</span>
            <span class="hljs-keyword">set</span> category = <span class="hljs-built_in">$GET</span>(<span class="hljs-symbol">^lookupTable</span>(key))
            <span class="hljs-keyword">w</span> !,key, <span class="hljs-string">" "</span>, category
            <span class="hljs-keyword">set</span> tSC =  xmlWriter.WriteLine(<span class="hljs-string">"&lt;entry table=""HIE.ICE.FilterOBR4"" key="""</span>_key_<span class="hljs-string">"""&gt;"</span>_category_<span class="hljs-string">"&lt;/entry&gt;"</span>)
            <span class="hljs-keyword">set</span> key = <span class="hljs-built_in">$O</span>(<span class="hljs-symbol">^lookupTable</span>(key))
        }
        <span class="hljs-keyword">set</span> tSC = xmlWriter.WriteLine(<span class="hljs-string">"&lt;/lookupTable&gt;"</span>)
   		<span class="hljs-keyword">set</span> tSC = xmlWriter.<span class="hljs-built_in">%Save</span>(<span class="hljs-string">"&lt;/lookupTable&gt;"</span>)
        <span class="hljs-keyword">set</span> tSC = xmlWriter.<span class="hljs-built_in">%Close</span>()
  }
    <span class="hljs-keyword">catch</span> (ex) {
        <span class="hljs-keyword">Write</span> <span class="hljs-string">"Error: "</span>_ex.DisplayString(), !
        <span class="hljs-keyword">s</span> tSC = ex
    }
   <span class="hljs-keyword">return</span> tSC
}</code></pre>
<p>&nbsp;</p>
<p>So what are the perceived gaps here?&nbsp;&nbsp;</p>
<p>So some potential implic<strong>ations </strong>came into my mind would be:<br><br>1. <strong>Productivity</strong>:&nbsp; The productivity reward would depend&nbsp;on the COS proficiency of the developer.&nbsp; This tool would further amplify any advantages in coding proficiency.&nbsp;&nbsp;<br><br>2. <strong>Gaps</strong>:&nbsp;My wondering would be:&nbsp; how could we drive up its accuracy by few-shot learning via prompts , with or even without fine-tuning?&nbsp; If you happen to explore this front across LLMs - I'd love to hear your thoughts, or dreams. &nbsp;<br>&nbsp;</p>
<p>&nbsp;</p>
<h2>Unit Tests Auto-generations<br>&nbsp;</h2>
<p>By talking about development, we&nbsp;can't get away from testing.&nbsp;&nbsp;<br><br>Things seem to be changing now.&nbsp;Very&nbsp;"simple" looking, sleek&nbsp;tools leveraging the power of GPTs, such as RubberDuck, mushroomed out to help.</p>
<p>So I tried RubberDuck extension in VSCode, then set it up with my OpenAI API key.<br><br>Then I open the Python function as mentioned above in VSCode, as below:<br>
</p>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6mr10y4jef7tnzg4rhgo.png)

<p>Then I&nbsp; selected click "Generate Unit Test ..." menu, and it would auto-generate some unit test coverage&nbsp;in a few seconds, at least&nbsp;100+&nbsp;times quicker than typing. It gives a quick place holder for tweaks.</p>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iorf5n3ro6t2rnw2xzdv.png)

<p>&nbsp;</p>
<p>&nbsp;</p>
<p>Can we do such for COS code too, even though neither RubberDuck nor ChatGPT truly understood COS&nbsp;yet (and it's not LLM's&nbsp;fault):<br>
</p>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/86etml6tik0s3a2001py.png)

<p>And it generated these unit test placeholder without understanding COS much yet - I'll leave its gaps open for advice for now, and particularly, how or will the gaps need to be addressed with a sense of purpose?<br>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bmtvylk15hzl7de7l5y0.png)</p>
<p>It would need some tweaks, but by harvesting ChatGPT, it seems&nbsp;now able to auto generate codes, analyse codes, auto-comment the code, and auto-generate unit test place holders for you, in today's reality.&nbsp;(Well, sort of, depending on what coding language we are using, and what we really want it to do next).</p>
<p>&nbsp;</p>
<h2>Conclusion?</h2>
<p><br>Again, I don't have quick conclusions, since I am not able to explore its boundaries and traps much yet.&nbsp;&nbsp;</p>
<p>&nbsp;</p>
<p><strong>Math foundation??</strong></p>
<p><br>Maybe one day, just as&nbsp;the concept of "entropy" in information theory was&nbsp;cast in 1948,&nbsp;another math genius might simply come out to enlighten us with another concept to simply&nbsp;quantify the "linguistic" gaps and distances among all sorts of languages, be it human or machine languages. Until then&nbsp;we would not really know the actual potential, limits. risks and traps of these LLMs. Nonetheless it doesn't seem to stop LLMs&nbsp;leap forward on monthly or weekly basis.&nbsp;</p>
<p>&nbsp;</p>
<p><strong>Other use cases??</strong></p>
<p><br><strong>Analytics dashboards driven by human natural language queries</strong>:&nbsp; A few weeks ago I was attempting some R&amp;D for an innovation project, and by chance I noticed another use case:&nbsp; &nbsp;it "can" turn clinical queries in human language into a machine query in SQL/MDX.&nbsp; Well, kind of, without any fine-tuning yet by today. It seems to start to bridge the abyss between human languages and machine tones, to a degree?&nbsp; &nbsp;<br><br>It would not be difficult to imagine such a&nbsp;scenario:&nbsp; that a clinician just types in a clinical query into a chat in her natural language, and automatically a clinical dashboard is generated, highlighting those patients who might have missed the early detection of heart failure, and grouping and ranking&nbsp;them across regions, gender, and age groups. The differences in care quality would be highlighted in minutes instead of months of engineering efforts.&nbsp;</p>
<p>&nbsp;</p>
<p>And certainly,<strong> personalised AI care assistant.</strong>&nbsp; It seemed to be so remote and so complex last year this time, and with GPT4, it rapidly becomes a reality, it seems.&nbsp; Theoretically, nothing would stop GPT4 alike LLMs looking into my care records, just bits and pieces of structured and semi-structure data (like Lab tests and my medicines), unstructured data (my clinical notes and reports etc) and image data (X-Rays, CTs and MRIs), to start to make the best sense of it, and able to coordinator my cares and appointments in the coming soon. &nbsp;</p>
<p>​</p>
<h2><br><strong>Disclaimer</strong>&nbsp;<strong> </strong><br>&nbsp;</h2>
<p><strong>Sorry to state the obvious</strong>, but please note that the purpose of this post is not about&nbsp;how&nbsp;can we do XML Lookup tables quickly, manually or not, and actually I am pretty good at Excels and Notepad++ too. Instead,<strong> it's meant to get into touch with minds who'd like to share use cases, reflections, challenges, risks and experiences on the journey to LLM etc assisted automations</strong>.&nbsp;<br><br>And LLMs' power came from developers, from every public repository and posting everybody made in public forums. GPTs are not born out of air. This is not being fully appreciated and acknowledged yet in recent months.<br><br>There are risks to humanity by AGI at this speed, but personally I felt slightly lucky, relieved and excused since&nbsp;&nbsp;we are&nbsp;in healthcare services.&nbsp;<br><br>Other routine risks include&nbsp;data privacy compliances per HIPPA, GDPR and DPAs as usual.&nbsp;&nbsp;</p>
<p>&nbsp;</p>