Generating meaningful test data using Gemini
Tags: ai, tutorial, beginners, programming
Published: 2024-06-22 15:36:49

<p>We all know that having a set of proper test data before deploying an application to production is crucial for ensuring its reliability and performance. It allows to simulate real-world scenarios and identify potential issues or bugs before they impact end-users.&nbsp;Moreover, testing with representative data sets allows to optimize performance, identify bottlenecks, and fine-tune algorithms or processes as needed. Ultimately, having a comprehensive set of test data helps to deliver a higher quality product, reducing the likelihood of post-production issues and enhancing the overall user experience.&nbsp;</p>
<p>In this article, let's look at how one can use generative AI, namely <a href="https://gemini.google.com/" target="_blank">Gemini by Google</a>, to generate (hopefully) meaningful data for the properties of multiple objects. To do this, I will use the RESTful service to generate data in a JSON format and then use the received data to create objects.</p>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/maat0y04y4mgziig1xtu.jpeg)
<p><!--break--></p>
<p>This leads to an obvious question: why not use the methods from <code>%Library.PopulateUtils</code> to generate all the data? Well, the answer is quite obvious as well if you've seen the list of methods of the class - there aren't many methods that generate meaningful data.</p>
<p>So, let's get to it.</p>
<p>Since&nbsp;I'll be using the Gemini API, I will need to generate the API key&nbsp;first&nbsp;since I don't have it beforehand. To do this, just open&nbsp;<a href="https://aistudio.google.com/app/apikey" target="_blank">aistudio.google.com/app/apikey</a>&nbsp;and click on Create API key.</p>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8etpiqr080bdfuc6f4j2.png)
<p>and create an API key in a new project</p>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/414jpqdgccszflwevdt3.png)
<p>After this is done, you just need to write a REST client to get and transform data and come up with a query string to a Gemini AI. Easy peasy&nbsp;&#x1f601;</p>
<p>For the ease of this example, let's work with the following simple class</p>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t8p4znztrlnyxi6c1ory.png)
<pre class="codeblock-container" idlang="0" lang="ObjectScript" tabsize="4"><code class="language-cls hljs cos"><span class="hljs-keyword">Class</span> Restaurant.Dish <span class="hljs-keyword">Extends</span> (<span class="hljs-built_in">%Persistent</span>, <span class="hljs-built_in">%JSON.Adaptor</span>)
{
<span class="hljs-keyword">Property</span> Name <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span><span class="hljs-comment">;</span>
<span class="hljs-keyword">Property</span> Description <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span>(MAXLEN = <span class="hljs-number">1000</span>)<span class="hljs-comment">;</span>
<span class="hljs-keyword">Property</span> Category <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span><span class="hljs-comment">;</span>
<span class="hljs-keyword">Property</span> Price <span class="hljs-keyword">As</span> <span class="hljs-built_in">%Float</span><span class="hljs-comment">;</span>
<span class="hljs-keyword">Property</span> Currency <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span><span class="hljs-comment">;</span>
<span class="hljs-keyword">Property</span> Calories <span class="hljs-keyword">As</span> <span class="hljs-built_in">%Integer</span><span class="hljs-comment">;</span>
}</code></pre>
<p>In general, it would be really simple to use the built-in <code>%Populate</code> mechanism and be done with it. But in bigger projects you will get a lot of properties which are not so easily automatically populated with meaningful data.</p>
<p>Anyway, now&nbsp;that we have the class, let's think about the wording of a query to Gemini. Let's say we write the following query:</p>
<pre class="codeblock-container" idlang="3" lang="JSON" tabsize="4"><code class="language-json hljs">{<span class="hljs-attr">"contents"</span>: [{
    <span class="hljs-attr">"parts"</span>:[{
      <span class="hljs-attr">"text"</span>: <span class="hljs-string">"Write a json object that contains a field Dish which is an array of 10 elements. Each element contains Name, Description, Category, Price, Currency, Calories of the Restaurant Dish."</span>}]}]}</code></pre>
<p>If we send this request to&nbsp;<a href="https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?key=APIKEY" target="_blank">https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?key=APIKEY</a> we will get something like:</p>
<div class="spoiler">
<div class="spoiler-title">
<div class="spoiler-toggle show-icon">&nbsp;</div>
Spoiler</div>
<div class="spoiler-content" style="display: none;">
<pre class="codeblock-container" idlang="3" lang="JSON" tabsize="4"><code class="language-json hljs">{
  <span class="hljs-attr">"Dish"</span>: [
    {
      <span class="hljs-attr">"Name"</span>: <span class="hljs-string">"Dish 1"</span>,
      <span class="hljs-attr">"Description"</span>: <span class="hljs-string">"A delicious dish with a unique flavor."</span>,
      <span class="hljs-attr">"Category"</span>: <span class="hljs-string">"Main Course"</span>,
      <span class="hljs-attr">"Price"</span>: <span class="hljs-number">15</span>,
      <span class="hljs-attr">"Currency"</span>: <span class="hljs-string">"$"</span>,
      <span class="hljs-attr">"Calories"</span>: <span class="hljs-number">500</span>
    },
    {
      <span class="hljs-attr">"Name"</span>: <span class="hljs-string">"Dish 2"</span>,
      <span class="hljs-attr">"Description"</span>: <span class="hljs-string">"A flavorful dish with a spicy kick."</span>,
      <span class="hljs-attr">"Category"</span>: <span class="hljs-string">"Appetizer"</span>,
      <span class="hljs-attr">"Price"</span>: <span class="hljs-number">10</span>,
      <span class="hljs-attr">"Currency"</span>: <span class="hljs-string">"$"</span>,
      <span class="hljs-attr">"Calories"</span>: <span class="hljs-number">300</span>
    },
    {
      <span class="hljs-attr">"Name"</span>: <span class="hljs-string">"Dish 3"</span>,
      <span class="hljs-attr">"Description"</span>: <span class="hljs-string">"A hearty dish with a comforting flavor."</span>,
      <span class="hljs-attr">"Category"</span>: <span class="hljs-string">"Main Course"</span>,
      <span class="hljs-attr">"Price"</span>: <span class="hljs-number">20</span>,
      <span class="hljs-attr">"Currency"</span>: <span class="hljs-string">"$"</span>,
      <span class="hljs-attr">"Calories"</span>: <span class="hljs-number">600</span>
    },
    {
      <span class="hljs-attr">"Name"</span>: <span class="hljs-string">"Dish 4"</span>,
      <span class="hljs-attr">"Description"</span>: <span class="hljs-string">"A refreshing dish with a zesty flavor."</span>,
      <span class="hljs-attr">"Category"</span>: <span class="hljs-string">"Salad"</span>,
      <span class="hljs-attr">"Price"</span>: <span class="hljs-number">12</span>,
      <span class="hljs-attr">"Currency"</span>: <span class="hljs-string">"$"</span>,
      <span class="hljs-attr">"Calories"</span>: <span class="hljs-number">250</span>
    },
    {
      <span class="hljs-attr">"Name"</span>: <span class="hljs-string">"Dish 5"</span>,
      <span class="hljs-attr">"Description"</span>: <span class="hljs-string">"A sweet dish with a decadent flavor."</span>,
      <span class="hljs-attr">"Category"</span>: <span class="hljs-string">"Dessert"</span>,
      <span class="hljs-attr">"Price"</span>: <span class="hljs-number">8</span>,
      <span class="hljs-attr">"Currency"</span>: <span class="hljs-string">"$"</span>,
      <span class="hljs-attr">"Calories"</span>: <span class="hljs-number">400</span>
    },
    {
      <span class="hljs-attr">"Name"</span>: <span class="hljs-string">"Dish 6"</span>,
      <span class="hljs-attr">"Description"</span>: <span class="hljs-string">"A savory dish with a smoky flavor."</span>,
      <span class="hljs-attr">"Category"</span>: <span class="hljs-string">"Main Course"</span>,
      <span class="hljs-attr">"Price"</span>: <span class="hljs-number">18</span>,
      <span class="hljs-attr">"Currency"</span>: <span class="hljs-string">"$"</span>,
      <span class="hljs-attr">"Calories"</span>: <span class="hljs-number">550</span>
    },
    {
      <span class="hljs-attr">"Name"</span>: <span class="hljs-string">"Dish 7"</span>,
      <span class="hljs-attr">"Description"</span>: <span class="hljs-string">"A light dish with a fresh flavor."</span>,
      <span class="hljs-attr">"Category"</span>: <span class="hljs-string">"Appetizer"</span>,
      <span class="hljs-attr">"Price"</span>: <span class="hljs-number">9</span>,
      <span class="hljs-attr">"Currency"</span>: <span class="hljs-string">"$"</span>,
      <span class="hljs-attr">"Calories"</span>: <span class="hljs-number">200</span>
    },
    {
      <span class="hljs-attr">"Name"</span>: <span class="hljs-string">"Dish 8"</span>,
      <span class="hljs-attr">"Description"</span>: <span class="hljs-string">"A hearty dish with a comforting flavor."</span>,
      <span class="hljs-attr">"Category"</span>: <span class="hljs-string">"Soup"</span>,
      <span class="hljs-attr">"Price"</span>: <span class="hljs-number">11</span>,
      <span class="hljs-attr">"Currency"</span>: <span class="hljs-string">"$"</span>,
      <span class="hljs-attr">"Calories"</span>: <span class="hljs-number">350</span>
    },
    {
      <span class="hljs-attr">"Name"</span>: <span class="hljs-string">"Dish 9"</span>,
      <span class="hljs-attr">"Description"</span>: <span class="hljs-string">"A refreshing dish with a zesty flavor."</span>,
      <span class="hljs-attr">"Category"</span>: <span class="hljs-string">"Salad"</span>,
      <span class="hljs-attr">"Price"</span>: <span class="hljs-number">14</span>,
      <span class="hljs-attr">"Currency"</span>: <span class="hljs-string">"$"</span>,
      <span class="hljs-attr">"Calories"</span>: <span class="hljs-number">300</span>
    },
    {
      <span class="hljs-attr">"Name"</span>: <span class="hljs-string">"Dish 10"</span>,
      <span class="hljs-attr">"Description"</span>: <span class="hljs-string">"A sweet dish with a decadent flavor."</span>,
      <span class="hljs-attr">"Category"</span>: <span class="hljs-string">"Dessert"</span>,
      <span class="hljs-attr">"Price"</span>: <span class="hljs-number">10</span>,
      <span class="hljs-attr">"Currency"</span>: <span class="hljs-string">"$"</span>,
      <span class="hljs-attr">"Calories"</span>: <span class="hljs-number">450</span>
    }
  ]
}</code></pre>
</div>
</div>
<p>Already not bad. Not bad at all! Now that I have the wording of my query, I need to generate it as automatically as possible, call it and&nbsp;process the result.</p>
<p>Next step - generating the query. Using the very useful article on <a href="https://community.intersystems.com/post/how-get-property-definitions-written-class-programmatically" target="_blank">how to get the list of properties of a class</a> we can generate automatically most of the query.</p>
<pre class="codeblock-container" idlang="0" lang="ObjectScript" tabsize="4"><code class="language-cls hljs cos"><span class="hljs-keyword">ClassMethod</span> GenerateClassDesc(classname <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span>) <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span>
{
    <span class="hljs-keyword">set</span> cls=<span class="hljs-keyword">##class</span>(<span class="hljs-built_in">%Dictionary.CompiledClass</span>).<span class="hljs-built_in">%OpenId</span>(classname,,.status)
    <span class="hljs-keyword">set</span> <span class="hljs-keyword">x</span>=cls.Properties
    <span class="hljs-keyword">set</span> profprop = <span class="hljs-built_in">$lb</span>()
    <span class="hljs-keyword">for</span> i=<span class="hljs-number">3</span>:<span class="hljs-number">1</span>:<span class="hljs-keyword">x</span>.Count() {
        <span class="hljs-keyword">set</span> prop=<span class="hljs-keyword">x</span>.GetAt(i)
        <span class="hljs-keyword">set</span> <span class="hljs-built_in">$list</span>(profprop, i-<span class="hljs-number">2</span>) = prop.Name        
    }
    <span class="hljs-keyword">quit</span> <span class="hljs-built_in">$listtostring</span>(profprop, <span class="hljs-string">", "</span>)
}

<span class="hljs-keyword">ClassMethod</span> GenerateQuery(qty <span class="hljs-keyword">As</span> <span class="hljs-built_in">%Numeric</span>) <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span> [ Language = objectscript ]
{
    <span class="hljs-keyword">set</span> classname = ..<span class="hljs-built_in">%ClassName</span>(<span class="hljs-number">1</span>)
    <span class="hljs-keyword">set</span> str = <span class="hljs-string">"Write a json object that contains a field "</span>_<span class="hljs-built_in">$piece</span>(classname, <span class="hljs-string">"."</span>, <span class="hljs-number">2</span>)_
        <span class="hljs-string">" which is an array of "</span>_qty_<span class="hljs-string">" elements. Each element contains "</span>_
        <span class="hljs-built_in">..GenerateClassDesc</span>(classname)_<span class="hljs-string">" of a "</span>_<span class="hljs-built_in">$translate</span>(classname, <span class="hljs-string">"."</span>, <span class="hljs-string">" "</span>)_<span class="hljs-string">". "</span>
    <span class="hljs-keyword">quit</span> str
}</code></pre>
<p>When dealing with complex relationships between classes it may be easier to use the object constructor to link different objects together or to use a built-in mechanism of <code>%Library.Ppulate</code>.</p>
<p>Following step is to call the Gemini RESTful service and process the resulting JSON.</p>
<pre class="codeblock-container" idlang="0" lang="ObjectScript" tabsize="4"><code class="language-cls hljs cos"><span class="hljs-keyword">ClassMethod</span> CallService() <span class="hljs-keyword">As</span> <span class="hljs-built_in">%String</span>
{
 <span class="hljs-keyword">Set</span> request = <span class="hljs-built_in">..GetLink</span>()
 <span class="hljs-keyword">set</span> query = <span class="hljs-string">"{""contents"": [{""parts"":[{""text"": """</span>_<span class="hljs-built_in">..GenerateQuery</span>(<span class="hljs-number">20</span>)_<span class="hljs-string">"""}]}]}"</span>
 <span class="hljs-keyword">do</span> request.EntityBody.<span class="hljs-keyword">Write</span>(query)
 <span class="hljs-keyword">set</span> request.ContentType = <span class="hljs-string">"application/json"</span>
 <span class="hljs-keyword">set</span> sc = request.Post(<span class="hljs-string">"v1beta/models/gemini-pro:generateContent?key=&lt;YOUR KEY HERE&gt;"</span>)
 <span class="hljs-keyword">if</span> <span class="hljs-built_in">$$$ISOK</span>(sc) {
    <span class="hljs-keyword">Set</span> response = request.HttpResponse.Data.<span class="hljs-keyword">Read</span>()	 
    <span class="hljs-keyword">set</span> p = <span class="hljs-keyword">##class</span>(<span class="hljs-built_in">%DynamicObject</span>).<span class="hljs-built_in">%FromJSON</span>(response)
    <span class="hljs-keyword">set</span> iter = p.candidates.<span class="hljs-built_in">%GetIterator</span>()
    <span class="hljs-keyword">do</span> iter.<span class="hljs-built_in">%GetNext</span>(.key, .value, .type ) 
    <span class="hljs-keyword">set</span> iter = value.content.parts.<span class="hljs-built_in">%GetIterator</span>()
    <span class="hljs-keyword">do</span> iter.<span class="hljs-built_in">%GetNext</span>(.key, .value, .type )
    <span class="hljs-keyword">set</span> obj = <span class="hljs-keyword">##class</span>(<span class="hljs-built_in">%DynamicObject</span>).<span class="hljs-built_in">%FromJSON</span>(<span class="hljs-built_in">$Extract</span>(value.text,<span class="hljs-number">8</span>,*-<span class="hljs-number">3</span>))
    
    <span class="hljs-keyword">set</span> dishes = obj.Dish
    <span class="hljs-keyword">set</span> iter = dishes.<span class="hljs-built_in">%GetIterator</span>()
    <span class="hljs-keyword">while</span> iter.<span class="hljs-built_in">%GetNext</span>(.key, .value, .type ) {
        <span class="hljs-keyword">set</span> dish = <span class="hljs-keyword">##class</span>(Restaurant.Dish).<span class="hljs-built_in">%New</span>()
        <span class="hljs-keyword">set</span> sc = dish.<span class="hljs-built_in">%JSONImport</span>(value.<span class="hljs-built_in">%ToJSON</span>())
        <span class="hljs-keyword">set</span> sc = dish.<span class="hljs-built_in">%Save</span>()
    }    
 }
}</code></pre>
<p>Of course, since it's just an example, don't forget to add status checks where necessary.</p>
<p>Now, when I run it, I get a pretty impressive result in my database. Let's run a&nbsp;SQL query to see the&nbsp;data.</p>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1anwpzygkejj0dnk9itv.png)
<p>The description and&nbsp;category&nbsp;correspond&nbsp;to the name of the dish. Moreover, prices and calories look correct as well. Which means that I actually get a database, filled with reasonably real looking data. And the results of the queries that I'm going to run are going to resemble the real results.</p>
<p>Of course, a huge drawback of this approach is the necessity of writing a query to a generative AI and the fact that it takes time to generate the result. But the actual data may be worth it. Anyway, it is for you to decide&nbsp;&#x1f609;</p>
<div class="spoiler">
<div class="spoiler-title">
<div class="spoiler-toggle show-icon">&nbsp;</div>
P.S.</div>
<div class="spoiler-content" style="display: none;">At this point Gemini API is available in a <a href="https://ai.google.dev/available_regions" target="_blank">limited number of countries and territories</a> listed below:
<ul>
<li>Algeria</li>
<li>American Samoa</li>
<li>Angola</li>
<li>Anguilla</li>
<li>Antarctica</li>
<li>Antigua and Barbuda</li>
<li>Argentina</li>
<li>Armenia</li>
<li>Aruba</li>
<li>Australia</li>
<li>Azerbaijan</li>
<li>The Bahamas</li>
<li>Bahrain</li>
<li>Bangladesh</li>
<li>Barbados</li>
<li>Belize</li>
<li>Benin</li>
<li>Bermuda</li>
<li>Bhutan</li>
<li>Bolivia</li>
<li>Botswana</li>
<li>Brazil</li>
<li>British Indian Ocean Territory</li>
<li>British Virgin Islands</li>
<li>Brunei</li>
<li>Burkina Faso</li>
<li>Burundi</li>
<li>Cabo Verde</li>
<li>Cambodia</li>
<li>Cameroon</li>
<li>Caribbean Netherlands</li>
<li>Cayman Islands</li>
<li>Central African Republic</li>
<li>Chad</li>
<li>Chile</li>
<li>Christmas Island</li>
<li>Cocos (Keeling) Islands</li>
<li>Colombia</li>
<li>Comoros</li>
<li>Cook Islands</li>
<li>Côte d'Ivoire</li>
<li>Costa Rica</li>
<li>Curaçao</li>
<li>Democratic Republic of the Congo</li>
<li>Djibouti</li>
<li>Dominica</li>
<li>Dominican Republic</li>
<li>Ecuador</li>
<li>Egypt</li>
<li>El Salvador</li>
<li>Equatorial Guinea</li>
<li>Eritrea</li>
<li>Eswatini</li>
<li>Ethiopia</li>
<li>Falkland Islands (Islas Malvinas)</li>
<li>Fiji</li>
<li>Gabon</li>
<li>The Gambia</li>
<li>Georgia</li>
<li>Ghana</li>
<li>Gibraltar</li>
<li>Grenada</li>
<li>Guam</li>
<li>Guatemala</li>
<li>Guernsey</li>
<li>Guinea</li>
<li>Guinea-Bissau</li>
<li>Guyana</li>
<li>Haiti</li>
<li>Heard Island and McDonald Islands</li>
<li>Honduras</li>
<li>India</li>
<li>Indonesia</li>
<li>Iraq</li>
<li>Isle of Man</li>
<li>Israel</li>
<li>Jamaica</li>
<li>Japan</li>
<li>Jersey</li>
<li>Jordan</li>
<li>Kazakhstan</li>
<li>Kenya</li>
<li>Kiribati</li>
<li>Kyrgyzstan</li>
<li>Kuwait</li>
<li>Laos</li>
<li>Lebanon</li>
<li>Lesotho</li>
<li>Liberia</li>
<li>Libya</li>
<li>Madagascar</li>
<li>Malawi</li>
<li>Malaysia</li>
<li>Maldives</li>
<li>Mali</li>
<li>Marshall Islands</li>
<li>Mauritania</li>
<li>Mauritius</li>
<li>Mexico</li>
<li>Micronesia</li>
<li>Mongolia</li>
<li>Montserrat</li>
<li>Morocco</li>
<li>Mozambique</li>
<li>Namibia</li>
<li>Nauru</li>
<li>Nepal</li>
<li>New Caledonia</li>
<li>New Zealand</li>
<li>Nicaragua</li>
<li>Niger</li>
<li>Nigeria</li>
<li>Niue</li>
<li>Norfolk Island</li>
<li>Northern Mariana Islands</li>
<li>Oman</li>
<li>Pakistan</li>
<li>Palau</li>
<li>Palestine</li>
<li>Panama</li>
<li>Papua New Guinea</li>
<li>Paraguay</li>
<li>Peru</li>
<li>Philippines</li>
<li>Pitcairn Islands</li>
<li>Puerto Rico</li>
<li>Qatar</li>
<li>Republic of the Congo</li>
<li>Rwanda</li>
<li>Saint Barthélemy</li>
<li>Saint Kitts and Nevis</li>
<li>Saint Lucia</li>
<li>Saint Pierre and Miquelon</li>
<li>Saint Vincent and the Grenadines</li>
<li>Saint Helena, Ascension and Tristan da Cunha</li>
<li>Samoa</li>
<li>São Tomé and Príncipe</li>
<li>Saudi Arabia</li>
<li>Senegal</li>
<li>Seychelles</li>
<li>Sierra Leone</li>
<li>Singapore</li>
<li>Solomon Islands</li>
<li>Somalia</li>
<li>South Africa</li>
<li>South Georgia and the South Sandwich Islands</li>
<li>South Korea</li>
<li>South Sudan</li>
<li>Sri Lanka</li>
<li>Sudan</li>
<li>Suriname</li>
<li>Taiwan</li>
<li>Tajikistan</li>
<li>Tanzania</li>
<li>Thailand</li>
<li>Timor-Leste</li>
<li>Togo</li>
<li>Tokelau</li>
<li>Tonga</li>
<li>Trinidad and Tobago</li>
<li>Tunisia</li>
<li>Türkiye</li>
<li>Turkmenistan</li>
<li>Turks and Caicos Islands</li>
<li>Tuvalu</li>
<li>Uganda</li>
<li>United Arab Emirates</li>
<li>United States</li>
<li>United States Minor Outlying Islands</li>
<li>U.S. Virgin Islands</li>
<li>Uruguay</li>
<li>Uzbekistan</li>
<li>Vanuatu</li>
<li>Venezuela</li>
<li>Vietnam</li>
<li>Wallis and Futuna</li>
<li>Western Sahara</li>
<li>Yemen</li>
<li>Zambia</li>
<li>Zimbabwe</li>
</ul>
<p>If you're not in one of these countries or territories, you will get an error&nbsp;<code>{"error": {"code": 400, "message": "User location is not supported for the API use.", "status": "FAILED_PRECONDITION"}</code>. In this case, try&nbsp;<a href="https://cloud.google.com/vertex-ai#build-with-gemini">Gemini Pro in Vertex AI</a>.</p>
<p>&nbsp;</p>
</div>
</div>
<p>P.P.S. The first image is how Gemini imagines the "AI that writes a program to create test data"&nbsp;&#x1f606;</p>