2024-04-02: Prompt optimizations
Tags: devjournal
Published: 2024-04-02 17:27:11

Hey guys. So yesterday was a holiday, and I requested the day off so I'm writing this blog post for today.

Anyway, last week was pretty productive. My job as of late has been to optimize the prompts along with the two psychologists so that we can present it in acceptable form to investors.

Monday was a continuation of our Friday meeting. We met with the CEO to bring forth our ideas, and he thought it was all good. We explained our plans for the architecture and what we want to do with the fine-tuning, RAG, etc. In that regard, it was pretty fruitful.

The next day we continued with our planning, and I met with the CEO in one-on-one to showcase my thoughts directly, and also discuss compensation. He did say that he may be able to reach what I'm asking for, but time will tell how successful the funding will be. I also discussed with him about organizational optimizations because I felt that there was quite literally no structure to how we're working and everything feels somewhat nebulous, especially with how cagey the other developers are. They're extremely reluctant to share anything for whatever reason. Maybe it's a cultural/lingual barrier, because they are from Germany, but either way, it's not pleasant at all. I tend to thrive in environments where communication flows freely.

On Wednesday, I had the technical interview with Zillow. I had been somewhat preparing by doing Leetcode problems, since I'd read online that they mostly just ask mediums. I will admit that Leetcode is not a strong suit of mine, and I definitely do not believe it's an accurate representation of coding skill or problem solving ability. I certainly cannot work that fast. And plus, for most people, it's just a matter of memorizing the patterns. Either way, when I started the interview, I was met with a very easy string parsing problem, which was definitely not a Leetcode medium.

I got stuck with edge cases, so I didn't get to pass all the test cases, but I did well enough, in my opinion. After that was a system architecture question as to how I'd implement the previous problem. I also had my architecture for that pretty fleshed out, but I somehow forgot that queuing services exist (mostly because I've never had a reason to use them in my work), so I might have gotten dinged for that.

Regardless, the next day, I was met with a rejection. Which is fine, I am always open to doing interviews even just as practice and to learn more.

Thursday I went ahead and implemented Geekbot workflows myself because I didn't want to keep waiting on the lead dev to finally get it done. I also talked to the psychologists about what they'd think about me becoming the Product Owner in terms of Agile, and they were definitely receptive to the idea. I'd already somewhat squeezed myself into that role since I was the one constantly talking to the non-technical folks as a developer myself, so it wasn't much of a leap. I really like organizing tasks and keeping a smooth operation.

I then conferred to the CEO again, and he said it sounds like a good idea but we should wait until we can get a good version of the prompt up and running before we start reorganizing the roles and whatnot. I figured that that was reasonable, so that's what I continued to work on.

The backend developer brought to my attention that the chat history was being shared among every single client, which was bad for obvious reasons, and the chat history was also carrying over between chats. I immediately got to work on that.

I thought of keeping a running dictionary with user IDs as keys with the message history as values, and that worked out pretty fine. However, the bigger issue was getting it to clear out the message history when a new chat started. This involved me going into frontend to change the buttons' functionality and pass along the user ID, and then creating a new Django backend endpoint to handle the button clicks and delete it.

It was pretty rough to do considering all the typing that had to take place, but eventually I got it all working fine and it came out good. I then pushed it out and got to work on pushing out some prompt changes. The previous night I found a cool Reddit post about how to fix up the language that ChatGPT uses, so I used some of the suggestions there (particularly about using Flesch reading scores to make it simple but not simplistic), and it came out pretty decent. The major issue left was the annoying "mirroring" that ChatGPT tends to do when it acknowledges someone's input, e.g. "I like turtles" to which the bot replies, "**It's great that you like turtles!** What do you like about them?" The bold part is what we want to avoid.

We've been working on culling that behavior but it's been a hassle since it's so baked in to the system. No matter what we say in the prompt, it ignores it. We may need to resort to fine-tuning, but it's hard to come up with data at this stage. We have to write it manually which sucks.

On Friday, I was determined to keep working on the prompt with one of the psychologists (the other called in sick) but I was hit with a sinus headache myself which forced me to also call in sick.

Anyway, that's a recap of this past week. Until next time, cheers.