Prompt:
Job Description:
has-scraping-experience-a

Candidate A Resume:
{"data": {"awards": ["1st (Winner) AIR 8", "Among top 100", "CSA Hackathon", "National Engineering Olympiad (NEO)", "UP Science Talent Search Examination Merit"], "certifications": [], "education": [{"GPA": "8.8", "degree": "B.Tech", "endYear": "2024", "major": "Computer Science Engineering", "school": "UPES", "startYear": "2020"}, {"GPA": "", "degree": "Grad", "endYear": "2019", "major": "", "school": "St. Francis School", "startYear": ""}], "projects": [{"endYear": "", "projectDescription": "Pioneering Text-to-Image Synthesis: Merged advanced AI with artistic ingenuity for groundbreaking Text-to-Image Generation. Innovative Pipeline: Crafted a robust PyTorch foundation and executed Stable Diffusion Symphony using OpenVINO. Versatile Model Architecture: Showcased creative brilliance in Unet Denoising and Text Encoder, defining new possibilities in AI artistry.", "projectName": "MiragePalette", "startYear": ""}, {"endYear": "", "projectDescription": "Advanced NLP Techniques: Implemented bidirectional LSTM with attention mechanisms and GloVe embedding. Innovative Experimentation: Differentiated the project by exploring diverse LSTM architectures and fine-tuning alternatives to BERT, including RoBERTa to successfully categorized tweet sentiments.", "projectName": "Sentiment Analysis on US Airlines Tweets", "startYear": ""}, {"endYear": "", "projectDescription": "Analytical Mastery: Delved into the Telco dataset, decoding intricate patterns to predict customer churn and uncover pivotal factors. Statistical Proficiency: Applied advanced probability concepts, conducted meticulous hypothesis tests, and executed precise logistic regression analyses, ensuring a comprehensive exploration of data nuances.", "projectName": "Customer Churn Prediction", "startYear": ""}], "publication": [], "skills": ["Machine Learning", "Data Analytics", "Natural Language Processing", "Web Scraping", "Microsoft Excel", "Python", "C++", "LATEX", "MySQL", "AWS", "Streamlit", "Flask"], "workExperience": [{"company": "Accenture", "description": "In-depth Training: Acquired comprehensive expertise in the Workday platform, specializing in Adaptive Planning, Forecasting, Data Conversion, and Workday Integration. Hands-on Experience: Configured the system, modeled real-life business processes, and prepared Workday EIB files, honing practical skills. Valuable Insights: Developed insights into effective enterprise management and employee on-boarding processes.", "endYear": "2023", "location": {"city": "Gurugram", "country": "India"}, "role": "Summer Intern (ASE)", "startYear": "2023"}, {"company": "IIT Roorkee", "description": "Project Contribution: Collaborated on the development of a GUI for automated detection of gas pipeline changes via satellite imagery surveillance. Innovative Integration: Spearheaded the integration of Geographic Information Systems (GIS) tools, leveraging spatial data for automated gas pipeline change detection. Data Analysis Mastery: Analyzed and interpreted satellite imagery within GIS frameworks, contributing crucial insights to enhance surveillance accuracy.", "endYear": "2022", "location": {"city": "Roorkee", "country": "India"}, "role": "Summer Intern (Web Developer)", "startYear": "2022"}]}, "status": "success"}

Candidate B Resume:
{"data": {"awards": [], "certifications": ["Introduction to Artificial Intelligence (AI) - IBM Coursera - Nov 2023", "Python Data Structures - Coursera - Mar 2022"], "education": [{"GPA": "9.5", "degree": "Pre University College: Science (PCMB)", "endYear": "2020", "major": "Science (PCMB)", "school": "Excellent PU College", "startYear": "2018"}, {"GPA": "9.6", "degree": "Secondary Schooling", "endYear": "", "major": "", "school": "Excellent High School", "startYear": ""}, {"GPA": "8.37", "degree": "Bachelor's in Engineering", "endYear": "2024", "major": "Information Science and Engineering", "school": "R.N.S Institute of Technology", "startYear": "2020"}], "projects": [{"endYear": "2023", "projectDescription": "Built a dynamic local marketplace using React JS (front-end), Tailwind CSS (styling), Node.js (backend), and MongoDB (database).", "projectName": "Farmers Market Website", "startYear": "2023"}, {"endYear": "", "projectDescription": "Created a classic Snake Game for PC with Python's Tkinter. Players control a growing snake, avoiding collisions for an engaging gaming experience.", "projectName": "Snake Game Application", "startYear": "2023"}, {"endYear": "", "projectDescription": "Built an advanced AI Assistant with Bard and ChatGPT APIs for a dynamic, conversational user experience, enhancing productivity.", "projectName": "Personal AI Assistant", "startYear": "2023"}], "publication": [], "skills": ["Python", "Java", "C", "HTML", "CSS", "JavaScript", "SQL", "MongoDB", "Git", "Canva", "Figma", "Problem Solving", "React JS", "Flask", "Django", "Tailwind CSS", "Bootstrap"], "workExperience": [{"company": "CommScope", "description": "Actively participated as an intern, collaborating on real-time projects for the Oncell 5G Broadband product.", "endYear": "Present", "location": {"city": "Bengaluru", "country": "India"}, "role": "Intern", "startYear": "2023"}, {"company": "Across the Globe", "description": "As a Python Developer Intern, my responsibilities were writing and troubleshooting technical issues. My major responsibilities included Backend Development, Web Scraping and Automation.", "endYear": "2023", "location": {"city": "Bengaluru", "country": "India"}, "role": "Python Developer Intern", "startYear": "2022"}]}, "status": "success"}

Candidate A Transcript:
{'pairs': [['Interviewer: Hello and welcome to the AI interview! This interview will consist of basic questions about your background as well as some high-level questions about the skills you listed on your application. Ensure that you minimize long pauses during your responses, otherwise you may be cut off prematurely. Are you ready to start the interview?\n', 'Interviewee: Review? Yes.\n'], ["Interviewer: Great! I see from your resume that you've worked on a project involving sentiment analysis on US Airlines Tweets. Could you walk me through the NLP techniques you used for this project and how you implemented them?  \n", "Interviewee: Sure. Uh, so initially, the after the dataset extraction that we did from Kaggle, basically, uh, we had tweets from various US airlines. I think it was 6 US airlines. Uh, so we had to filter out the text of the each tweet because now we need to determine the, um, kind of content or the that sentiment, basically, of those, uh, tweets. So the content has to be filtered. Now the filtration steps or, basically, the NLP steps that we use initially was the first of all, was the stemming. We had to make the word back to its stem form or the root form, um, so that the so that, uh, there could be single let's say, there's a word called as, um, jump and jumping. So both of their the meaning is same, but there's an I n g added to the jumping part. So we don't want our model to get confused between the meanings of the context of both of the words. So we perform the stemming and convert the words to back to their root forms, uh, stem forms. Uh, then the next step was the lemmatization. Now lemmatization is basically mapping, uh, checking their values, uh, whether in the dictionary, the stemmed word exists or does not exist. So what happens is it also converts the word back to the root form or the stem form, but it also checks that the stem form or, uh, uh, the form that has been generated in the word that has actually some meaning in the dictionary. So that mapping part is extremely important there because it could happen is that, uh, let's say there's a word intelligent and intelligence. Now meaning of both them is same, but it could happen that, uh, while stemming, it removes the t from the intelligent at the end and c e from intelligence. So it becomes intelligent, which actually or technically is not a word in a dictionary. Although we humans can understand, but our models will become ambiguous and, uh, get confused. So what we do is that we perform a a step of limitation instead of showing there. What we do is that we check whether, uh, a word exists, let's say, intelligent. So that word does not exist, so we'll eliminate it. So what we will do is that intelligent will be kept as the word, uh, lemmatized word. So that is the second step. Then we went for the tokenization. Basically, tokenization was the first part as well, so it was evolving around mixing of them. Uh, the to tokenization is when we convert the whole sentence or the whole tweet into various multiple tokens. Let's say the sentence is my name is Shivanshu, so it will convert my name is Shivanshu. Or let's say the movie that I watched today was good, so it will just separate them all back into the, uh, individual tokens. Then we perform the stop word removal. Now this is very important because there are various words which are called as stop words. These stop words are those words which does not actually contribute any contextual or meaningful, um, impact to the model. Let's say there's and, there's or, there's but, these types of sentences these types of words or tokens do not contribute, um, in a very high level impact to the NLP model. So we try to remove them and just try to contain maximum information and minimum number of words or minimum amount of words. Then we performed, um, then we also performed the, um, mapping them with the, um, I forgot the name, um, it was around. So after the the the tokenization, uh, stemming lemmatization, uh, we performed yeah. We are the vectorization. Then the vectorization we also performed small techniques such as small, uh, converting them to lowercase capitals, um, removing few various links that was done through regular expression. Uh, because there could be hyperlinks and there could be, uh, Twitter handles, which we don't want. We could, like, basically convert the links into another column and, uh, write another column such as tweets, uh, text of the tweets, um, links, and then the handles. So that that was the then there was the vectorization techniques. I think I used the TFID vectorization technique of the inverse document frequency dome frequency, inverse document frequency. There, we try to see how many times a word has occurred in a particular tweet and a in the complete corpus and what does it contribute, and how much does that, uh, word's contextual meaning contribute towards the whole of the tweets. I think that was overall, um, um, the process that we used there.\n"], ["Interviewer: It sounds like you've implemented a comprehensive preprocessing pipeline for the text data. You mentioned stemming, lemmatization, tokenization, stopword removal, and vectorization using TF-IDF. Can you elaborate on how you evaluated the performance of your sentiment analysis model and what metrics you used to determine its accuracy?  \n", "Interviewee: Yeah. Sure. Um, the sentiment analysis metrics that we use to basically um, analyze, uh, the, um, the evaluation metrics that we use to analyze were, um, of course, it's a classification thing that we um, identify the tweet as a positive tweet or a negative tweet. So we or even a neutral, so 3. Um, so we use confusion metrics. That was one thing that we implemented, and we tried to find out the false positives, the false negatives, um, and all the true positives, true negatives so that we can better understand the question. Then we have also implemented the accuracy, the, um, loss, whatever loss rate was coming out. Then we implemented the f one score basically using the accuracy and precision. We implemented the precision recall f one score. Um, so all in all, we we wanted to explore because in classification, accuracy is not always what matters. We also have to see what kind of accuracy that we are getting. There could be a true positive, true negative accuracy that we are getting, so we have to understand that as well. So what we use is f one score that we also implemented in our model. This f one score is based on precision and recall and gives us a very high idea about, um, a balanced idea about how our model is performing. So we use f one score, error rate, um, precision recall, and accuracy, uh, and the confusion metrics as well. Yeah.\n"], ["Interviewer: That's a solid approach to evaluating a sentiment analysis model. Using the F1 score to balance precision and recall is particularly important in imbalanced datasets. Shifting gears a bit, I noticed you have experience with web scraping. Can you describe a project where you utilized web scraping, and which tools or libraries you used to extract the data?  \n", "Interviewee: Sure. Um, I don't think it's mentioned in my resume, but I actually worked on a project. Um, that project was basically an idea that came into our mind when we when I participated with my team in a cyber police hackathon that held that was held by the police. Uh, so the the the the the topic was about dark web investigation and cryptocurrency analysis. We just wanted to mix both of them as on dark web. Uh, the problem is that we cannot perform steps directly as it requires a Tor browser to be opened. So there's a lot of complexity involved there. So even web scraping there is a bit difficult because you cannot directly or you cannot automate the opening of the browser in the next, um, then automate the, uh, web scraping stuff. The the, uh, this is just one thing. The other thing was that on dark web, there are no, uh, information that we can actually gather from the website. Because of web websites, however, being very simple, they have absolutely no content or no related information to the user. The only thing sometime we can find is, let's say, there's a marketplace, illegal marketplace on dark web. Uh, it will have a cryptocurrency address, uh, a wallet address which we can use which they basically ask for us to pay them, and then they'll provide the services. So what we wanted to do was we wanted to extract or we wanted to find those IP addresses. Oh, sorry. We wanted to find those cryptocurrency addresses, uh, wallet addresses, and then fetch those addresses. And then we had few OSINT tools. OSINT tools are basically, um, open source intelligence, um, intelligence tools. So these tools are used in, uh, various offensive, um, analysis of, uh, let's say, a lot of malicious activities or to check trails of cryptocurrencies. So what we use was we basically got those crypto addresses, and then we put them or basically automated them. Uh, after getting them, we would forward that request to the to those who send tools, which are available online, of course. Um, and then we would gather the information about the, uh, crypto crypto wallet address, basically, uh, getting a trail of, uh, the how much the Bitcoins or the how much the payment has been following. And from one user to another, what kind of, um, stuff has been following. So there, I use the web scraping part. I use the Selenium library of Python, um, because I initially was used used to use Beautiful Soup, but, apparently, there are a few complex issues in Beautiful Soup, such as FindAll, which basically, uh, there's an ambiguity in string and, um, the function of, um, the beautiful soup itself. So I don't prefer that. I use Selenium. I we use that to basically fetch or scrape the information. When we passed, uh, we created an API which would pass the where you would pass the crypto address, then we'd get the then it would be passed to a website where all the information will be there about the crypto trail, about the crypto wallet address, about the money sent there, about how many bit packets are traced, and what and a various lot of informations. That that information was to be scraped. So I created a scraping script there, and that would fetch all of that information and get me back the results.\n"], ["Interviewer: That's a very innovative use of web scraping and automation for a critical purpose. Using Selenium for such a complex task is quite impressive. Given your experience with both web scraping and NLP, how would you approach a project that involves scraping text data from websites and then performing NLP tasks on that data, such as topic modeling or text summarization?  \n", "Interviewee: um, yeah, I think the web scraping of course is a very what should I say a very important factor or a very important tool in the whole idea of automation because come on what do we need to automate is basically to get the data either we sit down and manually get down the data which is not at all automation or we can sorry or we can get the Again, sorry and we can get the complete automation through web scraping. So let's take the example of the text summarization that you talked about. I actually worked on this project. I think I didn't I didn't get the chance to finish it. But yeah, I knew that so basically the approach was to that. We had a few blog articles of a particular guy. I don't exactly remember the link or the website of his blog. So what do you wanted was to basically get a summarized text of all his blogs so that he can like put it on a on his LinkedIn post or in on an Instagram post for a social media promotion. So he asked us to do that. So what we did was we basically use a website. I created a script for web scraping using selenium there. I used multiple web stripping tools to extract his blog articles and then all the text has to be passed to the NLP model. First of all, of course, you'll have to apply them to the pre-processing chain, of course involving the various steps of stemming limitations the vectorization even small we have to remove various stop for removal, of course. And then a small converting them back to a small Capital small letters all of these. Precepts will be first performed and then the extracted text or the extracted passage. The automatic web scraped passage will be then passed to our model and then our model will basically learn about these and then at the end give us a textual summary about that complete the complete passage. Now, there are I think two types of the summary text summarization techniques one is the exhaustive summary and one is the implemented or something. I don't remember the name, but the one is where it basically checks the it just basically extract is the most important sentences. The other is where it completely understands the text and then rewrite those as a complete summary. So I think that will be the first approach that I'll be scrap the data from the website and using selenium various tools and get the text important texts and all the text that is required for my and more than pass them through the natural language processing steps such as token tokenization stemming limitation stopped removal and then convert them to vectorization because of course It's a very important step. We have to see whether TF idea works there or they could also be counted by transition that could work there after that. We apply them and pass them through a model we could also use Board model there. Basically, I mean come on. It's a sequence to sequence model. So it will help us in it's a Transformer which is a bad directional and coder. I think it could work very well. I think that's that would be my Approach.\n"], ["Interviewer: Your approach to text summarization using web scraping and NLP is quite thorough. It's clear you understand the importance of preprocessing and the different types of summarization techniques. The use of a transformer model like BERT for more advanced summarization is also a forward-thinking strategy. Given your experience with AWS, can you discuss a time when you had to use AWS services in a project or work setting, and how you leveraged them to achieve your goals?  \n", "Interviewee: Okay. Uh, so this was very recently. 1 of so, basically, I used to, like, do small steps of, um, college professors or, uh, of friends. Basically, they would ask me when they were, like, working on some website setups and all. So they had to, like, work on a project where, um, they had done the part about the project where they, like, had to find out the species of the animal. So they wanted me to perform that for them. So what I did was that, uh, I used the AWS's recognition, um, what is it called, service. Yeah. AWS's recognition service to basically identify the image, uh, and recognize the species of the image. Uh, we I had to modify a lot of things because it gives information in multiple layers. Like, it will tell that it is a mammal, and then any mammal also, let's say if it's a cow, then you'll say that it's a cow. Then even if it's a cow, let's say it's a spotted cow. So it could even go down to that that it's a black spotted cow or something. Even we are telling the multi, um, species of that cow as well. So we're not interested in that. We just wanted to know whether it's a cow or not. So I modified that layering and asked just to fetch the name of the animal. So it would basically fetch the name of the animal whenever we'll click on the submit button where the image is uploaded, and then the user has to wait for the answer from the AWS site. So the AWS recognition species, uh, recognition service was connected. Uh, I also had to implement the AWS's API end module because, uh, I had to, of course, create various endpoints for the a AWS API to handle. So, uh, I had to create Lambda functions to handle these AWS API endpoints. But the the initial setup was, uh, that the user will upload the animal image. He'll basically click on the submit button. He'll wait, and then this request, after clicking on the submit button, will be passed on to the, um, to the to the AWS Lambda function where the AWS first AWS endpoint will be hit, and that AWS endpoint will trigger the, uh, will create a trigger function or the trigger function. Yes. Um, that function will, um, find the specie fetch the image from the, uh, user, and then store it in the s 2 bucket s 3 bucket. Sorry. And that s 3 bucket, after storing that image in the s 3 bucket, it will gather the, um, uh, it will basically trigger another function, which was actually initiated in the s 3 bucket itself that whenever an image is stored in that, so what it will do is that it will basically trigger the, uh, AWS's recognition species and find out the species of the name, uh, final species of the animal, and then return back the results, uh, where the other, um, return back the results into a, uh, database that we were maintaining uh, with the serial number and the number of, um, the request from the user that has been submitted and the name of the animal with the image of the animal. So that was there. And we also had another DynamoDB table, which I I had to initially create. That DynamoDB table had multiple animals and their information required in them. So what happened was that, uh, after we got the animal name, that will trigger another API. And what it will do is that that function will go ahead and fetch the name of the animal and the information of the animal from the DynamoDB table. All of this will be then returned back to the user. We also implemented Amazon Polly. Uh, that was for, um, voice, uh, answer. He wanted to also get a voice result where the voice announces that the species is this and the, um, whatever the information we have faced from the DynamoDB, that is also announced. So that was something that I had implemented in Amazon Polly. So whenever we got the information as well back, then when we sent it back to the user to display it on his out out, um, his printing or the output of the display screen, we also triggered an API, uh, to Amazon Polly, which would fetch us the voice note icon as well where he could click and just get the voice result as well.\n"], ["Interviewer: That's a very practical application of AWS services, combining Rekognition for image analysis, Lambda for serverless computing, S3 for storage, DynamoDB for a NoSQL database, and Polly for text-to-speech capabilities. It's clear you have a strong understanding of how to architect solutions using AWS. Given your experience with these services, how do you ensure the security of data when using AWS, especially when handling sensitive information like images and user data?  \n", "Interviewee: Yeah. Uh, of course. See, uh, whenever we go ahead in AWS, the first thing, uh, the the declined or the the person who was ready to give me his credentials, which I, of course, declined because that is not how I work. I told him that the first and foremost step of security in AWS is to create another account and then give me the rights for those particular account. Basically, create a user group or assign me a role where I have the, um, necessary permissions for the services. So in AWS, there's a quote we say that, um, we should only give services we should only give permissions to the minimum number of, uh, services that is required to the, um, to to the user or the programmer. So, um, I was given these services. So I asked him to design, and I asked him to, uh, basically, um, allow me the permissions for those particular services such as the recognition, the Lambda function, the, uh, poly, and the DynamoDB. Um, so after those, I have that. Now I had to make sure that the data, um, that the data privacy and the data security concerns are, of course, being handled there. So what we used to do was, uh, I used to I, first of all, created an s three bucket with the, um, um, completely, um, no public access so that nobody could access only the API points where the user would trigger. User would upload and when the submit button is clicked, that it will go into that SDK bucket. Also, I had a simple folder, which was called the input folder that would have the images in them and the output folder, which would have the multiple information in them as well. So that was one of the things that I made sure that, um, it is make sure. I also assigned various, um, I I also initiated the, uh, two factor authorization, um, because multiple authorization is very important, uh, MFA, um, because, uh, I didn't wanted anybody to access that private data. So whenever I had to access that data, I also had to contact that guy because he was the owner. So he would have to tell me the, um, of course, the the second, uh, factor authorization, and then only I could log in in that. So that was one of the privacy security concern concern that I made sure that not even I could make misuse of that data. Um, in the sense of DynamoDB tables also, I made them extremely, um, under extremely restricted to the public access. They were only available for the, uh, they were only available, and the effects that they were allowed were the triggering of the, um, APIs and the fetching of the information. There was nothing that the user could, um, that me or another person, um, with even a a smaller access of with a smaller access could write into them the, um, in the the in an element table. He could not make any edits without the permission of the, of course, the owner. So I think I assigned various roles and groups security groups, and, um, assigned each of the service with the, um, only the permission only the necessary permission that was required.\n"], ['Interviewer: Thanks for taking the time to interview with us! The time allotted for our interview has come to an end. We will reach out to you with next steps shortly. Please click the end interview button to end the interview.', 'Interviewee:']]}

Candidate B Transcript:
{'pairs': [['Interviewer: Hello and welcome to the AI interview! This interview will consist of basic questions about your background as well as some high-level questions about the skills you listed on your application. Ensure that you minimize long pauses during your responses, otherwise you may be cut off prematurely. Are you ready to start the interview?\n', 'Interviewee: Yes. Yes. Great.\n'], ["Interviewer: Great! I see from your resume that you've completed a project involving web scraping. Could you tell me about the technologies you used for web scraping and any challenges you faced during that project?  \n", 'Interviewee: I have, uh, for the web scraping, I have used, uh, Selenium. And, uh, the challenges I have faced was, like, using proxy websites because many have searched have a cloud fair which, uh, detects the bots, so it was very difficult to avoid those things and I have to scrape the data from there. So most of the website level of site use those things Cloudflare and it would it would detect the bot moment. So I had to, like, slow down the space. I have to recently change the network IP address. So in the these, I had to overcome these things. And after this, all these things, so I was able to overcome with the help of Stack Overflow. So this were all, uh, before the Chargegbt. So there were not, uh, no direct answers. We have to search for the answers. So the best way was all only you could get all the answers in the stack workflow. So this is how I overcame all the problems which I faced during the web screen. Thank you.\n'], ["Interviewer: It sounds like you've had some hands-on experience with web scraping and have a good understanding of the challenges that come with it, such as anti-bot measures like Cloudflare. Using Selenium is a common approach for handling dynamic content, but have you also worked with any other libraries or tools specifically designed for web scraping, such as Beautiful Soup or Scrapy? And how did you ensure that your scraping activities were ethical and legal?  \n", "Interviewee: I have used the beautiful soup for scraping. I have not used scrappy. I have used beautiful soup. Beautiful soup is also good, but it is not for the dynamic website. It's only usable for the static website. So for for dynamic website, so I have to move for the selenium part selenium. So regarding the epic ethics, so there are some fine lines when you are collecting the data. So you should not cross those limits. So if it is in the limits, then it is fine. It's because your web web web browser and web service is free for everyone, so anyone can gain the data. So it is not restricted that you should not, uh, take the data from the websites. So, uh, you are free to do anything with the web data. You you, uh, you search or you find it. Anything you can do. But there is a limit. There is a silver lining between the ethical and nonethical parts. So you should not cross those things. So you have to keep in mind that, uh, while scraping the data, is this data publicly available? Is this data, uh, available for everyone? It is not for a particular only for me or only for particular people group of people. So you have to be careful regarding that. And, also, you have to look into the concerns like which company are you going to and what is this data. Is this the company's related data or it is the general profitably available data, or it's the leaked data? If it is a leaked data, then you should definitely not work on with it because it is already breached, and you will also be breaching the ethical contents in that. Thank you.\n"], ["Interviewer: You've given a thoughtful response on the ethical considerations of web scraping. It's clear you understand the importance of respecting the terms of service and the public availability of data. Shifting gears a bit, I noticed you've worked with Django. Can you describe a project where you used Django and how you implemented features like user authentication or data modeling in your application?  \n", "Interviewee: I have used Django mainly for 2 projects. So one is my personal projects. Uh, I have worked on a personal project like bookstore management, where I I have made a online bookstore where you can buy or purchase the books, all those things. So for the authentication in this one, in the particular, uh, this in the bookstore man bookstore management project, I use the inbuilt Java authentication function. So with the basic username and password stored in it, we'll be storing all the password with the hashing function, and it will be stored. And with the Django's inbuilt authentication function, I used to authenticate. And another project I worked on, it was in the internship project. So I can say it was like an, uh, finance based company, which project I'm was working on. So there I have worked with Django. The, uh, and there, the authentication part was done based on the OTP. We're generating the tokens. We are generating a of tokens, and then we use it was valid for the particular amount of time. If you enter the token validly in the web browser, then it will allow you to work further with the website. So regarding the model building, yeah, Django the for the model building, Django is one of the best and easiest methods for easily handling data between different apps. So, you know, as the Django works based on applications or app set of apps, you can, uh, differentiate into, uh, different apps. So if it's a big project, you can divide it into separate separate apps. And each each apps has its own model. So you have to think about, uh, building a model search that the model in particular app must not conflict with another application. So model building, uh, those things are very necessary during the the developing of model. So you have to look into that. And another thing, like, uh, Django has given us the free, uh, independence to take the primary key or unique key. We can anything. So you are the given the full authority to work with it. So you have to build a web, uh, data or model search that it is not conflicting with any other model as well as it is not conflicting with any other applications with it. Thank you.\n"], ["Interviewer: It's great to hear about your experience with Django, especially how you've implemented different authentication methods and handled data modeling within the framework. The use of OTP for authentication adds a layer of security, which is quite important in e-commerce and finance-related projects. Since you've also mentioned working with JavaScript and React, can you discuss how you've integrated a Django backend with a React frontend in any of your projects, and what challenges you might have faced during that integration?  \n", "Interviewee: React integration part. So I'm going to say that, uh, React, I'm basically, uh, Django is used for API API, uh, development. So and with the use of express, I used to, uh, connect to the API. So all these, uh, connect integration of the React front end and the Django back end were done through the APIs. So I used to integrate the particular API using the express routing methods of express. So I used to, uh, access those data, and I was used to work with those things. Thank you.\n"], ["Interviewer: It's clear you understand the role of APIs in connecting the frontend and backend. Using Express to handle the API requests is a common approach. Could you elaborate on how you structured your API endpoints in Django and how you ensured that the data flow between the frontend and backend was efficient and secure? Additionally, how did you handle CORS (Cross-Origin Resource Sharing) issues that might arise when making requests from a React frontend to a Django backend?  \n", "Interviewee: Regarding the the for express, I'll the the endpoints regarding the endpoints. So during the API creation in the Django itself, we are for each application, like, if it is a sales or invoice or it is a purchase purchase, uh, related applications, So then I would create a separate endpoint for each of the applications. So if it is a sales letter, then I would put all the sales related, uh, part in the sales endpoints. So in in the sales endpoints, uh, in the sales part, there are so many other sub endpoints. Like, in the sales there, we can find the invoices, slips, purchases, orders, those things. And under the purchases, there are many other things. So this is how I used to manage all the different endpoints together. And so how I, uh, how I did the authentication part with the Chorus. So in the Django, there is a part for Chorus where we can directly implement Chorus for authentication purpose. So understanding that it is the user, uh, authentic, uh, user is perfect, and it is being used with a different data while sharing. So and regarding the data sharing in the cross platform, it was a bit difficult for me in the first because, uh, React, I have not must work earlier before. I mostly worked on the back end part, so it was a bit challenging for me as I had got the help from my, uh, friend with most of the React, uh, any doubts I had with the React, I used to ask him. So he was he he's a friend in developer, and we used to, uh, he used to, uh, show me the path for how to work with the React. So regarding the power data, uh, uh, using of the data in the cross platform, so I used to request the API. The API would use to give us the answer response in the the JSON format. And in the JSON, there are all the datas which I require required. So using the JavaScript, I used to transfer all the required data only the required data which I have to show in the particular, uh, states or particular who who using the states and notes, I used to transfer the data. So dynamically loading websites, uh, has to dynamically load all the data based on the data which we receive. So I used to do this.\n"], ["Interviewer: You've provided a comprehensive explanation of how you've structured your API endpoints in Django and managed CORS using the built-in Django CORS headers. It's also good to hear that you've leveraged JSON responses and state management in React to efficiently handle data flow. Given your experience with both backend and frontend technologies, how do you approach testing in your full-stack applications? Can you describe your experience with any testing frameworks or strategies you've used to ensure the reliability of your code?  \n", "Interviewee: So regarding the testing, uh, I'll I'm I'm going to be honest that I'm not, uh, I don't know much about the testing part because, uh, in the internship, uh, my the testing part used to be looked after by the testers. So in the internship, there was nothing testing related to work with me. But for my particular, uh, my own personal process, I used to do the testing. So regarding the front end front end, I have also used Selenium testing. Selenium automation for testing purposes, like how the click is the click working or is the clicks are being rendered or is the alignment of the, uh, elements are particular orderly aligned or not. Those things I use Selenium for the testing. And regarding the features of the, uh, back end API, I used to test with the Django's, uh, testing. Uh, in Django, there is a particular part for testing itself. So I used to put all the dummy data into those in the test for the test file test dotpy file. I used to pass the test to dummy data, and using using those things, I used to verify the APIs.\n"], ["Interviewer: It's understandable that your testing experience might be more limited, especially with the focus on development during your internships. However, it's good to know that you've at least had some exposure to testing with Selenium for frontend and Django's built-in testing framework for backend. As you continue to develop your skills, testing will become an important aspect, especially for full-stack developers. For now, let's pivot to your experience with state management in React. Can you discuss how you've managed state in your React applications, and have you used any state management libraries or context API for this purpose?  \n", "Interviewee: Uh, regarding this, the state management. Uh, I will not use any other libraries for the state management. Uh, I only work with, uh, because it was my my own mini personal project where where I use the React. So, uh, there, uh, you only use the JavaScript, React state management or React inbuilt state management where I used to pass the data. So states are like you can't directly change the data, uh, in state by using the new you yeah. Sorry. You can't directly change the values in React. So if you want to change something or you have you can't increment the value directly. So you have to use the state for those things. So if, like, I'm going to increase the rate of the books or in the in my bookstore management project, if I'm going to increase the rates or increase the price of a book, then I use the states to do it because I can't directly increment the price based on the, um, based on the discounts. I can't directly I can't directly increment it. So I am going I I use the states that reacts in built state management, uh, to increment the values.\n"], ["Interviewer: Using React's built-in state management is a solid approach, especially for smaller applications. It's good to hear that you understand the importance of immutability in React state updates. As you've worked on a dynamic local marketplace website, can you tell me more about how you handled the data flow in this application? Specifically, how did you manage the data for the products, user interactions, and the shopping cart? Did you use any specific patterns or techniques to keep the data flow efficient and maintainable?  \n", "Interviewee: Regarding the data flow from, uh, shopping, uh, shopping to the shopping cart and to the checkout, so what I have done is, like, I used created a separate model for the carts carts and the orders. So what are the the uh, carts is separate and order is separate. Cards are cards in the cards, the elements which he has put into the cards are only stored. So using those things, I used to manage the cards elements separately and the ordered element separately. So from the cart, if you place the order, then only the elements from the cart will be moved to the orders orders, or else it will be in the cart itself so that the, uh, it may not conflict with the already shipped order or not ordered parts. So this is how I manage the data flow with the shopping carts. So, uh, using, uh, directly sending the data from the back end, I used to do, Like, if you if you select a particular item, then the ID of the item from the normal home page, the each book each book element has a particular letter ID to it. So the ticked I uh, the selected item's ID will be transferred to the next part. So based on the ID, I you based on the ID, I used to locate the book as well as its details like pricing, its author, its availability, store number of stocks. Those I used to check. And in the order list, I used to keep all those data. And if the order is available, uh, I mean, it's not out of stock, then it can be, uh, from the cart, it can be moved into if he is going to purchase, it can be moved. If it is out of stock, then in the web page itself, it will show us that if the quantity is less than 1, then it is showing it it is not in the market right now. So we have, uh, I manage those things. If it is in the stock or not, I used to manage with those things. So, uh, the data how did I add the data? So for my project, uh, I had to so it was not feasible to add all the, uh, hundreds of the 50 to 100 of books manually. So I implemented the scraping over here to add all the data to the my database. So I used the selenium here. I used the Sapna Bookhouse, uh, website for book reference, and from there, I used to scrape the book data along with the, uh, I'll scrape, uh, book name, book author name, and, uh, book name, author name, and publication dates, and publishers. Those things I used to scrape, and the price also yeah. Price also I used to scrape, but the price I have given my custom prices for for my reference over there. So these five things I used to scrape and I have to store in the database. This is how I store the data in the database. Regarding the, uh, number of available products, I used to write manually. Since this was a mini project, I, uh, each books were of only 5 5 limit 5 limited stocks itself. Only 5 limited stocks. So, uh, each book was 5 limited stocks. So if anyone orders a particular from the cart, it is ordered. If it is in the cart, then it can also be sold out. If it is not ordered, it can be sold out. So if someone orders from cart, it if the product is moved from cart to the orders, then only the, uh, quantity of the product will be reduced, or else it won't be reduced. So these were used to manage both the data, uh, element flowing from, uh, web page to the orders.\n"], ["Interviewer: You've outlined a clear strategy for managing the data flow in your marketplace application, using separate models for the cart and orders to maintain transactional integrity. The use of web scraping to populate your database with book data is a clever way to automate the process and ensure your marketplace has a comprehensive inventory. It's also good to see that you've implemented stock management logic to handle product availability. Given your experience with web scraping and data management, how do you ensure the consistency and accuracy of the scraped data, and what measures do you take to handle any changes in the source website's structure that might affect your scraping process?  \n", "Interviewee: When we whenever we are building a scraping model, the most difficult situation is for for us is that if the website has been updated or if, uh, changes in the webs website has been done. So this this thing cannot be, uh, completely automated. So we have to constantly check the website for if, uh, regarding if you have to constantly monitor the website. If there are even a slight changes, like only the class names are changed, if the element's class names are changed, IDs are changed, those things can be easily implemented because we store all the important x path or the parallel elements, uh, elements, class name, path name, path ID name, all are stored in a separate variable, required variable so that if in the in the future, if anything changes related to particular element, I can easily, uh, copy the path of that particular element and store in that particular variable. So by doing this, most of the code remains same. Only the, uh, thing that is changing is only the x path or the path of the element. So this is how I manage the automation. So but even though after doing that, we have to constantly monitor the website, like, uh, 2 weekly ones, 2 week uh, if it is not of, uh, big big big website like LinkedIn or LinkedIn, we can monitor it, like, weekly once. If it is LinkedIn, then we have to do it weekly once at least weekly once. If it is, uh, other website, not such large websites, then we can do it monthly once. So we have to monthly once we have to monitor. If there are changes, then we have to pop up, find the particular path, and then we have to implement it.\n"], ["Interviewer: You've highlighted a key aspect of web scraping, which is the need for robust monitoring and adaptability to changes in the source website's structure. Storing selectors and paths in variables is a good practice for maintainability, and your approach to monitoring frequency is well-considered based on the size and frequency of updates to the website. To further understand your technical skills, let's talk about your experience with Python. Can you walk me through how you've used Python in your backend development, perhaps in the context of your Django projects, and how you've leveraged its features for efficient and secure application development?  \n", "Interviewee: Yeah, sure. So Django is basically a python framework for backend. So everything you write for about by core is written in the python itself. So in the similar way how we use JavaScript for normally for backend purposes in node.js similar. The python does the same work python is a very vast language and it is a well well organized Community where everyone is contributing to it. So it is one of the best language which is present in all the fields it can handle AI domain it can handle web domain everything can be held in it in it. So during the back it's similarly how we are writing a python core it similarly it is in Django. But only the thing changes is like we have to create a models. We have structuring the structuring the code based on the particular Frameworks. So only those things are changing other than that all the basic python course Remains the Same so regarding the robustness of the Court. and usability of the court, I would say that like using the python the one of the most difficult situation is that it would take a lot of time like it is it is a bit time complexities with high for all the programming python programs compared to other C or C plus plus programs, but the memory management is good in Python. So the most important thing which we face is Time management. So for the time management, we can order always rely on those things. Like if the data comes then we can't so we have to be we have to be a synchronous. So we we can adapt to any changes if the based on the incoming output if the even if the incoming output is fast or slow due to the network errors, those things must be also handled. So by using async and a synchronous functions in Python, I used to handle those those things delay in that time. And other than that for error handling I used to use error handling methods try and try except methods for handling the errors because the dynamic applications we won't be able to understand. It there may be due to anything because it is dynamic website. So the real time in the real time data, there may be any unexpected any expected issues which we have previously not built for a program. So to handle all those things you I used try and I mean error handling methods to handle all those things as well as with the error handling. It's also difficult. I know because we won't be able to find all those things directly. So error handling must be in search will it we are able to understand and during a after the project has been developed. We decent already deployed and the devil of development and deployed face. It has already published at that time. We won't be able to monitor the console. So at those thing times we have to be. We have to use logs. So in the python, I used to use loggers logger modules to store all my logs in a particular log folders where I used to monitor if any error has been if any errors are there I used to add it to the log log and if it is a debug I used to specify it is an error message or a debug message so that I would be able to understand if it is due to error or it is to error the debug debug. So this is how I managed to understand during the Production environment, so and one more important thing is they are using python. So most of the time the browser doesn't understand the true or false because in Python that true or false is the true tea in the true is capital, but in the Json for JavaScript for format the team in the tourist small letter, so, okay, so these case Institute things. Must also be handled. So if I am rating a program in Python, then I must not use the I must not use the normal JavaScript true or false for Mac because I'm writing the code in Python. So I have to look into it. But before sending the data to the apis or to the front end users, I have to convert the Python structure data into the Json JavaScript structure data because I am going to send it to the other users in the web. The web won't understand the difference between capital D and small T. It is it is strict with the case sensitivity strict. So I have to understand and I have to change those things. This is how I used to manage.\n"], ['Interviewer: Thanks for taking the time to interview with us! The time allotted for our interview has come to an end. We will reach out to you with next steps shortly. Please click the end interview button to end the interview.', 'Interviewee:']]}

Preferred Candidate:


Prompt:
Job Description:
ml-engineer-v3

Candidate A Resume:
{"data": {"awards": [], "certifications": ["Machine Learning", "Applied Data Science w/ Python, Specialization"], "education": [{"GPA": "", "degree": "B.Tech", "endYear": "2017", "major": "Civil Engineering", "school": "Indian Institute of Technology", "startYear": "2013"}], "projects": [], "publication": [], "skills": ["Data Analytics", "Predictive Modelling", "Behavioral Analytics", "Customer Segmentation", "Engagement/Campaign Analytics", "Team Management", "Mentoring/Training", "Image Processing", "Image Segmentation", "Named Entity Recognition", "Personalization & Recommendation", "RAG-LLMs", "Model Deployment", "Model Explainability", "ML Project Management", "Microservices", "CI/CD", "numpy", "pandas", "scikit-learn", "scipy", "shaply", "tensorflow", "pytorch", "detectron2", "mm-labs", "opencv", "mlflow", "transformers", "peft", "langchain", "llama-index", "pytorch lightining", "docker", "kubeserve", "open-ai", "pyspark", "mllib"], "workExperience": [{"company": "Seekho", "description": "Analyzed strategies for customer acquisition and built a recommendation system for content recommendation, enhancing user engagement and conversion, reducing CAC.", "endYear": "Present", "location": {"city": "Bangalore", "country": "India"}, "role": "AI/ML Professional", "startYear": "2023"}, {"company": "Tidings Ltd", "description": "Created a English Grammar Assistant utilizing a GPT-4 open-ai with streamlit to aid students. Used Cambridge Dictionary Online as Knowledge Base.", "endYear": "2023", "location": {"city": "London", "country": "United Kingdom"}, "role": "Contract Project: Language Model - Grammar Assistant Tool", "startYear": "2022"}, {"company": "Fashinza Pvt Ltd", "description": "Developed a trend monitoring engine for the sales team, enabling them to identify the latest apparel industry trends from online retailers and publishers. Created a recommendation engine that leveraged product attributes and trend scores to enhance product search-ability. Constructed an image data-based system for identifying apparel attributes, to ensure quality control and defect detection in the manufacturing process. Designed and executed pipelines to improve data quality by product information extraction from descriptions using GPT. Oversaw the development of internal data tools and pipelines, supporting Product Analytics and Business Analytics initiatives.", "endYear": "2022", "location": {"city": "Gurugram", "country": "India"}, "role": "Lead ML Engineer", "startYear": "2021"}, {"company": "Cars24", "description": "Constructed a damage detection system (DDS) utilizing car images to estimate repair costs. Conceptualized the system using instance segmentation and object detection. Devised a feedback & monitoring system for DDS incorporating human-in-loop interaction to benchmark and simulate the model using real-world data. Created an object detection model to evaluate engineers' adherence to uniform compliance, resulting in a significant improvement from 20% to 75%. Implemented a data annotation server with auto-annotation capabilities for seamless annotation of image data. Built & deployed a classification model to reduce procurement of bad conditioned cars. Developed and managed various event-driven serverless cloud functions and microservices on Google Kubernetes Engine. Utilized KServe and Nvidia-Triton-Inference-Server to optimize inference time and system latency for image models.", "endYear": "2021", "location": {"city": "Gurugram", "country": "India"}, "role": "ML Engineer", "startYear": "2020"}, {"company": "SyncMedia", "description": "Designed and implemented a cutting-edge Generative AI model for generating personalized selfies with custom hairstyles. Employed the UNET architecture for efficient image segmentation tasks, and utilized the StyleGAN, CycleGAN and WGAN architecture for image translation and image inpainting purposes. Developed and oversaw a range of OpenCV Algorithms & functions, integrated them as microservices.", "endYear": "2018", "location": {"city": "Mumbai", "country": "India"}, "role": "Data Consultant", "startYear": "2017"}, {"company": "EXL", "description": "Led a team of 4 analysts in consumer behavior analytics for a healthcare industry client, resulting in a promotion based on exceptional leadership and analytical skills. Created and trained an insurance claim denial classification model using XGBoost, incorporating demographics and member behavior data to improve predictive accuracy. Implemented engagement tracking and conducted comprehensive behavior analyses to enhance customer retention and loyalty programs for retail stores. Designed a suite of BI dashboards using PySpark, Hive, and Tableau to track user journey across digital health products and channels, providing valuable insights. Mentored junior analysts through regular training sessions and one-on-one coaching to enhance their technical and analytical expertise, fostering professional growth.", "endYear": "2018", "location": {"city": "Noida", "country": "India"}, "role": "Data Scientist", "startYear": "2018"}, {"company": "Capgemini", "description": "Developed IoT-based proof-of-concepts at AI Labs, on both Android and iOS platforms. Created a sophisticated computer vision model using the TensorFlow Object Detection API to accurately recognize hand gestures, thereby improving user interface interactions for mobile applications.", "endYear": "2018", "location": {"city": "Mumbai", "country": "India"}, "role": "Data Scientist", "startYear": "2017"}]}, "status": "success"}

Candidate B Resume:
{"data": {"awards": ["Data Science Certificate from Amazon"], "education": [{"GPA": "", "degree": "Project", "endYear": "", "major": "Mathematics", "school": "Kashmir University", "startYear": ""}], "projects": [{"endYear": "", "projectDescription": "Understanding the problem and Objectives and development EDA and Data Cleaning Model Building and Visualization to find best combination by Python Data Visualization with Python. Programming Analyse by making graphs and plots Removing outliers, data cleaning and processing.", "projectName": "Match Win Prediction", "startYear": ""}, {"endYear": "", "projectDescription": "Data cleaning and data preprocessing Data Analysis by Logistic Regression using Python. Using confusion matrix, classification, ROC curve and algorithms to detect it by Python. Factor Analysis by PCA Analysing data by confusion matrix. correlation matrix and Eigen vectors Dimension has been reduced with 95% accuracy", "projectName": "Credit Risk Data Testing", "startYear": ""}, {"endYear": "", "projectDescription": "Data Mining from S3. EDA & Data cleaning via Sagemaker using Python. Balancing Data to prevent data leakage. Trained Data on Google Bert. Used RNN for analysis.", "projectName": "Amazon Ratings NLP Projects", "startYear": ""}], "publication": [], "skills": ["Python", "Machine Learning", "Deep Learning", "Neural Network", "AWS", "Data Analysis & Data Visualization", "NumPy", "Pandas", "Tableau", "TensorFlow", "Business Analytics", "MS Office/Excel"], "workExperience": [{"company": "Amazon", "description": "Amazon Ratings NLP Projects. Data Mining from S3. EDA & Data cleaning via Sagemaker using Python. Balancing Data to prevent data leakage. Trained Data on Google Bert. Used RNN for analysis.", "endYear": "Present", "location": {"city": "", "country": ""}, "role": "Development", "startYear": "2022"}, {"company": "Tech Mahindra", "description": "ML Data Associate. Amazon Ratings NLP Projects. Data Mining from S3. EDA & Data cleaning via Sagemaker using Python. Balancing Data to prevent data leakage. Trained Data on Google Bert. Used RNN for analysis.", "endYear": "2022", "location": {"city": "", "country": ""}, "role": "ML Data Associate", "startYear": "2021"}]}, "status": "success"}

Candidate A Transcript:
{'pairs': [['Interviewer: Hello and welcome to the AI interview! This interview will consist of basic questions about your background as well as some high-level questions about the skills you listed on your application. Ensure that you minimize long pauses during your responses, otherwise you may be cut off prematurely. Are you ready to start the interview? \n', 'Interviewee: Yes. Yes. Yes, I\n'], ["Interviewer: Great! I see from your resume that you've worked on a variety of AI and ML projects, including the recommendation system for Seekho. Can you elaborate on the specific techniques you used to build the recommendation system, and how you evaluated its effectiveness?  \n", "Interviewee: Okay. Uh, so for this particular project, I was doing, uh, I was building a recommendation engine for better conversion of customers. So Sequo is a platform. Uh, It's an online elearning platform which with short video courses. So when a new customer lands in, we may, uh, suggest him some particular video so that he's more interested into paying us for the subscription, and, uh, we can get the customer onboarded. So we wanted to recommend, uh, the video courses in which the customer is most likely to convert or most likely to pay for our subscription plan. So I employed collaborative filtering, and, uh, it is a multistep process for payment. So I, uh, assign different steps of the payments as ratings that particular customer wants to pay for a particular, uh, series. So using collaborative filtering and assigning different steps, the incremental steps for, uh, rating system as, uh, as the different rating rating given by the customer to a particular series. Now when, uh, using, uh, and building the collaborative system, I check for its evaluation through CTR rate in which, uh, I was checking whether they recommend this recommended series top 5 recommended series for a particular customer. Has, uh, is there are there any impression on those series? Are there any conversion rate through, uh, through the any of those series? Are are customers paying through or clicking through our recommendations or our suggestions.\n"], ["Interviewer: That's a comprehensive approach. When you evaluated the CTR for the recommended series, how did you handle the cold start problem, especially for new users or new courses with limited data? And in terms of the multi-step payment process, did you encounter any challenges in modeling the recommendation system to account for these incremental payment steps, and how did you address them?  \n", "Interviewee: Okay. Uh, so first of all, uh, to answer the first question where I am, uh, I am evaluating the, uh, incremental steps, uh, incremental payment steps as a rating system. So, uh, there were 4 different steps, uh, in which, uh, whether a customer which are whether a customer has visited a series page, whether whether a customer has initiated the payment, whether a customer has visited the payment, uh, bill gateway, made the payment and bought a subscription plan to us. So these are 4 different ratings that a that a particular user can, uh, like it is a proxy for for the payment steps, the rating system. And if the user has paid for a particular series, then perhaps he has given, uh, it a rating of 4. It is a proxy system. And, uh, Yeah. Uh, I'm not sure what was the other question.\n"], ["Interviewer: Regarding the cold start problem, how did you handle recommendations for new users who didn't have enough historical data or for new courses that had limited or no interaction data? And specifically, did you implement any techniques to address this issue within the collaborative filtering model?  \n", 'Interviewee: Okay. Uh, for for the cold start problem, I think, uh, my problem resolved revolved around the, uh, the new users only. So we had different campaigns to which, uh, uh, different campaign funnels, uh, and marketing campaign funnels through which we were receiving different kind of users. So based on the, uh, ideation of that campaign and based on the, uh, based on the designing of the campaign, we, uh, decided upon the target category for a particular customer. And, uh, we started recommendation engine for, uh, for new customers from those target categories. So new video courses from the target, uh, cat category of the campaign and, uh, like, the category of the series is. We use that to, uh, address the cold start problem. And specifically in collaborative filtering, I think, uh, I used a day t minus 1 data to see whether the customer has, uh, visited any, uh, series or checked for any payment step or went through any payment steps in a particular series, uh, we we can take those as the initial, rating given by the user, uh, as the payment steps. And, uh, and we can, like, move on with the collaborative filtering on the next day for and suggest series for that particular user.\n'], ["Interviewer: That's a strategic use of campaign data to mitigate the cold start problem. Shifting focus to your GPT-4 based English Grammar Assistant tool, could you describe the challenges you faced in fine-tuning GPT-4 for grammar correction, and how you measured the tool's accuracy in grammar suggestions?  \n", 'Interviewee: Okay. Uh, uh, for this particular, uh, project, I did not use fine tuning the GPT, but rather I used prompt completion and prompt engineering techniques in which I used chain of thoughts and, uh, different reasoning steps to check for a particular to to check for the context uh, while answering a particular question about English grammar. And for for x for measuring the accuracy and relevancy of the answer generated by the, uh, by by by the chaining of prompts. I employed, uh, blue score and road score to check for the, uh, to check for the precision and recall, uh, from the knowledge data that I was using, which was essentially the grammar articles, uh, from Cambridge dictionary online.\n'], ['Interviewer: Using prompt engineering with GPT-4 is an interesting approach, especially for context-rich tasks like grammar correction. How did you manage the response coherence and relevance, given that prompt engineering can sometimes lead to disjointed responses? And in terms of the evaluation metrics like BLEU score and ROUGE score, did you have a target threshold for these scores to ensure the responses met a certain level of accuracy and relevance?  \n', "Interviewee: Okay. For this particular, uh, measuring the relevancy of about blue score and Roche score, uh, we did not use as such any threshold. It's just a a measure that we use to evaluate our responses. It was a POC that went for a a feedback and review process, uh, that we built.\n"], ["Interviewer: Understood, using BLEU and ROUGE scores as evaluative measures without specific thresholds makes sense for a POC. Now, let's delve into the technical skills listed on your resume. Can you explain how you've used MLflow for experiment tracking and model management in your projects, and specifically, any challenges you faced with its implementation in a production environment?  \n", "Interviewee: Okay. Uh, I've used MLflow, uh, in tracking the experiments. While I was at CAS24, I was working on a damage detection system, which is an image based inspection system for cars damages. And we were detect we were using object detection and strength segmentation to, uh, to, uh, to check for the kind of damage that is there on the car body and to check for onto which part the damage is associated with, so using mask segmentation for that. So, uh, while doing, uh, while training all this model, uh, this was not an out of the box, uh, model which we we can use. So we use faster r CNN and mask r CNN while, uh, experimenting with different kind of data, with different, uh, datasets and different labeled datasets. Uh, we wanted to ensure that whatever hypothesis we are making during the experimentation, we, uh, are ensuring that we are aligned with the starting hypothesis and the evaluation evaluation measures or different statistical test tests that we'll have to do for a particular hypothesis. Summarizing all of this in an experiment, I used MLflow to track all of this information along with the type of data used, along with the type of model used, along with the hyperparameter information, uh, for tuning the, uh, for tuning and training the model. So I use MLflow over there. Uh, as per technical challenges, I did not, uh, I do not remember as of now. Uh, there were a couple of challenges while, uh, while keeping the different data versions. It's a little bit confusing, but, yeah, not nothing specific as such because, uh, I used it in research and development.\n"], ["Interviewer: Your use of MLflow for tracking and managing the experimentation process in a damage detection system is insightful. Regarding the management of different data versions in MLflow, did you encounter any issues with reproducibility, especially ensuring that a specific model's performance could be replicated with the same accuracy when retrained with the same data and hyperparameters? And in terms of the statistical tests and evaluation measures you mentioned, how did you integrate those into the MLflow tracking to ensure that the model's performance was not just technically sound but also statistically significant?  \n", "Interviewee: Uh, I'm I'm I'm not sure about that.\n"], ['Interviewer: No problem. Given your experience with TensorFlow and PyTorch, could you discuss a scenario where you had to choose one framework over the other for a specific project, and what were the key factors that influenced your decision?  \n', 'Interviewee: Okay. Uh, I have experience around TensorFlow and PyTorch working on different kind of AI problems, ML problems. I have used TensorFlow and PyTorch in different projects, but, uh, there was one project where I preferred PyTorch over TensorFlow because a lot of new research area, uh, that comes up, they have their working, uh, or their demo code working into PyTorch since PyTorch is most prep more preferred by a researcher. So, uh, one experimental, uh, product that I was working on during Stylex, which was, uh, generating customized, uh, human selfies with different hairstyles. So, uh, for this particular project, uh, I think I used PyTorch and I preferred PyTorch because a lot of, uh, frameworks like style and w again, cycle again. These were available, uh, with with some initial implementation, which can be extended further into PyTorch only. So, uh, the reason, uh, I prefer PyTorch is something, uh, which is, like, more closer to research. And also I use PyTorch Lightning Library, which is like another, uh, quick wrapper for PyTorch, which can initialize training in very few steps and, uh, increases productivity.\n'], ['Interviewer: Your decision to use PyTorch for the StyleX project based on its alignment with research and the availability of models and frameworks is well-founded. Regarding the use of PyTorch Lightning for project management, how did it specifically help in managing the training process and ensuring code cleanliness and readability, especially in a project like StyleX which might involve complex GAN or generative modeling architectures?  \n', "Interviewee: Right. Uh, I think I used, uh, while I was working at Stylex, it was a 2 step process which included, uh, image segmentation and then, uh, training a a GAM, uh, after the image segmentation process. So, like, during the, uh, I I used PyTorch Lightning during the process of, uh, image segmentation. So we used a unit model in the back end, and, uh, Lightning ensures that we are, uh, that some particular steps like selecting, uh, an an optimizer and then, uh, loading a data loader and then loading the model architecture and then, uh, defining a training loop in terms of epochs as to how many epochs has to go before the training stops or what is the evaluation metric, uh, we we need to check for during, uh, CV, cross evaluation, and what is the, uh, what is the state at which at which we will, uh, store the model checkpoint. So all of these things, PyTorch Lightning helped us and helped helped me in writing the writing a more efficient and readable code in in in terms of performance. Also yeah. I think that's it.\n"], ["Interviewer: That's a clear explanation of how you utilized PyTorch Lightning for the image segmentation and GAN training processes. Given your experience with both TensorFlow and PyTorch, could you discuss a scenario where you had to choose one framework over the other for a specific project, and what were the key factors that influenced your decision?  \n", "Interviewee: The, uh, I'm, uh, I think I've, uh, already answered this particular question where I where I suggested the use of PyTorch over TensorFlow because of the, uh, research repository on GitHub or research areas that are, like, the for for which the initial implementation is available over GitHub's, which you can extend. They are mostly present in PyTorch framework. Also, I really like the oops, uh, oops methodology too for oops methodology, which is followed in PyTorch framework. But it also comes with PyTorch lightning framework wrapper, which is, like, quite efficient. So I use PyTorch over TensorFlow.\n"], ['Interviewer: Your detailed explanation about choosing PyTorch over TensorFlow for its research-focused ecosystem and the object-oriented approach, especially with the added efficiency of PyTorch Lightning, provides a clear understanding of your decision-making process. How do you see the landscape of AI and ML tools evolving, and what factors do you think will drive the adoption of new tools and frameworks in the industry?  \n', "Interviewee: Uh, I I think there are, uh, new tools coming in. Uh, a lot of a lot of new deals are coming in. A lot of use cases. Uh, but I think the most important thing is to check whether the use case is relevant enough or the effort that we are doing behind building an ML model, uh, and the research that we are doing. Uh, we need to make we need to ensure that we'll have some impactful, uh, results after the research and after the effort for the business that we are working for. So, essentially, uh, first assessing the AI use case and how how much value it will generate for the business is most important. While talking about the tools, there are a lot of new tools coming in, but I believe, uh, some super tools like, uh, GCP, AWS, and, uh, super tools from Meta and some, uh, some big organizations like OpenAI. They are much more efficient in, uh, in producing tools which are more general or which which are more generalized in nature. So, uh, hence, I believe it's it's a very volatile space, and we should assess, uh, what we are how much value we are generating for the effort we are putting in, uh, in all different AI and ML use cases.\n"], ['Interviewer: Your emphasis on the relevance and impact of AI and ML solutions to business use cases as a primary evaluation criterion is insightful. Given this strategic approach, how do you stay updated with the latest tools and techniques in the AI/ML field, and what criteria do you use to assess whether a new tool or framework is worth exploring for your projects?  \n', "Interviewee: I usually go around reading blogs first and foremost. I look for some particular benchmarks from papers with code or or someplace else from Academia website where I am able to assess or I am reading research paper from archive for a particular problem say for image retrieval say for text to imagination text to speech whatever the area is AI. Ml it is I usually check for the Benchmark from papers with code and different that they the particular approach is performing on And once after a single performance through the benchmarks, I usually assess whether how difficult it is to implement. And what is the how difficult it is to implement? And what is the result that we can Target from a particular approach? What is that level of accuracy that we can Target from particular approach. These are two important factors that are usually go before like looking at which tool to use or which technology to use how difficult it is to implement and what? Yeah, I'm like that how difficult it is to implement. And what is the\n"], ['Interviewer: Thanks for taking the time to interview with us! The time allotted for our interview has come to an end. We will reach out to you with next steps shortly. Please click the end interview button to end the interview.', 'Interviewee:']]}

Candidate B Transcript:
{'pairs': [['Interviewer: Hello and welcome to the AI interview! This interview will consist of basic questions about your background as well as some high-level questions about the skills you listed on your application. Ensure that you minimize long pauses during your responses, otherwise you may be cut off prematurely. Are you ready to start the interview?\n', "Interviewee: Yes, John. I'm ready. Great.\n"], ['Interviewer: Great! I see from your resume that you have a strong background in data analytics and predictive modeling. Can you tell me about a particularly challenging project you worked on at Amazon and how you applied machine learning to solve the problem?  \n', 'Interviewee: For asking. Is the challenging project one? It was, um, it is confidential project. However, I will give you an overview of it. So it is basis on the real time analysis. So what happens, a video comes from a data stream, and, uh, we have to do some analytics on it that how much video flow is in the stream and how much workforce we need for it. So it is, uh, like the real time analysis. So what we do, we put it into the, uh, firehomes and then apply the Lambda function to check what is the current flow of the video streams. And we also, uh, in Lambda function, we also provide the input to do the trim for the videos for the sensor that, uh, it occupies less space. And, uh, the adequate amount the proper amount of video or the video stream goes to the associate or whoever is processing it. That was very challenging and we have to use Amazon Data Fire Tools, Amazon SageMaker in that, uh, we use Pandas and NumPy. And, uh, we also use 2 phase information using, uh, seaborne heat maps or correlations like that. It sounds\n'], ["Interviewer: It sounds like you've had hands-on experience with real-time data processing and AWS services. Given your use of Amazon SageMaker, can you elaborate on how you utilized SageMaker in this project, particularly in terms of model development and deployment? And could you also explain how you integrated the machine learning models with the real-time data stream?  \n", "Interviewee: Uh, actually, we were working as a a machine learning engineer, and there's a different team for the development. So what happens, we process the input for it and fed it into the data streams via Amazon SageMaker. So what happens there, uh, we just, uh, give the input data to the data stream. It just goes through the, uh, Amazon data files, uh, files, where whether it is Amazon Redshift or is it in s 3. And it does the data mining from there. And, uh, that is also the test data. So what happens in the test data, they do not provide us the real data in the first instance. We have to use the test data for that. So we just, um, for example, if it is, uh, real time analysis, so what happens there, we face the data and, uh, put it into the Amazon SageMaker using, uh, the Panda stream. So whatever, it is cyclic learn. And, uh, from there, we import the SVM, sport vector machine, and, um, do the modeling from there. And sometimes we also leverage the AI model like CNN RNNs. It's clear.\n"], ["Interviewer: It's clear you've had a collaborative role in the machine learning process, focusing on data preprocessing and model integration. Since you mentioned using SVM, could you discuss how you approached feature selection and engineering for your models? Also, I'm interested in how you evaluated the performance of your models in the context of real-time data, and what metrics were most important for your use case.  \n", 'Interviewee: Thanks for asking. So in terms of, uh, SVM, support vector machine, we, uh, primarily use PC, principal component analysis. So it penalizes the features which are are mostly related or it has high correlation with each other. So they just, uh, trimmed out those features. Features. And sometimes we also manually check whether, uh, some important data is not missing from there or PCA, uh, had PCA done the exact analysis for it. And regarding the matrices, mostly it was precision and recall. So, for example, we had to feed out false negative values from that mostly. And first, for uh, regarding cost, was it if they went into the suite, that is going to be put in as the icon for the job. And regarding the, uh, first negatives, it should not be there because we cannot miss out the important data from not being visualized by the associate or not being, um, I mean, the same process by the associate. Using\n'], ["Interviewer: Using PCA for feature selection is a solid approach, and it's good to hear that you were mindful of feature correlation and missing data. Precision and recall are indeed critical for a classification problem like the one you described. Can you tell me about a specific instance where you had to balance precision and recall, and how you made the decision to favor one over the other in your model tuning? Additionally, how did you handle the trade-off between model complexity and interpretability, especially in a real-time system?  \n", "Interviewee: That is a good challenging question. So regarding, uh, since we are talking about, uh, Amazon guidances, so I'm going with that thing only. So what happened there? Uh, the beta was up 2 types. Uh, they were up the 2 types. 1 was high confidence, and another was low confidence. High confidence means machine learning, uh, already answered by itself. However, regarding the low confidence, it was not confident to them to answer. So what happened there, uh, when we classified the these jobs into LC or HC, LC by means, uh, low confidence, HC means high confidence. And, uh, for example, in the PC, we are talking about PC. In case it did not provide it, uh, educate input. For example, an easy job was marked as l c or an l c, important job. It was parked as, uh, like, false negative. So in those cases, we missed out the important data. So I had to go through it whether it is working pro uh, we just took the little little sample from the data and checked whether it is working fine or not. So it used to come around 90 to 95%. So we were good with it. Because, uh, important data is there, then we can consider it. And, uh, regarding the trade off between these two, so what happened, uh, mainly because some of the trimmed videos right there or empty videos right there, there was nothing going on in the videos that was the noise. We consider that as noise or trade off. So we usually used to fetch the data and check the little samples if something is there. If, uh, it was there, then we need to check the code or the machine since, uh, in Amazon, you do not need to code again and again. So you can understand it is all the tools running in the behind. You just have to test the data and do a little ETA or ETA and ETL, sorry, extract transform and load. And, uh, it is a little buzz, but, uh, regarding the more hands on coding in Amazon, uh, we do not take that out, uh, on the last scale, but first, uh, little little step we used to employ it. Balance\n"], ["Interviewer: Balancing high and low confidence predictions is crucial, especially in a system where human intervention might be required for low confidence cases. Your approach to sampling and evaluating the model's performance is commendable. Given the complexity of the models you've worked with, how did you ensure that your models remained interpretable, especially when presenting your findings to stakeholders who may not have a technical background? And could you also touch on how you communicated the potential risks of model predictions, such as false negatives, to these stakeholders?  \n", "Interviewee: John, so regularly, since I was in the team, we were the team of 5 data engineers, and there were also many teams working in the behind. So I can talk about my team at Hippo too. So I worked under the data scientist. Uh, he was very experienced, and he had a large of a large amount of experience there. So regarding the false negative, since, uh, most as you told, you are very right. Most of the stakeholders, they don't have any technical knowledge what we were talking about. Right? Like, this is what we call our first negatives, first positives. So we have to tool them, uh, make them understand that, uh, look, uh, since, uh, during the data processing for example, if we use human for all the data, whether it is HCLC or anything, Uh, it will cost you millions because you have to employ a large number of people, and they should be well versed with the knowledge, but they'll see what is HCLC or to answer. So, uh, we want to make them understand, uh, how AI is helping us out in it. And we just, uh, provided a thought put for them. They're like, uh, what is a false positive? And what's a false negative? We told them that false positives, for example, if something is there and we tell that it is not there, it is a false negative, and we are already ignoring it. So that means we are missing an important input. So we are losing on data and it may not be accurate. So we have to be careful about the false negatives. Regarding the first positives, if it is there, for example, as something was there and we told that it is not there. Uh, sorry. It is there. But if it was it if it was not there and we told that if it is there, it is a false positive case. And it is okay if it goes for the processing. But the the things which, uh, for example, in if in a beta, nothing is there. Considering with nothing is there. So and we tell them that something is there. That makes a problem because we are, uh, wasting the time of the associate who is working on the system in the background. And, uh, similarly, it was easy for them to understand what we were telling, and it was very communicable for them also. It's great.\n"], ["Interviewer: It's great to hear that you and your team were able to effectively communicate the implications of model predictions to stakeholders. Making the AI's role in decision-making clear and using relatable examples like the cost of human labor versus AI solutions is a smart approach. Given your experience with AWS, can you discuss a time when you had to optimize a machine learning model for better performance or cost efficiency on the cloud? What specific AWS services or strategies did you employ to achieve this?  \n", "Interviewee: Here's John regarding my challenging project or when I had the optimize them already existing project. It was mining the data from Amazon redshift. So what happens there initially when the data is reprocess it you may know that it goes into S3. Then as separate coffee command. Command is given and it goes to the Amazon ratio. So at some point, I don't know whether it was a glitch or not. I faced the data from density, but I found that and I think two features that they were not being processed it. I don't know why but you know them they considered that it is good. Finally doesn't need any optimization. So I will make you understand since it is a confidential project. I will tell you how it so whatever working on a data and we found that Associates who processed the data in very low time. They used to get it's a high confidence jobs, so it should not be like that because since you have a group of Associates working in the background station get that equal amount of time. They answer the queries or the videos, which they have been provided. So I found that and I implemented the classification like K nearest neighbors and then made the groups and using the paid plots. I found this and all the and I then informed the same to the stakeholders and my team manager and they took the decision accordingly and then after that. we just took all the I mean whatever features were there, we considered all of them and only few of them were then drop on the list and process it\n"], ["Interviewer: It's interesting to hear how you identified and addressed the issue of unequal workload distribution among associates by implementing a classification model. Using KNN and pair plots for feature selection is a practical approach. When you say you considered all the features and then dropped some, how did you determine which features to drop? Also, how did you ensure that the model you implemented was both accurate and fair in its distribution of work among the associates?  \n", "Interviewee: It's John. Uh, since it is a confidential project, I will not go in-depth in it. But, uh, two features right there. As I already told you, the associates were getting for those who answered in the less time, they were getting easy easy workflow, and it was not very complex for them. And, uh, using that, they were in the top of their metrics because no one knew in the background, uh, using the these things that since they answered or, uh, processed the job in low time, they were getting less jobs. So we just fine tuned our Amazon client That's the real time. It was all, uh, real time data analysis there. So we had to fine tune it, and we just, uh, made sure that, uh, the lambda function is working correctly and, uh, using the input like shards and how they would deface the data from our stream. And, uh, it went uniformly, uh, to all of the associates. I under\n"], ["Interviewer: I understand the sensitivity of the project details. Focusing on the technical aspects, it's clear that you leveraged AWS services to ensure fair distribution of work. When you were working with Lambda functions and real-time data, how did you handle the monitoring and logging of these functions to ensure they were performing as expected? And in terms of model validation, what techniques did you use to test the fairness and accuracy of the workload distribution model you implemented?  \n", "Interviewee: Yes, John. Uh, I forgot to answer this in the previous question. I missed it out. Sorry for that. So regarding the, uh, fairness of this model, we just checked the f one score. And, uh, we also sometimes, uh, went to the model score using, uh, like, we take the model score from scikit learn. Because we for example, if we cross the data, there's a very large amount of data. We just took some little sample from it and then, uh, used the model score or Apple score for it. From then, we did the null hypothesis also. That is also a way of doing it, whether it is funding in the path or not. So regarding the Lambda function inside the Amazon Kinases, so, um, it was already predefined there. We have to do little change. Like, we have to adjust the time. And, uh, sometimes we used to, uh, go through the loops. For example, we, uh, use the conditional of our conditional modeling there and give it a function like, uh, while, uh, the associate answers this job, and then it should go again there, and it should be uniformed. The same job should not go there. And, like, uh, it was like first come, first serve reason. Like, easy jobs who are processing the jobs in little time. He's getting all the easy jobs and getting the admin addresses, and that was not fair like that. No problem.\n"], ["Interviewer: No problem, and thank you for clarifying. Using the F1 score and model scoring from scikit-learn are good measures for assessing model performance. It's also reassuring to hear that you considered the null hypothesis to ensure the model's predictions were statistically significant. For the Lambda functions, it's clear you had to tailor them to meet the specific needs of workload distribution. Can you elaborate on how you implemented the conditional logic within the Lambda functions to ensure fair distribution, and what kind of testing or simulation you performed to validate that the new logic was working as intended?  \n", "Interviewee: John, uh, when we reported this anomaly to our, uh, I'm reporting to the manager and he reported to them as stakeholders or whoever was in the background he, uh, reporting to. So, uh, but in the testing mode, uh, we'll just place the data. And, uh, in the Amazon guide system, the Lambda function, we adjust the time flow. Like, uh, when we assign a pair to our that associates were getting the easy job flow. We're answering it in the little time. So we had to adjust that because it was just going 1st come, 1st serve basis. Whoever is answering and the little time is getting the easy ones. So we just made a distribution there, like, uh, using the central limit theorem. For example, if it is not a good distribution there, if it is not uh, symmetrical distribution, we use the, uh, central limit in the normal cases. Why, uh, however, in there, we use the while while loops there. For example, while the associate is answering the job in the less time, we should still get the, uh, same complexity, complexity, uh, as the other ones are getting. And once we implemented that in the, uh, Firehouse Amazon Kinetic, we just took this, uh, little data from there and checked ourselves whether it is working fine or not. So similarly, we went on with model score. If it was 95% and above since it was very easy, it should be 95% and above. So we just implemented it like that. Implementing\n"], ["Interviewer: Implementing a fair distribution system using the central limit theorem and while loops within the Lambda functions is a clever solution to ensure equal workload complexity for associates. It's also good to hear that you performed manual checks to validate the system's performance. Given the complexity of this task, how did you ensure that the system remained scalable and efficient as the workload increased? And what kind of automated testing or monitoring did you put in place to catch any regressions in the workload distribution?  \n", "Interviewee: John, regarding the complete, uh, complexity and the regression thing which you're asking, how we have monitored that? So first of all, I will go with the distributions. So I will make you understand, uh, the job flow was same. It was not different. So for example, if there are a 100 jobs in the queue, consider the video as jobs. I'm telling that, uh, there were 100 jobs in the queue. So what was getting, uh, if what was happening if there is a set a and a set b, Uh, if associate a answers the job in the less time, uh, so I repeated it many times, you will get the easy one. So when I use the Lambda function, what happens, they are the same under jobs by uniformly distributed. For example, if, uh, an, uh, a machine predicts an LC and it is very, uh, critical. Uh, for example, if there are a 100 LCs and the the total job flow is 200 jobs, so it will divide 50 ML 1 and 50 50 ML, other associate. So regarding the fairness of this, uh, in background, we employed the k nearest neighbors and we checked, um, this thing whether if it is going uniformly across all the associates or not. So groups right there, and we face the data from the clusters. It's cleared.\n"], ["Interviewer: It's clear that you've put a lot of thought into ensuring that the workload is evenly distributed among associates. Using KNN for clustering and ensuring that the system remains scalable is a good approach. To further understand the scalability of your solution, could you discuss how you handled the dynamic nature of the workload? For instance, how did your system adapt to sudden spikes in the number of jobs in the queue, and what measures did you put in place to ensure that the system could handle an increasing number of associates or jobs without significant performance degradation?  \n", 'Interviewee: Yes, John, that was a very good question because I have faced this values. So what happens I already told you it is regarding the videos which were loading in the background Associates have to answer them or the process them. So sometimes the workflow increase at your rightly mentioned. So in those cases, we always have a backup Associates because the same number of SSS can also work on this data, but we cannot delete or drop some data from the input so that there is a loss to the company or whatever we are working there is lost to the crosses. So we have a backup amount of associated with in the different processes, but they were trained on this process all so in case there is a sentence Spike we used to get the input from the key and and which is already there. He that processes that can and there is also Amazon quicksight quicksight Q. Basically we use it that also showed that there is a number of backlog or we are missing out this number of data or slee and accordingly if we need more Associates, we took the associates and they were the hands on the system. However, sometimes what happens the jobs were processing being repeatedly, so that was one concern. It happens sometimes ridiculous or something. We also had to address that and remote using the lasting approach.\n'], ['Interviewer: Thanks for taking the time to interview with us! The time allotted for our interview has come to an end. We will reach out to you with next steps shortly. Please click the end interview button to end the interview.', 'Interviewee:']]}

Preferred Candidate: