How I Built an AI Scholarship Finder with Olostep
Some applications failed at the first eligibility check. Others ended in rejection after I had spent hours reviewing requirements and preparing documents.
Eight years later, I built ScholarScope to make that process easier.
ScholarScope is an AI scholarship finder that searches direct provider pages, ranks relevant opportunities, explains why they may fit, and flags details that still need verification. It does not promise certainty; it helps students decide where to focus their effort.
Project links:
- Try the live ScholarScope app
- View the source code
- Open the beginner notebook
- Browse the ScholarScope dataset
How I Designed the Workflow
ScholarScope uses Olostep for web search and scraping, Hugging Face Inference for AI analysis, and Hugging Face Spaces for deployment.
Each search follows a bounded pipeline:
Student profile
→ 5 focused searches
→ filter and rank links
→ scrape up to 8 provider pages
→ 1 AI review
→ validate deadlines and output
→ ranked opportunities
The main design choice was simple: use regular code for predictable tasks and AI only when it adds value. Code builds queries, removes duplicate and weak sources, handles failures, and filters expired deadlines. The model reads the remaining pages, compares them with the student’s profile, and explains uncertain details.
This kept the app fast, testable, and inexpensive. During development, my combined spending on web APIs and model inference remained below one dollar.
Why I Chose Olostep
I needed to find relevant pages quickly and turn the strongest ones into clean text for the model. Olostep handled search, JavaScript rendering, proxies, scraping, and page cleaning through one API.
The examples in this article are simplified versions of the app’s code. They leave out retries, error handling, and other production details so the main ideas are easier to follow.
Here is the basic Olostep flow:
import os
from olostep import Olostep
client = Olostep(api_key=os.environ["OLOSTEP_API_KEY"])
search = client.searches.create(
query="fully funded PhD scholarships for Pakistani students",
limit=3,
)
page = client.scrapes.create(
url_to_scrape=search.links[0]["url"],
formats=["markdown"],
remove_images=True,
)
print(page.markdown_content[:500])
The important part is the order: search first, filter the candidates, and scrape only the best pages.
My first instinct was to collect as much information as possible. That produced scholarship directories, social posts, old articles, and pages with no official application details. More data created more noise.
ScholarScope runs five short searches in parallel and initially collects only URLs, titles, and snippets. It removes duplicates, social links, known aggregators, and obvious list pages, then ranks direct sources from universities, governments, foundations, and recognised organisations. Only the strongest eight pages are scraped.
Here is a small example of that filtering step:
BLOCKED_DOMAINS = {"facebook.com", "instagram.com", "scholarshipportal.com"}
def keep_result(result):
url = result["url"].lower()
title = result["title"].lower()
is_blocked = any(domain in url for domain in BLOCKED_DOMAINS)
is_list_page = "top scholarships" in title or "scholarship list" in title
return not is_blocked and not is_list_page
strong_results = [item for item in search_results if keep_result(item)]
pages_to_scrape = strong_results[:8]
The real app uses more checks, but the idea is the same: remove obvious noise before paying to scrape pages.
Search results are candidates, not product results.
Filtering before scraping improved speed, reduced API costs, and gave the model better evidence.
Why I Chose Hugging Face Inference
I initially tested ScholarScope with Groq and Qwen3.6-27B, but structured output became inconsistent when I sent several pages for review. I also encountered rate and token limits.
Hugging Face Inference let me compare open models and providers without tying the app to one model company. I selected DeepSeek-V4-Flash and pinned Fireworks AI as the provider for consistent production behaviour.
import os
from huggingface_hub import InferenceClient
client = InferenceClient(
api_key=os.environ["HF_TOKEN"],
timeout=120,
)
response = client.chat.completions.create(
model="deepseek-ai/DeepSeek-V4-Flash:fireworks-ai",
messages=[{
"role": "user",
"content": "In one sentence, explain what an AI scholarship finder does.",
}],
max_tokens=80,
temperature=0.2,
)
print(response.choices[0].message.content)
The normal workflow uses one model call after the app has selected the strongest evidence. If the JSON-mode request itself fails, the app can make a second inference request with a stricter instruction to return only JSON.
Model selection is not just an AI decision. It affects product speed, reliability, and cost.
How I Divided the Work Between Code and AI
ScholarScope asks for a student’s origin, target degree, field of study, and preferences.
Not every task needs AI. Many parts of the workflow follow clear rules, so regular Python code can handle them more quickly and consistently.
Before the model sees any pages, the app uses code to:
- Cleans the profile and builds five focused queries
- Runs searches in parallel
- Removes duplicate URLs and weak sources
- Ranks candidate pages using simple rules
- Handles timeouts and partial failures
- Removes clearly expired opportunities
For example, a small function can turn the student’s profile into focused searches:
def build_queries(profile):
degree = profile["applying_for"]
field = profile["field_of_study"]
return [
f'{degree} {field} scholarships for international students',
f'{profile["origin"]} students {degree} scholarships abroad',
f'fully funded {degree} {field}',
f'university {degree} {field} financial aid',
f'government fellowship {field} {profile["preferences"]}',
]
profile = {
"origin": "Pakistan",
"applying_for": "PhD",
"field_of_study": "Computer Science",
"preferences": "Europe",
}
queries = build_queries(profile)
This task is predictable: the same profile should produce the same queries. A model would add cost and complexity without improving the result.
After that preparation, the model handles work that requires interpretation. It:
- Extracts funding, deadline, and eligibility details
- Compares each opportunity with the student’s profile
- Explains why it may fit
- Flags missing or unclear information
- Returns structured JSON
This division keeps the workflow clear. Code handles fixed rules; AI reads evidence, compares details, and explains unclear information.
How I Validated Output and Handled Uncertainty
ScholarScope needs consistent fields for every opportunity: title, organisation, source URL, deadline, funding, eligibility, match score, reasons, and concerns. Free-form text would make result cards, saved records, and search history unreliable.
I therefore treat the model’s JSON output as an API contract: define required fields, validate them, handle missing values, and keep the structure stable when models or prompts change.
The app uses a Pydantic model to validate each opportunity. Here is a simplified version:
from pydantic import BaseModel, Field, HttpUrl
class Opportunity(BaseModel):
title: str = Field(min_length=2)
source_url: HttpUrl
match_score: int = Field(default=70, ge=0, le=100)
concerns: list[str] = Field(default_factory=list)
model_result = {
"title": "Example Scholarship",
"source_url": "https://example.org/scholarship",
"match_score": 85,
}
validated = Opportunity(**model_result)
print(validated)
This catches missing titles, invalid URLs, and scores outside the allowed range. The real schema includes the remaining scholarship fields and sensible defaults. Invalid records are skipped; if none remain, the app raises an error instead of sending malformed data to the interface.
Deadlines required special care. Scholarship pages may describe rolling applications, annual cycles, or no clear closing date. ScholarScope applies cautious rules:
- Clearly expired opportunities are removed.
- Rolling opportunities can remain.
- Unclear deadlines remain with a warning.
- The provider page is always linked for final verification.
The same principle applies to eligibility and funding. A result can be useful even when one detail is missing, but the app must label it as a partial match.
Missing information should become a warning, not a made-up answer.
How I Kept Costs Predictable
The normal workflow has a small, predictable base budget:
5 search calls
→ up to 8 scrapes
→ 1 model call
These are logical operations rather than a strict limit on network requests. The Olostep client may retry a failed search or scrape once, and the app may make a second inference request if the first JSON-mode request fails. Even with that fallback behavior, the workflow remains bounded and does not let an agent search indefinitely. This made it possible to test the complete pipeline for less than one dollar.
I could have built ScholarScope with an agentic framework. In that design, an AI agent would decide which tools to call, when to search again, which pages to scrape, and when it had enough evidence to produce a result. That flexibility sounds useful, but it introduced clear tradeoffs in my tests.
The agentic version took about 2.5 minutes to return results. The fixed ScholarScope pipeline took about 35 seconds. It was also easier to predict its cost because every session had a clear limit.
An autonomous agent could make extra model and web API calls, especially if it became stuck, repeated a search, or kept looking for better evidence. Those calls could quickly increase both latency and cost.
For this product, the steps were already known. A controlled pipeline was therefore a better fit than giving an agent full control over the workflow.
How I Moved From a Notebook to a Live App
I built the core workflow in a notebook so I could inspect each stage independently: search results, source filters, scraped pages, prompts, and structured output. The beginner notebook shows the pipeline without interface code.
I then deployed the app on Hugging Face Spaces. The interface handles profiles, progress, results, saved opportunities, and errors. The backend handles search, filtering, scraping, AI review, deadline checks, and storage.
When dataset storage is configured, ScholarScope automatically stores the top result from each completed session and also stores opportunities explicitly saved by users. Session IDs and source URLs prevent duplicate records.
The public dataset does not include names, contact details, or account information. It stores only the search inputs, such as country of origin, target degree, field of study, and preferences, alongside the scholarship results.
This helps other students discover relevant opportunities while making past research easier to search, compare, and review.
The Four Lessons That Mattered Most
These four decisions had the biggest impact on ScholarScope’s result quality, speed, and reliability.
1. More context does not always mean better context
Sending more pages to the model did not automatically improve the answer. In practice, searching and scraping too many sources added noise and made it harder for the model to identify the most relevant scholarship details. A smaller set of strong provider pages produced clearer evidence and more accurate results.
2. Keep model and provider options open
Using routing platforms such as Hugging Face Inference, OpenRouter, and Requesty makes it easier to compare models and providers without tying the application to one company. This gives more flexibility when speed, cost, availability, or reliability changes.
3. Control latency at every stage
Response time is affected by every part of the pipeline, not just the final model call. Running independent searches in parallel, reducing unnecessary scraping, sending less context, and choosing a fast inference provider all helped make the app feel faster. Every unnecessary request and extra token adds delay for the user.
4. Treat structured output as a core requirement
Reliable JSON was essential because the interface depended on predictable fields for titles, links, scores, and concerns. Defining the schema early made results easier to render, store, validate, and test. It also helped catch incomplete or malformed model output before it reached the user.
Final Thoughts
ScholarScope is the tool I wish I had after my master’s degree. I did not need more links; I needed help identifying which opportunities deserved my time.
The same idea applies beyond scholarships. Useful AI products do not depend on the largest prompt or the most API calls. They use each tool for the work it does best, keep costs predictable, and remain honest about what they know.