Muhammad Imran Zaman PRO

ImranzamanML

AI & ML interests

Results-driven Machine Learning Engineer with 7+ years of experience leading teams and delivering advanced AI solutions that increased revenue by up to 40%. Proven track record in enhancing business performance through consultancy and expertise in NLP, Computer Vision, LLM models and end-to-end ML pipelines. Skilled in managing critical situations and collaborating with cross-functional teams to implement scalable, impactful solutions. Kaggle Grandmaster and top performer in global competitions, dedicated to staying at the forefront of AI advancements.

Recent Activity

posted an update 25 days ago
Deep understanding of (C-index) evaluation measure for better model Lets start with three patients groups: Group A Group B Group C For each patient, we will predict risk score (higher score means higher risk of early event). Step 1: Understanding Concordance Index The Concordance Index (C-index) evaluate that how well the model ranks survival times. Understand with sample data: Group A has 3 patients with actual survival times and predicted risk scores: Patient Actual Survival Time Predicted Risk Score P1 5 months 0.8 P2 3 months 0.9 P3 10 months 0.2 Comparable pairs: (P1, P2): P2 has a shorter survival time and a higher risk score → Concordant ✅ (P1, P3): P3 has a longer survival time and a lower risk score → Concordant ✅ (P2, P3): P3 has a longer survival time and a lower risk score → Concordant ✅ Total pairs = 3 Total concordant pairs = 3 C-index for Group A = Concordant pairs/Total pairs= 3/3 = 1.0 Step 2: Calculate C-index for All Groups Repeat the process for all groups. For now we can assume: Group A: C-index = 1.0 Group B: C-index = 0.8 Group C: C-index = 0.6 Step 3: Stratified Concordance Index The Stratified Concordance Index combines the C-index scores of all groups and focusing on the following: Average performance across groups (mean of C-indices). Consistency across groups (low standard deviation of C-indices). Formula: Stratified C-index = Mean(C-index scores) - Standard Deviation(C-index scores) Calculate the mean: Mean=1.0 + 0.8 + 0.6/3 = 0.8 Calculate the standard deviation: Standard Deviation= sqrt((1.0-0.8)^2 + (0.8-0.8)^2 + (0.6-0.8)^/3) = 0.16 Stratified C-index: Stratified C-index = 0.8 - 0.16 = 0.64 Step 4: Interpret the Results A high Stratified C-index means: The model predicts well overall (high mean C-index).
updated a model about 2 months ago
ImranzamanML/arabert_finetuned_model
updated a model about 2 months ago
ImranzamanML/arabic_finetuned_model
View all activity

Articles

Organizations

MISATO-dataset's profile picture Masakhane NLP's profile picture GEM benchmark's profile picture BigScience Biomedical Datasets's profile picture LangChainDatasets's profile picture DeepGHS's profile picture Blog-explorers's profile picture MLX Community's profile picture Cognitive Computations's profile picture

Posts 8

view post
Post
462
Deep understanding of (C-index) evaluation measure for better model
Lets start with three patients groups:

Group A
Group B
Group C
For each patient, we will predict risk score (higher score means higher risk of early event).

Step 1: Understanding Concordance Index
The Concordance Index (C-index) evaluate that how well the model ranks survival times.

Understand with sample data:
Group A has 3 patients with actual survival times and predicted risk scores:

Patient Actual Survival Time Predicted Risk Score
P1 5 months 0.8
P2 3 months 0.9
P3 10 months 0.2
Comparable pairs:

(P1, P2): P2 has a shorter survival time and a higher risk score → Concordant ✅
(P1, P3): P3 has a longer survival time and a lower risk score → Concordant ✅
(P2, P3): P3 has a longer survival time and a lower risk score → Concordant ✅
Total pairs = 3
Total concordant pairs = 3

C-index for Group A = Concordant pairs/Total pairs= 3/3 = 1.0

Step 2: Calculate C-index for All Groups
Repeat the process for all groups. For now we can assume:

Group A: C-index = 1.0
Group B: C-index = 0.8
Group C: C-index = 0.6
Step 3: Stratified Concordance Index
The Stratified Concordance Index combines the C-index scores of all groups and focusing on the following:

Average performance across groups (mean of C-indices).
Consistency across groups (low standard deviation of C-indices).
Formula:
Stratified C-index = Mean(C-index scores) - Standard Deviation(C-index scores)

Calculate the mean:
Mean=1.0 + 0.8 + 0.6/3 = 0.8

Calculate the standard deviation:
Standard Deviation= sqrt((1.0-0.8)^2 + (0.8-0.8)^2 + (0.6-0.8)^/3) = 0.16

Stratified C-index:
Stratified C-index = 0.8 - 0.16 = 0.64

Step 4: Interpret the Results
A high Stratified C-index means:

The model predicts well overall (high mean C-index).
view post
Post
675
Easy steps for an effective RAG pipeline with LLM models!
1. Document Embedding & Indexing
We can start with the use of embedding models to vectorize documents, store them in vector databases (Elasticsearch, Pinecone, Weaviate) for efficient retrieval.

2. Smart Querying
Then we can generate query embeddings, retrieve top-K relevant chunks and can apply hybrid search if needed for better precision.

3. Context Management
We can concatenate retrieved chunks, optimize chunk order and keep within token limits to preserve response coherence.

4. Prompt Engineering
Then we can instruct the LLM to leverage retrieved context, using clear instructions to prioritize the provided information.

5. Post-Processing
Finally we can implement response verification, fact-checking and integrate feedback loops to refine the responses.

Happy to connect :)