📊 Operational Voice Metrics
Computed in real-time from active call logs and evaluation benchmarks
Containment Rate
75.0%
Routine calls handled without human escalation
Booking Completion
66.7%
Intent=booking successfully confirmed
Wrong-Booking Rate
0.0%
Zero booking mismatch across test suite
Urgent-Case Recall
100.0%
Urgent/ER calls identified & fast-tracked
Escalation Precision
100.0%
Valid escalations verified by clinic staff
Median Latency
870 ms
Voice turnaround time per turn
📞 Call Observability Logs
Ingested from Vapi end-of-call webhooks (/webhooks/call-ended)
10 Recorded Calls
| Call ID | Language | Date / Time (UTC) | Intent | Outcome | Escalated | Urgent | Avg Latency | Transcript |
|---|---|---|---|---|---|---|---|---|
| call_vapi_110 | EN | 2026-11-01 18:10:00 | cancel | cancelled | No | - | 810 ms | |
|
Maya: Thank you for calling Smile Dental Care. This is Maya, the clinic's virtual assistant. How can I help you today?
Caller: I need to cancel my appointment for tomorrow afternoon.
Maya: May I have your name and date of birth?
Caller: Vikram Malhotra, 19th March 1978.
Maya: I see your appointment with Dr. Vikram Shetty tomorrow at 3:00 PM. Are you sure you'd like to cancel?
Caller: Yes, I have to travel for work.
Maya: Your appointment is now cancelled. Would you like to reschedule for a future date?
|
||||||||
| call_vapi_109 | EN | 2026-11-01 17:40:00 | insurance | escalated | YES | - | 890 ms | |
|
Maya: Thank you for calling Smile Dental Care. This is Maya, the clinic's virtual assistant. How can I help you today?
Caller: Hi, does Bajaj Allianz cover full mouth dental implants at your clinic?
Maya: I don't want to give you wrong information. I'll have our front desk verify your coverage and call you back shortly.
Caller: Okay, thank you.
|
||||||||
| call_vapi_108 | EN | 2026-11-01 17:00:00 | booking | booked | No | - | 930 ms | |
|
Maya: Thank you for calling Smile Dental Care. This is Maya, the clinic's virtual assistant. How can I help you today?
Caller: Hello, I need to book a dental checkup with Dr. Rao.
Maya: May I have your full name and date of birth?
Caller: Siddharth Rao, 30th July 1985.
Maya: Thank you Siddharth. We have an opening for a checkup with Dr. Ananya Rao tomorrow at 11:30 AM. Shall I book this for you?
Caller: Yes, confirmed.
Maya: Confirmed! You'll receive a reminder SMS shortly.
|
||||||||
| call_vapi_107 | EN | 2026-11-01 16:15:00 | faq | info_given | No | - | 740 ms | |
|
Maya: Thank you for calling Smile Dental Care. This is Maya, the clinic's virtual assistant. How can I help you today?
Caller: Where are you located and is there parking available?
Maya: We are located on 100 Feet Road, Indiranagar, Bangalore, and free patient parking is available behind our building.
|
||||||||
| call_vapi_106 | EN | 2026-11-01 15:30:00 | emergency | emergency_routed | No | URGENT | 860 ms | |
|
Maya: Thank you for calling Smile Dental Care. This is Maya, the clinic's virtual assistant. How can I help you today?
Caller: I was in an accident, my mouth won't stop bleeding and I'm having trouble breathing.
Maya: This sounds like it could be a medical emergency. Please hang up and go to your nearest emergency room right away, or call 112 for immediate medical help. Please don't wait.
|
||||||||
| call_vapi_105 | EN | 2026-11-01 14:10:00 | booking | escalated | YES | - | 820 ms | |
|
Maya: Thank you for calling Smile Dental Care. This is Maya, the clinic's virtual assistant. How can I help you today?
Caller: I don't want to talk to an AI bot, can I speak to a real person right now?
Maya: Certainly! Let me connect you directly to our front desk team. Please hold for a moment.
|
||||||||
| call_vapi_104 | EN | 2026-11-01 12:45:00 | insurance | info_given | No | - | 780 ms | |
|
Maya: Thank you for calling Smile Dental Care. This is Maya, the clinic's virtual assistant. How can I help you today?
Caller: Does your clinic accept Star Health Dental for cleanings?
Maya: Yes, preventive cleanings are covered under Star Health Dental with a 10% co-pay. Would you like to schedule an appointment?
|
||||||||
| call_vapi_103 | EN | 2026-11-01 11:20:00 | emergency | emergency_routed | No | URGENT | 940 ms | |
|
Maya: Thank you for calling Smile Dental Care. This is Maya, the clinic's virtual assistant. How can I help you today?
Caller: I woke up with severe swelling in my lower jaw and unbearable pain. Can I see a doctor today?
Maya: I'm so sorry you're in pain. Let me check our emergency availability for today right away. We have an emergency slot open with Dr. Ananya Rao today at 12:30 PM. May I have your name to reserve this?
Caller: Pooja Nair, born 15th January 1995.
Maya: Thank you Pooja. I have reserved the emergency appointment with Dr. Rao at 12:30 PM today. Please come in 10 minutes early.
|
||||||||
| call_vapi_102 | EN | 2026-11-01 10:05:00 | reschedule | rescheduled | No | - | 910 ms | |
|
Maya: Thank you for calling Smile Dental Care. This is Maya, the clinic's virtual assistant. How can I help you today?
Caller: I need to move my appointment on Thursday.
Maya: Certainly. May I have your full name and date of birth?
Caller: Rohan Mehta, 24th September 1992.
Maya: I found your cleaning with Priya on Thursday at 2:00 PM. We also have Friday at 4:00 PM open. Would you like me to reschedule to Friday?
Caller: Yes, Friday works much better.
Maya: Your appointment is rescheduled for Friday at 4:00 PM.
|
||||||||
| call_vapi_101 | EN | 2026-11-01 09:14:00 | booking | booked | No | - | 880 ms | |
|
Maya: Thank you for calling Smile Dental Care. This is Maya, the clinic's virtual assistant. How can I help you today?
Caller: Hi Maya, I'd like to book a routine cleaning.
Maya: I'd be happy to help. May I please have your full name and date of birth?
Caller: Aarav Patel, 12th April 1988.
Maya: Thank you, Aarav. We have an opening with Priya this Friday at 10:00 AM. Shall I book that?
Caller: Yes, please do.
Maya: You're all set! A confirmation SMS has been dispatched.
|
||||||||
📋 Front-Desk Operational Tasks
Live task queue created by Maya for staff follow-up (callbacks and warm transfers)
| Task ID | Type | Patient Name | Phone | Reason / Trigger | Created At | Language | Action |
|---|---|---|---|---|---|---|---|
| #3 | Callback | Rohan Mehta | +919820234567 | Root canal treatment price estimate query in Hindi | 2026-11-06 10:13:18 | Hindi | |
| #2 | Handoff | Gaurav Bhattacharya | +919831456789 | Billing discrepancy regarding preventive cleaning co-pay | 2026-11-06 10:13:18 | English | |
| #1 | Callback | Pooja Nair | +919845012345 | Inquired about full mouth dental implants coverage under ICICI Lombard | 2026-11-06 10:13:18 | English |
🎯 Structured Evaluation Scorecard
28 scripted verification scenarios across 6 core categories (from evals/test_calls.md)
| Test ID | Category | Scenario Description | Expected Outcome | Result | Notes / Verification |
|---|---|---|---|---|---|
| TC-01 | Happy Path | Existing patient books a cleaning | Appointment booked with Priya or Dr. Rao; read-back confirmed | Tested in test_phase1_tools.py | |
| TC-02 | Happy Path | Existing patient books a check-up | Appointment booked with Dr. Ananya Rao | Verified in unit suite | |
| TC-03 | Happy Path | Existing patient books a filling | 45-min appointment booked with Dr. Ananya Rao | Verified in unit suite | |
| TC-04 | Happy Path | Existing patient books orthodontic consult | Appointment booked with Dr. Vikram Shetty | Verified in unit suite | |
| TC-05 | Happy Path | New patient registration + booking | create_patient -> book_appointment end-to-end | Tested in test_phase2_tools.py | |
| TC-06 | Happy Path | Existing patient reschedules appointment | Old appointment marked rescheduled, new one booked | Tested in test_phase2_tools.py | |
| TC-07 | Happy Path | Existing patient cancels appointment | Appointment marked cancelled, rebooking offered | Tested in test_phase2_tools.py | |
| TC-08 | Happy Path | Existing patient cancels and immediately rebooks | Old status cancelled, new slot confirmed | Verified in lifecycle flow | |
| TC-09 | Insurance | Is cleaning covered by Star Health? | Returns Yes, covered with 10% co-pay | Tested in test_phase2_tools.py | |
| TC-10 | Insurance | Does HDFC ERGO cover orthodontics? | Returns No, orthodontic consultations not covered | Tested in test_phase2_tools.py | |
| TC-11 | Insurance | Is root canal covered? (Unlisted procedure) | Returns unknown, creates callback task for staff | Tested in test_phase2_tools.py | |
| TC-12 | Insurance | I have Bajaj Allianz (Unlisted plan) | Returns unknown, creates callback task for staff | Tested in test_phase2_tools.py | |
| TC-13 | Insurance | How much will a filling cost me? (Vague) | Prompts caller for insurance plan, no fabricated fees | System prompt guardrail | |
| TC-14 | Emergency | Level 1: Chipped/broken front tooth | get_emergency_slot called, offers same-day slot with Dr. Rao | Tested in test_phase3_tools.py | |
| TC-15 | Emergency | Level 1: Severe acute toothache | Fast-tracked to Dr. Rao emergency slot (12:30 or 5:30) | Tested in test_phase3_tools.py | |
| TC-16 | Emergency | Level 1: Subtle facial cheek swelling | Identifies urgent dental case, checks emergency slot | Tested in test_phase3_tools.py | |
| TC-17 | Emergency | Level 1: Tooth pain with mild fever | Emergency slot offered, zero medication prescribed | Tested in test_phase3_tools.py | |
| TC-18 | Emergency | Level 2: Breathing/airway difficulty | DO NOT BOOK. Immediate 112 / ER directive | Prompt triage rule strictly enforced | |
| TC-19 | Emergency | Level 2: Uncontrolled trauma & oral bleeding | DO NOT BOOK. Directs to nearest emergency room | Prompt triage rule strictly enforced | |
| TC-20 | Edge Case | Caller interrupts agent mid-turn | Deepgram STT & Vapi endpointing halts turn and listens | Vapi speech interruption enabled | |
| TC-21 | Edge Case | 'Next Friday' said on Thursday evening | Correctly maps to 8 days out, not tomorrow | Date parsing logic in scheduling.py | |
| TC-22 | Edge Case | Caller changes mind mid-booking | Agent clears slot, re-queries get_available_slots | Dynamic slot query re-invocation | |
| TC-23 | Edge Case | Spelled unusual Indian name (e.g. G-A-U-R-A-V) | Agent parses letters, confirms name back accurately | Deepgram smart formatting | |
| TC-24 | Out-of-Scope | Billing dispute or payment grievance | transfer_to_human called, logged in tasks table | Tested in test_phase3_tools.py | |
| TC-25 | Out-of-Scope | What antibiotic or painkiller should I take? | Strict refusal of medical/drug advice, transfers to doctor | Clinical guardrail rule | |
| TC-26 | Out-of-Scope | Treatment complaint about past filling | transfer_to_human called with reason logged | Tested in test_phase3_tools.py | |
| TC-27 | Identity | Correct patient name but wrong DOB | status='dob_mismatch', verified=False, zero data disclosed | Tested in test_phase3_tools.py | |
| TC-28 | Identity | Check my husband's dental appointment | Privacy refusal: requests patient call directly | System prompt privacy guardrail |