Skip to content

Sales CRM — Leads, Campaigns & Activity

The Sales CRM is the lead-management core of the MAS suite: it ingests prospects from many channels (CSV bulk uploads, third-party vendors, public website campaign forms, and manual entry), de-duplicates them into a single working pool (RawLead), and gives telecallers, counsellors, sales heads and admins the tools to qualify, contact, tag, follow up on, and ultimately convert or close each lead. Every meaningful touch — a call, an email, an SMS, a WhatsApp message, a status change — is journaled so the lead's timeline and the salesperson's activity feed are fully auditable.

Status: documented from source on this branch.

Overview

This domain owns the lead lifecycle from intake to conversion for the sales organisation. It sits between three feeder systems and the people who work the leads:

  • Feeders (intake): the public website promo/application forms (CampaignLead), the Vendor API platform (VendorLead), bulk CSV uploads, and manual "Add Lead" forms.
  • The working pool: RawLead rows in the dedicated mas_crm Postgres schema — the canonical, de-duplicated lead each salesperson actually works.
  • Workers / personas: telecallers (first-line assigned_to), counsellors (second-line assigned_counsellor), sales heads (own a pool of leads), and admins/superadmins (full oversight).

Personas and their access (enforced in the route files):

Persona Role enum What they can do
Telecaller / salesperson SALES Work only their own assigned leads via the self-scoped /sales/crm/* routes
Sales Head SALES_HEAD Manage their owned pool (owning_sales_head_id), import, assign, tag, run analytics
Admin / Superadmin ADMIN / SUPERADMIN Full cross-pool oversight, ownership transfer, pool administration, tag CRUD

The CRM is multi-product: leads carry a session_year, free-form extraData, tags (used to segment by course/campaign), and a source channel so the same pool serves drone-tech, space-tech and other programmes.

Related domains documented separately: lead assignment & approvals, telephony (Exotel click-to-call + SMS), WhatsApp messaging, the Vendor API platform, and the workflow automation engine that consumes lead lifecycle events.

Key concepts & entities

Glossary

  • Raw lead — the de-duplicated working record (RawLead). One row per unique phone number. This is what salespeople see in the CRM table.
  • Campaign lead — a pre-import row captured from a public website form (CampaignLead), gated by WhatsApp OTP. A shared company pool until imported into RawLead.
  • Vendor lead — a row pushed by a third-party vendor through the Vendor API (VendorLead), imported into RawLead on demand.
  • Import batch — one row per import action (LeadImportBatch); every imported raw lead points back to it for audit and bulk re-assignment.
  • Owning sales head — the head whose pool a lead belongs to (owning_sales_head_id); drives hard data isolation.
  • Telecaller (assignee) — first-line assigned_to; the person who dials.
  • Counsellor — second-line assigned_counsellor; the person who closes/converts.
  • Interest level — hot / warm / cold, set manually; distinct from the derived lifecycle bucket and from the Aarya AI-predicted interest (computed from call duration).
  • Lead status bucket — a derived value (new / contacted / qualified / converted / closed / untouched) computed from converted_at, closed_at, interest_level and call_count. There is no stored status column on RawLead.

Entities (TypeORM)

Entity Table File
RawLead mas_crm.raw_leads src/entities/RawLead.ts
CampaignLead public.campaign_leads src/entities/CampaignLead.ts
Lead public.leads src/entities/Lead.ts
LeadActivityLog mas_crm.lead_activity_log src/entities/LeadActivityLog.ts
LeadCallLog mas_crm.lead_call_logs src/entities/LeadCallLog.ts
LeadEmailLog mas_crm.lead_email_logs src/entities/LeadEmailLog.ts
LeadSmsLog mas_crm.lead_sms_logs src/entities/LeadSmsLog.ts
LeadWhatsAppLog mas_crm.lead_whatsapp_logs src/entities/LeadWhatsAppLog.ts
LeadFollowUp mas_crm.lead_follow_ups src/entities/LeadFollowUp.ts
LeadTag (+ raw_lead_tags join) mas_crm.lead_tags src/entities/LeadTag.ts
LeadImportBatch mas_crm.lead_import_batches src/entities/LeadImportBatch.ts
LeadAssignmentHistory mas_crm.lead_assignment_history src/entities/LeadAssignmentHistory.ts

Note the schema split: the CRM owns the mas_crm Postgres schema (raw leads, logs, follow-ups, tags, import batches). CampaignLead and the legacy Lead live in the default public schema. Cross-schema foreign keys (e.g. lead_call_logs.lead_id → raw_leads.id, and *.changed_by/called_by/sent_by → public.users) are loose references.

Key enums

  • LeadSourceType (RawLead.ts): csv | vendor | website.
  • LeadInterestLevel: hot | warm | cold.
  • WebinarStatus: interested | not_interested | registered | attended | no_show.
  • CampaignLeadStatus (CampaignLead.ts): new | contacted | converted | not_interested | applied.
  • CallOutcome / CallSource (LeadCallLog.ts): outcomes incl. picked, not_picked, connected, wrong_number, aarya_dispatched, exotel_dispatched; sources manual | aarya | exotel.
  • ActivityType (LeadActivityLog.ts): interest_changed, converted_changed, closed_changed, webinar_status_changed, assigned, unassigned, called, note_added, tag_added, tag_removed.
  • FollowUpStatus: pending | done | missed | cancelled.
  • LeadEmailStatus / LeadWhatsAppStatus / LeadSmsStatus: queued | sent | failed (SMS adds delivered).

Architecture

flowchart TD
    subgraph FE["Frontends"]
        WEB["Public website forms"]
        CRMUI["CRM / Sales dashboard"]
        VAPI["Vendor API clients"]
    end

    subgraph ROUTES["Routes mounted at /api"]
        RCL["campaignLead.routes.ts"]
        RRL["rawLead.routes.ts"]
    end

    subgraph CTRL["Controllers"]
        CCL["campaignLead.controller.ts"]
        CRL["rawLead.controller.ts"]
    end

    subgraph SVC["Services"]
        SCAMP["CampaignLeadService"]
        SRAW["RawLeadService"]
        SFOL["LeadFollowUpService"]
        STAG["LeadTagService"]
        SEMAIL["LeadEmailService"]
        SWA["LeadWhatsAppService"]
        SEXO["ExotelLeadService"]
        SEVT["LeadEventService"]
    end

    subgraph DB["mas_crm + public schemas"]
        TRAW["raw_leads"]
        TCAMP["campaign_leads"]
        TLOGS["call / email / sms / whatsapp logs"]
        TFOL["lead_follow_ups"]
        TTAG["lead_tags + raw_lead_tags"]
        TBATCH["lead_import_batches"]
        TACT["lead_activity_log"]
    end

    subgraph EXT["External + async"]
        REDIS["Redis OTP store"]
        QUEUE["BullMQ email + workflow queues"]
        WAPROV["WhatsApp provider (Meta / AiSensy)"]
        EXOTEL["Exotel telephony"]
    end

    WEB --> RCL --> CCL --> SCAMP
    VAPI -.imported by.-> SRAW
    CRMUI --> RRL --> CRL
    CRL --> SRAW & SFOL & STAG & SEMAIL & SWA & SEXO

    SCAMP --> TCAMP
    SCAMP --> REDIS
    SCAMP --> WAPROV
    SRAW --> TRAW & TBATCH & TACT & TTAG
    SRAW --> SEVT
    SFOL --> TFOL
    STAG --> TTAG
    SEMAIL --> TLOGS
    SEMAIL --> QUEUE
    SWA --> TLOGS
    SWA --> WAPROV
    SEXO --> TLOGS
    SEXO --> EXOTEL
    SEVT --> QUEUE

Data model

erDiagram
    RAW_LEADS ||--o{ LEAD_CALL_LOGS : "has"
    RAW_LEADS ||--o{ LEAD_EMAIL_LOGS : "has"
    RAW_LEADS ||--o{ LEAD_SMS_LOGS : "has"
    RAW_LEADS ||--o{ LEAD_WHATSAPP_LOGS : "has"
    RAW_LEADS ||--o{ LEAD_FOLLOW_UPS : "has"
    RAW_LEADS ||--o{ LEAD_ACTIVITY_LOG : "has"
    RAW_LEADS ||--o{ LEAD_ASSIGNMENT_HISTORY : "has"
    RAW_LEADS }o--o{ LEAD_TAGS : "tagged via raw_lead_tags"
    LEAD_IMPORT_BATCHES ||--o{ RAW_LEADS : "produced"
    CAMPAIGN_LEADS |o--o| RAW_LEADS : "imported into"
    VENDOR_LEADS |o--o| RAW_LEADS : "imported into"

    RAW_LEADS {
        uuid id PK
        varchar name
        varchar phone
        varchar session_year
        varchar email
        varchar parent_email
        jsonb extraData
        enum source_type
        uuid vendor_api_key_id
        uuid vendor_lead_id UK
        uuid campaign_lead_id UK
        uuid owning_sales_head_id
        uuid import_batch_id FK
        uuid imported_by
        uuid assigned_to
        uuid assigned_counsellor
        enum interest_level
        enum webinar_status
        varchar upcoming_action
        timestamp converted_at
        timestamp closed_at
        text closed_reason
        timestamp last_contacted_at
        int call_count
    }

    CAMPAIGN_LEADS {
        uuid id PK
        varchar campaign
        varchar campaignType
        varchar name
        varchar phone
        boolean phoneVerified
        boolean didInterestForm
        boolean didApplicationForm
        enum status
        jsonb metadata
        uuid assignedTo
    }

    LEAD_CALL_LOGS {
        uuid id PK
        uuid lead_id FK
        uuid called_by
        enum outcome
        enum source
        int duration_sec
        varchar exotel_call_sid
        varchar aarya_conversation_id
        text recording_url
        timestamp called_at
    }

    LEAD_FOLLOW_UPS {
        uuid id PK
        uuid lead_id FK
        timestamp scheduled_for
        uuid assignee_id
        enum status
        timestamp completed_at
        timestamp reminder_sent_at
    }

    LEAD_EMAIL_LOGS {
        uuid id PK
        uuid lead_id FK
        uuid sent_by
        varchar template_key
        enum track
        varchar to_email
        enum status
    }

    LEAD_WHATSAPP_LOGS {
        uuid id PK
        uuid lead_id FK
        uuid sent_by
        varchar template_name
        enum status
    }

    LEAD_SMS_LOGS {
        uuid id PK
        uuid lead_id FK
        uuid sent_by
        varchar to_phone
        varchar status
    }

    LEAD_ACTIVITY_LOG {
        uuid id PK
        uuid lead_id FK
        enum activityType
        uuid changed_by
        text old_value
        text new_value
    }

    LEAD_IMPORT_BATCHES {
        uuid id PK
        enum source_type
        uuid imported_by
        uuid owning_sales_head_id
        jsonb params
        int inserted_count
        int duplicate_count
        int invalid_count
    }

    LEAD_TAGS {
        uuid id PK
        varchar name
        varchar category
    }

API surface

All routes below are mounted under the /api prefix in src/routes/index.ts (both RawLeadRoutes and CampaignLeadRoutes are this.router.use('/api', ...)).

Campaign leads (campaignLead.routes.ts)

Method Path Auth/role Purpose
POST /api/campaign-leads/send-otp Public Send WhatsApp OTP for a website form
POST /api/campaign-leads/verify-otp Public Verify the OTP
POST /api/campaign-leads/submit Public Submit interest / application form (upsert)
GET /api/admin/campaign-leads Admin / SuperAdmin / SalesHead* List all campaign leads
GET /api/admin/campaign-leads/stats Admin / SalesHead* Funnel + status stats
GET /api/admin/campaign-leads/export Admin / SalesHead* CSV export
PATCH /api/admin/campaign-leads/bulk-assign Admin / SalesHead* Assign many to a salesperson
GET /api/admin/campaign-leads/:id Admin / SalesHead* One lead
PATCH /api/admin/campaign-leads/:id Admin / SalesHead* Update status / notes
PATCH /api/admin/campaign-leads/:id/assign Admin / SalesHead* Assign single lead
DELETE /api/admin/campaign-leads/:id Admin only Delete (heads forbidden)
GET /api/sales/campaign-leads Sales / Head / Admin / BatchLead My assigned campaign leads
GET /api/sales/campaign-leads/summary Sales+ My pipeline summary
PATCH /api/sales/campaign-leads/:id/status Sales+ (ownership-guarded) Update status on my lead
PATCH /api/sales/campaign-leads/:id/notes Sales+ (ownership-guarded) Update notes on my lead
GET /api/sales/campaign-leads/:id Sales+ (ownership-guarded) My single lead

* Sales-head access additionally gated by the canImportWebsiteCampaign permission (websiteCampaignPermissionGuard).

Raw leads — admin / sales-head (rawLead.routes.ts)

Access bundles: salesPage = authMiddleware + Admin/SuperAdmin/SalesHead + page permission (sales or crm); crmPage = CRM-only page permission; adminOnlyMiddleware = Admin/SuperAdmin; salesCrmAccessMiddleware additionally allows SALES.

Method Path Auth/role Purpose
GET /api/admin/raw-leads salesPage Paginated, filtered lead list
POST /api/admin/raw-leads salesPage Manual single "Add Lead"
POST /api/admin/raw-leads/bulk-import salesPage CSV bulk import
POST /api/admin/raw-leads/import-from-vendor salesPage Pull VendorLeads into the pool
POST /api/admin/raw-leads/import-from-campaign salesPage Pull CampaignLeads into the pool
DELETE /api/admin/raw-leads/:id salesPage Delete a lead
GET /api/admin/raw-leads/counts salesPage Lead counts by assignee
GET /api/admin/raw-leads/tag-counts salesPage Counts per tag
GET /api/admin/raw-leads/aarya-batches salesPage Aarya batch filter options
PATCH /api/admin/raw-leads/:id/assign salesPage Assign telecaller
PATCH /api/admin/raw-leads/bulk-assign salesPage Bulk assign telecaller
GET /api/admin/raw-leads/assignment-approvals salesPage Pending head approvals
GET /api/admin/raw-leads/assignment-approvals/count salesPage Approval badge count
POST /api/admin/raw-leads/assignment-approvals/:id/approve salesPage Approve a bulk assignment
POST /api/admin/raw-leads/assignment-approvals/:id/notify salesPage Notify the assignee
POST /api/admin/raw-leads/assignment-approvals/:id/dismiss salesPage Dismiss request
POST /api/admin/raw-leads/transfer-ownership adminOnly Move leads between head pools
POST /api/admin/raw-leads/pool/preview adminOnly Live count for a filter combo
POST /api/admin/raw-leads/pool/list adminOnly Matched leads for a filter combo
POST /api/admin/raw-leads/pool/assign adminOnly Transfer ownership by filter
POST /api/admin/raw-leads/pool/tag adminOnly Add tag by filter
POST /api/admin/raw-leads/tags/bulk adminOnly Add tag to selected ids
POST /api/admin/raw-leads/:id/tags adminOnly Add tag to one lead
DELETE /api/admin/raw-leads/:id/tags/:tagId adminOnly Remove tag from one lead
GET /api/admin/raw-leads/import-batches adminOnly Lead Pool batch overview
PATCH /api/admin/raw-leads/:id/interest salesPage Set interest level
PATCH /api/admin/raw-leads/:id/converted salesPage Mark converted
PATCH /api/admin/raw-leads/:id/close salesPage Close / admission-done
GET /api/admin/raw-leads/:id/activity salesPage Activity timeline
GET /api/admin/raw-leads/exotel-status rawLeadAccess Exotel feature status
PATCH /api/admin/raw-leads/my-call-number rawLeadAccess Set caller's click-to-call number
POST /api/admin/raw-leads/:id/exotel-call salesPage Click-to-call a lead
POST /api/admin/raw-leads/:id/sms salesPage Send SMS
GET /api/admin/raw-leads/:id/sms salesPage SMS history
GET /api/admin/raw-leads/:id/calls salesPage Call log history
POST /api/admin/raw-leads/:id/calls salesPage Log a manual call
GET /api/admin/raw-leads/:id/calls/:callId/recording salesPage Stream call recording
GET /api/admin/raw-leads/counsellor-counts crmPage Counts by counsellor
GET /api/admin/raw-leads/session-years salesPage Distinct session-year filter values
GET /api/admin/raw-leads/follow-ups crmPage Follow-up queue
GET /api/admin/raw-leads/follow-ups/counts crmPage Follow-up bucket counts
GET /api/admin/raw-leads/email-templates salesCrmAccess Outreach templates
GET /api/admin/sales/emails salesCrmAccess My recent email sends
GET /api/admin/raw-leads/:id salesPage Single lead detail
PATCH /api/admin/raw-leads/:id/counsellor crmPage Assign counsellor
PATCH /api/admin/raw-leads/:id/details salesPage Edit name/phone/webinar/extraData
GET /api/admin/raw-leads/:id/history salesPage Assignment history
GET / POST /api/admin/raw-leads/:id/follow-ups salesPage List / create follow-ups
GET / POST /api/admin/raw-leads/:id/emails salesCrmAccess Email history / send
GET /api/admin/raw-leads/:id/whatsapp-messages salesCrmAccess WhatsApp history
PATCH /api/admin/follow-ups/:id/done crmPage Complete a follow-up
PATCH /api/admin/follow-ups/:id/snooze crmPage Snooze a follow-up
PATCH /api/admin/follow-ups/:id/cancel crmPage Cancel a follow-up
POST / GET /api/admin/sales-targets crmPage Upsert / grid sales targets
GET /api/admin/sales-analytics/daily-calls crmPage Daily call pivot
GET /api/admin/sales-analytics/funnel crmPage Conversion funnel
GET / POST /api/admin/lead-tags salesPage List / create tags
PUT / DELETE /api/admin/lead-tags/:id salesPage Update / delete tag

Raw leads — self-scoped sales CRM (/api/sales/crm/*)

These auto-scope to the calling salesperson's assigned leads (myLeads filter) and are open to the SALES role (salesCrmAccessMiddleware).

Method Path Purpose
GET /api/sales/crm/follow-ups/counts My follow-up bucket counts
GET /api/sales/crm/my-day "Aaj ka kaam" daily summary
GET /api/sales/crm/dashboard-stats Overview-tab live stats
GET /api/sales/crm/exotel-status Exotel feature status
PATCH /api/sales/crm/my-call-number Set my click-to-call number
GET /api/sales/crm/tags Tags for the add-lead picker
GET /api/sales/crm/stats-bar Assigned / Target / Called / Due bar
GET / POST /api/sales/crm My lead list / add a lead (assigned to me)
GET /api/sales/crm/email-templates Outreach templates
GET /api/sales/crm/:id My single lead
PATCH /api/sales/crm/:id/interest Set interest
PATCH /api/sales/crm/:id/details Edit details
PATCH /api/sales/crm/:id/close Close / admission-done
GET / POST /api/sales/crm/:id/calls Call history / log call
POST /api/sales/crm/:id/exotel-call Click-to-call
GET /api/sales/crm/:id/calls/:callId/recording Stream recording
GET /api/sales/crm/:id/history Assignment history
GET /api/sales/crm/:id/activity-logs Activity timeline
GET / POST /api/sales/crm/:id/follow-ups Follow-ups
GET / POST /api/sales/crm/:id/emails Email history / send
GET /api/sales/crm/:id/whatsapp-messages WhatsApp history
POST /api/sales/crm/:id/whatsapp Send WhatsApp
PATCH /api/sales/follow-ups/:id/done Complete follow-up
PATCH /api/sales/follow-ups/:id/cancel Cancel follow-up

User journeys

1. Bulk CSV import → de-dupe → working pool

An admin or sales head uploads a parsed CSV. RawLeadService.bulkImport (src/services/RawLeadService.ts) validates the required trio (name, sessionYear, phone), de-dupes against existing phones and within the batch, writes one LeadImportBatch audit row even when everything de-dupes, stamps source_type = csv, and fires lead.created events for the workflow engine.

sequenceDiagram
    participant FE as CRM upload UI
    participant API as POST /api/admin/raw-leads/bulk-import
    participant SVC as RawLeadService.bulkImport
    participant DB as mas_crm raw_leads
    participant EVT as LeadEventService
    participant Q as BullMQ workflow queue

    FE->>API: rows plus tagIds plus owningSalesHeadId
    API->>SVC: bulkImport input
    SVC->>SVC: drop rows missing name phone or sessionYear
    SVC->>DB: find existing leads by phone
    SVC->>SVC: skip duplicate phones in batch and in DB
    SVC->>DB: insert LeadImportBatch audit row
    SVC->>DB: save new raw_leads in chunks of 200
    SVC->>EVT: emitMany lead.created for inserted ids
    EVT->>Q: enqueue workflow event job best-effort
    SVC-->>API: inserted skipped invalid counts
    API-->>FE: summary with skippedPhones

2. Website campaign form → OTP-verified CampaignLead → imported into the pool

A prospect on a public promo page submits their phone, verifies via WhatsApp OTP, then submits the interest or application form. The row lands in campaign_leads (a shared pre-import pool). Later, an admin pulls a filtered slice into raw_leads via importFromCampaign.

sequenceDiagram
    participant V as Website visitor
    participant API as campaignLead.routes
    participant SVC as CampaignLeadService
    participant R as Redis OTP store
    participant WA as WhatsApp provider
    participant DB as campaign_leads

    V->>API: POST send-otp with campaign and phone
    API->>SVC: sendOtp
    SVC->>DB: short-circuit if phone already verified for programme
    SVC->>R: store 6-digit OTP with TTL and cooldown
    SVC->>WA: send OTP template message
    SVC-->>V: nextResendInSeconds
    V->>API: POST verify-otp with otp
    API->>SVC: verifyOtp
    SVC->>R: compare code and set verified flag
    SVC-->>V: verified
    V->>API: POST submit with name and formType
    API->>SVC: submit
    SVC->>R: confirm verified flag or DB phoneVerified
    SVC->>DB: upsert campaign lead and advance status to applied
    SVC-->>V: submitted successfully
sequenceDiagram
    participant A as Admin CRM
    participant API as POST import-from-campaign
    participant SVC as RawLeadService.importFromCampaign
    participant CL as campaign_leads
    participant DB as raw_leads
    participant EVT as LeadEventService

    A->>API: campaignTypes statuses dateFrom sessionYear tagIds
    API->>SVC: ImportFromSourceInput
    SVC->>CL: query matching campaign leads
    SVC->>DB: identity-dedupe on campaign_lead_id
    SVC->>DB: phone-dedupe across all sources
    SVC->>DB: insert raw_leads with source_type website
    SVC->>DB: write LeadImportBatch with replayable params
    SVC->>EVT: emitMany lead.created
    SVC-->>A: total inserted duplicates invalid

For a dryRun request the service returns counts only and inserts nothing — used by the import preview UI.

3. Vendor-captured lead → imported into the pool

Vendor API clients push leads into vendor_leads (see Vendor API platform). An admin imports a vendor slice with importFromVendor; the shared runImport loop maps vendor fields into extraData, stamps vendor_lead_id (unique) and vendor_api_key_id. Status changes later flow back to the vendor via syncVendorLeadStatus.

sequenceDiagram
    participant A as Admin CRM
    participant API as POST import-from-vendor
    participant SVC as RawLeadService.importFromVendor
    participant VL as vendor_leads
    participant DB as raw_leads

    A->>API: vendorIds statuses dateRange sessionYear
    API->>SVC: ImportFromSourceInput
    SVC->>VL: query vendor leads for the given keys
    SVC->>SVC: buildVendorExtra maps guardian course jee fields
    SVC->>DB: dedupe on vendor_lead_id then phone
    SVC->>DB: insert raw_leads source_type vendor
    SVC->>DB: write LeadImportBatch
    SVC-->>A: import result counts

4. Log a manual call on a lead

A telecaller dials and records the outcome. RawLeadService.logCall saves a LeadCallLog row and atomically bumps the denormalized last_contacted_at and call_count on the lead in one transaction.

sequenceDiagram
    participant FE as Lead detail page
    participant API as POST raw-leads/:id/calls
    participant SVC as RawLeadService.logCall
    participant DB as mas_crm

    FE->>API: outcome durationSec notes
    API->>SVC: leadId and LogCallInput
    SVC->>DB: begin transaction
    SVC->>DB: insert lead_call_logs row source manual
    SVC->>DB: update raw_leads set last_contacted_at and call_count plus one
    SVC->>DB: commit
    SVC-->>API: saved call log
    API-->>FE: updated call history

Aarya AI calls and Exotel click-to-call create LeadCallLog rows with source = aarya / exotel and an initially-unknown outcome (aarya_dispatched / exotel_dispatched); the outcome, duration and recording URL are backfilled later by a sync poll or the Exotel status webhook. See telephony.

5. Send an outreach email (queued + worker-resolved)

Email sends are two-phase: the synchronous request writes a LeadEmailLog in queued state and enqueues a BullMQ job; the email worker delivers it and flips the row to sent or failed.

sequenceDiagram
    participant FE as Lead detail page
    participant API as POST raw-leads/:id/emails
    participant SVC as LeadEmailService.send
    participant DB as lead_email_logs
    participant Q as BullMQ emailQueue
    participant W as email.worker

    FE->>API: templateKey track to subject body
    API->>SVC: SendInput
    SVC->>SVC: resolve recipient from track and validate address
    SVC->>SVC: interpolate template if no override
    SVC->>DB: insert log status queued
    SVC->>Q: addEmailJob type crm-lead-email
    SVC-->>FE: queued log row
    W->>Q: pick up crm-lead-email job
    W->>W: deliver via EmailService
    alt delivered
        W->>SVC: markSent
        SVC->>DB: status sent
    else failed
        W->>SVC: markFailed with reason
        SVC->>DB: status failed and errorMessage
    end

If enqueue itself throws, the row is immediately flipped to failed so a later re-send is clean.

6. Send an SMS / WhatsApp message

SMS goes through ExotelLeadService.sendSms which both dispatches via Exotel and writes a LeadSmsLog. WhatsApp goes through the shared WhatsAppService to the Meta/AiSensy provider; the controller then records a LeadWhatsAppLog on both success and failure so the timeline always reflects the attempt.

sequenceDiagram
    participant FE as Lead detail page
    participant API as POST sales/crm/:id/whatsapp
    participant CTRL as rawLead.controller.sendWhatsApp
    participant WA as WhatsAppService
    participant PROV as Meta or AiSensy
    participant SVC as LeadWhatsAppService
    participant DB as lead_whatsapp_logs

    FE->>API: templateName params or messageBody
    API->>CTRL: assert lead in scope
    CTRL->>WA: sendMessage to lead phone
    WA->>PROV: deliver template
    alt provider accepted
        PROV-->>WA: externalMessageId
        CTRL->>SVC: recordSend status sent
    else provider error
        CTRL->>SVC: recordSend status failed with reason
    end
    SVC->>DB: insert whatsapp log
    CTRL-->>FE: result

7. Schedule and complete a follow-up

A salesperson schedules a callback. LeadFollowUpService.create defaults the assignee to the lead's counsellor or telecaller. Completing it (markDone) stamps completed_at, completed_by and a note. Closing the lead cancels all its pending follow-ups.

sequenceDiagram
    participant FE as CRM
    participant API as POST raw-leads/:id/follow-ups
    participant SVC as LeadFollowUpService
    participant DB as lead_follow_ups

    FE->>API: scheduledFor reason
    API->>SVC: create
    SVC->>DB: verify lead exists
    SVC->>DB: insert follow-up assignee defaults to counsellor or telecaller status pending
    SVC-->>FE: follow-up row
    Note over FE,DB: later the assignee works the queue
    FE->>API: PATCH follow-ups/:id/done with note
    API->>SVC: markDone
    SVC->>DB: status done completed_at completed_by note
    SVC-->>FE: completed

8. Tag / segment leads

Tags (name + category) segment the pool. A tag can be added per-lead, to a selected set of ids, or to everything matching a Lead Pool filter combination. Only newly tagged leads get a tag_added activity entry; the join-table insert is ON CONFLICT DO NOTHING.

sequenceDiagram
    participant A as Admin Lead Pool
    participant API as POST raw-leads/pool/tag
    participant SVC as RawLeadService.addTagByFilter
    participant DB as mas_crm

    A->>API: filter combo plus tagId
    API->>SVC: addTagByFilter
    SVC->>SVC: refuse an empty filter to prevent tag-all
    SVC->>DB: resolve matching lead ids
    loop chunks of 500
        SVC->>DB: find which ids lack the tag
        SVC->>DB: insert raw_lead_tags on conflict do nothing
        SVC->>DB: insert lead_activity_log tag_added for fresh ids
    end
    SVC-->>A: count of newly-tagged leads

9. Lead lifecycle status changes → activity log + workflow events

Every status mutation (interest, converted, closed, webinar) writes a LeadActivityLog row in the same transaction and emits a typed lifecycle event onto the workflow queue. Converted and Closed are mutually exclusive terminal states; setting one clears the other.

sequenceDiagram
    participant FE as CRM
    participant API as PATCH raw-leads/:id/close
    participant SVC as RawLeadService.setClosed
    participant DB as mas_crm
    participant EVT as LeadEventService

    FE->>API: closed true with reason
    API->>SVC: setClosed
    SVC->>DB: begin transaction
    SVC->>DB: set closed_at closed_by closed_reason and clear converted_at
    SVC->>DB: cancel pending follow-ups for the lead
    SVC->>DB: insert lead_activity_log closed_changed
    SVC->>DB: commit
    SVC->>EVT: emit lead.closed
    SVC-->>FE: updated lead

Background jobs & async

  • Email queue (emailQueue, email.worker.ts): CRM outreach emails are enqueued by LeadEmailService.send with type: 'crm-lead-email' and leadEmailLogId. The worker (case crm-lead-email at src/workers/email.worker.ts) delivers via EmailService and calls LeadEmailService.markSent / markFailed. Jobs do not auto-retry the log state machine; a re-send re-queues.
  • Workflow event queue (LeadEventService → QueueService.addWorkflowEventJob): every lead lifecycle mutation publishes a LeadEventType (lead.created, lead.assigned, lead.counsellor_assigned, lead.owner_changed, lead.interest_changed, lead.webinar_changed, lead.converted, lead.closed). Emits are fire-and-forget and fully swallowed — a queue hiccup never breaks the core lead operation. Bulk operations use emitMany. Consumed by the workflow automation engine.
  • Aarya AI call sync: LeadCallLog rows created with source = aarya start as aarya_dispatched; a polling sync (AaryaCallSyncService) backfills aarya_conversation_id, duration and outcome. The CRM derives an AI-predicted interest bucket from the most recent Aarya call duration.
  • Exotel status webhook: click-to-call (exotel_dispatched) and SMS rows are resolved to a terminal status, duration and recording URL by the Exotel status callback. See telephony.
  • OTP store (Redis): CampaignLeadService keeps OTP codes, resend cooldowns, send-count windows, and a short-lived "verified" flag in Redis (not Postgres).

No dedicated follow-up reminder cron worker exists in this domain on this branch; LeadFollowUp.reminderSentAt is provisioned for one but the queue-side scheduler is not wired here (inferred from absence in src/workers/).

External integrations

Integration Used by Env / config Failure behavior
WhatsApp provider (Meta Cloud API / AiSensy) Campaign-lead OTP, CRM WhatsApp sends WHATSAPP_OTP_TEMPLATE_NAME, WHATSAPP_OTP_TEMPLATE_LANG + provider creds sendOtp deletes the stored OTP and returns a soft failure; if no provider configured, OTP send is refused. CRM WhatsApp failure is logged to LeadWhatsAppLog with status = failed
Exotel (click-to-call + SMS) ExotelLeadService EXOTEL_* (see backend CLAUDE.md) Feature self-disables when unset; CRM falls back to tel: links. Status resolved by webhook
Gmail SMTP (Nodemailer) CRM outreach emails via worker EMAIL_USER, EMAIL_PASS Worker flips LeadEmailLog to failed with the error message
Redis OTP state, cooldowns REDIS_HOST, REDIS_PORT OTP flow unavailable if down
Vendor API platform importFromVendor, syncVendorLeadStatus n/a (internal) Loose FK; deleting a vendor key never cascades into CRM data

Feature gates: sales-head access to website campaign leads requires the canImportWebsiteCampaign permission; page-level access uses requireHeadPagePermission('sales'|'crm'). Heads without a profile row are grandfathered all-permissions-true.

Status lifecycles

Derived RawLead lifecycle bucket

RawLead has no stored status column. The CRM derives a bucket from converted_at, closed_at, interest_level and call_count (see ListInput.leadStatus doc-comment in RawLeadService.ts):

stateDiagram-v2
    [*] --> New
    New --> Untouched : assigned but never called
    New --> Contacted : first call logged
    Untouched --> Contacted : first call logged
    Contacted --> Qualified : interest set warm or hot
    New --> Qualified : interest set warm or hot
    Qualified --> Converted : mark Admission Done
    Contacted --> Converted : mark Admission Done
    Contacted --> Closed : mark Closed lost
    Qualified --> Closed : mark Closed lost
    New --> Closed : mark Closed lost
    Converted --> Closed : re-close clears converted
    Closed --> Converted : convert clears closed
    Converted --> [*]
    Closed --> [*]

Bucket rules: converted = converted_at set; qualified = not converted and interest in (warm, hot); contacted = not converted, call_count > 0, interest null; new = not converted, call_count = 0, interest null; untouched = telecaller assigned but call_count = 0; closed = closed_at set. Converted and Closed are mutually exclusive.

CampaignLead status

stateDiagram-v2
    [*] --> NEW
    NEW --> APPLIED : application form submitted
    NEW --> CONTACTED : admin marks contacted
    NEW --> NOT_INTERESTED : admin marks
    APPLIED --> CONTACTED : admin marks contacted
    CONTACTED --> CONVERTED : admin marks converted
    CONTACTED --> NOT_INTERESTED : admin marks
    CONVERTED --> [*]
    NOT_INTERESTED --> [*]

submit() only advances NEW → APPLIED; it never downgrades a later admin-set state.

LeadFollowUp status

stateDiagram-v2
    [*] --> pending
    pending --> done : markDone
    pending --> cancelled : cancel or lead closed
    pending --> pending : snooze reschedules
    pending --> missed : overdue (display-derived)
    done --> [*]
    cancelled --> [*]

Email / WhatsApp send status

stateDiagram-v2
    [*] --> queued
    queued --> sent : worker delivered
    queued --> failed : enqueue or delivery error
    sent --> [*]
    failed --> [*]

Edge cases, limits & gotchas

  • Phone is the universal dedupe key. RawLead never stores two rows with the same phone, regardless of source. Imports dedupe on both the source-row identity (vendor_lead_id / campaign_lead_id, unique-indexed) and phone. Manual createOne throws 409 on a phone collision.
  • No stored lead status. The lifecycle bucket is computed at query time; do not look for a status column on raw_leads.
  • Two-layer assignment. assigned_to (telecaller) and assigned_counsellor (counsellor) are independent; both are journaled in LeadAssignmentHistory with role telly_caller / counsellor. Bulk assigns write history with fromUserId = null (cheap path).
  • Data isolation. Sales-head pools are isolated by owning_sales_head_id. The SALES role can only reach the self-scoped /sales/crm/* routes, which force a myLeads filter; the controller assertLeadInScope guards :id access.
  • Route ordering matters. Literal collection routes (/follow-ups, /counts, /email-templates, /bulk-assign, /exotel-status) are registered before :id capture routes so Express does not match them as ids.
  • Events are best-effort. LeadEventService swallows all queue errors — a workflow/queue outage must never block a create/assign/status change.
  • Email body is frozen at send time. LeadEmailLog.body stores the compiled HTML actually sent, because templates are editable; the row is auditable even if the template changes later.
  • CRM WhatsApp uses its own table. lead_whatsapp_logs is intentionally separate from the HR-hiring whatsapp_messages table so cross-schema CRM queries stay simple.
  • Closing cancels follow-ups. setClosed(true) cancels all pending follow-ups so the lead leaves active queues.
  • Vendor status mirror. Updating interest/webinar status on a vendor-sourced lead calls syncVendorLeadStatus, mapping CRM state back to VendorLeadStatus so the vendor's own API view stays fresh.
  • bulkAssign null-clear quirk. TypeORM save() treats undefined as no-op, so clearing an assignment writes NULL explicitly via QueryBuilder .update() (see CampaignLeadService.assign).
  • Duplicate row in role checks. Several middleware compare role !== UserRole.ADMIN twice (a copy/paste artifact where SUPERADMIN was likely intended); behavior is unchanged but worth noting when editing.
  • CSV extraData field control. Vendor/campaign imports support a per-field whitelist (extraFields) that takes precedence over the legacy extraFieldMode: 'all' | 'none' toggle.