Skip to content

Prompt Suggestion — Service Documentation

Overview

The prompt-suggestion service generates suggested prompts for brands. It is a self-contained FastAPI microservice that lives entirely inside apps/prompt_suggestion_service/.

Given a brand's context the service:

  1. Retrieves the top-N most semantically similar reference prompts from a vector database, filtered by industry, intent tag, and branded tag.
  2. Synthesizes clean, well-formed suggested prompts by sending a rendered prompt template to the LLM microservice in apps/ai-service and parsing the structured JSON output.

Both steps are exposed through POST /suggestions. The apps/ai-service LLM microservice is intentionally a thin completion wrapper: it accepts a prompt string and returns the raw output string. All template construction, output parsing, and tag-modifier logic lives in this service — apps/ai-service never sees the brand schema, the references, or the structured response shape.

This service covers the request-time system only. The offline pipeline that ingests, cleanses, embeds, and clusters the 50M-prompt clickstream dataset is a separate workstream and is not implemented here.

Scope

Everything in this project lives in apps/prompt_suggestion_service/. External services (e.g. apps/server) are consumers — they call POST /suggestions and receive the final generated prompts.

Endpoints:

  • POST /suggestions — primary endpoint; retrieves references and synthesizes prompts in one call. External consumers only ever call this.
  • POST /retrieve — internal helper for testing; returns top-N references only.
  • POST /synthesize — internal helper for testing; takes a supplied list of references and returns synthesized prompts.
  • GET /health — liveness check.

Out of scope:

  • The actual LLM call (delegated to apps/ai-service)
  • The offline ingestion, cleansing, clustering, and embedding pipeline
  • Any frontend or UI
  • Database migrations — the service is intended to be read-only against the shared PostgreSQL database; schema ownership and migrations belong to apps/server via Prisma
  • Authentication (removed for now; reintroduce once the scheme is decided)
  • Structured logging / request middleware / readiness checks (removed for now; reintroduce as needed)

Architecture

[External Consumer (e.g. apps/server)]
        │ POST /suggestions
        ▼
[apps/prompt_suggestion_service (FastAPI)]
        │
        ├──► [SuggestionService]
        │         │
        │         ├──► [Embedder] ──► [OpenAI Embeddings (default)]
        │         ├──► [VectorStore] ──► [pgvector (default, currently a stub)]
        │         └──► [LLMOrchestrator]
        │                   ├──► [PromptBuilder + tag_modifiers]   (local)
        │                   ├──► HTTP ──► [apps/ai-service /complete]
        │                   └──► [SuggestedPromptListParser]       (local)
        │
        └── /retrieve  → Embedder + VectorStore only
        └── /synthesize → LLMOrchestrator only

Request flow for POST /suggestions:

  1. Consumer sends {brand, industry, existing_prompts?, tag_filters?, limit?}.
  2. SuggestionService embeds "{brand} {industry}" via the Embedder and calls VectorStore.similarity_search(...) to pull top-N references (currently limit * 5).
  3. SuggestionService calls LLMOrchestrator.synthesize(...):
  4. PromptBuilder.build(...) renders the template with brand context, reference list, and any tag-modifier fragments.
  5. The orchestrator POSTs {"prompt": <rendered>} to {AI_SERVICE_URL}/complete.
  6. SuggestedPromptListParser.parse(...) extracts the JSON array from the completion text and validates it against SuggestedPrompt.
  7. Response: {suggestions: [...], meta: {reference_count, llm_model, duration_ms}}.

When LLM_PROVIDER=mock, the orchestrator short-circuits to deterministic dummy prompts and never opens an HTTP connection — useful for local dev when apps/ai-service isn't running.

Directory structure

apps/prompt_suggestion_service/
├── pyproject.toml              # uv-managed; single source of truth for deps
├── Dockerfile
├── .env.example
├── docs/
│   └── documentation.md        # This file
└── app/                        # Python package root
    ├── __init__.py
    ├── main.py                 # FastAPI app + router registration + /health
    ├── settings.py             # pydantic-settings config — use this, not os.getenv
    ├── schemas/                # Pydantic v2 request/response models
    │   ├── __init__.py
    │   ├── shared.py           # IntentTag, BrandedTag, TagFilter, HealthResponse
    │   ├── retrieval.py        # RetrieveRequest, ReferencePrompt, RetrieveResponse
    │   ├── synthesis.py        # SynthesizeRequest, SuggestedPrompt, SynthesizeResponse
    │   └── suggestions.py      # SuggestionRequest, ResponseMeta, SuggestionResponse
    ├── api/                    # FastAPI routers
    │   ├── __init__.py
    │   ├── suggestions.py      # POST /suggestions  ← primary external endpoint
    │   ├── retrieve.py         # POST /retrieve     ← internal helper
    │   └── synthesize.py       # POST /synthesize   ← internal helper
    ├── services/               # Business logic
    │   ├── __init__.py
    │   └── suggestion_service.py   # Orchestrates retrieval + synthesis
    ├── adapters/               # Pluggable infrastructure interfaces
    │   ├── __init__.py
    │   ├── vector_store.py     # VectorStore ABC + PgVectorStore (stub)
    │   └── embedder.py         # Embedder ABC + OpenAIEmbedder + NoOpEmbedder
    └── llm/                    # Synthesis composition (everything LLM-adjacent
                                #   except the call itself)
        ├── __init__.py
        ├── tag_modifiers.py    # IntentTag/BrandedTag → instruction fragments
        ├── prompt_builder.py   # PromptBuilder.build() — base template assembly
        ├── output_parser.py    # JSON output parser → list[SuggestedPrompt]
        └── orchestrator.py     # builder + httpx call + parser

LLM service contract

The LLM call is delegated to apps/ai-service. The current placeholder contract — to be confirmed with the apps/ai-service team — is:

POST {AI_SERVICE_URL}/complete
Content-Type: application/json
Body:     {"prompt": "<string>"}
Response: {"output": "<string>"}

Update _COMPLETE_PATH in app/llm/orchestrator.py (and this section) once the real shape is known.

Data model

The service is intended to have read-only access to the shared PostgreSQL database. The canonical schema is defined in apps/server/prisma/schema.prisma; the Pydantic models in app/schemas/ mirror it for read access.

The DB connection is not yet wired — PgVectorStore.similarity_search() currently returns an empty list. The connection pool and readiness checks were removed during the early-stage cleanup and will be reintroduced when the vector store implementation lands.

Tables

Topic — a single search dimension; prompts belong to at most one topic.

Column Type Notes
id String cuid primary key
name String
industry String indexed
created_at DateTime

Cluster — groups semantically similar prompts; produced by the offline pipeline.

Column Type Notes
id String cuid primary key
topic_id String FK → Topic
size Int number of prompts in cluster
dominant_intent IntentTag
dominant_branded BrandedTag
created_at DateTime

The cluster centroid embedding lives in the vector store, referenced by cluster id.

Prompt — a real cleansed clickstream prompt (used as a synthesis reference).

Column Type Notes
id String cuid primary key
cluster_id String FK → Cluster
text String
intent_tag IntentTag
branded_tag BrandedTag
language String
frequency Int default 1
is_truncated Boolean default false
created_at DateTime

Enums

IntentTag:  INFORMATIONAL | COMMERCIAL | TRANSACTIONAL
BrandedTag: BRANDED | NON_BRANDED

Synthesis prompt design

Base prompt template (app/llm/prompt_builder.py)

You are generating clean, well-formed search prompts that real buyers would type
into AI search tools, based on real-user reference prompts from a clickstream.

Brand: {brand}
Industry: {industry}
Existing prompts already tracked:
{existing_prompts}

Reference prompts from real users (these may be messy, multilingual, or truncated):
{reference_prompts}

{tag_modifier_fragments}

Generate {n} distinct prompts. Each prompt must:
- Be well-formed and free of typos
- Represent a different angle or stage within the topic (research, comparison, decision)
- Not duplicate the existing tracked prompts
- Be in English

Return as a JSON array, one object per prompt:
[{"prompt": "...", "topic": "...", "intent_tag": "INFORMATIONAL|COMMERCIAL|TRANSACTIONAL", "branded_tag": "BRANDED|NON_BRANDED", "source_cluster_id": "..."}]

Tag modifier fragments (app/llm/tag_modifiers.py)

Tag Fragment injected into prompt
INFORMATIONAL Phrase prompts as research-stage questions seeking information.
COMMERCIAL Phrase prompts as comparison-stage queries evaluating options.
TRANSACTIONAL Phrase prompts as decision-stage queries with clear buying intent.
BRANDED Each prompt must mention the brand: {brand}.
NON_BRANDED Prompts must NOT mention any brand name.

Deferred decisions (⚠ research items for meta team)

  1. Vector DB — Default: pgvector (no extra infra). Alternatives: Qdrant, Pinecone, Weaviate. Abstracted behind VectorStore in app/adapters/vector_store.py.

  2. Embedding model — Default: OpenAI text-embedding-3-small. Document embeddings are pre-computed offline; the Embedder interface is used only to embed the incoming query context at request time. Abstracted behind Embedder in app/adapters/embedder.py. EMBEDDING_PROVIDER=none activates NoOpEmbedder for local dev without an API key.

  3. LLM service endpoint — Placeholder is POST {AI_SERVICE_URL}/complete with {"prompt": str} → {"output": str}. Confirm exact shape with the apps/ai-service team and update _COMPLETE_PATH in app/llm/orchestrator.py to match.

  4. Inter-service auth — Currently no auth. Confirm with security/meta team what scheme to use (API key, mTLS, OAuth2, …) before exposing publicly.

Environment variables

See .env.example for the full list with descriptions.

Variable Default Purpose
LLM_PROVIDER mock mock for dummy data; remote to call apps/ai-service over HTTP
LLM_MODEL gpt-4o-mini Label echoed in response metadata
AI_SERVICE_URL — Base URL of apps/ai-service; required when LLM_PROVIDER=remote
EMBEDDING_PROVIDER openai Embedding provider (openai or none)
EMBEDDING_MODEL text-embedding-3-small Embedding model identifier
OPENAI_API_KEY — Required when EMBEDDING_PROVIDER=openai
VECTOR_STORE pgvector Vector store backend
REQUEST_TIMEOUT_MS 30000 Per-upstream-call timeout in ms (applied to the apps/ai-service POST)

Common commands

# Sync dependencies
uv --project apps/prompt_suggestion_service sync

# Run locally (from repo root)
uv --project apps/prompt_suggestion_service run uvicorn app.main:app --reload --port 8000

# Run tests
uv --project apps/prompt_suggestion_service run pytest

# Lint + format check
uv --project apps/prompt_suggestion_service run ruff check apps/prompt_suggestion_service
uv --project apps/prompt_suggestion_service run ruff format --check apps/prompt_suggestion_service

FastAPI auto-generates docs at:

  • Swagger UI: http://localhost:8000/docs
  • ReDoc: http://localhost:8000/redoc
  • Raw JSON: http://localhost:8000/openapi.json