Extraction Pipeline

Extraction Pipeline

Detailed pipeline order for both initial and event extraction, including which extractors make LLM calls and which are computed locally.

Initial Extraction Pipeline

Runs on the first message to create the initial snapshot.

#ExtractorCategoryLLM Call?Dependencies
1TimetimeYes (unless card default)None
2LocationlocationYes (unless card default)None
3ForecastclimateNo (computed)Location + Time
4Characters PresentcharactersYesNone
5Character ProfilescharactersYes (per character)Characters Present
6Character OutfitscharactersYes (per character)Characters Present
7RelationshipsrelationshipsYesCharacters Present
8PropspropsYesLocation + Character Outfits
9Topic & TonesceneYesNone
10TensionsceneYesTopic & Tone

Total LLM calls: 6 + (2 × number of characters) minimum

For example, with 2 characters: 6 + 4 = 10 LLM calls for initial extraction.

Card defaults can reduce this:

  • Time from card → skips step 1 (-1 call)
  • Location from card → skips step 2 (-1 call)

Event Extraction Pipeline

Runs on every subsequent message. Organized in phases.

Phase 1: Core State

ExtractorCategoryLLM Call?Notes
Time ChangetimeYes
Location ChangelocationYes
ForecastclimateNo (computed)Triggers on area change or time exceeding range
Topic/Tone ChangesceneYes
Tension ChangesceneYes

Phase 2: Character Presence

ExtractorCategoryLLM Call?Notes
Presence ChangecharactersYesDetects appeared/departed

Phase 3: Per-Character State

Runs once per present character:

ExtractorCategoryLLM Call?Notes
Position/Activity ChangecharactersYesCombined in one call
Mood ChangecharactersYes
Outfit ChangecharactersYes

Phase 4: Props

ExtractorCategoryLLM Call?Notes
Props ChangepropsYesAfter outfits (clothing ↔ props)
Props ConfirmationpropsYesConfirms props still present

Phase 5: Relationship Subjects (Global)

ExtractorCategoryLLM Call?Notes
Subject DetectionrelationshipsYesIdentifies interaction types

Phase 6: Per-Pair Relationships

Runs once per character pair:

ExtractorCategoryLLM Call?Notes
Relationship ChangerelationshipsYesUpdates feelings/wants/secrets/status

Phase 7: Narrative

ExtractorCategoryLLM Call?Notes
Narrative DescriptionnarrativeYesBrief summary of what happened
Milestone DescriptionnarrativeYes (conditional)Only for first-occurrence milestone subjects

Phase 8: Chapters

ExtractorCategoryLLM Call?Notes
Chapter EndednarrativeYesDetects chapter boundaries
Chapter DescriptionnarrativeYes (conditional)Only when chapter ends

Total LLM Calls per Message

With all modules enabled, N characters, and P pairs (P = N×(N-1)/2):

Core:           4  (time + location + topic/tone + tension)
Presence:       1
Per-character:  3N (position/activity + mood + outfit per character)
Props:          2  (change + confirmation)
Subjects:       1
Per-pair:       P
Narrative:      1-2 (description + optional milestone)
Chapters:       1-2 (detection + optional description)
────────────────────
Total:          10 + 3N + P  (minimum, all modules on)

Examples:

  • 2 characters, 1 pair: 10 + 6 + 1 = 17 calls
  • 3 characters, 3 pairs: 10 + 9 + 3 = 22 calls
  • 4 characters, 6 pairs: 10 + 12 + 6 = 28 calls

Reducing Call Count

Disable modules you don’t need:

DisableCalls Saved
props2 per message
scene (topic/tone + tension)2 per message
narrative (events + chapters)2-4 per message
relationships1 + P per message
characters1 + 3N per message

shouldRun Logic

Not every extractor runs every time. Each has a shouldRun check:

  • Track disabled → extractor is skipped entirely
  • Run strategy → some extractors only run on certain conditions (e.g., forecast only when area changes)
  • No relevant changes → some extractors check if upstream state actually changed

Computed vs Extracted

Some state is computed locally rather than extracted by the LLM:

StateSource
Climate (temperature, conditions, etc.)Computed from forecast + time + location
ForecastComputed from climate normals + seeded RNG
Daylight phaseComputed from latitude + time
Indoor temperatureComputed from outdoor temp + building type
Narrative event witnessesDerived from projection state at each message
Chapter boundaries (partial)Detected from location changes + time jumps