DIG-574 Mart Design: Mary’s questions in CONFORMED_DM
Mart Design Document
Template version: 2 | Updated: 2026-10-11 UTC
Branch Delivery Document - Level 3 - Design Document
| Effort | DIG-574 - DIGEST course: Data Vault (datavault4dbt) - branch claude_ws/dig-574-arrival-and-identity; built on the fix to release 1 (DIG-594), branch claude_ws/dig-594-release-1-keys-rebased-2 |
| Governed by | Standard: Branch Delivery Documents (knowledge 12, item 27); Data Mart Design Principles, v4 (knowledge 12, item 104); the workgroup datamart standard (knowledge 17, item 118). |
| References | Profiling: branch profile (source ERD, 2.1 to 2.6); related records: branch design (consumer questions Q-7 to Q-16, D-1 to D-52); QA item (DIG-594); DIG-583 (the connector); test plan: branch test plan , its mart section 2.3. |
| Audience | Ray (reviewer and approver); Mary Mink, the consumer, through DV Digest |
| Author | Ray Serrano; edited by ChatGPT - 2026-10-27 |
| Classification | Internal - Data Management |
| Official copy method | Instantiate this governed template through AGENTIC.PILOT.KNOWLEDGE_ITEM_FROM_TEMPLATE; retain the source template number and SHA-256 recorded by the procedure. |
How to complete this template
This template specializes the Branch Delivery Design Document template (knowledge 12, item 175) for mart design. The design owns WHAT and WHY, including all source-to-column mappings. The implementation guide owns HOW; the test plan owns the detailed runnable proof. Neither adds an unresolved design decision.
Complete the workflow in order: consumer questions and metric definitions; required grains and existing-fact search; missing measures and dimensions; MetricFlow definitions and consumer views; validation and approval. Read the existing branch code and warehouse objects before declaring an object reusable, insufficient, or absent. Use current evidence with its environment and clock. Do not propose new physical objects before completing the metric, grain, and existing-fact analysis. Record not applicable with a reason instead of omitting a required field. Replace every placeholder before instantiation.
A measure describes one row at a declared fact grain. A metric answers a consumer’s question over a population. A calculated value can still be a measure at the row grain. A qualifier describes, filters, or groups the rows and belongs in a dimension. For example, an amount applied by one payment to one invoice is a measure; its total across the selected population is a metric; payment status is a qualifier.
0. Target ERD
The mart this design builds, drawn from section 2’s grains and section 3’s maps (Standard: Branch Delivery Documents v6.2, section 0): four facts, one per grain (client_policy_role_fact, agent_policy_fact, policy_terms_fact, finding_fact), each carrying its keys, its event dates and, for policy_terms_fact, the one measure; the qualifiers live in the dimensions. People and organizations follow the party model (D-10, D-11): two base dimensions, person_dim and organization_dim, with role views over them, client_dim, insured_dim and agent_dim over person_dim and carrier_dim over organization_dim, each drawn joined to its base; policy_dim, source_dim and day_dim complete the dimensions. Each event date is on its fact as a date that is also its key to day_dim in its role (sold, first known, valid from, valid to, effective, detected), drawn as a note rather than lines; agent_policy_fact, person_dim, organization_dim and policy_dim also refer to source_dim, left undrawn for the same reason. The insured’s birth date and address are shown under the masking mock of D-9. Every line holds by construction from the vault’s keys and is proved by section 5.3’s dimension integrity tests.
1. Consumer questions and metrics
1.1 Purpose, scope, and concept
The consumer is Mary Mink, Director of Data Architecture and Governance, asking her predicted questions from her own Claude through the DV Digest connector, which serves governed metrics only (list-metrics, get-dimensions, query-metrics: a metric and its group-by, no filter). The surface supports her first look at what the acquired agency’s and the carrier’s data say about clients, policies and agents, and how much of it arrives in a form the vault rejects.
Scope: the six questions the raw vault answers today, Q-9, Q-10, Q-11, Q-13, Q-14 and Q-15. Out of scope: Q-7, Q-8 and Q-16, which need the data integration layer and a second acquired agency; Q-12 is deferred until its rule is decided (DD-1). The concepts: a client (a client number Sunbelt Senior sends), a policy (a carrier’s policy number), an agent (a national producer number), a role (what a client is on a policy: owner, insured, payor), a version of a policy’s terms (what the carrier said about it from one load until the next), and a finding (a record an inspector or auditor flagged).
This design replaces the five MART views built on 2026-10-26 and 27 without a design (client_policy_fact, agent_policy_fact, policy_fact, finding_fact, and time_spine_day), which break the datamart form: facts carrying qualifiers, no dimensions, timestamps in place of day keys, and the sunsetting MART schema. Their numbers were right; their shape is retired here (D-6).
Name the consumer, the decision the surface supports, and the business concepts in plain language. State the design’s boundary and cite the profile results that establish it.
1.2 Questions from consumers
| ID | Asked by | Asked at | Origin | Question in the asker’s words | Answer and reason | Design decision | Test probe | Answer state |
|---|---|---|---|---|---|---|---|---|
| Q-9 | Mary Mink (prediction, design ) | 2026-10-25 | DIG-574 | For each client, which policies are they on, with which carrier, and in what role? | The client-policy-role fact, grouped by the client, policy and carrier qualifiers and the role | D-1, D-3 | P-9 | Served today in MART (to be rebuilt) |
| Q-10 | Mary Mink (prediction) | 2026-10-25 | DIG-574 | How much coverage does each carrier provide through us? | The current face amount of the policy-terms fact by carrier; only Northfield Life sends amounts today, Pinecrest Mutual through Ray’s labs | D-1, D-4 | P-10 | Served today in MART (to be rebuilt) |
| Q-11 | Mary Mink (prediction) | 2026-10-25 | DIG-574 | Which agent sold each policy, and where do our sources disagree about it? | The agent-policy fact: agents named per policy; a policy with more than one agent named is where the sources disagree | D-1 | P-11 | Served today in MART (to be rebuilt) |
| Q-12 | Mary Mink (prediction) | 2026-10-25 | DIG-574 | Which independent agents sell through us without an agency? | Deferred: needs the rule for independent (DD-1) | DD-1 | none yet | Deferred |
| Q-13 | Mary Mink (prediction) | 2026-10-25 | DIG-574 | Where did this client’s record come from, and when did we first know it? | The client-policy-role fact’s metrics grouped by the client’s source and first-known date; a client with no policy role does not appear until release 2 | D-1, D-2 | P-13 | Not yet served |
| Q-14 | Mary Mink (prediction) | 2026-10-25 | DIG-574 | What did we know about this policy on a given date, and what changed since? | The policy-terms fact grouped by policy and by each version’s valid-from and valid-to days; the version covering a date, and every change after it, read from the rows | D-1, D-4 | P-14 | Not yet served |
| Q-15 | Mary Mink (prediction) | 2026-10-25 | DIG-574 | How many records arrive in a form we don’t accept, from which source? | records_not_accepted by source and finder type, the form findings; rows_missing_key beside it counts rows kept and flagged for a missing key | D-1, D-5, D-13 | P-15 | Served today in MART (to be rebuilt) |
Use Q-n identifiers. A predicted initial question is identified as a prediction, rather than attributed to a consumer who has not asked it.
1.3 Questions to consumers and source owners
| ID | Asked of | Raised by and when | Question | Decision blocked | Interim rule | Measured stakes, environment, and clock | State, answer, respondent, and answer date |
|---|---|---|---|---|---|---|---|
| C-1 | Ray, for Mary | Claude, 2026-10-26 | What makes an agent independent? Proposed: only carriers name the agent, no acquired agency’s data does | Q-12 | Q-12 not served | 6 agents in DV_QA, 1 policy with two agents named (2026-10-27 01:04 UTC) | Open |
Use C-n for questions to consumers and S-n for questions to source owners. This design is their live home; other branch documents cite these entries. An unanswered question carries its interim treatment and disclosed consequences.
1.4 Metric definitions
| Metric | Consumer question | Business definition and owner | Population and exclusions | Calculation, unit, and sign | Reporting grain | Period, date role, and time zone | Unknown, absent, and known-zero treatment | Comparison basis | Additivity restrictions |
|---|---|---|---|---|---|---|---|---|---|
| policies_held | Q-9 | Distinct policies Sunbelt Senior’s clients are on; owner Ray | Client-policy roles in the sales data; ghost rows out; a sale with a blank policy number counts once as the unknown policy | Distinct count of policy keys; policies; positive | Any grouping of client, carrier, policy, role, first-known day | First-known day of the role (the link’s load), UTC | A client with no sale is on none (absent, not zero) | none | Not additive across roles or clients: one policy can have several |
| client_policy_roles | Q-9 | Client, policy and role combinations; owner Ray | As above | Distinct count of role-row keys; roles; positive | As above | As above | As above | none | Additive across roles |
| face_amount_current | Q-10 | The face amount a carrier currently states for its policies; owner Ray | Current versions of carrier-sent terms; policies a carrier never sent terms for are absent | Sum of face amount; US dollars as sent; positive | Carrier, policy, status | Current version only (valid-to empty); UTC | A policy without terms contributes nothing and is not zero; disclosed as policies without terms | none | Additive across policies within the current version |
| policy_versions | Q-14 | Versions of a policy’s terms the vault holds; owner Ray | All versions, current and past | Distinct count of version keys; versions | Policy with valid-from and valid-to days | Valid-from day (the version’s load), UTC | none | none | Additive |
| face_amount | Q-14 | Face amount per version, to read what was stated on a date; owner Ray | All versions | Sum of face amount; US dollars | Policy with valid-from and valid-to days | Valid-from day, UTC | As face_amount_current | Before and after a change, by version | Not additive across versions of one policy: read per version |
| policies_sold | Q-11 | Distinct policies with an agent named as the one who sold them; owner Ray | Agent-policy pairs from both sources | Distinct count of policy keys | Agent, policy, carrier, first-named source | First-known day of the pair, UTC | A policy no source names an agent for is absent | none | Not additive across agents |
| agents_named | Q-11 | Distinct agents named for the policies in the group; owner Ray | As above | Distinct count of agent keys | Grouped by policy: 2 or more means the sources disagree | As above | none | none | Not additive across policies |
| records_not_accepted | Q-15 | Records that arrived in a form the vault does not accept; owner Ray | Findings whose finding_kind is form, in the landing’s CONFIG.DATA_FINDING | Distinct count of finding keys; records; positive | Source, table, column, finder, finder type, status | Detected day, UTC | A source with no such finding is absent, not zero | none | Additive |
| rows_missing_key | Q-15 | Rows that arrived without a business key, kept and flagged; owner Ray | Findings whose finding_kind is missing_key | Distinct count of finding_row; rows; positive | Source, table, finder | Detected day, UTC | As records_not_accepted | none | Not additive across columns: one row can miss two keys |
Define each displayed number before deciding its physical model. State filters, numerator and denominator where applicable, period boundaries, and whether partial periods are allowed. Distinguish reporting grain from the supporting fact grain identified in Section 2. A comparison names both sides and their interpretation. Do not silently supply a business judgment that requires a consumer answer.
2. Grains and fact reuse
Grain is foundational. Declare exactly what one row represents in business terms and the keys that prove its uniqueness before selecting fact columns, joins, or physical objects. Distinguish the consumer’s reporting grain from each required supporting fact grain. Population restrictions are separate from grain; a restricted population does not by itself justify another fact at the same grain. Establish the actual grain from existing code and warehouse rows, not from selected columns, grouping, or an assumed model name.
2.1 Data reality
| Profile result | Measured fact and read clock | Existing code or warehouse evidence | Design consequence |
|---|---|---|---|
| Profile 2.1, source ERD | client_policy_l 13 rows, 13 keys, 13 distinct (client, policy, role), 11 distinct (client, policy), 1 row to the unknown policy; DV_QA, read 2026-10-27 01:04 UTC | dbt/models/raw_vault/client_policy_l.sql; role is a dependent child key (design D-33) | The role is part of the row’s identity |
| Profile 2.1 | agent_policy_l 17 rows, 17 keys, 17 distinct (agent, policy), 16 policies; first named by SSI 10, NFL 7; 1 policy with two agents; DV_QA, read 2026-10-27 01:04 UTC | agent_policy_l.sql | One row per pair, not per source: the record source is the first source to name the pair |
| Profile 2.1 | policy_northfield_s 10 rows, 10 distinct (policy, load), 9 policies; the dated view gives 9 current versions; DV_QA, read 2026-10-27 01:04 UTC | policy_northfield_s.sql and policy_northfield_s1.sql (valid-to 8888-12-31 when current) | A version is a policy and its load |
| Profile 2.6 | client_h 35 clients, policy_h 15 policies, agent_h 6 agents (ghost rows out); DV_QA, read 2026-10-27 01:04 UTC | the hubs | The hubs supply the dimensions |
| Profile 2.5, findings | LANDING_QA.CONFIG.DATA_FINDING 3 rows, 3 keys; DV_QA, read 2026-10-27 01:04 UTC | inspectors and auditors, merge_into | One row per finding |
Cite the profile’s result identifiers. Measurements are dated evidence, not fixed expected values for future tests.
2.2 Metric-to-fact map
| Metric | Consumer reporting grain | Required supporting fact grain: what one row represents in business words | Row identity and uniqueness keys | Existing fact and repository path | Evidence of actual grain and population | Existing measures | Missing measures or population support | Reuse, extend, consumer view, or new fact | Justification and decision |
|---|---|---|---|---|---|---|---|---|---|
| policies_held, client_policy_roles | client, carrier, policy, role | One role a Sunbelt Senior client has on one policy, as the sales data reports it | client_policy_role_key (the link’s hash key); unique, and one per (client, policy, role) | None in CONFORMED_DM; MART.client_policy_fact retired | 13 rows = 13 keys = 13 triples | none: factless | none | New factless fact | No existing fact has this relationship grain |
| policies_sold, agents_named | agent, policy, carrier | One agent named by at least one source as the one who sold the policy | agent_policy_key; unique, one per (agent, policy) | None; MART.agent_policy_fact retired | 17 rows = 17 keys = 17 pairs | none: factless | none | New factless fact | A distinct relationship grain; no existing fact serves it |
| face_amount_current, face_amount, policy_versions | carrier, policy, version days | One version of a carrier’s terms for one policy, from its load until the next | policy_terms_key (policy key and valid-from); unique | None; MART.policy_fact retired, which was at policy grain and mixed sales-only policies | 10 rows = 10 (policy, load) | face_amount | none | New fact | Q-10 and Q-14 share this grain: one fact serves both (Ray, one fact per grain) |
| records_not_accepted, rows_missing_key | source, table, finder | One record an inspector or auditor flagged | finding_key (finding_pk); unique | None; MART.finding_fact retired | 3 rows = 3 keys | none: factless | none | New factless fact | A distinct grain; the two metrics filter it by finding_kind (D-13) |
One fact serves each group of metrics at the same required supporting grain. Reuse an existing fact at that grain; extend it when measures or necessary population support are missing. Preserve existing consumers’ declared populations through explicit semantic filters and regression proof. A dashboard, comparison purpose, new consumer, or different reporting grain does not justify another shared fact by itself.
Before proposing a new fact, show whether the existing facts and dimensions can supply the required output through semantic calculations and consumer views. A new fact requires evidence that a distinct underlying business grain is necessary and that no existing fact serves it; the same requirement applies to a factless relationship fact. State the actual row identity, not merely a list of selected columns. For any upstream computation required, specify whether an existing source supplies it or a separately designed change is necessary.
Complete the review in this order: existing facts and actual grains; required supporting grain; reuse or extension; consumer-view necessity; object classification and schema-specific naming. A naming or suffix correction rechecks object necessity and classification; it must not preserve an unvalidated object proposal. For example, a deal/month output does not by itself justify another deal fact: assess the existing deal fact and attendance relationships before designing its consumer projection.
2.3 Decisions
| ID | Decision | Rationale | Supporting profile and code or data evidence | Affected map rows |
|---|---|---|---|---|
| D-1 | Four facts, one per grain: client_policy_role_fact, agent_policy_fact, policy_terms_fact, finding_fact | Ray, 2026-10-27: one fact table per grain; the principles’ fact reuse by grain | Section 2.1 grains, each proven rows = keys | 2.2 rows 1 to 4 |
| D-2 | Dimensions: person_dim and organization_dim as base dimensions, with role views client_dim, insured_dim and agent_dim over person_dim and carrier_dim over organization_dim (D-10, D-11); policy_dim; source_dim, from the source system seed; day_dim, which is also MetricFlow’s time spine | Qualifiers belong in dimensions; one day dimension in its roles | Hubs and the seed | 3.3 |
| D-3 | The role stays on client_policy_role_fact as a degenerate code: it is part of the row’s identity, and three values need no dimension | Datamart form allows a judicious degenerate code | client_policy_l, D-33 | 2.2 row 1 |
| D-4 | A version’s valid-to day is empty while it is current; face_amount_current filters on that in its semantic definition | Q-10 needs now, Q-14 needs any date; one fact | policy_northfield_s1 | 2.2 row 3, 4.1 |
| D-5 | The finding’s finder name and type, table, column, value and status stay on finding_fact as degenerate codes and identifiers of the flagged record; only the source is a dimension key, since two finders need no dimension | They identify the one record, not a population | DATA_FINDING | 2.2 row 4 |
| D-6 | All objects in CONFORMED_DM (models/marts/conformed_dm), views; the five MART views retired in the same pull request; DV Digest is their only reader and reloads from the QA build | : MART empties by pull; no duplicate copy | MART objects of 2026-10-26 and 27 | 4.4 |
| D-7 | No workgroup datamart in release 1: Mary’s metrics sit on CONFORMED_DM, the mart layer’s shared schema. Her workgroup, Data Architecture and Governance, gets its datamart, GOVERNANCE_DM, with release 2, whose compliance questions serve governance; it reads CONFORMED_DM alone | Created by need; release 1 needs no workgroup schema (Ray, 2026-10-31) | none | 4.4 |
| D-8 | Disagreement about a policy’s agent is read from agents_named grouped by policy, not a SQL flag | If MetricFlow can express it, MetricFlow does it; the retired view’s window function is gone | 4.2 | |
| D-9 | The insured dimension’s birth date and address are shown under role-based conditional masking: a secure view returns the real values to the governance role and a mask to every other role (birth year only; the address reduced to its state). This mocks a Snowflake masking policy, which needs the Enterprise edition; this Snowflake rejects masking policies as an unsupported feature (verified 2026-10-31). The view says so, and the policy replaces it once the edition allows | A basic personal-data handling capability, under Mary’s governance remit; a mock in a view can be bypassed by another view, which a column policy cannot (Ray, 2026-10-31, answer 2d) | CREATE MASKING POLICY refused, 2026-10-31 | 3.3 |
| D-10 | The party model in release 1: person_dim holds one row per person record the raw vault receives (each client, insured and agent record is its own person, unmatched), and client_dim, insured_dim and agent_dim are role views over it; an agent’s name is the person’s attribute, from the most recent sale that names the agent. Matching the same human across roles is left to release 2’s business vault, and the unmatched people are shown to Mary as the reason for it | Ray, 2026-10-31: role dimensions are views over the person dimension; release 1 has only the raw vault, release 2 brings the business vault | client_h, insured_h, agent_h; client_sunbelt_s, insured_northfield_s, client_policy_sunbelt_s | 3.3 |
| D-11 | A carrier is an organization: organization_dim, with carrier_dim as its role view. Each carrier sends its own organization record, code and name, through its feed into landing, then staging and the raw vault; no carrier is known by a code alone | Ray, 2026-10-31: no source system gives organizations without names (answer 1b) | policy_h carries the carrier code only today | 3.1, 3.3 |
| D-12 | client_arrival_fact is removed: it had one row per client, the client dimension’s own grain, and no metric is assigned to the arrival. Its two attributes move to the client: source_key and first_known_date become columns of person_dim for client records, shown by the client_dim role view. Q-13 (“Where did this client’s record come from, and when did we first know it?”) is answered in release 1 from client_policy_role_fact’s metrics grouped by the client’s source and first-known date; a client with no policy role does not appear, and release 2 enhances this. Release 2 also reviews the decision | Ray, 2026-10-31 | none | 1.2, 1.4, 2.2, 3.2, 3.3, 4.1, 4.1a, 4.4, 5.6 |
| D-13 | Q-15 is answered by two metrics on finding_fact: records_not_accepted, the findings whose finding_kind is form, and rows_missing_key, the distinct rows named by findings whose finding_kind is missing_key. finding_fact carries finding_kind, taken from the finder that wrote the finding (form, missing_key or audit), and finding_row, the source table and the name of the row, as degenerate codes beside those of D-5 | A count of every finding answered Q-15 with 10 instead of 3 once the missing-key inspectors existed; release 1 already serves the two metrics (branch design D-47), so its readers keep the same answers | Branch design , D-47; development vault at commit 2a921a1: 3 form findings, and 7 missing-key findings naming 6 rows | 1.2, 1.4, 2.2, 3.2, 4.1, 4.1a |
| D-14 | Each ghost key appears once in person_dim, with no role, and every role view, client_dim, insured_dim, agent_dim and carrier_dim, carries the two ghost rows of its base dimension, so a fact’s unknown or error key always finds exactly one member | client_h, insured_h and agent_h share the same two ghost keys; one row per role would repeat a key and break person_dim’s uniqueness | Test plan uq_person_dim_person_key (DU1); the hubs’ ghost records | 3.3 |
Use D-n identifiers in sequence. Proposed mappings are shown to the reviewer and resolved through numbered decisions. Changed mappings are re-proposed, never silently substituted.
3. Measures and dimensions
3.1 Upstream and UDM map
| Profile verdict | Source concept and column | Existing or proposed UDM destination table and column | Identity recipe | Derivation and pattern | Additive change? | Decision | Deliberate omission or deferral and reason |
|---|---|---|---|---|---|---|---|
| Integrate | A carrier’s own organization record: carrier code and carrier name, as each carrier sends it; Northfield Life sends NFL and Pinecrest Mutual sends PMC | New landing table in each carrier’s feed, one row per organization record | The carrier code | As sent | Yes | D-11 | none |
| Integrate | The same record in staging | New staging model over that landing table, with the source system named | The carrier code | As sent; the source system added | Yes | D-11 | none |
| Integrate | The organization in the raw vault | New hub organization_h keyed by the carrier code, fed by every carrier’s staging model; one satellite per carrier source, such as organization_northfield_s and organization_pinecrest_s, each with the carrier’s name as that carrier sends it | The hash of the carrier code | Standard hub and satellite from the staging model | Yes | D-11 | none |
| Not applicable | No other integration change: this design consumes the raw vault as built (hubs client_h, policy_h, agent_h, insured_h; links client_policy_l, agent_policy_l, insured_policy_l; satellites client_sunbelt_s, insured_northfield_s, client_policy_sunbelt_s and policy_northfield_s, with their dated views) and the landing’s CONFIG.DATA_FINDING | none | none | none | No | D-6 | The data integration layer is out of scope (Q-7, Q-8, Q-16) |
Consume every column in the profile’s Integrate list; name Defer and Reject as absences. If this mart design requires no integration change, explicitly state that conclusion and map the existing upstream inputs it consumes. A missing consumer view does not by itself establish a missing integration capability.
3.2 Fact measures and keys
Each event date is carried on its fact as the date itself, which is also its key to day_dim in that role ( as corrected 2026-10-31). People and organizations are reached through the role views of section 3.3 (D-10, D-11).
| Fact and declared grain | Destination column | Classification: measure, dimension key, or permitted degenerate identifier | Source object and column | Derivation | Unit, sign, and null meaning | Additivity and date role | Existing or new | Decision |
|---|---|---|---|---|---|---|---|---|
| client_policy_role_fact: one client role on one policy | client_policy_role_key | permitted degenerate identifier (row key) | client_policy_l.hk_client_policy_l | as is | key | none | new | D-1 |
| client_policy_role_fact | client_key, policy_key | dimension keys: client_dim, policy_dim | client_policy_l.hk_client_h, hk_policy_h | as is | keys; the unknown policy’s key is the ghost key | none | new | D-1, D-10 |
| client_policy_role_fact | role | permitted degenerate code | client_policy_l.role | as is | owner, insured, payor as sent | none | new | D-3 |
| client_policy_role_fact | sold_date | event date and day_dim key, role sold | client_policy_sunbelt_s1.sold_on, current version | to date | UTC day; empty when the sale gives none | date role: sold | new | D-1 |
| client_policy_role_fact | first_known_date | event date and day_dim key, role first known | client_policy_l.ldts | to date | UTC day | date role: first known | new | D-2 |
| agent_policy_fact: one agent named for one policy | agent_policy_key; agent_key, policy_key; first_named_by_source_key; first_known_date | row key; dimension keys: agent_dim, policy_dim; source_dim key; event date and day_dim key | agent_policy_l.hk_agent_policy_l, hk_agent_h, hk_policy_h, rsrc, ldts | as is; ldts to date | the source is the first to name the pair | date role: first known | new | D-1, D-10 |
| policy_terms_fact: one version of a policy’s terms | policy_terms_key; policy_key; valid_from_date; valid_to_date | row key (policy key and load); dimension key: policy_dim; event dates and day_dim keys in two roles | policy_northfield_s1.hk_policy_h, ldts, ledts | key is the hash of policy key and load; valid-to empty when ledts is 8888-12-31 | UTC days | date roles: valid from, valid to | new | D-1, D-4 |
| policy_terms_fact | effective_date | event date and day_dim key, role effective | policy_northfield_s1.effective_date | to date | the day the carrier says the policy took effect | date role: effective | new | D-1 |
| policy_terms_fact | face_amount | measure | policy_northfield_s1.face_amount | as is | US dollars; null when not sent | additive across policies in one version state | new | D-1 |
| policy_terms_fact | status | permitted degenerate code of the version | policy_northfield_s1.status | as is | as sent | none | new | D-1 |
| finding_fact: one flagged record | finding_key; source_key; detected_date; finder_name, finder_type, source_table, source_column, column_value, status, finding_kind, finding_row | row key; dimension key: source_dim; event date and day_dim key; degenerate codes and identifiers | DATA_FINDING.finding_pk, source_system, detected_at, finder_name, finder_type, source_table, source_column, column_value, status | detected_at to date; finding_kind is audit when finder_type is audit, missing_key for a finder name ending _key_missing, form for one ending _form; finding_row is the source table and column_value | UTC day | date role: detected | new | D-5, D-13 |
Every written column traces to a source column or an explicit derivation. Facts carry measures and keys; qualifiers belong in dimensions. Date keys refer to the shared day dimension in their named roles. Explain any judicious degenerate identifier. Population metrics do not become precomputed fact columns.
3.3 Dimensions and qualifiers
Release 1 uses the party model (D-10, D-11): two base dimensions, person_dim and organization_dim, with role views over them. Dates that describe the entity itself, such as a birth date, live in its dimension; event dates live in the facts.
| Dimension and grain | Qualifier or key | Business meaning | Source object and column or derivation | Existing or new | Fact foreign key and dimension key | Unknown member and history behavior | Decision |
|---|---|---|---|---|---|---|---|
| person_dim: one person record, as one source sent it, unmatched | person_key, person_role (client, insured or agent), source_key, first_known_date, person_number, first_name, last_name, full_name, state, birth_date, address | A person as one source knows them; the same human under two roles is two rows until release 2 | client: client_h (client_number; ldts to first_known_date, the hub’s load, for client records), client_sunbelt_s1 current version (first_name, last_name, state); insured: insured_h, insured_northfield_s1 current version (insured_name to full_name, birth_date, address); agent: agent_h (agent_npn to person_number), agent name from the most recent sale (client_policy_sunbelt_s1.agent_name) of a policy that agent_policy_l joins to the agent | new | through the role views | Each ghost key kept once, with no role (D-14); current satellite version only | D-10 |
| client_dim, role view: one client record | person_key as client_key; client_number, first_name, last_name, state, source_key, first_known_date | A client of Sunbelt Senior | person_dim where person_role is client, with the two ghost rows (D-14) | new | client_policy_role_fact.client_key | the two ghost rows of its base dimension (D-14) | D-10, D-14 |
| insured_dim, role view: one insured record | person_key as insured_key; full_name; birth_date and address under the masking mock | The person a Northfield Life policy covers | person_dim where person_role is insured, through the secure view of D-9, with the two ghost rows (D-14) | new | joins policy_dim through insured_policy_l in release 1 | the two ghost rows of its base dimension (D-14) | D-9, D-10, D-14 |
| agent_dim, role view: one agent record | person_key as agent_key; agent_npn, full_name | An agent, by national producer number | person_dim where person_role is agent, with the two ghost rows (D-14) | new | agent_policy_fact.agent_key | the two ghost rows of its base dimension (D-14) | D-10, D-14 |
| organization_dim: one organization | organization_key, organization_code, organization_name, source_key | An organization as it names itself | each carrier’s own organization record, through its feed: landing, staging, a new organization hub, and one satellite per carrier source | new | through carrier_dim | Ghost rows kept as unknown and error members | D-11 |
| carrier_dim, role view: one carrier | organization_key as carrier_key; carrier_code, carrier_name | A carrier Digest sells for | organization_dim for organizations sent through carrier feeds, with the two ghost rows (D-14) | new | policy_dim.carrier_key | the two ghost rows of its base dimension (D-14) | D-11, D-14 |
| policy_dim: one policy number per carrier | policy_key, carrier_key, policy_number, plan_name, first_named_by_source_key | A carrier’s policy; its plan as the most recent sale names it | policy_h; carrier_code resolved to carrier_key; plan_name from the most recent client_policy_sunbelt_s1 row for the policy | new | facts’ policy_key | Ghost rows kept as unknown and error members | D-2, D-11 |
| source_dim: one source system | source_key, source_name, owner | Who sent the data | seed_source_system | new | the facts’ source keys | none | D-2 |
| day_dim: one calendar day | date_day | The day dimension, and MetricFlow’s time spine under this name | generated, 2026 and 2027 | new, replacing MART.time_spine_day | every event date key | none | D-2 |
Reuse dimensions where they supply the required qualifiers. Qualifiers include codes, names, states, flags, and descriptive attributes.
3.4 Joins and cardinality
| Left object and grain | Right object and grain | Join columns and predicate | Expected cardinality | Unmatched-row treatment | Evidence or planned proof | Effect on measures |
|---|---|---|---|---|---|---|
| client_policy_role_fact, role row | client_dim, client | client_key = client_key | many to one | none: every link row’s key exists in its hub (assert_increment_2_link_hub_keys_exist) | layer 4 assertion; relationships tests | none: factless |
| client_policy_role_fact and agent_policy_fact | policy_dim, policy | policy_key = policy_key | many to one | none, as above | as above | none |
| policy_terms_fact, version | policy_dim, policy | policy_key = policy_key | many to one | none: D-40’s relationship test | rel_policy_northfield_s_hk_policy_h | face_amount stays at version grain |
| any fact | day_dim | date key = date_day | many to one | days outside 2026 and 2027 unmatched; none today | relationships test | none |
| finding_fact | source_dim | source_key = source_key | many to one | the findings’ source names (SUNBELT, NORTHFIELD) are not seed codes: mapped in the view, SUNBELT to SSI, NORTHFIELD to NFL | relationships test | none |
Show how each join preserves row identity and measures. A total over a view must first establish how often each measure-bearing key occurs.
3.5 Dictionary and domain rules
| Term | Business meaning | Model and column mapping | Mapping state | Decision | Verified example |
|---|---|---|---|---|---|
| client | A client number Sunbelt Senior sends | client_dim.client_number | PROPOSED | D-2 | A-1007 |
| role | What a client is on a policy | client_policy_role_fact.role | PROPOSED | D-3 | owner |
| version of terms | What a carrier said about a policy from one load until the next | policy_terms_fact | PROPOSED | D-4 | NF-100003 with two versions |
| finding | A record the vault flagged at arrival | finding_fact | PROPOSED | D-5 | A1007, client-number form |
Ghost members: the unknown and error rows of each hub are dimension members, never fact rows. A sale with a blank policy number is joined to the unknown policy, whose carrier reads (unknown).
Teach terms in dependency order. Newly designed dictionary mappings begin PROPOSED and become VERIFIED through built-data evidence.
4. MetricFlow and consumer views
4.1 Semantic definitions
| Metric | Semantic model and repository path | Fact and measures | Aggregation or expression | Entities and dimensions | Time dimension and filters | Definition owner and record | Decision |
|---|---|---|---|---|---|---|---|
| policies_held, client_policy_roles | client_policy_role_fact, dbt/models/marts/conformed_dm/_conformed_dm.yml | none (factless); keys | count_distinct policy_key; count_distinct client_policy_role_key | entity client_policy_role; dimensions role; joins to client, policy | first_known_date | Ray, this design | D-1 |
| policies_sold, agents_named | agent_policy_fact, same file | none; keys | count_distinct policy_key; count_distinct agent_key | entity agent_policy; joins to agent, policy, source | first_known_date | Ray | D-1, D-8 |
| face_amount_current, face_amount, policy_versions | policy_terms_fact, same file | face_amount | sum face_amount, filtered to an empty valid_to_date for the current one; sum face_amount; count_distinct policy_terms_key | entity policy_terms; dimensions status, valid_to_date; joins to policy | valid_from_date | Ray | D-1, D-4 |
| records_not_accepted, rows_missing_key | finding_fact, same file | none | count_distinct finding_key where finding_kind is form; count_distinct finding_row where finding_kind is missing_key | dimensions source_table, source_column, status, finder_name, finder_type, finding_kind; joins to source | detected_date | Ray | D-5, D-13 |
Every published metric is defined in semantic YAML. A consumer view preserves that shared definition; it does not become a second independent definition.
4.1a The agent’s guide: descriptions and saved queries
The semantic layer is the agent’s guide to this data (Data Mart Design Principles, , “The agent’s guide”). Every semantic model, metric and column of sections 3.2 and 3.3 carries a description in the semantic YAML: a model states what one row is and which questions it serves; a metric states what it counts, how to read it, what it leaves out, what to pair it with, and the decision that defined it; a column states what it holds, including when it is empty or unknown. A saved query stands for each question release 1 answers:
| Saved query | Question | Metrics | Grouped by |
|---|---|---|---|
| q09_client_policies | Q-9 | policies_held, client_policy_roles | client, carrier, policy, role |
| q10_coverage_by_carrier | Q-10 | face_amount_current | carrier |
| q11_agent_per_policy | Q-11 | policies_sold, agents_named | agent, policy; a policy with agents_named of 2 or more is where the sources disagree |
| q13_client_arrivals | Q-13 | policies_held, client_policy_roles | client’s source, client’s first-known date |
| q14_policy_versions | Q-14 | policy_versions, face_amount | policy, valid-from day, valid-to day, status |
| q15_records_not_accepted | Q-15 | records_not_accepted, rows_missing_key | source, finder type |
Proof: for each saved query, the test plan runs it and checks its answer against the figures on Mary’s release page.
4.2 SQL exceptions
| Exception name | Required per-entity computation | Why MetricFlow cannot express it | Single derivation site | Unit-test reference | Layer-6 reconciliation reference | Capability or model change that retires it | Decision |
|---|---|---|---|---|---|---|---|
| none | none | Every metric is a MetricFlow aggregation; the retired agent view’s per-policy window flag is replaced by agents_named grouped by policy (D-8) | none | none | none | none | D-8 |
Use none with a reason if no exception is required. SQL exceptions are limited to computations allowed by Data Mart Design Principles (knowledge 12, item 104); dashboard convenience is not a justification.
4.3 Consumer-view and card map
| Consumer question or card | Metric or evidence supplied | View and repository path | Report grain and row key | Upstream objects | Population and period filters | Summary or finer-grain detail | Repeated-measure prevention | Decision |
|---|---|---|---|---|---|---|---|---|
| DV Digest, every question | the metrics of 4.1 | none: the connector reads the semantic manifest and MetricFlow compiles over the facts and dimensions | the metric’s group-by | the facts and dimensions | in the metric definitions only (no tool filter exists) | summary | MetricFlow joins facts to dimensions many to one | No consumer view: the connector is the surface |
One measures view serves a report’s measures at the report grain. Finer-grain child views serve separate detail cards. The measures view never joins its children; child records are not flattened into numbered columns.
4.4 Physical placement and materialization
| Object | Schema | Upstream schemas and objects | View or table and justification | Existing readers affected | Move, retirement, and reader changes | Decision |
|---|---|---|---|---|---|---|
| person_dim, organization_dim, policy_dim, source_dim, day_dim; role views client_dim, insured_dim, agent_dim, carrier_dim | CONFORMED_DM | RAW_VAULT hubs, satellites and the new organization hub; REFERENCE.seed_source_system; generated days | views: rebuilt each run from the vault, which keeps history | none | new | D-6 |
| client_policy_role_fact, agent_policy_fact, policy_terms_fact, finding_fact | CONFORMED_DM | RAW_VAULT links and the dated satellite view; landing CONFIG.DATA_FINDING through landing_storage() | views | none | new | D-6 |
| MART.client_policy_fact, agent_policy_fact, policy_fact, finding_fact, time_spine_day | MART | as built | dropped by the pull request: models removed; the QA build’s DV_QA.MART views dropped by setup script | DV Digest, which reloads from the QA build | retired | D-6 |
Warehouse schemas and repository folders are distinct. Workgroup marts read CONFORMED_DM alone. A needed MART object moves to CONFORMED_DM once, with its MART copy retired and readers repointed; no duplicate copy is introduced. Datamart tables are facts or dimensions with the corresponding _fact or _dim suffix. Apply the Object Naming standard (knowledge 12, item 147) before agreement.
4.5 Consumer interpretation
| Surface | Plain-language question or finding | Definition and methods shown at point of use | Unknown and no-answer display | Exclusions and limitations | Evidence drill and clocks |
|---|---|---|---|---|---|
| DV Digest | Mary’s six questions | each metric’s description, from the semantic YAML, shown by list-metrics | absent groups are absent, not zero | Q-7, Q-8, Q-12, Q-16 not served; amounts only where a carrier sends them | the QA build’s run id and time, which the connector loaded |
Apply the Data Consumption Design standard (knowledge 17, item 72). Show what was measured, over which population and period, with which exclusions and as-of clocks. An unexplained discrepancy or unresolved interpretation remains visible rather than receiving an invented answer.
5. Validation and approval
5.1 Instance models
Policy NF-100003 in DV_QA (read 2026-10-27 01:10 UTC): policy_dim holds it once (carrier NFL). policy_terms_fact holds two versions: valid from 2026-10-01 02:00 to 2026-10-02 01:59:59.999 UTC with status IF and face amount 15,000.00, and valid from 2026-10-02 02:00, current, with status LP and the same amount. face_amount_current reads 15,000.00 from the second; policy_versions grouped by policy reads 2; grouped by status, the change from IF to LP on 2026-10-02 is the answer to what changed. Figure M-1:
flowchart LR
S1["policy_northfield_s1: NF-100003, IF, 15,000.00, loaded 2026-10-01 02:00"] ==> V1["policy_terms_fact: valid 2026-10-01 to 2026-10-02, IF, 15,000.00"]
S2["policy_northfield_s1: NF-100003, LP, 15,000.00, loaded 2026-10-02 02:00"] ==> V2["policy_terms_fact: valid from 2026-10-02, current, LP, 15,000.00"]
V1 ==> P["policy_dim: NFL NF-100003"]
V2 ==> P
V2 ==> C["face_amount_current: 15,000.00"]
V1 ==> N["policy_versions for NF-100003: 2"]
V2 ==> N
Show verified real records, source read clocks, and their rows through the proposed model to the consumer projection. Include a diagram for each trace. An instance model is evidence.
5.2 Worked examples
A client Ada is on policy P-1 as owner and insured: two rows in client_policy_role_fact, one policy. policies_held for Ada reads 1; client_policy_roles reads 2. Figure M-2:
flowchart LR
R1["client_policy_role_fact: Ada, P-1, owner"] ==> P["policy_dim: P-1"]
R2["client_policy_role_fact: Ada, P-1, insured"] ==> P
R1 ==> A["client_dim: Ada"]
R2 ==> A
A ==> H["policies_held for Ada: 1"]
A ==> K["client_policy_roles for Ada: 2"]
Use small invented names and values to explain the design, with a diagram. A worked example teaches the shape and does not replace real-data evidence.
5.3 Required proof and test-plan references
| Requirement | Model, metric, or view | Source-to-target comparison or invariant | Expected relationship | Test-plan artifact and layer | Consumer question or design decision |
|---|---|---|---|---|---|
| Grain uniqueness | each fact and dimension | rows equal keys | equal | unique tests, layer 1 | D-1, D-2 |
| Dimension integrity | every fact key | each key exists in its dimension | no orphans | relationships tests, layer 1 | D-2 |
| Measure conservation | policy_terms_fact.face_amount | sum over versions equals the satellite’s sum | equal | layer 4 assertion | D-1 |
| Two-route reconciliation | every metric | MetricFlow’s answer equals a direct query on the vault | equal | layer 6 | 4.1 |
| Consumer probes | Q-9, Q-10, Q-11, Q-13, Q-14, Q-15 | DV Digest’s answer equals the direct query on DV_QA | equal | consumer certification | 1.2 |
Walk the seven layers of the current Testing Methodology (knowledge 2) in order in the paired test plan; state not applicable with a reason. Do not copy an older template’s layer numbering. Include grain uniqueness, dimension integrity, join cardinality, measure conservation, unknown treatment, metric reconciliation through two independent routes, existing-consumer regression, and consumer-question probes as applicable. Tests precede implementation under the Test-Driven Development standard (knowledge 23). Fixed observed counts or amounts are dated examples, not enduring pass conditions.
5.4 Deferred decisions
| ID | Deferred decision | Reason | Interim treatment | Unblocking condition | Owner |
|---|---|---|---|---|---|
| DD-1 | The rule for an independent agent (Q-12) | Needs Ray’s word (C-1) | Q-12 not served | Ray answers C-1 | Ray |
5.5 Compliance record and completion checklist
| Requirement | Passage, map row, or evidence | Met, not met, or not applicable | Reason and unresolved decision |
|---|---|---|---|
| Every consumer question maps to a metric | 1.2, 1.4 | Met for six; not applicable for Q-12 (DD-1) | none |
| Every fact declares its grain and proves it | 2.1, 2.2 | Met: rows equal keys on DV_QA | none |
| One fact per grain; reuse before new | 2.2, D-1 | Met: no existing fact in CONFORMED_DM; four distinct grains | none |
| Facts carry measures and keys; qualifiers in dimensions | 3.2, 3.3, D-3, D-5 | Met, with three judicious degenerate codes | none |
| Metrics in MetricFlow; no SQL metric | 4.1, 4.2 | Met | none |
| Placement: CONFORMED_DM, _fact and _dim, MART retired | 4.4 | Met | none |
| Every template section and field filled | Sections 0 to 5.6 and the header | Met | none |
| Reporting grain distinguished from supporting fact grain | 2.2 | Met | none |
| Every written column has a source or a derivation | 3.2, 3.3 | Met | none |
| Joins preserve row identity and measures | 3.4 | Met: every fact joins its dimensions many to one | none |
| Consumer views assessed before another shared fact | 4.3 | Met: no consumer view is needed | none |
| Every instance model and worked example carries its diagram | 5.1, 5.2 | Met: Figures M-1 and M-2 | none |
| Every decision has proof in the test plan | Test plan , section 2.3; implementation guide , steps 52 to 78 | Met for D-1 to D-6 and D-8 to D-14: D-13 by SR8, SR9, SQ6 and P-15; D-14 by DG1 and DS1 to DS4; D-6 by the guide’s steps 60 and 78; not applicable for D-7, which builds nothing | none |
Before agreement, verify every template section and field is filled; every applicable standard has evidence; every consumer question maps to a metric or evidence answer; every fact declares what one row represents and the keys that prove its actual grain; reporting grain and supporting fact grains are distinguished; every metric maps to a required supporting grain and fact; every new fact proves a distinct underlying business grain is necessary and no existing fact serves it; every missing measure or necessary population extension reuses an existing fact when its grain matches; consumer views are assessed before proposing another shared fact; object necessity and classification are checked before schema-specific naming, including after a suffix correction; every written column has a source or derivation; dimensions carry qualifiers; joins preserve measures; semantic definitions and SQL exceptions comply; physical placement meets schema boundaries; and every decision has corresponding proof in the test plan. Resolve every internal section and item reference. A requirement without evidence is not met.
5.6 Handoff boundary
The implementation guide adds the mart steps: the conformed_dm folder and schema entry, the thirteen models (four facts, five dimensions and four role views), the semantic YAML, the tests of 5.3 written first, the MART retirement, and the QA run. Agreement of this design does not authorize code; the build follows Ray’s answer on it.
The implementation guide implements the agreed decisions and maps, naming exact files and patterns. It adds no model, column, metric definition, consumer interpretation, or mapping decision absent from this design. Agreement of this document does not authorize code or database execution; execution follows the separately approved numbered decisions or agreed execution goal.