CheckMySchema

Structured data explained

Schema-on-read vs schema-on-write: a practical explainer

A clear, engineer-friendly comparison of schema-on-read and schema-on-write, with concrete examples, trade-offs, and how this differs from JSON Schema readOnly/writeOnly.

    Minimalist full frame diagram comparing ingest to warehouse with validation and ingest to lake with schema applied at read showing small JSON snippets

    Short answer: schema-on-write enforces a schema when data is stored; schema-on-read defers applying a schema until data is queried. Use schema-on-write for predictable, governed queries over structured data; use schema-on-read to ingest diverse or evolving data quickly, then shape it at query time. Many teams combine both patterns.

    What each approach means and when to use it

    Schema-on-write structures data according to a predefined schema before it is written, which can support consistent data and predictable query behaviour; it does not by itself guarantee a particular query speed, typical of relational warehouses and transactional systems. Schema-on-read postpones the schema to analysis time, which increases flexibility for semi-structured or unstructured data and fast landing of raw files, common in data lakes and exploratory analytics. These definitions and trade-offs are outlined in vendor documentation and primers such as Dremio’s overview of schema-on-read vs schema-on-write.

    Diagram on midnight background of storage with raw json files feeding a pipeline into a query node

    Concrete examples: write-time vs read-time schema

    Schema-on-write (illustrative PostgreSQL table)

    The PostgreSQL constraints reference explains how declared constraints govern stored rows. Other database dialects can enforce or convert inputs differently.

    In this PostgreSQL example, the declared types and constraints restrict accepted rows. Some inputs can be converted to the declared type; do not assume that every database rejects every differently typed input. These illustrative SQL examples are for an isolated demo database, not commands to run over existing production tables.

    -- Define table with required columns and types
    CREATE TABLE orders (
      order_id     BIGINT PRIMARY KEY,
      customer_id  BIGINT NOT NULL,
      total_cents  BIGINT NOT NULL,
      created_at   TIMESTAMP NOT NULL
    );
    
    -- Type conversion and constraint checks follow PostgreSQL rules.
    -- Omitting a required value without a default is rejected.
    

    This upfront enforcement establishes a contract for the stored rows; actual query cost still depends on layout, indexing, the engine and workload, which is the core benefit Dremio notes for schema-on-write Dremio: Schema-on-Read vs Schema-on-Write.

    Schema-on-read (semi-structured data)

    With Snowflake’s VARIANT type, you can store parsed JSON in a VARIANT column and project fields at query time. The JSON must still parse, the table still has a declared column type, and platform limits still apply, illustrating schema-on-read. Snowflake documents that applying structure during reads can carry a performance penalty versus pre-structured columns, though their platform provides features to query semi-structured data directly Snowflake: Making Schema-on-Read a Reality.

    -- One-column table for semi-structured data
    CREATE TABLE events (v VARIANT);
    
    -- Store a valid illustrative JSON value
    INSERT INTO events SELECT PARSE_JSON('{
      "fullName": "Johnny Appleseed",
      "age": 42,
      "phoneNumber": {"areaCode": "415", "subscriberNumber": "5551234"}
    }');
    
    -- Project fields at read time
    SELECT
      v:fullName::string AS full_name,
      v:phoneNumber.areaCode::string AS area_code
    FROM events;
    

    For event streams, Confluent’s Schema-on-Read pattern describes leaving validation to consumers that select which schema version to apply at read time, often aided by a centralized schema registry and compatibility rules Confluent developer: Schema-on-Read.

    A schema registry and compatibility rules can support versioned event contracts. They do not make every old and new event automatically compatible, and schema-on-read does not mean there are no structural checks anywhere in the pipeline.

    Operational trade-offs to weigh

    • Read performance: Applying structure at query time can add work; pre-structured columns may avoid repeated parsing or conversion. Neither pattern guarantees a speed advantage for every engine and workload, a point Snowflake calls out when contrasting reads over semi-structured data vs columns Snowflake: Making Schema-on-Read a Reality.
    • Ingestion speed and agility: Schema-on-read can make landing evolving data easier; schema-on-write moves more modelling and transformation before storage. Measure ingestion and query costs for the actual platform rather than assuming a universal winner Dremio: Schema-on-Read vs Schema-on-Write.
    • Governance: Central registries and compatibility policies can help manage schema-on-read event streams, while relational constraints and DDL provide enforcement in schema-on-write systems Confluent developer: Schema-on-Read.
    • Hybrid: Land raw data for exploration, then publish curated, typed tables for analytics and BI; a view alone does not mean enforcement happened at storage time Dremio: Schema-on-Read vs Schema-on-Write.
    Minimalist 2D vector flowchart showing ETL into a data warehouse with a schema on write validation path and a schema on read path into indexed tables on a midnight background

    How this differs from JSON Schema readOnly and writeOnly

    JSON Schema’s readOnly and writeOnly are annotation keywords typically used for API semantics. They communicate producer or consumer behavior (e.g., a field is settable but not returned), not storage-time vs query-time schema enforcement. The JSON Schema reference clarifies these as annotations rather than validation keywords that change instance acceptance JSON Schema: Annotations and illustrates usage A Tour of JSON Schema: readOnly and writeOnly.

    Related reading on CheckMySchema: What JSON Schema is used for in practice and What does additionalProperties mean in JSON Schema?

    Decide schema-on-read or schema-on-write for this workload

    FAQs

    Can I mix schema-on-read and schema-on-write?

    Yes. A common approach lands raw data for exploration (schema-on-read) and writes curated, typed datasets for analytics (schema-on-write); merely projecting a raw table through a view is still read-time interpretation, as noted in lakehouse guidance Dremio: Schema-on-Read vs Schema-on-Write.

    Do I need a schema registry for schema-on-read?

    Not always, but for event streams it helps. A registry centralizes schema versions and can enforce compatibility so readers can safely choose which version to apply at runtime Confluent developer: Schema-on-Read.

    CheckMySchema logo

    Related guides

    Official sources and further reading