Skip to content

CLI Reference

plan

Preview what apply and load would do: every table, column, type mapping, write strategy, route and cursor. Changes nothing.

sqlload plan [SELECTOR ...] [OPTIONS]

sqlload plan prints what apply and load would do: tables to create or alter, each column’s type mapping, write strategy, route, batches and cursor. It reads the sources’ and destinations’ catalogs and changes nothing. apply and load print the same plan before they run. See Projects for how the three fit together.

In a Kafka exporter configuration, plan shows each topic’s table and the objects it needs, and refuses the options below.

A selector is a source name, a destination table name, or a source’s schema.table. Give selectors positionally or with --select, and leave tables out with --exclude. With none, every table is selected. See Selecting tables for examples.

A source’s problems, such as a refused setting or a server it can’t reach, fail only a command that selects something from it. A source that can’t list its tables also fails a command that selects a table by name, since that table could be one of its own.

plan, apply and load take the same options.

Option Meaning
SELECTOR ..., -s, --select <SELECTOR>... Sources or tables to select: a source name, a destination table name, or a source’s schema.table. All when none is given.
--exclude <SELECTOR>... Sources or tables to leave out. Wins over --select.
--start-cursor <VALUE> Read from this cursor value instead of the destination’s maximum. Typed from each table’s cursor: an integer, an RFC 3339 timestamp or a date.
--end-cursor <VALUE> Read only cursor values below this one.
--start-cursor-ts <TIMESTAMP>, --end-cursor-ts <TIMESTAMP> The range for the selected date and timestamp cursor tables only: an RFC 3339 timestamp or a date. A date cursor takes a date. Not with --start-cursor/--end-cursor. See Backfilling a range.
--start-cursor-int <INTEGER>, --end-cursor-int <INTEGER> The range for the selected integer cursor tables only. Can be given with the -ts flags, so one run ranges tables of both kinds.
--reload Rebuild the selected tables from the whole source, with the current columns and types. Asks first, unless --yes. Refuses postgres_cdc tables, which --resnapshot copies again; see Reloading.
--resnapshot postgres_cdc sources: drop and recreate the replication slot, then copy every table again. Asks first, unless --yes. See Postgres changes.
--follow load of one postgres_cdc source only: keep streaming after catching up, writing a batch every batch_interval, until --max-duration passes or Ctrl-C or SIGTERM. See Keep streaming.
--max-duration <DURATION> With --follow: stop after this long, such as 10m, once the changes read so far are written.
--yes Rebuild with --reload or --resnapshot without asking. Needed when not on a terminal.
--sync-deletes After each selected table with deletes merges, read every key of its source and mark its rows of the keys the source no longer holds. Fails when no selected table sets deletes. See Rows deleted at the source.
--allow-type-change-rebuild Allow table rebuilds that type_change_rebuild = "require_confirmation" would block.
--concurrency <N> The most tables loading at once. Default: the project’s concurrency, else 4.
--memory-limit <SIZE|off> The memory the run stays under, such as 4GiB, or off for no budget. Default: the project’s memory_limit. See Memory budget.
--fail-fast load: once a table fails, start no other table. See Stopping at the first failure.
--events jsonl, --events-file <PATH> load: write each load event as one line of JSON, to stderr or to a file. See Events.
--progress <auto|always|never> load: when to draw the live block of running tables on stderr. Default auto: on a terminal, without --json or --log-format json. See Events.
--memory-per-table Linux only: load loads one table at a time and reports each table’s peak memory. See Peak memory.
-v, --verbose Print every line of the plan: each table’s source, route and memory lines and the defaults it follows, each column of a new table, and every connection’s and discovery’s own line. See Reading a plan.
--json Print one JSON document on standard output. For plan, the plan. For apply and load, the result: see Results as JSON.

plan shows only the lines that are a table’s own or ask you to act. A table that keeps every default takes one line; -v shows everything else.

$ sqlload plan
✓ Connected local DuckDB · app PostgreSQL 0.03s
✓ Discovered 3 tables in 2 sources 0.02s
Objects (3) sqlload apply
├── raw.app__customers create table, 6 columns
├── raw.app__orders create table, 7 columns
└── raw.events create table, 8 columns
Loads (3) into DuckDB local sqlload load
├── app postgres into raw
│ ├── customers merge on customer_id
│ └── orders merge on order_id updated_at: first run, everything
└── events ./lake/events/ into raw
└── events delete_insert on dt dt: first run, everything · 3 new of 3 files
└── route DuckDB reads directly from ./lake/events/ (nothing through the loader)
✓ Plan complete 3 objects to create, 3 to load

Loads names the target, then each source with its kind (or its Python file or folder) and the schema its tables land in, when that isn’t the source’s name, and the target, when the plan has more than one. Each table’s line has its write strategy and key, then where its read starts, and for a files table, after ·, how many of the files it lists it reads. Once a table has rows, an incremental table’s read starts at the warehouse’s maximum:

$ sqlload plan
✓ Connected local DuckDB · app PostgreSQL 0.02s
✓ Discovered 3 tables in 2 sources 0.02s
Loads (3) into DuckDB local sqlload load
├── app postgres into raw
│ ├── customers merge on customer_id
│ └── orders merge on order_id updated_at ≥ 2026-11-02 08:04:00+00 (max)
└── events ./lake/events/ into raw
└── events delete_insert on dt dt ≥ 2026-11-03 (max) · 1 new of 3 files
└── route DuckDB reads directly from ./lake/events/ (nothing through the loader)
✓ Plan complete 3 to load

The start is ≥ when the bound row is read again and > when it isn’t, followed by where it comes from: (max), the warehouse maximum, (max − 5s) with a lookback, --start-cursor, or from state "…" (run 20d2b872) for a logged state.

A project with an http connection starts with a line for each, showing the limits every request to it shares in the run, its retries and its auth scheme, never its credentials:

http shop rate_limit 10/s, max_in_flight 4, retries 5, auth bearer

A child line under a table means it does something unlike the defaults, or needs a look:

Line Means
NULL cursors included, NULL cursors skipped null_cursor is set.
commits every 30 days The read commits after each window (batch_size).
repeated keys fail, newest by updated_at on_duplicate_keys = "fail", or newest_by.
route … (nothing through the loader) The warehouse reads the files itself.
memory ~1.9 GiB per fetch, over the 512 MiB fetch buffer An ODBC table’s row doesn’t fit its fetch buffer; the next line is the max_batch_memory that does.
deletes, transform, filter, flatten, parse, note The table’s own settings and notes.

Skipped tables get one line per reason:

Skipped (7)
├── no new files customers, prices, receipts, stock_counts, targets
└── up to date estimates (3 files), kpis (1 file)

sqlload plan -v shows every line: each table’s source, route, memory, fetch and buffer lines, the rules its cursor follows by default, and a new table’s columns one by one, as each portable type -> the warehouse type and what that type loses. Select one table with -v to also see its key, columns and checks, with times to the millisecond:

$ sqlload plan -v app__orders
✓ Target connected local DUCKDB (0.02s)
✓ Source connected app POSTGRES (0.01s)
✓ Discovered app 2 tables (0.02s)
✓ Discovered events 1 table (0.00s)
Plan ready 1 selected, 1 to load
Loads (1) sqlload load
└── raw.app__orders merge (unique_key: order_id ← {primary_key})
├── source postgres app public.orders
├── route read as Arrow -> DuckDB takes Arrow
├── key order_id, the primary key the source reports (unique_key = "{primary_key}", sqlload.toml:22)
├── reads updated_at >= 2026-11-02 08:04:00.000+00 (warehouse maximum)
├── columns 5
└── commits once, at the end of the read (updated_at isn't read in order)
✓ Plan complete 1 selected, 1 to load

The sections, as -v shows them; without it, Objects counts a new table’s columns, and Loads and Skipped keep only the lines above:

Section Lists
Objects Tables apply creates or alters: each column as its portable type -> the warehouse type, then in brackets what that type loses, if anything (for an array, the array’s own loss, then its elements’ after elements:), added and widened columns, with the same note, excluded and masked columns.
Loads Each table load loads, with its write strategy, and lines for source, key, reads, commits, files, schema, route, why, batch and note. A plan of one table also shows columns (how many, and the names naming changes), names (every column rename named or numbering numbered; see Naming) and checks, and key names the sqlload.toml line that sets unique_key.
Skipped Tables left out on purpose, with the reason: a files table with no new file (up to date: 0 new of 3 files in drop/orders/ (csv)), or one left out by skip_tables_with_columns or new_tables = "skip". load reads nothing for them.
Errors Tables that can’t load as configured, with a short headline, the reason and the fix. A source that can’t be read is one item, with what it holds up under blocks.
Warnings Tables that load, but slower or with less than you might expect.

A table under Errors is left out; the others still run, and the command exits 1.

plan, and the plan apply and load start with, times each target’s and source’s connection and each source’s discovery of its tables. Once they are done they collapse into two lines on standard error:

✓ Connected warehouse DuckDB · erp OpenEdge · app PostgreSQL 0.06s
✓ Discovered 17 tables in 9 sources 0.05s

A step that took a second or more keeps its own line, so a slow catalog stands out:

✓ Discovered erp 2 tables (21.30s)

A step that fails keeps its line too and says how long it ran, as ✗ Source connection failed erp POSTGRES (after 30.01s). -v prints every step’s line instead of the two summary lines. A step still running after 30 seconds writes a row every 30 seconds, such as erp listing tables RUNNING 30.0s, so logs that aren’t a terminal show it too. Postgres and ODBC sources have a connection step; other sources connect as their tables are found, within the discovery time. --json and --log-format json leave the lines out, and plan --json has the times under phases: one entry per connection ("phase": "connect") and discovery ("discover"), with its role (target or source), name, ok and elapsed_ms.

On a terminal, a spinner names the step running once it has run 300 ms, such as erp describing PUB.absence (2/5)… 18s, or between steps Planning checking sources, 1 of 3 done. --progress never leaves it out. It clears when the plan prints.