How do you stop it from touching data a user shouldn't see?

Several independent layers. Sign-in at the product level. Then policies that scope a user to particular data through the semantic layer, so the agent can work only with what that user could. Operational sources are read-only unless you say otherwise. And a check on the structured response can reject an answer before it is returned if it contains fields you have disallowed.

Can it act on operational systems without a person watching?

Not on anything that matters. A step that changes something plans the change, freezes that plan with a fingerprint of its inputs, routes it for approval with timeouts and escalation, runs only on approval, and then writes an append-only record.

If something goes wrong three months from now, can we reconstruct what happened?

Yes. Look up the workflow and the run id and you get each step, the SQL it ran, the tools it called, the tokens and cost it spent, the outcome of every gate, and which person approved what.

Where does our intellectual property live?

In your own object storage. Workflow specs, the skill library, custom tools, project memory, run history and the audit log are all versioned there. None of it is held in our systems.

Do prompts or data leave our boundary to reach a model?

Only if you route them there. Models are chosen per step from a registry your admin controls. A step that touches sensitive data can be pinned to a model you host, and inside your VPC with your own keys, nothing calls home.

Turn live data into immediately queryable tables

Bring high volume live data from databases, streams, applications, and files into your lakehouse. Shape and enrich records as they arrive and deliver fresh data near-instantly to open table formats.

Book a demo

WHY THE INGESTION ENGINE

Replace three tools with one. Get usable tables, faster.

Replace separate connector services, transformation jobs, and table writers with one product.

ONE SYSTEM

Streams and files, together

Bring streams, database changes, HTTP events, and files into one product and operating system.

NO SEPARATE JOBS

Shape data as it arrives

Filter, join, aggregate, enrich, and reshape records without waiting for a separate transformation job.

NO LOCK-IN

Output for the rest of the platform

Write to open tables that your existing tools can use.

HANDLE HIGH VOLUME

Proven on 1M+ events / sec

Production throughput on a single pipeline, with headroom for bursts and volume spikes.

DATA MOVEMENT

Connect sources. Push events. Publish results.

Data movement

The ingestion engine reads from connected sources, accepts events from applications, and writes the results to tables, streams, or files.

Read

Read database changes, topics, streams, and files in batches.

Push

Send webhooks and application events directly over HTTP or gRPC. No broker required.

Publish

Write processed data to supported tables, streams, and files.

Three paths · one runtime

Data movement

records read continuously

Connected sources

CDC · streams · files The engine readsfrom sources Applications pushHTTP · gRPC · webhooks events sent directly The engine acceptsevents The engine processesRecords processed records TablesStreamsFiles

Built-in CDC

Replicate data from hundreds of MySQL and PostgreSQL databases into your lake. The ingestion engine captures existing data, keeps tables up to date as records change, and sends each table to the right destination.

Replicate

Many tables from many databases in a single pipeline.

Reuse

The same database changes feed multiple pipelines.

Direct

Database to Iceberg, without Debezium and Kafka in between.

One service

Connect databases without a separate CDC service for every pipeline.

Many databases · one CDC path

CDC data path

PostgreSQL · orders

The ingestion engine

Built-in CDC Snapshot first changes continuously

Continuous processing

Transform data with SQL or call an external service without taking it out of the live pipeline.

orders.sql

Transform before the data lands

Continuous SQL Filter, join, aggregate, and reshape records as they arrive, including event-time windows and stateful updates.

Incoming record

User-defined functions Use UDFs for custom transformations and enrichments when built-in SQL is not enough.

Built-in table management

The ingestion engine keeps open lakehouse tables organized as new data arrives, without a separate maintenance job.

Partition and organize files as they are written.
Keep tables healthy as data grows.
Keep data ready for the query engines you already use.

Automatic · continuous

Table maintenance cycle

Small files accumulate

Records Arrive

Live records Records become files Compaction Occurs Files combine automatically Table Made Query-Ready Fewer, larger files

Result

Queries scan fewer, larger files.

Runtime and deployment

Run the ingestion engine in the environment and compute model that fits your stack.

01
Where can it run?
Broad environment support AWS Google Cloud Azure Private infrastructure
02
What does the ingestion engine need to run?
Kubernetes · Object Storage K8S Kubernetes Store Object Storage
03
What stays consistent?
Spread pipeline work across available machines. Increase throughput by running more work in parallel. Resume processing without starting over.

INTEGRATIONS

See where the ingestion engine fits

The engine works with the systems, storage, and catalogs already in your data platform.

Connect · process · deliver

Integration topology

PostgreSQL

Sources

MySQL KafkaConfluent KinesisHTTP gRPC Files The ingestion engine One livedata path ConnectProcessDeliver Outputs Apache IcebergDelta Lake KafkaFiles

Storage and catalog foundation

AWS S3

AWS GlueGoogle Cloud StorageBigLakeAzure BlobREST catalogsHive Metastore

BEFORE YOU DECIDE

Ingestion Engine Questions

Common questions about adding the ingestion engine to the data stack you already have.

Do we need to replace our current stack?

No. Start with one pipeline and keep the tools you already use. Move more workloads to the ingestion engine when you are ready.

Is the ingestion engine a database or query engine?

Neither. The ingestion engine gets live data ready and sends it where it needs to go. Keep using your existing query, analytics, and BI tools.

Does the ingestion engine use its own storage?

No. Your data lands in your storage and open table formats, where other tools can use it directly.

Does our data have to leave our environment?

No. The ingestion engine runs inside your cloud or private environment. It processes data there and sends results only to destinations you configure.

Does the ingestion engine replace Kafka?

It can remove Kafka from pipelines where Kafka is only moving data from one system to another. Keep Kafka where it is already the shared event backbone.

Does the ingestion engine replace dbt, Spark, or Flink?

The ingestion engine can take over live transformations that currently need separate jobs. Keep those tools for batch work, analytics, or other jobs they already handle well.

What will our team still need to operate?

You still own your source systems, storage, and analytics tools. The ingestion engine handles the live data path between them.

How can we evaluate the ingestion engine?

Pick one real source and one destination. We’ll build it with your data so you can judge the speed, effort, and fit for yourself.

TALK TO US

Book a demo on your own workloads

Reach out to book a demo, share challenges you’re facing, and tell us how this fits into what you’re currently working on or thinking about.

Problems we’re solving