# DataPlot

> DataPlot is a company simulator for data-engineering practice. It plays the source systems of a simulated business (OLTP databases in Postgres, event streams in Kafka or Redpanda, daily file drops on disk or S3) so you can build ingestion, CDC, streaming and bronze, silver and gold layers on data that behaves like a real company's. It ships free as one Docker image (amd64 and arm64) with a web UI, a CLI and a guide in English and Portuguese.

## Run it

```
docker run -p 8080:8080 -v dataplot-data:/data gartito/dataplot
```

Then open http://localhost:8080, pick a company, size it and choose where the data goes. The `dataplot-data` volume keeps saved runs and the files they write. The same image runs the CLI:

```
docker run --rm gartito/dataplot scenario list
```

CLI commands: `scenario run`, `scenario estimate` (rows, bytes, memory and recommended destination sizes before a run) and `scenario manual` (a business manual of the company's systems and columns).

## Companies (packs)

- Supermarket chain (batch + stream): hyper, super and express stores, e-commerce, stock with replenishment and stock-outs, weekly promotions, loyalty, in one or several countries, each with its prices, taxes, opening hours and retail calendar. Systems: point of sale, NFC-e in Brazil (files), SAP-style ERP (Postgres), CRM and loyalty (Postgres), VTEX-style e-commerce (Kafka).
- Music streaming (stream): listening sessions with skips, high-volume and out-of-order app events, Friday releases, a long-tail catalogue with each country's genres, subscriptions in local currencies and monthly royalties per territory. Systems: app events (Kafka), Stripe-style subscriptions and billing (Postgres), distributors, DDEX and sales reports (files).
- Fast-food franchise (batch + stream): franchised and company-owned restaurants in one or several countries, counter, drive-thru, kiosk, app and delivery orders, each market's menu, prices and taxes. Systems: restaurant POS (Postgres), kiosk, app and drive-thru events (Kafka), delivery aggregators and royalty statements (files), supply, inventory and labour (Postgres).

Countries: the United States, the United Kingdom, Germany, France, Italy, Spain, India and Brazil; a run can mix them. Tables and columns are in English with the vendors' own names; documents are valid for each country (EIN, VAT numbers, GSTIN, CNPJ and CPF, EAN-13).

## Behaviour

- Loads history first, then streams live from a cutover date.
- Keyed tables get inserts, updates and deletes; rows carry `_op` (I/U/D). Postgres upserts per window, Kafka keys messages and sends tombstones for deletes, files append every version.
- Clean by default; dirty on demand (nulls, duplicates, badly formatted columns); late and out-of-order app events.
- Deterministic: the same seed writes exactly the same data.
- Destinations: Postgres, Kafka or Redpanda, and Parquet, CSV, NDJSON, JSON or XML files on disk or S3, with an optional Glue catalogue for Athena.
- DataPlot does not grade pipelines; how you build them is up to you.

## Terms

- Free to download and use. For use at a company, in a course or for demos, ask about a commercial licence on the website.
- Closed source; distributed only as a Docker image.
- The image has no telemetry and calls no server.

## Links

- [Website, English](https://dataplot.dev/)
- [Website, Portuguese](https://dataplot.dev/pt/)
- [Challenges waitlist and commercial licence](https://dataplot.dev/#next)
- [Contact](https://dataplot.dev/#contact)
- [Privacy](https://dataplot.dev/privacy.html)