How to backfill tick-level trade history

Years of individual prints and book updates, some from venues that no longer trade. Where the tape is sold, what a year of it weighs, what you may do with it.

Buy the archive rather than page it out of a live endpoint. Tardis.dev sells recorded exchange websocket messages as CSV from 2019, including venues that no longer trade; Binance, Bybit and OKX give away their own flat files under non-commercial terms; Kaiko, Amberdata and Coin Metrics deliver into your own bucket or warehouse. Acquisition is the easy half. Reconstruction, clocks and the licence are the rest.

The short way

Two questions settle this before any vendor does: does the venue still exist, and do you need the book or only the prints.

If the venue is alive and trades are enough, take its own archive. Binance, Bybit and OKX each publish per-symbol zipped CSV on an open file host — no key, no account, and nothing metered that is worth planning around:

data.binance.vision/data/spot/monthly/trades/{SYMBOL}/{SYMBOL}-trades-{YYYY}-{MM}.zip
public.bybit.com/trading/{SYMBOL}/

Backfilling from those is a loop over filenames rather than a paged REST crawl, which is the whole reason to prefer them: pulling four years of prints out of a recent-trades endpoint is a rate-limit problem that the file host simply does not have. See rate limit for what the endpoint route actually spends.

If you need order-book depth, or the venue is gone, you are buying. Tardis.dev is the self-serve end of that: recorded exchange websocket messages, replayed as CSV, from 30 March 2019. Read the access tier before the data plan — the card records that Academic and Solo get downloadable CSV only, and the replay API and the local replay server start at Professional.

What the options are

The venue's own dumps. Free, unkeyed and licensed for you alone. Binance API carries monthly and daily klines, trades, aggregate trades and futures funding history on its file host. Bybit API publishes per-symbol daily trade files with the BTCUSDT perpetual running from 25 March 2020 — and blocks US and mainland China IP addresses at the API, which for many readers decides this before any feature does. OKX API's historical-data portal starts later than people expect: tick trades from September 2021, candlesticks from July 2023, order-book L2 from March 2023. An archive is younger than the exchange, always.

The recorded tape. Tardis.dev, and it is the only self-serve product here that sells raw message history rather than a normalised summary of it. Nine CSV data types — incremental L2 updates, 25-level and 5-level book snapshots, trades, quotes, book ticker, derivative ticker, liquidations and options chains — across 64 exchange feeds, twelve of which are frozen at the date their venue stopped. The billing interval, not the plan, decides how far back you can reach: monthly four months, quarterly twelve, yearly four years, and only Business on yearly billing reaches 2019.

The institutional normalised set. Kaiko and Amberdata sell one symbol scheme over many venues with a contract behind it. Kaiko goes deepest on trades — 2010 — and is the one to ask about pre-2013 history; Amberdata is the one of the two you can start with a card, at 600 USD a month for a single exchange and a single market, with bulk delivery as Parquet into S3, Snowflake or Databricks. Neither publishes a self-serve price for the whole archive, which is the priced on request pattern this end of the market runs on.

The warehouse drop. When the destination is a research cluster rather than a laptop, the useful shape is a daily file in your own bucket. Coin Metrics publishes ten datasets as Apache Parquet in Amazon S3 and as Snowflake tables — market trades, candles, quotes, order-book snapshots and updates, reference rates and the catalogs — and its coverage narrows as the data gets finer: 48 exchanges for trades and candles, 29 for quotes and book snapshots, 5 for order-book updates. CoinAPI sells the same idea as Flat Files over an S3-compatible API and through Snowflake, which is the right product there rather than the REST one: its credits bill one per 100 data points returned when you use the limit parameter, so paging years of history through the API is the expensive way to do it.

Where this breaks

The venues you most want are the ones nobody can re-issue. A dead exchange cannot serve you its own archive, and the tape of it survives only where somebody was recording at the time. Tardis.dev retains FTX end to end, 1 August 2019 to 13 November 2022, and freezes a dozen feeds at the day they stopped — FTX and FTX US and Serum on 13 November 2022, Mango on 14 October 2022, CoinFLEX on 7 April 2023, dYdX v3 on 30 October 2024, OKCoin on 1 October 2025, AscendEX on 7 July 2026. Coin Metrics keeps Mt. Gox, FTX, Bittrex, Liquid and Gatecoin in its reference data for the same reason, and Amberdata still lists FTX, FTX US, OKCoin and ZB among its venues. This also runs forwards: Amberdata's card records that BitMEX data ends on 23 September 2026 because the exchange does. Download while the host is still up; that is the whole of the advice.

Two clocks, and the difference between them is data rather than noise. Tardis.dev's CSV schema carries both: timestamp is "timestamp provided by the exchange in microseconds since epoch" and local_timestamp is "message arrival timestamp in microseconds since epoch", both UTC. A venue's own dump usually gives you one clock, in milliseconds, and does not say which of the two it is. That matters twice. Ordering: inside a millisecond bucket the sequence you see is the order the file was written in, not a fact about the market, so anything that depends on which of two prints came first is unreliable at that resolution. And latency: exchange time minus arrival time is the only way to tell a slow venue from a slow collector, and a dataset with a single clock has thrown that distinction away before you got it. Joining a venue archive to a vendor archive without deciding which clock is authoritative produces a series nobody can reproduce, including you.

An order book is a replay, not a file. Depth arrives as increments, and the procedure for turning them back into a book is exact. Binance publishes it as numbered steps: buffer the stream, fetch a depth snapshot, "drop any event where u is < lastUpdateId in the snapshot", require the first processed event to satisfy U <= lastUpdateId and u >= lastUpdateId, and then check each new event's pu against the previous event's u — "otherwise initialize the process from step 3". Quantities are absolute rather than deltas, and a zero removes the level. Read step 3 again with a historical file in mind: there is no snapshot to re-request from 2021. One missed sequence number does not spoil one row, it invalidates every book state after it until the next snapshot, and if the dataset has no periodic snapshots in it, that means the rest of the day. This is what Tardis.dev's is_snapshot column on incremental_book_L2 exists for, and it is the single field worth checking before you buy anyone's depth archive.

The size is the part that surprises people who priced the subscription first. The public bucket listing for Binance spot BTCUSDT trades shows twelve monthly zips for 2025, from 501 MB in September to 1.26 GB in March, about 11.1 GB compressed for the year. One pair. One venue. One market. Trades only, with no order book, no perpetual, no option and none of the other thousands of symbols. Incremental depth is a much larger message count than trades, and multiplying by venues and instruments is how a research backfill becomes a storage bill. The vendors have already conceded this in their product shapes: Amberdata's REST API rejects a result over 10 MB with a 400 and times out at 28 seconds, which is why its deep history is a bulk drop rather than an endpoint, and Kaiko's full order book is a once-a-day CSV rather than a REST route at all.

And what you may do with it afterwards is a separate agreement from what you paid for. The free archives are the sharpest case: Binance's Vision dataset terms license the downloads CC BY-NC-SA 4.0, bind you "upon downloading, caching, querying, or otherwise utilizing" them, and bar hosting or commercially exploiting the datasets or any derivative feed. Paid does not dissolve it either — Tardis.dev's licence permits reselling derived data only as aggregates at ten-minute resolution or coarser, with Coinbase carved out of even that exception, and CoinAPI's usage policy counts a customer-facing dashboard as redistribution. If a model is the plan rather than a chart, read the machine-learning clause specifically: Tardis.dev permits internal quantitative models and prohibits training, fine-tuning, distilling or benchmarking anything else on the data. See redistribution for what the word covers, and what exchange API terms actually let you do for the terms themselves — this page does not repeat them.

If you outgrow this

If the problem is that the download has become the job, stop calling endpoints. Bulk delivery into your own bucket is a different product rather than a bigger quota: Parquet in S3, tables in Snowflake, one file per day, and a restart that resumes instead of re-paging. Coin Metrics, Amberdata's CloudSync and CoinAPI's Flat Files are all that shape, and the cost is that you now operate storage.

If the problem is that the reconstruction has to be defensible, buy depth that ships snapshots beside its increments, and store the snapshot cadence as part of the dataset rather than as a note. A book you rebuilt from increments alone is only as good as the last sequence check that passed, and the honest version of that is a gap log you can show somebody.

If the problem is that trades are not enough and the candles you derived disagree with the vendor's, that is not a bug in the backfill. A bar is an aggregation with somebody's interval boundaries inside it — OHLCV is why two feeds labelled one-minute for the same pair are routinely two different series, and how to pull OHLCV candles from a crypto exchange is the shallower version of this job.

And if the problem is the licence rather than the data, the route out is written into the same documents that block you: the enterprise or redistribution agreement is a product these vendors sell, it has no published price, and it is the reason a paid archive layer exists over feeds anybody can read for free. Budget a procurement cycle, not an afternoon, and see the rest of Crypto Market Data APIs for who is in that layer.

The tools named above

In the order this page puts them in, which is an editorial judgement and not a ranking anyone paid for.

  1. Tardis.dev

    Recorded websocket messages as CSV from 30 March 2019, dead venues included — but the replay API starts at the Professional tier, not the cheapest one.

    Tick-by-tick order books, trades and options chains from 64 exchange feeds since 2019.

    $350/moFree tier

  2. Binance Market Data API

    One venue's own zipped CSV dumps, free and unkeyed on an open file host, licensed CC BY-NC-SA 4.0 — which decides what you may do with them afterwards.

    One venue's own tape — prices, books, trades and funding, free and unkeyed.

    FreeFree tier

  3. Amberdata Market Data API

    One exchange and one market bought with a card, with REST order-book history capped at 18 months and anything deeper sold as a bulk drop.

    Spot, futures and options data — and one exchange's feed you can buy with a card.

    $600/mo

  4. Kaiko Market Data API

    Tick trades and OHLCV back to 2010 on REST, but full order book only as a once-a-day CSV from August 2023, and only on a quoted contract.

    Tick history back to 2010, by REST, gRPC stream or cloud delivery.

  5. Coin Metrics

    Ten datasets as daily Parquet in S3 and Snowflake — note the coverage falls from 48 exchanges for trades to 5 for order-book updates.

    Institutional reference rates, exchange market data and network metrics behind one API.

    Free tier onlyFree tier

FAQ

How far back does tick history actually go?

Further for trades than for order books, everywhere, and that gap is the thing to check first. Kaiko sells tick trades and OHLCV back to 2010, but its full order book is a once-a-day CSV starting August 2023 and top of book starts December 2022. Amberdata's earliest order-book snapshot on any venue is January 2018 against OHLCV from August 2011. Tardis.dev starts at 30 March 2019 for its founding venues and at the collection date for everything added since. A pitch that says "since 2010" is describing the trade tape, not the book.

Can I just download the exchange's own files instead of paying a vendor?

For a live venue and for trades, often yes — Binance, Bybit and OKX all publish zipped CSV on open file hosts with no key. Three things decide whether that is enough — the archive usually starts later than the venue did, order-book depth is mostly absent from it, and the licence is non-commercial. Binance's Vision datasets are CC BY-NC-SA 4.0 and bind you on downloading, caching or querying, so a private backtest is fine and a product is a separate negotiation.

Why can I not rebuild the order book from the file I downloaded?

Because an incremental feed is a sequence, not a set of rows. Binance's own procedure has you buffer the stream, fetch a depth snapshot, discard events older than it, align the first event across the snapshot's update id, and then check that every event's previous-final id matches the last one you applied — "otherwise initialize the process from step 3". In a historical file there is no step 3 to go back to unless the dataset carries periodic snapshots of its own, so one gap invalidates every state after it.

How much storage does a year of this need?

More than the price list suggests. The public bucket listing for BTCUSDT spot trades shows twelve monthly zips for 2025 running from 501 MB to 1.26 GB, about 11.1 GB compressed for the year. That is one pair, on one venue, in one market, with no order book, no perpetuals and no options in it. Incremental book updates are far more numerous than trades, so a multi-venue book archive is a storage plan before it is a data purchase.

Sources

  1. Downloadable CSV files — data types and column schemas Tardis.dev, read
  2. How to manage a local order book correctly, USDⓈ-M futures websocket market streams Binance, read
  3. Monthly spot trade archives for BTCUSDT: data.binance.vision bucket listing Binance, read

The catalogue next door

This page names a handful of products. The rest of them are in Crypto Market Data APIs, each filled in against the same schema, with the fields to narrow it yourself.

Last updated . Corrected in place — an endpoint that moves is a bug on this page, not a new post.