# Data Catchups/Backfill

It's common in a contracts-centered development workflow to first deploy the contract and make sure things just work vs deploying the indexer first. We understand this crucial requirement and thus have set up a fast and efficient data catchup infrastructure that builds artificial ledger transitions.

This allows to backfill older data into your program ensuring that you index also actions performed before your indexer's deployment.

{% hint style="info" %}
Note that you'll only be allowed to catchup with contract events of a subset of contracts that you arbitrarily specify in the CLI when running the catchup. You cannot catchup will all of the emitted events for instance, you must know the contract's addresses beforehand.
{% endhint %}

### 1. Subscribe to the contract's events.

{% hint style="info" %}
After the last CLI update it's no longer needed to subscribe to the relevant events before the catchup: the CLI will perform the task automatically.
{% endhint %}

Before you can request a catchup with events from a given contract or contract, you must be subscribed to these events first. See [Introduction](/mercury-classic/introduction.md). Contract event subscriptions can be easily performed through the Mercury webapp ([testnet](https://test.mercurydata.app/subscriptions), [mainnet](https://main.mercurydata.app/subscriptions)).

The subscription is **only** required for catchups, not standard execution. Additionally, it doesn't currently mean higher resource usage or higher costs.

### 2. Deploy your program

Define your indexing logic beforehand and deploy the program. When you run the catchup, Zephyr will generate artificial ledger transitions to execute that same logic even for past data.

### 3. Call the catchup

To initiate a catchup you can use the following mercury CLI command.

* Mainnet:

{% code overflow="wrap" %}

```bash
mercury-cli --jwt $MERCURY_JWT --local false --mainnet true catchup --contracts "CONTRACT_1" --contracts "CONTRACT_2" --project-name "YOUR ZEPHYR PROJECT NAME"
```

{% endcode %}

* Testnet:

{% code overflow="wrap" %}

```bash
mercury-cli --jwt $MERCURY_JWT --local false --mainnet false catchup --contracts "CONTRACT_1" --contracts "CONTRACT_2" --project-name "YOUR ZEPHYR PROJECT NAME"
```

{% endcode %}

Executing this command will yield a catchup id which can be used to retrieve the status of the catchup.

#### RetroShades Catchup

To perform the catchup of a RetroShades program just add the `--retroshades true` flag:

{% code overflow="wrap" %}

```bash
mercury-cli --jwt $MERCURY_JWT --local false --mainnet true/false catchup --retroshades true --contracts "YOUR_CONTRACT" --project-name "YOUR_PROJECT_NAME"
```

{% endcode %}

{% hint style="success" %}
We almost always recommend to use scoped event catchups in this step instead of the above standard catchup. Learn more below in the [#scoped-event-catchups](#scoped-event-catchups "mention") section.
{% endhint %}

### 3. Reading catchup status.

To query the catchup status:

```
curl -X GET https://testnet.mercurydata.app/catchups/ID
```

{% hint style="info" %}
You can see the status of all catchups. This is intentional and not an IDOR.
{% endhint %}

## Reading catchup logs

Catchups logs, as well as other Mercury logs, can be found by clicking on the "User Logs" section in the dashboard's sidebar.

***

## Scoped Event Catchups

Sometimes we don't need to catchup all the events of a contract or at all past ledgers. For example, assuming that we're indexing a pool contract's deposits/withdrawals, but the pool also emits many airdrop or claim events, with a standard catchup we'd end up spinning a vm, and executing our program for artificial close metas that don't hold any of the information we're looking for.

To trigger a catchup scoped by event topics (this shows for mainnet but it's the same for testnet, just with `--mainnet false` ):

{% code overflow="wrap" %}

```
./mercury-cli --jwt $MERCURY_JWT --mainnet true catchup --contracts "CBP7NO6F7FRDHSOFQBT2L2UWYIZ2PU76JKVRYAQTG3KZSQLYAOKIF2WB" --topic1s "AAAADwAAABFzdXBwbHlfY29sbGF0ZXJhbAAAAA==" --topic1s "AAAADwAAABN3aXRoZHJhd19jb2xsYXRlcmFsAA==" --topic1s "AAAADwAAAAZib3Jyb3cAAA==" --topic1s "AAAADwAAAAVyZXBheQAAAA=="  --project-name "ybx-pool"
```

{% endcode %}

> Note that this was used to catchup Blend's YieldBlox pool on mainnet.

Alternatively, imagine a scenario where you need to index events from after a certain ledger because a new token was added to the pool and your indexer wasn't written with that in mind. In such a scenario, with standard catchups you'd need to ingest again all of your data, however scoped catchups allow to catchup starting from a certain ledger.

To trigger a catchup after a certain ledger (this shows for mainnet but it's the same for testnet, just with `--mainnet false` ):

{% code overflow="wrap" %}

```
./mercury-cli --jwt $MERCURY_JWT --mainnet true catchup --contracts "CBP7NO6F7FRDHSOFQBT2L2UWYIZ2PU76JKVRYAQTG3KZSQLYAOKIF2WB" --start 52810830 --project-name "ybx-pool"
```

{% endcode %}

{% hint style="info" %}
Note that you can combine both the topics and the ledger start in scoped event catchups
{% endhint %}

***

## Resources

* <https://blog.xycloo.com/blog/indexing-blend-ybx-pool>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.mercurydata.app/zephyr-full-customization/learn/data-catchups-backfill.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.