Jobs API - Getting information about jobs

There’re some operations that can start a job in Tinybird:

  • Import data via URL.

  • Populate a Data Source from a materialized node.

  • Run background queries.

When any of these operations start, the response contains a job_id field.

With the Jobs API, you can list the jobs for the last 7 days or the last 100 jobs.

You can also get the details for a job. Depending on the job kind (import, query, or populate), it will return certain information related with the specific job, along with the status of the job.

The Job status can be one of the following:

  • waiting: The initial status of a job. When creating a job, it has to wait if there’re other jobs running.

  • working: Once the job operation has started.

  • done: The job has finished successfully.

  • error: The job has finished with an error.

GET /v0/jobs/?

We can get a list of jobs for the last 7 days or the last 100 jobs:

Getting the latest jobs
curl \
-H "Authorization: Bearer <token>" \
"https://api.tinybird.co/v0/jobs" \

You will get a list of jobs with the kind, status, id, and the url to access the specific information about that job.

Jobs list
{
    "jobs": [
        {
            "id": "c8ae13ef-e739-40b6-8bd5-b1e07c8671c2",
            "kind": "import",
            "status": "done",
            "created_at": "2020-12-04 15:08:33.214377",
            "updated_at": "2020-12-04 15:08:33.396286",
            "job_url": "https://api.tinybird.co/v0/jobs/c8ae13ef-e739-40b6-8bd5-b1e07c8671c2"
        },
        {
            "id": "1f6a5a3d-cfcb-4244-ba0b-0bfa1d1752fb",
            "kind": "import",
            "status": "error",
            "created_at": "2020-12-04 15:08:09.051310",
            "updated_at": "2020-12-04 15:08:09.263055",
            "job_url": "https://api.tinybird.co/v0/jobs/1f6a5a3d-cfcb-4244-ba0b-0bfa1d1752fb"
        }
    ]
}
POST /v0/jobs/(.+)/cancel

With this endpoint you can try to cancel an existing Job. All jobs can be cancelled if they are in the “waiting” status, but you can’t cancel a Job in “done” or “error” status.

In the case of the job of type “populate”, you can cancel it in the “working” state.

After successfully starting the cancellation process, you will see two different status in the job: - “cancelling”: The Job can’t be immediately cancelled as it’s doing some work, but the cancellation will eventually happen. - “cancelled”: The Job has been completely cancelled.

A Job cancellation doesn’t guarantee a complete rollback of the changes being made by it, sometimes you will need to delete new inserted rows or datasources created.

The fastest way to know if a job is cancellable, is just reading the “is_cancellable” key inside the job JSON description.

Depending on the Job and its status, when you try to cancel it you may get different responses: - HTTP Code 200: The Job has successfully started the cancellation process. Remember that if the Job has now a “cancelling” status, it may need some time to completely cancel itself. This request will return the status of the job. - HTTP Code 404: Job not found. - HTTP Code 403: The token provided doesn’t have access to this Job. - HTTP Code 400: Job is not in a cancellable status or you are trying to cancel a job which is already in the “cancelling” state.

Try to cancel a Job
curl \
-H "Authorization: Bearer <token>" \
-X POST "https://api.tinybird.co/v0/jobs/:job_id/cancel"
Populate Job in cancelling state right after the cancellation request.
{
    "kind": "populateview",
    "id": "32c3438d-582e-4a6f-9b57-7d7a3bfbeb8c",
    "job_id": "32c3438d-582e-4a6f-9b57-7d7a3bfbeb8c",
    "status": "cancelling",
    "created_at": "2021-03-17 18:56:23.939380",
    "updated_at": "2021-03-17 18:56:44.343245",
    "is_cancellable": false,
    "query_id": "2cf10c06-cc9c-4520-8829-90e24db9a476",
    "pipe_id": "t_7fa8009023a245b696b4f2f7195b23c3",
    "pipe_name": "top_product_per_day",
    "queries": [
        {
            "query_id": "23c0badd-02da-4219-aba5-f9e0f25c9c1a",
            "status": "done"
        },
        {
            "query_id": "f1a55ded-1862-49ef-80c5-b8288583b1a7",
            "status": "done"
        },
        {
            "query_id": "1862ee52-5a0e-4986-ac43-24f480e7bd8a",
            "status": "working"
        },
        {
            "query_id": "25e72b0c-90e8-4051-a7f7-deeb2f92b49f",
            "status": "waiting"
        },
        {
            "query_id": "f4311fb6-400c-47ce-b11e-fcb2250e371a",
            "status": "waiting"
        },
        {
            "query_id": "4921da51-0489-4d03-a3b1-2e410040d605",
            "status": "waiting"
        }
    ],
    "progress_percentage": 50.0
}
GET /v0/jobs/(.+)

You can get the information for a job as follows:

Getting information about a job
curl \
-H "Authorization: Bearer <token>" \
"https://api.tinybird.co/v0/jobs/:job_id" \

You will receive specific information about the job apart from the common fields found in the jobs listing.

Import job has finished successfully
{
    "id": "c8ae13ef-e739-40b6-8bd5-b1e07c8671c2",
    "job_id": "c8ae13ef-e739-40b6-8bd5-b1e07c8671c2",
    "kind": "import",
    "status": "done",
    "statistics": {
        "bytes": 1913,
        "row_count": 2
    },
    "datasource": {
        "id": "t_0ab7a11969fa4f67985cec481f71a5c2",
        "name": "your_datasource_name",
        "cluster": null,
        "tags": {},
        "created_at": "2020-07-15 10:52:21.900886",
        "updated_at": "2020-07-15 10:52:22.335639",
        "statistics": {
            "bytes": 1913,
            "row_count": 2
        },
        "replicated": false,
        "version": 0,
        "project": null,
        "used_by": []
    }
}

If there’s been an error in the import operation, the job response will also include a detailed error:

Job has finished with errors
{
    "id": "1f6a5a3d-cfcb-4244-ba0b-0bfa1d1752fb",
    "job_id": "1f6a5a3d-cfcb-4244-ba0b-0bfa1d1752fb",
    "kind": "import",
    "status": "error",
    "statistics": null,
    "datasource": {
        "id": "t_02043945875b4070ae975f3812444b76",
        "name": "your_datasource_name",
        "cluster": null,
        "tags": {},
        "created_at": "2020-07-15 10:55:12.427269",
        "updated_at": "2020-07-15 10:55:12.427270",
        "statistics": null,
        "replicated": false,
        "version": 0,
        "project": null,
        "used_by": []
    },
    "quarantine_rows": 0,
    "invalid_lines": 0,
    "error": "There was an error with file contents",
    "errors": [
        "There are blocks with errors",
        "failed to normalize the CSV chunk: [Error] Cannot read DateTime: unexpected number of decimal digits for time zone offset: 6"
    ]
}