Introduction

Tinybird Analytics provides you with an easy way to ingest and query large amounts of data in real time, as well as to automatically create an API to consume those queries. This makes it extremely easy to build fast and scalable applications that query your data; no back-end needed!

Tinybird Analytics works with CSV files. If you have one around (hosted online or in your local drive), you can import it to your account directly through Tinybird’s UI or using our APIs. And if you don’t have any .CSV file around, you might want to try with a few million Taxi rides from the NYC Taxi Data or with hundreds of thousands of trees from the NYC Tree Atlas.

As an example, the code below illustrates how you can quickly build a simple frontend application which queries a few hundred thousand NYC taxi rides using Tinybird.js:

local file with Taxi rides
$ head green_tripdata_2018-12.csv
VendorID,lpep_pickup_datetime,lpep_dropoff_datetime,store_and_fwd_flag,RatecodeID,PULocationID,DOLocationID,passenger_count,trip_distance,fare_amount,extra,mta_tax,tip_amount,tolls_amount,ehail_fee,improvement_surcharge,total_amount,payment_type,trip_type

2,2018-12-01 00:23:25,2018-12-01 00:24:47,N,1,193,193,1,.00,3,0.5,0.5,0,0,,0.3,4.3,2,1
2,2018-11-30 23:58:31,2018-12-01 00:21:53,N,1,80,82,2,5.75,21,0.5,0.5,0,0,,0.3,22.3,2,1
2,2018-12-01 00:21:03,2018-12-01 00:30:15,N,1,225,225,1,1.55,8,0.5,0.5,1.86,0,,0.3,11.16,1,1
2,2018-12-01 00:15:56,2018-12-01 00:23:26,N,1,75,236,2,1.32,7,0.5,0.5,0,0,,0.3,8.3,1,1
2,2018-12-01 00:28:32,2018-12-01 00:30:33,N,1,75,75,2,.68,4,0.5,0.5,1.06,0,,0.3,6.36,1,1
2,2018-12-01 00:50:07,2018-12-01 00:55:51,N,1,75,263,2,.97,6,0.5,0.5,1,0,,0.3,8.3,1,1
2,2018-12-01 00:30:28,2018-12-01 00:46:46,N,1,256,37,1,2.69,12.5,0.5,0.5,2.76,0,,0.3,16.56,1,1
1,2018-12-01 00:03:42,2018-12-01 00:07:25,N,1,166,151,1,.90,5,0.5,0.5,0,0,,0.3,6.3,2,1
create a datasource with that file
$ curl -F "csv=@green_tripdata_2018-12.csv" \
  -H 'Authorization: Bearer TOKEN' \
  https://api.tinybird.co/v0/datasources?name=rides

Your data has now been imported into a Tinybird Analytics datasource called ‘rides’. And when you create a datasource in Tinybird Analytics, a transformation pipeline (or a pipe) will be created for you as well.

review the datasource you have just created
$ curl -H 'Authorization: Bearer TOKEN' \
  https://api.tinybird.co/v0/datasources/rides
  {
    "id": "t_8f531f1990894e3c96eec8a023a0a30a",
    "name": "rides",
    "created_at": "2019-07-16 16:07:01.387066",
    "updated_at": "2019-07-16 16:07:01.387072",
    "used_by": [
      {
        "id": "t_0c581f4947b046698b30337d2cae511b",
        "name": "rides_pipe"
      }
    ]
  }

A pipe is essentially a sequence of one or more SQL queries (transformations) applied to your datasource; you can publish the final transformation of any given pipe as an API endpoint. This gives you the ability to transform that intial raw CSV data into something that you can publish and use in another app with very little effort.

Let’s query the pipe you just created when importing your datasource; by default, Tinybird Analytics will name it just like the datasource plus a ‘_pipe’ suffix, so yours will be ‘rides_pipe’.

get information about the pipe itself
$ curl -H 'Authorization: Bearer TOKEN' \
  https://api.tinybird.co/v0/pipes/rides_pipe
  {
    "id": "t_0c581f4947b046698b30337d2cae511b",
    "name": "rides_pipe",
    "published_version": "t_28460f46fcfc40b4b3d735c0cd588e32",
    "published_date": "2019-07-16 16:07:01.387529",
    "nodes": [
      {
        "id": "t_28460f46fcfc40b4b3d735c0cd588e32",
        "name": "rides_pipe_0",
        "description": null,
        "sql": "select * from rides",
        "dependencies": [
          "rides"
        ],
        "materialized": false,
        "created_at": "2019-07-16 16:07:01.387514",
        "updated_at": "2019-07-16 16:07:01.387514"
      }
    ],
    "created_at": "2019-07-16 16:07:01.387386",
    "updated_at": "2019-07-16 16:07:01.387520"
  }

As you can see, there is only one transformation node in the pipe at first, which by default is a simple query against your datasource.

Let’s now add a transformation to get the number of rides per day.

adds a transformation to get number of rides per day
$ curl \
  -d 'SELECT toDate(lpep_pickup_datetime) date, count() nrides \
  FROM rides GROUP BY date ORDER BY date' \
  -H 'Authorization: Bearer TOKEN' \
  https://api.tinybird.co/v0/pipes/rides_pipe/append
  {
    "id": "t_11fb181f573b47b7a94153778ca2f0fb",
    "name": "rides_pipe_1",
    "description": null,
    "sql": "SELECT toDate(lpep_pickup_datetime) date, count() nrides FROM rides GROUP BY date ORDER BY date",
    "dependencies": [
      "rides"
    ],
    "materialized": false,
    "created_at": "2019-07-16 16:32:06.231977",
    "updated_at": "2019-07-16 16:32:06.231977"
  }

Notice how the query points to ‘rides’ (the datasource), but it would also be perfectly fine to do it against the result of the first transformation node ‘rides_pipe_0’.

Let’s now publish the node you just created to enable it as an API endpoint. For that, you just have to mark that node as ‘published’

publish the node
$ curl \
  -X PUT
  -d 't_11fb181f573b47b7a94153778ca2f0fb' \
  -H 'Authorization: Bearer TOKEN' \
  https://api.tinybird.co/v0/pipes/rides_pipe/published

You can now get the result of the transformation directly by issuing a GET request to the pipe.

get the data as a regular API
$ curl -H 'Authorization: Bearer TOKEN' \
  https://api.tinybird.co/v0/pipes/rides_pipe.json
  {
    "meta": [
      {
        "name": "date",
        "type": "Date"
      },
      {
        "name": "nrides",
        "type": "UInt64"
      }
    ],
    "data": [
      {
        "date": "2008-12-31",
        "nrides": 1
      },
      {
        "date": "2009-01-01",
        "nrides": 21
      },
      {
        "date": "2010-09-23",
        "nrides": 11
      },
    ...
  }
or with javascript
//norun
let t = tinybird('token').pipe('rides_pipe')
var res = await t.query(`select * from _`)
console.log(`rides on 2018-12-02 -> {res.data['2018-12-02'}`)
Simple tinybird.js application
<!doctype html>
<html lang="en">

  <head>
    <meta charset="utf-8">
    <title>Simple tinybird application</title>
    <script src='https://cdn.tinybird.co/v1/tinybird.js'></script>
  </head>

  <body>
    <script>
      let t = tinybird('<token>').pipe('<pipe>')
      var count = await t.single(`select count() from _`)
      console.log(`your pipe has ${count} elements`)
    </script>
  </body>

</html>

Everything you will find in the documentation is designed in a way that makes Tinybird Analytics:

  • Easy to use (no backend needed).

  • Fast in performing analytical workloads.

  • Secure.

  • Autoscalable.

When building applications, you can always use our REST API, but we highly recommend you use our official JS API library. If you are new here or coding is not your thing, we recommend you use Tinybird Analytics’s UI to import your data.

If you are already logged in, the documentation will be specifically generated for you; therefore your data and tokens, when shown, will be displayed just to you (this will save you tons of time when testing things!).

Next steps

Now that you know enough to create your first application with Tinybird Analytics, learn more about how to run queries using Tinybird.js or how to import your data with our REST API.