← All lessons
Querying data · Lesson 5 of 7

Window functions

GROUP BY collapses rows into one per group. A window function keeps every row and adds a value worked out over a set of related rows, the window. OVER (…) is what makes a function a window function.

Inside OVER, PARTITION BY splits the rows into groups (like GROUP BY, but nothing collapses) and ORDER BY sets the order within each group. ROW_NUMBER() numbers the rows 1, 2, 3 in that order. Each customer's orders, numbered by date:

SELECT customer_id, id, ordered_on,
  ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY ordered_on) AS nth
FROM orders
ORDER BY customer_id, nth;
Returns
customer_id | id  | ordered_on | nth
------------+-----+------------+----
1           | 101 | 2024-09-01 | 1
1           | 103 | 2024-09-10 | 2
1           | 108 | 2024-10-05 | 3
2           | 102 | 2024-09-03 | 1
2           | 106 | 2024-09-20 | 2
3           | 104 | 2024-09-12 | 1
3           | 110 | 2024-10-11 | 2
4           | 105 | 2024-09-15 | 1
5           | 107 | 2024-10-02 | 1
6           | 109 | 2024-10-07 | 1
(10 rows)

Ties are where the numbering functions differ. ROW_NUMBER always gives 1, 2, 3, 4, so with a tie the order between the tied rows is arbitrary unless you add a tie-breaker. RANK gives tied rows the same number and then skips (1, 2, 2, 4). DENSE_RANK doesn't skip (1, 2, 2, 3).

WITH counts AS (
  SELECT customer_id, COUNT(*) AS n FROM orders GROUP BY customer_id
)
SELECT customer_id, n,
  ROW_NUMBER() OVER (ORDER BY n DESC, customer_id) AS row_num,
  RANK() OVER (ORDER BY n DESC) AS rnk,
  DENSE_RANK() OVER (ORDER BY n DESC) AS dense
FROM counts
ORDER BY n DESC, customer_id;
Returns
customer_id | n | row_num | rnk | dense
------------+---+---------+-----+------
1           | 3 | 1       | 1   | 1
2           | 2 | 2       | 2   | 2
3           | 2 | 3       | 2   | 2
4           | 1 | 4       | 4   | 3
5           | 1 | 5       | 4   | 3
6           | 1 | 6       | 4   | 3
(6 rows)

LAG(x) reads x from the row before in the window, and LEAD(x) from the row after; the first row's LAG is NULL. Good for "change since last time" and "days between orders":

SELECT customer_id, ordered_on,
  LAG(ordered_on) OVER (PARTITION BY customer_id ORDER BY ordered_on) AS previous
FROM orders
WHERE customer_id IN (1, 2)
ORDER BY customer_id, ordered_on;
Returns
customer_id | ordered_on | previous
------------+------------+-----------
1           | 2024-09-01 | NULL
1           | 2024-09-10 | 2024-09-01
1           | 2024-10-05 | 2024-09-10
2           | 2024-09-03 | NULL
2           | 2024-09-20 | 2024-09-03
(5 rows)

An aggregate with OVER (ORDER BY …) gives a running total: each row adds up everything up to and including itself. (With ties in the ORDER BY the default frame treats tied rows as one step; write ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW to go strictly row by row.)

WITH totals AS (
  SELECT o.id, o.ordered_on, SUM(i.qty * p.price) AS total
  FROM orders o
  JOIN order_items i ON i.order_id = o.id
  JOIN products p ON p.id = i.product_id
  WHERE o.status = 'shipped'
  GROUP BY o.id, o.ordered_on
)
SELECT ordered_on, total,
  SUM(total) OVER (ORDER BY ordered_on) AS running_total
FROM totals
ORDER BY ordered_on;
Returns
ordered_on | total | running_total
-----------+-------+--------------
2024-09-01 | 49    | 49
2024-09-03 | 40    | 89
2024-09-10 | 47.25 | 136.25
2024-09-15 | 93.5  | 229.75
2024-09-20 | 59    | 288.75
2024-10-05 | 9.75  | 298.5
2024-10-07 | 50    | 348.5
2024-10-11 | 81.5  | 430
(8 rows)

Window functions are worked out after WHERE, GROUP BY and HAVING, so you can't filter on one in WHERE. Put the query in a CTE and filter outside it. Some warehouses (Snowflake, BigQuery, Databricks and DuckDB among them) add a QUALIFY clause for exactly this.

Practice

Number each customer's orders by date: customer_id, the order id, and the number (1 for their first order).

The tables
  • customers (id, name, city, joined)
  • products (id, name, category, price)
  • orders (id, customer_id, ordered_on, status)
  • order_items (order_id, product_id, qty)
  • signups (id, email, name, signed_up_at)
  • payments (id, order_id, amount, currency, paid_at)
More info

Quick reference

f() OVER (PARTITION BY a ORDER BY b)A value per row, over the rows that share a, in b order.
ROW_NUMBER()1, 2, 3, 4: always different.
RANK(), DENSE_RANK()Ties share a number: 1, 2, 2, 4 and 1, 2, 2, 3.
LAG(x), LEAD(x)x from the row before, the row after.
SUM(x) OVER (ORDER BY d)A running total.
QUALIFYFilter on a window function (some warehouses only).

The other AI and data engineering courses