(Cache) Congestion Control

Kian Bahadori∗, Mahdi Hosseini∗, Elie Bruno, Muhammad Abdullah, Pavlos Nikolopoulos, Katerina Argyraki · EPFL · HotNets ’26

∗ These authors contributed equally to this work.

Can a shared cache be fair and efficient?

Step 1 · Simple Example

Three providers, one small cache

A cache keeps copies of content close to users. A request for an object in the cache is a hit, served locally and fast. Otherwise it is a miss: the object comes from the provider’s distant origin server, slower and usually paid for by the provider.

A content provider, such as a video service, has a catalogue of objects (e.g. video chunks). Here three providers share a cache with 4 slots, one object per slot.

The request rate of each object is shown under each object.

Illustrative example

Shared cache · 4 slots

No request yet.

Overall hit rate

What is shown: the cache holds the four most-requested objects. 39 of 48 req/min are hits, but provider C gets none. Who should get the cache?

Step 2 · The trade-off

Whose objects should the four slots hold?

Click an object to add it to the cache or remove it. Two numbers update:

  • provider hit rate: the share of a provider’s requests that are hits;
  • overall hit rate (= efficiency): the share of all 48 requests that are hits.
Illustrative example

Shared cache · 4 slots

Overall hit rate (efficiency)

Step 3 · The model

Fairness-efficiency trade‑off

3.1

The problem

Fair allocations are not always maximally efficient.

max{ho}efficiency s.t.capacity constraint
max{ho}efficiency s.t.capacity constraint + fairness guarantee
max{ho}Σi Σo∈Oi λoho s.t.Σi Σo∈Oi soho ≤ C 0 ≤ ho ≤ 1
max{ho}Σi Σo∈Oi λoho s.t.Σi Σo∈Oi soho ≤ C 0 ≤ ho ≤ 1 Vi(hi) = V* ∀ i ∈ I

Capacity: object o uses soho on average; the total must fit in C.

ho
object o’s hit rate = share of time it is cached
λo, so
its request rate and size
C
cache capacity
I
the set of providers
Oi
provider i’s catalogue
Σi Σo∈Oi λoho
the aggregate hit rate
Λi = Σo∈Oi λo
provider i’s request volume
hi = Σo∈Oi λoΛiho
its provider hit rate
Vi(hi)
its net value from hits; fair = equal net values, V*. Here V(h) = h: equal hit rates
hi*
its hit-rate target under fairness; here one common h*

3.2

When is a fair allocation efficiency-friendly?

Two factors decide it.

Factor 1 · Request volumes of providers

Allocating a high hit-rate target to a large-volume provider is better for the aggregate hit rate than giving it to a small-volume provider.

Hfair = Σi Λihi*

Factor 2 · Popularity skew within a provider

The cache should spend space on the objects that increase the provider hit rate the most per byte, i.e. in decreasing order of λo/so; with equal sizes, most popular first.

This guarantees the minimum footprint to give provider i a target hit rate of hi*.

50 objects with Zipf popularity: object r gets a share ∝ 1/rζ (ζ = 0: all equal).

Provider hit rate against objects cached, most popular first (dashed: flat catalogue).

The cost of fairness

Two effects jointly determine whether fairness is efficiency-friendly: hit-rate targets that are larger for larger-volume providers, and within-provider popularity that gives those targets small minimum footprints. They can compensate for one another: concentrated popularity can offset poor alignment with volume, and the reverse. So the cost of fairness is not determined by the fairness rule alone; it depends on the full chain:

  1. fairness rule
  2. target net values
  3. hit-rate targets hi*
  4. their minimum footprints
  5. aggregate hit rate Hfair

Fairness and efficiency in a single optimisation problem

max{ho ∈ [0, 1]} Σi wi uα(Vi(hi)) + β · Σi Σo∈Oi λohoΣi Λi utilitised fairness normalised efficiency
subject to Σi Σo∈Oi soho ≤ C
wi > 0
provider i’s weight (e.g. higher for a premium service)
uα
the α-fair utility: uα(x) = x1−α/(1−α) for α ≠ 1; u1(x) = log x
α
how strongly small net values are protected: 0 maximises their sum, 1 is proportional fairness, larger is closer to max-min
β
how much efficiency counts relative to fairness

This page uses α = 1, β = 0 and wi = 1.

Step 4 · The policies

Caching policies

A policy decides what the cache keeps.

Efficiency-optimal

Caches the objects with the highest request rate per unit size, whatever the provider: the largest aggregate hit rate.

LRU

Least Recently Used, as in real caches: on a miss, insert the object; when full, evict the one requested longest ago.

Fairness-optimal

The most efficient allocation with equal net values: the largest common target h* whose minimum footprints fit in the cache together; each provider caches its most popular objects.

Fairness-TTL (online controller)

The paper’s controller. On each request it adjusts the object’s TTL (time-to-live: how long it stays cached) from request feedback, as congestion control does: a little longer if more cache time is worth its space, half as long if not. It approximates Fairness-optimal.

Step 5 · The experiment

Your turn: change the workload

It starts at the paper’s good case (§6): 20 providers with 50 equal-size objects each, a cache for 20% of all objects, 100 requests per second in total.

Two popularity distributions shape the requests: across providers (request volume Λi; provider 1 is the busiest) and within each catalogue (the same for all). Each gives rank r a share:

Zipf(ζ)
∝ 1/rζ, a long tail (3.2). The paper uses Zipf(1) for both.
Geometric(q)
∝ qr−1: each rank gets q times the one before, a thin tail.
Uniform
all equal.

At the paper’s two settings, the good case and the bad case (§5: 1000 providers × 1 object), the page shows the values reported in the paper. Other settings are computed with the paper’s model: not paper results.

Results

Loading…
The three policies
PolicyAggregate hit rateNet-value CoV

Net-value CoV (coefficient of variation): standard deviation / mean of the provider net values; 0 = all equal.

Plot 1 · Provider net value (as in the paper’s Fig. 2)

Computed with the paper’s model

Plot 2 · Object hit-rate allocation (as in the paper’s Fig. 3)

Computed with the paper’s model

Close-up: two providers

Provider 1