Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Redis Cache Architecture

Introduction

TaskMinder uses one Redis instance (redisClient) for four things:

  1. Data cache: read-through caches of database query results, namespaced under cache:.
  2. Auth gates: short-lived existence/validity flags that throttle account/class checks.
  3. Sessions: the express-session store plus a per-class reverse index.
  4. Job queue: a list used to hand file-processing jobs to a worker.

The connection, key-prefix constants, and queue helpers live in backend/src/config/redis.ts.


Key Naming Conventions

Data cache keys

All data-cache keys are built by generateCacheKey(baseKey, classId, accountId?):

cache:<baseKey>:<classId>                  // shared (class-wide) partition
cache:<baseKey>:<classId>:acc:<accountId>  // personal (per-account) partition
  • <baseKey> is one of the CACHE_KEY_PREFIXES values (see table below).
  • The :acc:<accountId> suffix only applies to caches with a personal partition (currently homework and events).
  • Omitting accountId addresses the shared partition; passing it addresses one account’s personal partition.
  • Exception: for substitutions, <classId> actually holds dsbMobileUser (the DSB Mobile login), since substitution data is keyed per school, not per class. site_statistics is a further exception; see below.

Other namespaces

NamespacePatternBuilt by
Auth gate (user)auth_user:<accountId>inline in access middleware
Auth gate (class)auth_class:<classId>inline in access middleware
Session payloadsess:<sid>RedisStore (prefix = sess:)
Class session setclass_sessions:<classId>RedisStore
Job queuefile_processing_queueQUEUE_KEYS

1. Data Cache

Read-through, write-on-miss caches of Prisma query results. A hit returns the parsed JSON directly; a miss queries the database, writes the result back with a TTL, then returns it. BigInt values are serialized to strings via a custom replacer.

Base keyFull keyValueTTL
homework_datacache:homework_data:<classId>[:acc:<id>]JSON array of rows3600 s
event_datacache:event_data:<classId>[:acc:<id>]JSON array of rows3600 s
event_type_datacache:event_type_data:<classId>JSON array of rows3600 s
event_type_stylescache:event_type_styles:<classId>Rendered CSS string3600 s
subject_datacache:subject_data:<classId>JSON array of rows3600 s
lesson_datacache:lesson_data:<classId>JSON array of rows3600 s
teams_datacache:teams_data:<classId>JSON array of rows3600 s
upload_metadatacache:upload_metadata:<classId>JSON array of rows3600 s
upload_requestscache:upload_requests:<classId>JSON array of rows3600 s
substitutions_datacache:substitutions_data:<dsbMobileUser>{ data, timestamp }60 s / 600 s
site_statisticscache:site_statisticsJSON object300 s

The standard TTL (cacheExpiration) is 3600 s (60 min). Three keys deviate:

  • event_type_styles stores a pre-rendered CSS string rather than a row array, so it can be served straight to the client.
  • substitutions_data uses stale-serve and a variable TTL; see below.
  • site_statistics is a single global key with no <classId> slot, built directly (cache:${CACHE_KEY_PREFIXES.STATISTICS}) rather than via generateCacheKey, since /stats aggregates across the whole database rather than one class. Its 300 s TTL (statisticsCacheExpiration) matches the route’s Cache-Control: public, max-age=300 header, so server- and client-side staleness stay in sync. Like substitutions, it is never explicitly invalidated: class/homework/event writes happen across too many unrelated code paths for a targeted del to be worthwhile, so it relies on the TTL alone.

Personal vs. shared partitioning (homework & events)

Homework and events are split into two partitions: a shared partition (accountId = null, team-scoped, one key per class) and a personal partition (one key per account, :acc:<accountId>).

The read path fetches both partitions, merges them, and re-sorts to preserve the canonical ordering. Anonymous/account-less viewers only ever receive the shared partition. Writes invalidate only the partition they touched: the shared key when accountId is null, otherwise just the affected account’s personal key, keeping cache busts surgical.

Substitutions cache

Substitution data is fetched from the external DSB Mobile service and relies entirely on TTL, stale-serve, and prefetch rather than explicit invalidation:

  • Value envelope: { data, timestamp } rather than a bare array, so freshness can be evaluated independently of the Redis TTL.
  • Variable TTL: 60 s during the peak window (weekdays 06:00-09:00 local), 600 s otherwise.
  • Stale-serve: if a live fetch from DSB Mobile fails, the previously cached entry is served instead of erroring.
  • Prefetch: a background job warms the cache for every DSB-activated school (deduplicated by dsbMobileUser, processed in batches of 5) so peak-time reads hit a warm cache.

2. Auth Gates

Negative-lookup-avoidance flags that throttle “does this account/class still exist?” checks to once per 15 minutes instead of once per request. On a miss, the middleware queries Postgres to confirm existence (and, for accounts, that deletedAt is null), then writes the flag.

KeyValueTTLExplicit invalidation
auth_user:<accountId>"true"15 minDeleted on account deletion (account.service.ts)
auth_class:<classId>"true"15 minDeleted on class deletion (class.service.ts) and class cleanup (db.cleanup.ts)

A soft-delete (deletedAt) without an accompanying del only takes effect once the flag expires, so a soft-deleted account can keep passing the gate for up to 15 minutes. Hard deletes invalidate the gate explicitly.


3. Sessions

The express-session store is implemented by RedisStore in backend/src/config/redis.session.ts.

KeyTypeContentsTTL
sess:<sid>String (JSON)Serialized SessionDataFollows cookie expiry; default 30 days
class_sessions:<classId>SetSession keys belonging to one classKept ≥ the longest member session

class_sessions:<classId> is a reverse index that lets the application find and destroy all sessions for a given class (e.g. on class cleanup). Its members are kept in sync on session set/destroy/touch, its TTL is bumped to at least the longest active member session, and stale members are lazily pruned when the set is read.


4. Job Queue

KeyTypeOperations
file_processing_queueListlPush (enqueue), rPop (dequeue, FIFO), lLen (length)

Producers enqueue file-processing jobs with queueJob; the upload worker drains them with dequeueJob. Jobs are JSON-serialized.