arbiter-core-0.1.0.0: Core types and logic for PostgreSQL-backed job queue
Safe HaskellNone
LanguageGHC2024

Arbiter.Core.Job.Schema

Description

SQL generation functions for job queue schemas. No database execution happens here.

Synopsis

Schema Creation

createSchemaSQL :: Text -> Text Source #

SQL to create the schema for Arbiter tables

defaultSchemaName :: Text Source #

Default PostgreSQL schema name for Arbiter tables

Using a dedicated schema prevents namespace pollution in the user's public schema.

Table Creation SQL

createJobQueueTableSQL :: Text -> Text -> Text Source #

SQL to create the main job queue table within a schema

This table stores pending and in-progress jobs. Completed jobs are deleted, failed jobs are moved to the DLQ.

createJobQueueDLQTableSQL :: Text -> Text -> Text Source #

SQL to create the dead letter queue table within a schema

Jobs that fail repeatedly (exceed max attempts) are moved here for inspection. This table contains ALL the Job fields (complete snapshot) plus DLQ-specific metadata.

Index Creation SQL

createJobQueueGroupKeyIndexSQL :: Text -> Text -> Text Source #

SQL to create a partial index on group_key for efficient per-group lookups.

Used by claim queries' LATERAL subqueries and by the DELETE/UPDATE triggers when recomputing in_flight_until. Composite (group_key, priority, id) so the DELETE trigger can recompute min values via index-only lookup.

createDLQGroupKeyIndexSQL :: Text -> Text -> Text Source #

SQL to create index on DLQ group_key for querying failed jobs by group

createDLQFailedAtIndexSQL :: Text -> Text -> Text Source #

SQL to create index on DLQ failed_at for time-based queries

createDLQParentIdIndexSQL :: Text -> Text -> Text Source #

SQL to create index on DLQ parent_id for efficient child lookups (DLQ child counts)

createDedupKeyIndexSQL :: Text -> Text -> Text Source #

SQL to create unique index on dedup_key for job deduplication

createParentIdIndexSQL :: Text -> Text -> Text Source #

SQL to create a partial index on parent_id for efficient child lookups.

NOTIFY Trigger SQL

createNotifyFunctionSQL :: Text -> Text -> Text Source #

SQL to create the NOTIFY function for a specific table

This function is triggered after job insertion and sends a notification on a table-specific channel. Each table gets its own function and channel for isolation.

Note: The channel name is quoted as a string literal (single quotes), not an identifier.

createNotifyTriggerSQL :: Text -> Text -> Text Source #

SQL to create the NOTIFY trigger for a specific table

This trigger fires AFTER INSERT on the job queue table and calls the table-specific notify function.

dropNotifyTriggerSQL :: Text -> Text -> Text Source #

SQL to drop the NOTIFY trigger

dropNotifyFunctionSQL :: Text -> Text -> Text Source #

SQL to drop the NOTIFY function for a specific table

Event Streaming Trigger SQL

createEventStreamingFunctionSQL :: Text -> Text Source #

SQL to create the event streaming notification function

This is a shared function (one per schema) that fires on INSERT, UPDATE, DELETE of any job table and INSERT on any DLQ table. It sends a lightweight JSON event via pg_notify on the arbiter_job_events channel with just the event type, table name, and job ID.

The function uses TG_TABLE_NAME and TG_OP to determine context, so it works for all tables without per-table copies.

createEventStreamingTriggersSQL :: Text -> Text -> Text Source #

SQL to create event streaming triggers for a table and its DLQ

Creates a combined INSERTUPDATEDELETE trigger on the main table and an INSERT trigger on the DLQ table, both calling the shared notify_job_event function. Also drops any legacy per-operation triggers left by older versions of setupEventTriggers.

Notification Channel Helpers

notificationChannelForTable :: Text -> Text Source #

Generate notification channel name for a table

Each table gets its own NOTIFY channel for job insertions. Example: notificationChannelForTable "email_jobs" -> "email_jobs_created"

eventStreamingChannel :: Text Source #

Channel name used by the event streaming (SSE) system.

Trigger / Function Name Helpers

notifyFunctionName :: Text -> Text Source #

Per-table NOTIFY trigger function name.

notifyTriggerName :: Text -> Text Source #

Per-table NOTIFY trigger name.

eventStreamingFunctionName :: Text Source #

Shared event streaming trigger function name (one per schema).

eventStreamingTriggerName :: Text -> Text Source #

Per-table event streaming trigger name.

eventStreamingDLQTriggerName :: Text -> Text Source #

Per-table DLQ event streaming trigger name.

Table Name Helpers

jobQueueTable :: Text -> Text -> Text Source #

Qualified table name: jobQueueTable "arbiter" "email_jobs" -> "arbiter"."email_jobs"

jobQueueDLQTable :: Text -> Text -> Text Source #

Qualified DLQ table name: jobQueueDLQTable "arbiter" "email_jobs" -> "arbiter"."email_jobs_dlq"

jobQueueResultsTable :: Text -> Text -> Text Source #

Qualified results table name: jobQueueResultsTable "arbiter" "email_jobs" -> "arbiter"."email_jobs_results"

jobQueueGroupsTable :: Text -> Text -> Text Source #

Qualified groups table name: jobQueueGroupsTable "arbiter" "email_jobs" -> "arbiter"."email_jobs_groups"

jobQueueReaperSeq :: Text -> Text -> Text Source #

Qualified reaper sequence name.

Reaper Coordination

createReaperSeqSQL :: Text -> Text -> Text Source #

SQL to create the reaper coordination sequence. Stores the epoch (seconds) of the last reaper run.

Results Table

createResultsTableSQL :: Text -> Text -> Text Source #

SQL to create the results table for storing child job results.

Child results are stored as individual rows (one per child), keyed by (parent_id, child_id). The ON DELETE CASCADE foreign key ensures cleanup when the parent is acked (deleted).

Groups Table

createGroupsTableSQL :: Text -> Text -> Text Source #

SQL to create the groups table for fast grouped claims.

Stores one row per distinct group_key with pre-computed min_priority, min_id, job_count, and in_flight_until. Maintained by statement-level AFTER triggers on the main job table (see createGroupsTriggerFunctionsSQL).

createGroupsIndexSQL :: Text -> Text -> Text Source #

SQL to create a ranking index on the groups table.

Non-partial so that UPDATE SET job_count, in_flight_until etc. never change indexed columns — enabling HOT updates on every ack and claim.

Groups Trigger SQL

createGroupsTriggerFunctionsSQL :: Text -> Text -> Text Source #

Three trigger functions maintaining the groups table via statement-level AFTER triggers. Uses incremental operations where possible.

createGroupsTriggersSQL :: Text -> Text -> Text Source #

SQL to create 3 statement-level AFTER triggers on the main job table that call the groups maintenance functions.

Uses REFERENCING NEW/OLD TABLE AS for efficient batch access to affected rows via transition tables.

Identifier Quoting

quoteIdentifier :: Text -> Text Source #

Quote a PostgreSQL identifier (schema name, table name, column name).

This escapes double quotes by doubling them and wraps the identifier in quotes. This prevents SQL injection when using dynamic identifiers.