Skip to main content

internal/formation/compaction.go

internal/formation · 139 lines · 8 declarations · source

Declarations​

type SegmentStore​

type SegmentStore interface {
	Subjects(ctx context.Context, scope string, limit int) ([]string, error)
	Formed(ctx context.Context, scope, subject string) ([]compaction.Turn, error)
	Segments(ctx context.Context, scope, subject string) ([]compaction.Segment, error)
	Material(ctx context.Context, scope, subject string, plan compaction.Plan) (compaction.Material, error)
	Write(ctx context.Context, scope, subject string, plan compaction.Plan, summary compaction.Summary) (compaction.Segment, error)
}

SegmentStore is what the compaction pass needs from the database.

source

type Summariser​

type Summariser interface {
	Write(ctx context.Context, m compaction.Material) (compaction.Summary, error)
}

Summariser writes one segment's prose from its material. A model, behind a port.

source

type Compaction​

type Compaction struct {
	store      SegmentStore
	summariser Summariser
}

Compaction rolls up every subject's history in a scope, one segment at a time.

Why this is a pass and not a read​

Writing a segment is a model call. A context assembly that triggered one would put inference on the read path and make one caller pay for a stretch of history nobody had asked to condense. So the pass writes segments off the read path, from formed turns, and a context is assembled from whatever segments exist when it is asked for: a history the pass has not reached yet is handed over verbatim and cut, never summarised on demand.

Why the pass is bounded by segments written​

A pass bounded by segments written is a pass whose cost an operator can predict: at most `limit` model calls, each over at most MaxMaterialCharacters. A subject with a long unrolled history catches up over several passes rather than in one, which is also what keeps one subject's backlog from holding every other subject's compaction for long.

source

func NewCompaction​

func NewCompaction(store SegmentStore, summariser Summariser) *Compaction

NewCompaction builds the pass. A nil summariser writes nothing and reports so.

source

type CompactionPass​

type CompactionPass struct {
	Subjects int
	Written  int
	Cut      int
	Errored  int
}

CompactionPass is what one call did.

source

method Compaction.Run​

func (c *Compaction) Run(ctx context.Context, scope string, limit int) (CompactionPass, error)

Run writes at most limit segments across the subjects of one scope, oldest gaps first.

source

method Compaction.one​

func (c *Compaction) one(ctx context.Context, scope, subject string) (int, int, error)

one writes the next segment a subject needs, or nothing when its history is rolled up.

source

func bound​

func bound(m compaction.Material) (compaction.Material, int)

bound cuts material over the ceiling from the oldest turn, and says how many turns went. The segment still covers the whole range, because its registrations must; what it was written from is smaller, and that is reported rather than hidden.

source