internal/semantic/surface.go
internal/semantic · 164 lines · 9 declarations · source
This file carries the package documentation, rendered on the package page.
Declarations
const SurfaceBudget
const SurfaceBudget = 2000
SurfaceBudget is the maximum UTF-8 bytes of the complete representation, including separators.
Why there is a bound
A hub entity has thousands of quotes behind it, and embedding all of them would make one entity's surface cost more than every other entity's together — and produce a vector that is an average of everything anybody ever said near that name, which resembles nothing in particular.
Why this number
About two thousand bytes is a small set of sentences: enough that an entity mentioned in several contexts is represented by more than one of them, and short enough that the surface still points at a thing rather than at a topic. It is a constant rather than a setting because nothing has asked to vary it, and the measurement that would justify changing it is the one #10 exists to take.
const MaxSurfaceAliases
const MaxSurfaceAliases = 64
Collection bounds cap validation, deduplication and sorting before any output is built.
const MaxSurfaceQuotes
const MaxSurfaceQuotes = 1024
const MaxSurfaceInputBytes
const MaxSurfaceInputBytes = 1 << 20
var ErrInvalidSurface
var ErrInvalidSurface = errors.New("semantic surface requires a canonical identity and valid UTF-8 without NUL")
var ErrSurfaceLimit
var ErrSurfaceLimit = errors.New("semantic surface input exceeds its identity or collection budget")
func Surface
func Surface(canonical string, aliases []string, quotes []string) (string, error)
Surface is the text an entity is found by: what it has been called, and what was said about it.
Why the stored quotes rather than a written description
A model-written description of a person, aggregated from many sources, cannot be erased by counting: deleting one source does not remove that person from prose about them, and there is nothing to count afterwards. The quotes are already stored, already scoped, already registered and already erased with the facts they belong to — so a surface built from them inherits the guarantee instead of needing an exception to it.
It also cannot drift from the corpus, because it IS the corpus: two rebuilds of one entity produce the same text, and a rebuild after an erasure produces exactly the text the surviving quotes give.
Why the names come first
A question usually contains the name. Putting the surface forms at the front means the thing being described is stated before the evidence for it, which is the order a sentence about anything takes — and when the budget cuts, it cuts evidence rather than identity.
func validateSurfaceInput
func validateSurfaceInput(canonical string, aliases, quotes []string) error
func distinct
func distinct(values []string, exclude string) []string
distinct returns the unique members, sorted, without a value the caller has already used.
Deduplicated because a relation asserted five times stores five identical quotes, and a surface repeating one sentence five times is a surface mostly about that sentence.