internal/api/api.go
internal/api · 1214 lines · 51 declarations · source
This file carries the package documentation, rendered on the package page.
Declarations
const Version
const Version = "v1"
Version is the contract version, and it is in the path.
In the path rather than negotiated in a header. A version a client must remember to send is a version a client will omit, and what it gets then is a default that silently changes meaning under it. In the path it is visible in a log line, in a curl, and in the one place a developer looks when something behaves unexpectedly. It is permanent from the day the first adapter ships.
type Server
type Server struct {
credentials *credential.Store
audit *pg.AuditStore
exporter *pg.Exporter
projects *pg.ProjectStore
observations *pg.ObservationStore
eraser *pg.Eraser
notifications *pg.NotificationStore
notifier *notify.Sender
recaller *recall.Recaller
citations *pg.CitationStore
records *pg.RecordStore
feedback *pg.FeedbackStore
artifacts *pg.ArtifactStore
subjects *pg.SubjectStore
contexts *pg.SegmentStore
passages *passage.Retriever
entityCandidates *entitycandidate.Retriever
reportCandidates *reportcandidate.Retriever
schema pg.Schema
log *slog.Logger
authAudits refusedAuthBudget
admission *Admission
}
Server is the HTTP surface over one instance.
type Stores
type Stores struct {
Audit *pg.AuditStore
Exporter *pg.Exporter
Projects *pg.ProjectStore
Observations *pg.ObservationStore
Eraser *pg.Eraser
Recaller *recall.Recaller
Citations *pg.CitationStore
Records *pg.RecordStore
// Feedback is reports about records that assert nothing, and the promotion that turns one into
// a correction or a retraction.
Feedback *pg.FeedbackStore
Artifacts *pg.ArtifactStore
Subjects *pg.SubjectStore
// Contexts serves a subject's history under a budget from the segments the compaction pass wrote.
Contexts *pg.SegmentStore
Passages *passage.Retriever
EntityCandidates *entitycandidate.Retriever
ReportCandidates *reportcandidate.Retriever
// Notifications and Notifier are the destinations a project nominated and the operator's policy
// about where this deployment may send. Both absent means the routes refuse, which is what a
// deployment that has not been configured for notifications should do.
Notifications *pg.NotificationStore
Notifier *notify.Sender
}
Stores names the memory dependencies explicitly so adding a capability cannot shift positional wiring.
func NewServer
func NewServer(credentials *credential.Store, stores Stores, schema pg.Schema, log *slog.Logger, admission ...*Admission) *Server
NewServer keeps the credential store separate because it uses the protected registry connection.
method Server.record
func (s *Server) record(r *http.Request, operation string, grant credential.Grant,
outcome string, magnitude int)
record appends to the ledger, and never fails the request that produced it.
A ledger write that could fail an operation would make it a second thing that has to be working for memory to work, and the failure would reach a customer as their write being refused for a reason unrelated to their data. A ledger that takes the system down is one an operator switches off, and a switched-off ledger records nothing at all.
So a failure here is logged at ERROR and swallowed. That is a real weakness — a gap in the record is exactly what the record exists to make impossible — and it is the better of the two, on a table whose value is completeness over time rather than atomicity per row.
type Operation
type Operation struct {
Name string
Method string
Path string
// Success is what the operation answers when it worked. Part of the contract because a client
// that treats any 2xx as success cannot tell an append that was stored from one that was
// accepted for later, and whether what it sent is readable yet is the whole of that difference.
Success int
}
Operation is one thing the surface can be asked to do: a name, and where it is reached.
The name is the ledger's name for the same operation. One identifier runs from the row in the audit table, through the wire, to the method an adapter exposes — so "which operation was that" has one answer everywhere it is asked, rather than a mapping somebody maintains between an operation's public name and its recorded one.
type route
type route struct {
Operation
handler func(*Server) grantedHandler
// request and response are zero values of the types this operation reads and writes. They are
// here so the contract document is rendered from the types the handlers actually use rather
// than written alongside them: a document describing a field the server does not serve is worse
// than no document, because it is believed.
//
// request is nil for an operation that reads no body.
request any
response any
}
The surface, declared once. A path per operation, because HTTP's own vocabulary is what every proxy, log and metric between a caller and us already reads — and dispatching on a body field would take that away from the operator to save an adapter four lines.
Declared as data rather than as five registration calls because the set has to be enumerable: a test asserts every one of these is refused without a credential and leaves a ledger row, and the contract is published from it. A route registered anywhere else is invisible to both, which is how a surface acquires an operation nobody checks — and had, before this list existed: an export was reachable, and neither test named "every operation" covered it.
var operations
60 lines of declaration
var operations = []route{
{Operation{domain.AuditMessageGet, http.MethodPost, "/messages/get", http.StatusOK}, func(s *Server) grantedHandler { return s.getMessage }, messageRequest{}, pg.SourceMessageWindow{}},
{Operation{domain.AuditEmbeddingSearch, http.MethodPost, "/passages/search", http.StatusOK}, func(s *Server) grantedHandler { return s.searchPassages }, passageRequest{}, passage.Page{}},
{Operation{domain.AuditEntityEmbeddingSearch, http.MethodPost, "/entities/candidates", http.StatusOK}, func(s *Server) grantedHandler { return s.searchEntityCandidates }, entityCandidateRequest{}, entitycandidate.Page{}},
{Operation{domain.AuditReportEmbeddingSearch, http.MethodPost, "/reports/candidates", http.StatusOK}, func(s *Server) grantedHandler { return s.searchReportCandidates }, reportCandidateRequest{}, reportcandidate.Page{}},
{Operation{domain.AuditEntityList, http.MethodPost, "/entities/list", http.StatusOK}, func(s *Server) grantedHandler { return s.listEntities }, entityListRequest{}, pg.EntityPage{}},
{Operation{domain.AuditEntityGet, http.MethodPost, "/entities/get", http.StatusOK}, func(s *Server) grantedHandler { return s.getEntity }, entityGetRequest{}, pg.EntityIdentity{}},
{Operation{domain.AuditSubjectRegister, http.MethodPost, "/subjects/register", http.StatusOK}, func(s *Server) grantedHandler { return s.registerSubject }, pg.SubjectRegistration{}, pg.SubjectWrite{}},
{Operation{domain.AuditSubjectGet, http.MethodPost, "/subjects/get", http.StatusOK}, func(s *Server) grantedHandler { return s.getSubject }, subjectGetRequest{}, pg.ManagedSubject{}},
{Operation{domain.AuditSubjectList, http.MethodPost, "/subjects/list", http.StatusOK}, func(s *Server) grantedHandler { return s.listSubjects }, subjectListRequest{}, pg.SubjectPage{}},
{Operation{domain.AuditSubjectUpdate, http.MethodPost, "/subjects/update", http.StatusOK}, func(s *Server) grantedHandler { return s.updateSubject }, pg.SubjectUpdate{}, pg.SubjectWrite{}},
{Operation{domain.AuditArtifactPut, http.MethodPost, "/artifacts/put", http.StatusOK},
func(s *Server) grantedHandler { return s.putArtifact }, pg.ArtifactPut{}, pg.ArtifactWrite{}},
{Operation{domain.AuditArtifactGet, http.MethodPost, "/artifacts/get", http.StatusOK},
func(s *Server) grantedHandler { return s.getArtifact }, artifactIDRequest{}, pg.Artifact{}},
{Operation{domain.AuditArtifactList, http.MethodPost, "/artifacts/list", http.StatusOK},
func(s *Server) grantedHandler { return s.listArtifacts }, artifactListRequest{}, pg.ArtifactPage{}},
{Operation{domain.AuditArtifactDelete, http.MethodPost, "/artifacts/delete", http.StatusOK},
func(s *Server) grantedHandler { return s.deleteArtifact }, artifactDeleteRequest{}, artifactDeleted{}},
{Operation{domain.AuditObserve, http.MethodPost, "/observations", http.StatusCreated},
func(s *Server) grantedHandler { return s.observe }, observeRequest{}, observeResponse{}},
{Operation{domain.AuditFreshness, http.MethodGet, "/freshness", http.StatusOK},
func(s *Server) grantedHandler { return s.freshness }, nil, freshnessResponse{}},
{Operation{domain.AuditRecall, http.MethodPost, "/recalls", http.StatusOK},
func(s *Server) grantedHandler { return s.recallHandler }, recallRequest{}, recallResponse{}},
{Operation{domain.AuditContextAssemble, http.MethodPost, "/contexts", http.StatusOK},
func(s *Server) grantedHandler { return s.assembleContext }, contextRequest{}, contextResponse{}},
{Operation{domain.AuditErase, http.MethodPost, "/erasures", http.StatusOK},
func(s *Server) grantedHandler { return s.erase }, eraseRequest{}, eraseResponse{}},
{Operation{domain.AuditExport, http.MethodPost, "/exports", http.StatusOK},
func(s *Server) grantedHandler { return s.export }, exportRequest{}, exportResponse{}},
{Operation{domain.AuditCitationResolve, http.MethodPost, "/citations/resolve", http.StatusOK},
func(s *Server) grantedHandler { return s.resolveCitation }, citationRequest{}, pg.Citation{}},
{Operation{domain.AuditNotificationRegister, http.MethodPost, "/notifications/endpoints/register", http.StatusOK},
func(s *Server) grantedHandler { return s.registerNotificationEndpoint }, notificationRegisterRequest{}, pg.Registered{}},
{Operation{domain.AuditNotificationList, http.MethodPost, "/notifications/endpoints/list", http.StatusOK},
func(s *Server) grantedHandler { return s.listNotificationEndpoints }, notificationEndpointListRequest{}, notificationEndpointList{}},
{Operation{domain.AuditNotificationDisable, http.MethodPost, "/notifications/endpoints/disable", http.StatusOK},
func(s *Server) grantedHandler { return s.disableNotificationEndpoint }, notificationEndpointRequest{}, notificationDisabled{}},
{Operation{domain.AuditNotificationDeliveries, http.MethodPost, "/notifications/deliveries/list", http.StatusOK},
func(s *Server) grantedHandler { return s.listNotificationDeliveries }, notificationDeliveryListRequest{}, notificationDeliveryList{}},
{Operation{domain.AuditEntityPurge, http.MethodPost, "/entities/purge", http.StatusOK},
func(s *Server) grantedHandler { return s.purgeEntity }, entityPurgeRequest{}, pg.EntityPurge{}},
{Operation{domain.AuditRecordList, http.MethodPost, "/records/list", http.StatusOK},
func(s *Server) grantedHandler { return s.listRecords }, recordListRequest{}, pg.RecordPage{}},
{Operation{domain.AuditRecordHistory, http.MethodPost, "/records/history", http.StatusOK},
func(s *Server) grantedHandler { return s.recordHistory }, recordHistoryRequest{}, pg.RecordHistoryPage{}},
{Operation{domain.AuditRecordAssert, http.MethodPost, "/records/assert", http.StatusOK},
func(s *Server) grantedHandler { return s.assertRecords }, recordAssertionRequest{}, pg.AssertionBatch{}},
{Operation{domain.AuditRecordCorrect, http.MethodPost, "/records/correct", http.StatusOK},
func(s *Server) grantedHandler { return s.correctRecords }, recordCorrectionRequest{}, pg.CorrectionBatch{}},
{Operation{domain.AuditRecordRetract, http.MethodPost, "/records/retract", http.StatusOK},
func(s *Server) grantedHandler { return s.retractRecords }, recordRetractionRequest{}, pg.RetractionBatch{}},
{Operation{domain.AuditFeedbackRecord, http.MethodPost, "/feedback/record", http.StatusOK},
func(s *Server) grantedHandler { return s.recordFeedback }, pg.FeedbackRequest{}, pg.Feedback{}},
{Operation{domain.AuditFeedbackList, http.MethodPost, "/feedback/list", http.StatusOK},
func(s *Server) grantedHandler { return s.listFeedback }, feedbackListRequest{}, pg.FeedbackPage{}},
{Operation{domain.AuditFeedbackPromote, http.MethodPost, "/feedback/promote", http.StatusOK},
func(s *Server) grantedHandler { return s.promoteFeedback }, pg.PromotionRequest{}, pg.Promotion{}},
}
func init
func init()
The declaration is checked before anything can serve it, and a bad one stops the binary.
Refusing at startup rather than at the request that hits the bad route: a surface is small, fixed and known at build time, so the failure is a programming error and the only question is whether an operator finds out from a crash or from a customer.
func checkOperations
func checkOperations(rs []route) error
checkOperations refuses a surface the ledger could not account for.
Two ways it can be wrong, and both are silent otherwise. A name the ledger does not know means every call to that operation writes an ERROR to the log and no row — the operation works and is unaccounted for, which is the one thing this system claims it is not. And two routes under one name means the ledger cannot tell them apart afterwards, so the record says an operation happened without saying which.
func Operations
func Operations() []Operation
Operations is the surface, for anything that has to enumerate it rather than remember it.
Paths are absolute and carry the version, because that is what a caller sends and what a contract document has to state. The slice is built fresh so a caller cannot rewrite the surface by holding onto it.
method Server.Handler
func (s *Server) Handler(readiness ...http.Handler) http.Handler
Handler builds the routes.
Recall and erase are POSTs despite reading rather than writing, and that is deliberate. A question is somebody's words and a data subject reference identifies a person; both would end up in an access log, a proxy log and a browser history if they travelled in a URL. This product's argument is that it can say where personal data went, so it does not scatter it into places it cannot account for.
type grantedHandler
type grantedHandler func(http.ResponseWriter, *http.Request, credential.Grant)
method Server.authenticated
func (s *Server) authenticated(next grantedHandler) http.Handler
authenticated refuses anything that does not resolve, without saying which way it failed.
Absent, malformed, unknown and revoked are one answer. Distinguishing them tells a stranger whether a token they hold is real, which is the single fact they cannot obtain any other way.
method Server.resolveGrant
func (s *Server) resolveGrant(w http.ResponseWriter, r *http.Request) (credential.Grant, bool)
resolveGrant is the credential half of authentication: the authentication slot, the bearer, the registry lookup and the check that the credential's project still exists. It writes the refusal itself and reports whether the caller may go on. It is separate from the work slot so that a door which loops back into the operations (the MCP route) can refuse a stranger at the door without holding a work slot the looped-back request then needs, which with a per-credential limit of one would refuse every tool call as busy.
func bearer
func bearer(header string) (string, bool)
type observeRequest
type observeRequest struct {
IdempotencyKey string `json:"idempotency_key"`
// No project. The credential decides it — a caller cannot name one, so there is nothing
// to check and nothing to get wrong.
DataSubjectID string `json:"data_subject_id"`
OccurredAt *time.Time `json:"occurred_at"`
Messages []messagePayload `json:"messages"`
}
type messagePayload
type messagePayload struct {
// If supplied, every message supplies one. Omitted groups mean standalone ordinary messages;
// tool exchanges require explicit boundaries because roles cannot reconstruct call identity.
GroupOrdinal *int `json:"group_ordinal"`
Role string `json:"role"`
Content string `json:"content"`
OccurredAt *time.Time `json:"occurred_at"`
}
type observeResponse
type observeResponse struct {
ID string `json:"id"`
Scope string `json:"scope"`
LogOffset int64 `json:"log_offset"`
}
method Server.observe
func (s *Server) observe(w http.ResponseWriter, r *http.Request, grant credential.Grant)
observe appends a turn and returns before it has been formed.
Returning after formation would make an append wait on a model call, which turns a write a caller is holding a request open for into a variable-latency operation whose worst case is a provider's worst case. So the append is durable when this returns, the offset is the caller's receipt, and freshness is the separate question of whether a recall would see it yet.
type freshnessResponse
type freshnessResponse struct {
Scope string `json:"scope"`
Stored *int64 `json:"stored"`
// Formed is absent until the scope's first turn has formed. Absent rather than zero, because
// zero is a real offset belonging to the first turn, and a caller polling for "formed >= my
// offset" would be told yes before anything had happened.
Formed *int64 `json:"formed"`
// Parked is how many turns the driver gave up on, and it is the exception to Formed. The
// watermark advances past a turn nobody can form so that one bad turn does not freeze a
// project's memory — which makes Formed mean "formed, except these". That is only acceptable
// while it is visible, so it is on the wire rather than left to be discovered.
Parked int `json:"parked"`
// Rebuilding is present only while an operator is reinterpreting this project's facts. Its
// presence is the message: a rebuild publishes source by source, so a project with one in
// flight answers from the old extractor's reading of what it has not reached and the new one's
// of what it has, and a bundle taken then is reproducible from neither alone.
//
// Here rather than on recall, deliberately. A lookup on the read path is paid by every caller on
// every question to report a state that changes a few times a year; this is the route a client
// already polls, and the MCP surface exposes it as a tool.
Rebuilding *rebuildingState `json:"rebuilding,omitempty"`
}
type rebuildingState
type rebuildingState struct {
ReinterpretedThrough int64 `json:"reinterpreted_through"`
ReinterpretingThrough int64 `json:"reinterpreting_through"`
// Acknowledged and Skipped are sources the job has finished with, never a frozen total: a source
// erased before the job reached it enters neither, so no remainder can be computed from them.
Acknowledged int64 `json:"sources_acknowledged"`
Skipped int64 `json:"sources_skipped"`
}
method Server.freshness
func (s *Server) freshness(w http.ResponseWriter, r *http.Request, grant credential.Grant)
type recallRequest
type recallRequest struct {
// Optional attribution filter, not a new authentication or project grant.
DataSubjectID string `json:"data_subject_id"`
Question string `json:"question"`
// AsOf and AsKnownAt are the two instants a bitemporal read is taken at: what was true of the
// world then, as this system understood things then. Both are optional and independent — the
// half a caller does not name is the open interval, which is what we believe now, and is not a
// timestamp anybody computed.
//
// Typed as times rather than strings, so a value that is not RFC 3339 is refused by the decoder
// with the rest of a malformed body. A string parsed later would be a second refusal path with
// its own error code, for a mistake the first one already catches.
AsOf *time.Time `json:"as_of"`
AsKnownAt *time.Time `json:"as_known_at"`
// Hops is how far from the entities the question names the answer may reach. One — the default —
// is facts about those entities. Two follows one relation further and returns the chain it
// followed, so "who does Marta's manager work for" is answerable and inspectable.
//
// Optional, because a second hop costs every caller who did not need one, and because at two hops
// a bundle contains facts about things the question never named. Whether that is the answer or
// noise depends on the question, and only the caller knows which they asked.
recall.Controls
}
method recallRequest.at
func (r recallRequest) at() domain.AsOf
at is the pair of instants this request asks about.
A request naming one and not the other asks about that time as things are understood now, which is the ordinary historical question: what was true in March, as best we know today.
type recallResponse
type recallResponse struct {
Controls recall.EffectiveControls `json:"controls"`
Anchors []anchorPayload `json:"anchors"`
Facts []factPayload `json:"facts"`
// Reports and Passages are the composed surfaces: themes the graph holds, and source
// words by similarity. Present and empty when a surface returned nothing, so a caller can
// tell an empty surface from a surface that did not run, which `degraded` names.
Reports []reportPayload `json:"reports"`
Passages []passagePayload `json:"passages"`
// Truncated says the limit cut the result, and it is not a ranking verdict: nothing has ranked
// these, so a cut bundle is missing arbitrary members rather than the least relevant ones. A
// caller told only the facts would read the cut as a judgement.
Truncated bool `json:"truncated"`
// Characters is what these facts cost, in the unit the cut was made in. A caller converts to
// their own tokeniser's units; we do not, because a token count computed with the wrong
// tokeniser is wrong in a way the caller cannot correct and would be believed.
Characters int `json:"characters"`
// Degraded names parts of the retrieval that did not run. Empty today and present anyway,
// because a bundle that silently comes back smaller is indistinguishable from a memory with less
// in it — and a field cannot be added to a published contract without a version.
Degraded []string `json:"degraded"`
// Reach is how this bundle was arrived at. Always present, because an empty bundle without it
// cannot say whether the question named something unknown or something known-and-silent.
Reach reachPayload `json:"reach"`
}
type reachPayload
type reachPayload struct {
Terms int `json:"terms"`
Anchored int `json:"anchored"`
// NamedNothingKnown separates "we have never heard of what you asked about" from "we know that
// and have nothing to say". Both produce an empty bundle and they are different answers.
NamedNothingKnown bool `json:"named_nothing_known"`
FactsPerAnchor map[string]int `json:"facts_per_anchor"`
}
reachPayload reports the shape rather than a verdict.
No relevance score, deliberately: scoring candidates is a ranking stage, and ranking waits for the measurement that would say whether it helps. What a caller gets instead is exact and needs no threshold — how many names the question offered, how many were found, and how the facts distribute across them. Everything arriving from one anchor when several matched is a question that named something and asked about something else, and it is visible without anybody choosing a number.
type anchorPayload
type anchorPayload struct {
EntityID string `json:"entity_id"`
Name string `json:"name"`
Type string `json:"type"`
// Matched is the term in the question that reached this entity. It is what makes the path
// inspectable: a caller can see why an entity is in the bundle, not only that it is.
Matched string `json:"matched"`
}
type factPayload
38 lines of declaration
type factPayload struct {
FactID string `json:"fact_id"`
Subject string `json:"subject"`
Predicate string `json:"predicate"`
Object string `json:"object"`
Statement string `json:"statement"`
Confidence float32 `json:"confidence"`
// ValidFrom and ValidUntil are when this was true of the world, which is not when it was
// recorded. ValidUntil is null for a fact nothing has superseded and a time for one that has
// been — so a fact returned by a read of last March says that it stopped holding, rather than
// arriving indistinguishable from a fact that holds now.
ValidFrom time.Time `json:"valid_from"`
ValidUntil *time.Time `json:"valid_until"`
// AnchoredOn is the entity this fact was reached through. A fact can name two anchors, and which
// one brought it in is the difference between a result and a traversal a caller can check.
AnchoredOn string `json:"anchored_on"`
// SourceRole is who said it. Always present, because a fact shown without saying it came from a
// fetched page asks the reader to trust our sources as if they were their user's words.
SourceRole string `json:"source_role"`
// Hops is how many relations away from the question this fact was found, and Path and Via are the
// chain that reached it: Path holds the entity names in order, Via the relations between them, so
// Path has one more member than Via.
//
// Empty for a one-hop fact, where the chain is a single name and no relations. Present for
// anything further, because a fact about something the question did not name is only believable
// if the route to it can be read.
Hops int `json:"hops"`
Path []string `json:"path"`
Via []string `json:"via"`
// CoDerivedWith names the other facts in this bundle from the same message about the same
// subject. They are not independent of each other, and a reader counting them as corroboration
// is counting one sentence twice.
//
// Always present, empty when there are none: an absent field says "we did not check", and this
// is checked on every bundle.
CoDerivedWith []string `json:"co_derived_with"`
Evidence evidencePayload `json:"evidence"`
}
type evidencePayload
type evidencePayload struct {
ObservationID string `json:"observation_id"`
SourceOrdinal int `json:"source_ordinal"`
Quote string `json:"quote"`
ByteStart int `json:"byte_start"`
ByteEnd int `json:"byte_end"`
// Context is the words around the quote, from the same message. A quote and a span prove a fact
// was in the message and say nothing about whether the sentence asserted it, hedged it or denied
// it — which is the difference between a receipt and a legible receipt.
//
// Empty when the message is gone, which is what an erasure leaves behind.
Context string `json:"context"`
// ContextStart is where the context begins in the message. The quote sits at
// `byte_start - context_start` inside it, which is what lets a reader highlight the cited words
// rather than search for them.
ContextStart int `json:"context_start"`
// ContextComplete says the context IS the whole message. Without it a window that happened to fit
// and one that was cut are indistinguishable, and a reader assuming the first has read a sentence
// that continues.
ContextComplete bool `json:"context_complete"`
}
method Server.recallHandler
func (s *Server) recallHandler(w http.ResponseWriter, r *http.Request, grant credential.Grant)
recallHandler answers a question against the scopes the credential authorises AND the caller asked for — the intersection, never the union.
A caller naming a scope they do not hold gets a refusal rather than a silent narrowing, because a bundle quietly missing a project looks exactly like a project with nothing in it.
type reportPayload
type reportPayload struct {
CommunityID string `json:"community_id"`
ReportID string `json:"report_id"`
Title string `json:"title"`
Summary string `json:"summary"`
Importance float32 `json:"importance"`
Similarity float64 `json:"similarity"`
Level int `json:"level"`
Parent string `json:"parent,omitempty"`
Sources []string `json:"sources"`
}
type passagePayload
type passagePayload struct {
ChunkID string `json:"chunk_id"`
SourceID string `json:"source_id"`
Ordinal int `json:"ordinal"`
Role string `json:"role"`
Quote string `json:"quote"`
Similarity float64 `json:"similarity"`
OccurredAt time.Time `json:"occurred_at"`
Complete bool `json:"complete"`
}
func renderBundle
func renderBundle(b domain.Bundle) recallResponse
type eraseRequest
type eraseRequest struct {
DataSubjectID string `json:"data_subject_id"`
SourceObservationIDs []string `json:"source_observation_ids"`
Reason string `json:"reason"`
}
eraseRequest names one thing: the person to erase, or the source observations to erase — a document observed project-wide has no person, and its manifest holds its observation ids.
const maxErasureSources
const maxErasureSources = 10000
maxErasureSources bounds an erasure by source. Each id is one element of one array predicate in every statement of the walk, so the cost is linear and small; the bound exists so that a request cannot be a project-wide delete spelled out one id at a time. Ten thousand segments is a document of two and a half megabytes cut at the smallest ceiling; a larger one is erased in more than one request, each with its own receipt.
type eraseResponse
type eraseResponse struct {
RequestID string `json:"request_id"`
Scope string `json:"scope"`
DataSubjectID string `json:"data_subject_id"`
SourceObservationIDs []string `json:"source_observation_ids,omitempty"`
CompletedAt time.Time `json:"completed_at"`
Deleted map[string]int `json:"deleted"`
// Residual is the product. "Done" is a claim; a count of what still matches, taken after the
// delete in the same transaction, is a measurement — and it is the one a data protection officer
// is actually asking for.
Residual map[string]int `json:"residual"`
Clean bool `json:"clean"`
}
method Server.erase
func (s *Server) erase(w http.ResponseWriter, r *http.Request, grant credential.Grant)
type exportRequest
type exportRequest struct {
DataSubjectID string `json:"data_subject_id"`
SourceObservationIDs []string `json:"source_observation_ids"`
}
exportRequest names one thing, as an erasure does: the person to export, or the turns to export. A document observed project-wide has no person, and its manifest holds its observation ids.
type exportResponse
type exportResponse struct {
Scope string `json:"scope"`
DataSubjectID string `json:"data_subject_id"`
SourceObservationIDs []string `json:"source_observation_ids,omitempty"`
// Sections keyed by what produced them, plus the messages everything derives from. Raw rows
// rather than a shaped view: an export has to include a column a migration added without
// anybody remembering to add it here.
Sections map[string][]json.RawMessage `json:"sections"`
// Rows is the count per section, so a reader can compare an export against an erasure receipt
// without parsing either. The two walks agreeing is what says neither has forgotten a projection.
Rows map[string]int `json:"rows"`
}
method Server.export
func (s *Server) export(w http.ResponseWriter, r *http.Request, grant credential.Grant)
export answers what is held about one person.
A POST although it reads, for the same reason recall and erase are: the subject reference identifies a person, and in a URL it would land in an access log, a proxy log and a browser history — places this product cannot account for.
func writeJSON
func writeJSON(w http.ResponseWriter, status int, body any)
type errorBody
type errorBody struct {
Error errorDetail `json:"error"`
}
type errorDetail
type errorDetail struct {
Code string `json:"code"`
Message string `json:"message"`
}
type errorCode
type errorCode string
errorCode is the vocabulary of refusals, and it is part of the contract.
A caller branches on the code and shows the message; that is the whole reason the code exists as a separate field. So the set is declared here rather than spelled at each call site, because a code invented in a handler is a value external code has to handle and nobody wrote down — and the day it appears is the day somebody's error handling falls through to a default it did not choose.
A named type rather than a string so a call site reads as a choice from a list. Go will still accept an untyped literal, which is why a test parses this package and refuses one.
const codeUnauthenticated, codeInvalidBody, codeInvalidTurn and 39 more
76 lines of declaration
const (
// codeUnauthenticated is every way a credential can fail to work: absent, malformed, unknown,
// revoked, or naming a project that is gone. One code, deliberately — see authenticated.
codeUnauthenticated errorCode = "unauthenticated"
// codeInvalidBody is a body that could not be read as this operation's request: malformed JSON,
// an unknown field, or one over the size limit.
codeInvalidBody errorCode = "invalid_body"
// codeInvalidTurn is a turn the domain refuses. The message carries the domain's own reason,
// because a caller can fix it and a generic refusal would send them reading source.
codeInvalidTurn errorCode = "invalid_turn"
// A key identifies one logical write, including after its content has been erased.
codeIdempotencyConflict errorCode = "idempotency_conflict"
// codeInvalidQuestion is a recall with nothing to answer.
codeInvalidQuestion errorCode = "invalid_question"
codeRateLimited errorCode = "rate_limited"
// Separate from rate_limited on purpose. A rate limit is this deployment deciding it is busy and
// is bounded by its own settings; this is the substrate having nothing left, which no per-request
// budget can prevent and which an operator fixes by changing max_connections or the replica
// count. A caller retries either the same way, and an operator reading a log needs to tell them
// apart.
codeNoCapacity errorCode = "no_database_capacity"
codeInvalidCitation errorCode = "invalid_citation"
codeCitationLimit errorCode = "citation_limit"
codeNotFound errorCode = "not_found"
// codeInvalidDestination is a notification destination this deployment will not send to, or one
// already registered. The message names the policy and never what is behind it.
codeInvalidDestination errorCode = "invalid_destination"
// codeNotificationsOff is every notification route on a deployment whose operator has named no
// destination. Refused rather than served empty: an application that registered an endpoint and
// got a receipt would reasonably believe it would be told.
codeNotificationsOff errorCode = "notifications_unavailable"
codeForbidden errorCode = "forbidden"
codeInvalidRecordPage errorCode = "invalid_record_page"
codeInvalidEntityPage errorCode = "invalid_entity_page"
codeInvalidPassageQuery errorCode = "invalid_passage_query"
codeInvalidEntityCandidateQuery errorCode = "invalid_entity_candidate_query"
codeInvalidReportCandidateQuery errorCode = "invalid_report_candidate_query"
codeInvalidMessageWindow errorCode = "invalid_message_window"
codeSourceChanged errorCode = "source_changed"
codePassageUnavailable errorCode = "passage_unavailable"
codeEntityCandidatesUnavailable errorCode = "entity_candidates_unavailable"
codeReportCandidatesUnavailable errorCode = "report_candidates_unavailable"
codeInvalidRecallControls errorCode = "invalid_recall_controls"
// codeInvalidContext is a context request the server refuses: no subject, or a budget outside the server's.
codeInvalidContext errorCode = "invalid_context"
codeInvalidFeedback errorCode = "invalid_feedback"
// A second promotion of one feedback is its own code, not a conflict: nothing about the request
// was wrong, and the caller's correct response is to stop rather than to re-read and retry.
codeFeedbackPromoted errorCode = "feedback_already_promoted"
codeInvalidRecordMutation errorCode = "invalid_record_mutation"
codeRecordConflict errorCode = "record_conflict"
codeRecordMutationLimit errorCode = "record_mutation_limit"
codeInvalidArtifact errorCode = "invalid_artifact"
codeInvalidSubject errorCode = "invalid_subject"
codeSubjectConflict errorCode = "subject_conflict"
codeArtifactConflict errorCode = "artifact_conflict"
codeStorageCapacity errorCode = "storage_capacity"
// codeNoSubject is an erasure or an export that names nobody. Refused rather than treated as
// scope-wide, which is the difference between a governance operation and an accident.
codeNoSubject errorCode = "no_subject"
// codeInvalidErasure is an erasure that names both a person and sources, too many sources, or a
// source that is not an observation id.
codeInvalidErasure errorCode = "invalid_erasure"
// codeInvalidExport is the same refusal for an export, which selects the same two ways.
codeInvalidExport errorCode = "invalid_export"
// codeExportTooLarge is a subject holding more rows than one export returns. Refused rather
// than truncated: an export missing rows nobody counted is the opposite of what it is for, and
// the caller's next move — export those turns by source — is one they can take.
codeExportTooLarge errorCode = "export_too_large"
// codeInternal is anything that went wrong here. It says nothing about what, because what went
// wrong is a description of how this system is built.
codeInternal errorCode = "internal"
// The management surface's refusals.
codeInvalidProject errorCode = "invalid_project"
codeInvalidCredential errorCode = "invalid_credential"
)
var errorCodes
var errorCodes = []errorCode{
codeInvalidMessageWindow, codeSourceChanged,
codeInvalidPassageQuery, codePassageUnavailable,
codeInvalidEntityCandidateQuery, codeEntityCandidatesUnavailable,
codeInvalidReportCandidateQuery, codeReportCandidatesUnavailable,
codeInvalidEntityPage,
codeUnauthenticated, codeInvalidBody, codeInvalidTurn,
codeInvalidQuestion, codeNoSubject, codeInvalidErasure, codeInternal, codeIdempotencyConflict, codeRateLimited, codeNoCapacity,
codeInvalidCitation, codeCitationLimit, codeInvalidExport, codeExportTooLarge, codeNotFound, codeInvalidDestination, codeNotificationsOff, codeForbidden, codeInvalidRecordPage, codeInvalidRecallControls, codeInvalidContext, codeInvalidRecordMutation, codeRecordConflict, codeRecordMutationLimit, codeInvalidFeedback, codeFeedbackPromoted, codeInvalidArtifact, codeArtifactConflict, codeStorageCapacity, codeInvalidSubject, codeSubjectConflict,
codeInvalidProject, codeInvalidCredential,
}
errorCodes is the declared set, in the order the contract publishes them.
What this does not cover: it says which codes exist, not which operation can return which. A per-operation set would document better and would be maintained by hand against handlers that change, so it would be wrong before it was useful. The honest statement is that any operation can return any of these, and that is what the contract says.
func writeError
func writeError(w http.ResponseWriter, status int, code errorCode, message string)
writeError says what the caller can act on and nothing about how the system is built.
method Server.failed
func (s *Server) failed(w http.ResponseWriter, err error, operation, project, message string)
failed answers a database error that no handler-specific arm recognised.
Why this is not just a 500
One database failure is the deployment's own doing and clears by itself: the server running out of connection slots. A caller did nothing wrong, the condition ends when a sibling process finishes, and an observation's idempotency key makes the retry safe. Answering it as an internal error tells a caller nothing they can act on — measured as ten documents lost on an eight-A100 node — so it is answered retryably, with the same status and header the admission gate uses for the same reason.
Why it is one function rather than an arm in every handler
Rule 8: a rule every future handler has to remember is broken by the handler written under pressure. Every default arm calls this, so a route added later is covered by having a default arm at all, which it cannot serve without.
What it does not cover
A handler that recognises the error in an earlier arm never reaches here. That is correct — a specific answer beats a general one — but it means a store that folds connection exhaustion into its own sentinel hides it from this, and the honest place to fix that is the store.
func names
func names(v []string) []string
names never renders null, for the reason coDerived does not: an absent list and an empty one read the same to a caller and mean different things.
func coDerived
func coDerived(ids []string) []string
coDerived never renders null. An absent list says "we did not check"; this is checked on every bundle, so the honest empty answer is an empty list.