Documentation / Configuration

StateLens: configuration

On this page

Everything StateLens needs is configuration, and only two things are required: a connection string for each store the installed services read, and how operators sign in. The rest have defaults that are right for a store installed under Memoria's own default names.

Settings are read the way ASP.NET Core reads any of them — appsettings.json, appsettings.{Environment}.json, environment variables, then command-line arguments — so a setting can be overridden without editing a file.

Every setting

As environment variables, replace each : with a double underscore: ConnectionStrings__Orders, Databases__Orders__Provider, Extensions__Directory, Authentication__Oidc__ClientSecret, Authorization__Roles__Administrator.

A host that gives a connection string a type of its own writes a second entry beside it. App Service does, for every type its Connection strings blade offers but Custom: a string named Orders there arrives as both ConnectionStrings:Orders and ConnectionStrings:Orders_ProviderName, the second holding System.Data.SqlClient, Npgsql or MySql.Data.MySqlClient. That second entry is an ADO.NET provider name rather than a store, so the tool passes over it and reads the engine off the string itself as ever.

When it is not configured

A tool told nothing about how operators sign in, or holding a connection string it cannot read, does not serve its pages. It answers every address with one page instead — at status 503, so a health check or a monitor reads it as a deployment that is not up — saying which settings would have let it start and linking back here. Nothing else is mapped while it does: not a page, not the upload form. The same is said in the log, as an error, for whoever is looking at the host rather than the browser. The messages quoted below are what that page and that log say.

A connection string that is missing is not that. The tool starts, and the service that named it is unreachable until the string is there.

The connection strings

A service's manifest names its connection string by name, and the string itself lives in the tool's configuration. Three relational engines, one document store and one stream store are read: PostgreSQL, SQL Server and SQLite through their keyword strings; Cosmos DB through its account string, AccountEndpoint=…;AccountKey=…, for a service whose store is declared with "kind": "cosmos"; and KurrentDB (formerly EventStoreDB) through the client's URL form, esdb://… or kurrentdb://…, for one declared with "kind": "kurrentdb".

Each service's manifest names the connection string it is read over — "connectionString": "Orders" in the manifest — and the configuration holds a string under that name:

{
  "ConnectionStrings": {
    "Orders": "Host=localhost;Port=5432;Database=orders;Username=postgres;Password=password",
    "Billing": "Data Source=C:\\stores\\billing.db"
  }
}

One instance holds as many strings as its services name, and two services may name one string and read one store. No name is required, Memoria — the one name the tool read before it had services — included. A service naming a string the configuration lacks is listed on the home page as unreachable, not configured, and each of its pages says so in place of its rows, until the string is added and the tool restarted. The manifest is not refused for it: the zip may well be uploaded before the deployment it is meant for is configured.

A string that is there but cannot be read is a different thing — a mistake in the file rather than a service ahead of its deployment — and the tool answers only the page that says so, whatever the string is called:

Connection string 'Orders' could not be read: …

Start-up logs one line per service, so a store that answers nothing can be traced to the string it was opened over, or to the string it was not:

info: StateLens[0]  Service Orders reads connection string Orders with PostgreSQL.
warn: StateLens[0]  Service Billing names connection string Billing, which is not configured.

The tool opens stores somebody else created. It creates nothing — no database, no container, no table — so each store has to exist and carry its schema already: whatever the framework that wrote it installed, which is what the manifest's store block describes — see What to put in a zip; for a store Memoria wrote, Install the store schema on the framework's site says how. The one database the tool does create is its own, the accounts, and that is never a store.

Which engine it is

The engine is read off the connection string. Most strings say plainly which one they are for, because each provider takes keywords the others do not:

Keywords all of them take — Database, Server, User Id, Password — settle nothing and are ignored for this purpose.

Set Databases:{name}:Provider, under the string's own name, when that string carries signals for more than one engine, or for none. The tool refuses to guess in either case, and says which it met:

The provider for connection string 'Orders' could not be read off it: it carries keywords for more than one provider. Set Databases:Orders:Provider to Npgsql, SqlServer, Sqlite or Cosmos.

A Cosmos DB string — AccountEndpoint=, AccountKey= — is recognised too, and read for a service whose store is declared with "kind": "cosmos" (see A store in Cosmos DB); under a service declaring tables it is refused, since a Cosmos account has neither. The service is listed, and every page under it says so.

The setting is not checked against the string. It is the way out of a string the tool cannot read, so second-guessing it would close the door it opens. The names are matched case-insensitively and without spaces, hyphens or underscores, so SQL Server, sql_server and sqlserver are one answer; postgres, postgresql and npgsql are another.

The string called Memoria also reads the older, unnamed Database:Provider, so a configuration written for the tool before it had services settles it as it always did. The named setting wins where both are set.

In-memory SQLite is refused

Data Source=:memory:

is rejected at start-up rather than opening an empty store. Such a database lives only as long as the connection that opened it, and the tool opens a connection per unit of work, so it would find nothing whatever was seeded. Point it at a file.

A store reached over a network

A PostgreSQL or SQL Server store fails in ways a store on the same disk does not, and most of them are over by the next attempt: a pooled connection the far end closed while it sat idle, a failover, a minute of maintenance. The tool tries such a failure again — three times, waiting no longer than a second — before it shows it to whoever opened the page. SQLite is not retried, having no connection to lose.

A second of retrying does not ride out a failover, and is not meant to. It covers the failure that is over immediately, which is the common one; anything longer is reported, and the next visit finds the store back. Retries also sit inside Stores:Patience, so a store that is simply slow is still given up on when the patience says, not later.

What retrying cannot fix from here is a pool that keeps handing out connections already closed at the other end. That is settled in the connection string:

Host=…;Database=…;Username=…;Password=…;Ssl Mode=Require;Keepalive=30;Minimum Pool Size=1;Command Timeout=30

Connection Idle Lifetime — 300 seconds unless set — prunes connections idle longer than it, but only those above Minimum Pool Size. The kept one is never pruned, which is why it needs the keepalive rather than a shorter lifetime.

A cancelled read is not a broken store. PostgreSQL does not report an abandoned read as a cancellation: Npgsql tears the connection down, the socket read fails on its own account, and EF Core wraps that as "An exception has been raised that is likely due to a transient failure". Seeing that sentence is a reason to look at Stores:Patience before looking at the store.

Signing operators in

Operators sign in one of two ways: through an OpenID Connect provider, or with accounts the tool keeps itself — and the tool answers nothing but the page saying so until it is told which, or told, in so many words, to run open. There is no default. The settings page takes an assembly and runs it, so "nobody said" cannot mean "anybody may". Exactly one of the three is set; two at once are refused naming both.

Through a provider

{
  "Authentication": {
    "Oidc": {
      "Authority": "https://login.example.com/realms/memoria",
      "ClientId": "statelens",
      "ClientSecret": "<from the provider>"
    }
  }
}

Authority is the issuer: the address the provider's discovery document is read from, at <Authority>/.well-known/openid-configuration. Any provider that publishes one will do — Microsoft Entra ID, Amazon Cognito, Google, Auth0, Okta, Keycloak, Zitadel, Authentik — and the tool never learns which. Whoever deploys it chooses the provider, and with it the cloud, rather than the tool choosing for them.

ClientId and ClientSecret are what the provider issued when the tool was registered there as a confidential web client. The secret is a secret: put it in an environment variable (Authentication__Oidc__ClientSecret) or the host's secret store, not in the file.

Scopes is what the sign-in asks the provider for, space-separated. The default asks for the identity, the name to show, and the email address. Add whatever scope your provider puts its groups or roles under when the next release starts reading them.

What the tool does with these: the authorization code flow with PKCE, tokens exchanged on the back channel and never handed to the browser, a session cookie that lasts as long as the identity the provider issued. The provider has to be told where to send the operator back to — see Deployment for the address to register.

A setting missing from the three is refused by name:

Authentication:Oidc:ClientSecret is not configured. The provider needs Authentication:Oidc:Authority, Authentication:Oidc:ClientId and Authentication:Oidc:ClientSecret all set.

Which provider was chosen is logged at start-up, next to which store:

info: StateLens[0]  Operators sign in through https://login.example.com/realms/memoria.

Keeping the accounts in the tool

{
  "Authentication": {
    "Local": true
  }
}

keeps the operators' accounts in the tool itself: ASP.NET Core Identity over a database of the tool's own. Nothing more said, that is a SQLite file, accounts.db under Accounts:Directory — App_Data/accounts beside the uploads unless the setting says otherwise, /data/accounts in the image — and nothing else is needed: no provider, no registration at one, no database server.

The accounts can live in a database of your choosing instead, named the way a service names its store: Accounts:ConnectionString is the name of an entry under ConnectionStrings, never the string itself, and its engine is read the way a store's is — PostgreSQL, SQL Server or SQLite, with Databases:{name}:Provider to say which when the string does not. The tool creates Identity's tables there at start-up and brings them to each release's shape, so give it an empty database, or a schema of its own; a string for KurrentDB or Cosmos DB is refused, since the tables are relational.

{
  "Authentication": { "Local": true },
  "Accounts": { "ConnectionString": "Operators" },
  "ConnectionStrings": {
    "Operators": "Host=db;Port=5432;Database=statelens_accounts;Username=statelens;Password=…"
  }
}

Either way it is the one database the tool writes and the one schema it creates, and it is written to no store the tool reads, even when the accounts share a server with one.

There is no registration. The first start, with no account yet, answers every address with a setup page at /setup that makes the first Administrator — an email and a password — and signs them in; once any account exists that page is gone, and every other account is made by an Administrator on the Users page of the admin area — an email, a password to pass on, which the operator changes to one of their own on the Password page under their name — and the roles to hold — where each account has a page of its own on which its roles are given and taken, its email changed, a password set without the old one, a lock lifted, and the account removed after one question; the last Administrator keeps the role whatever is ticked, since without one nobody could give it back, and neither the last Administrator nor the account one is signed in as can be removed. The cookie is checked against the account on every request, so a password set or an account removed ends the account's sessions at their next request. Operators sign in at /login, with the email and the password; five wrong passwords lock the account for five minutes, and the one refusal covers both fields so that a wrong password does not confirm an email. A session lasts twelve hours without a request and is renewed by each one; signing out ends it.

The roles are the roles: an account holds Administrator, Reader, or the name a manifest writes under roles.read, and the Authorization section does not apply — set beside Authentication:Local it is refused by name, since a mapping nobody reads is a setting somebody will trust. Every value the installed manifests name exists as a role from the moment the manifest is installed, without anyone making it, and stays when the manifest goes, since an account may hold it; the Roles page of the admin area lists each with where it came from and how many hold it, and an Administrator makes a role by name there and removes one that is neither built in nor named by a manifest still installed. Which way was chosen is logged at start-up:

info: StateLens[0]  Operators sign in with accounts the tool keeps itself, in /data/accounts. Roles are the roles an account holds.

or, with a connection string named, in the connection string Operators (PostgreSQL).

Roles

Signed in, an operator may hold one of two roles for a service, the second including the first:

There is no role between them because the tool writes to no store: every page under a service reads, and reading is what a Reader does.

A role is granted in two places. A service's manifest names, under roles.read, the claim values that may read that service alone. The tool's configuration maps claim values to the same roles for every service, Administrator among them — there is no per-service Settings:

{
  "Authorization": {
    "RoleClaimType": "roles",
    "Roles": {
      "Administrator": "memoria-admins",
      "Reader": "memoria-auditors, memoria-support"
    }
  }
}

Both places read the values off the same claim, so orders-team in a manifest means what memoria-admins means here. Nothing is granted until it is said, in one place or the other: an operator whose claims match no mapping and no manifest is nobody. They see no service on the home page — which says instead that nothing their sign-in carries names one, and who to ask — and an address they type under a service sends them to the page that says which service and which role.

The Roles page of the admin area, at /admin/roles, lists all of this back: the two roles with the claim values mapped to each, or the word that none is, and every value the installed manifests name with the services that name it. It changes nothing — the values are decided at the provider and in the files — and the Users page beside it says the same of the accounts, naming the provider. Running open, both say nobody signs in. With accounts the tool keeps, the same page is where roles are made and removed.

RoleClaimType names the claim the provider puts its groups or roles in. Every provider does this differently — Entra ID sends app roles under roles and group ids under groups, Cognito sends cognito:groups, Keycloak sends realm roles nested under realm_access unless a mapper flattens them into a claim of their own — so the tool asks rather than guesses. The default is roles.

Each of the two lists is comma-separated, so it fits in one environment variable:

Authorization__RoleClaimType=cognito:groups
Authorization__Roles__Administrator=memoria-admins

An operator whose claim carries a mapped value holds that role for every service; one whose claim carries a value a manifest names holds that role for that service. A group the provider happens to call Administrator grants nothing until it is mapped here. An operator without Administrator does not see the Settings link at all. An operator who types an address they may not use is told which role it needed, which service it was under, and where they were going, on a page that says so. The log lines at start-up say what was mapped:

info: StateLens[0]  Roles are read off the roles claim: Administrator for memoria-admins, Reader for memoria-auditors, memoria-support.

With no Authorization section at all, only the manifests grant anything — nobody can use Settings — and start-up says so:

info: StateLens[0]  No roles are mapped: a signed-in operator sees only the services whose manifest
      names a claim value they hold, and nobody can use Settings. Set Authorization:Roles:Administrator
      and Authorization:Roles:Reader to the claim values that grant each role for every service.

Running open, roles do not apply — the manifests' as much as the configuration's: there is nobody to hold one, and every service is listed and every page answers.

Running open

{
  "Authentication": {
    "Disabled": true
  }
}

runs the tool with nobody signed in and every page answering anyone who can reach it — including the upload form. It is how the image is run on your own machine — see Run it locally — and it is a choice that has to be written down: the tool told neither this nor a way to sign in refuses,

Authentication is not configured. Set Authentication:Oidc:Authority, Authentication:Oidc:ClientId and Authentication:Oidc:ClientSecret to sign operators in through an OpenID Connect provider, or Authentication:Local to true to keep their accounts in this tool.

and a tool told two of the three refuses too, rather than guessing which was meant. The refusal names the two ways of signing in and not this flag, on purpose: it is shown to whoever asks the tool, and the way to run it open is written here rather than advertised there. While open, every start-up says so:

warn: StateLens[0]  Running open: nobody is signed in and every page, including the upload form,
      answers anyone who can reach it, because Authentication:Disabled is true.

Editions

The licence meters the tool by services: one entry in the services list of an installed statelens.json. The tool counts them itself; there is no key and nothing to activate. Edition says which edition the instance runs as, and so how many services it reads:

Edition=Standard dotnet StateLens.dll

An upload that would take the instance past its edition's ceiling is refused, and Settings says so: "The Community edition reads one service; installing billing.zip would make 2." Should the installed archives already declare more than the edition reads — the setting was lowered, or archives were copied into the extensions directory by hand — the services up to the ceiling are read, oldest archive first, and the rest are named on Settings as installed and not read; nothing lists them and no page opens over them. A name that is not one of the four runs the instance as Community and says so on Settings. The About page says which edition is running.

Extensions

Uploaded archives and the assemblies taken out of them are kept on disk:

<Extensions:Directory>/
  zips/              the archives, exactly as uploaded
  lib/<archive>.zip/ the assemblies each archive brought, one directory per archive, loaded at start-up

The archive is kept whole, so its manifest is its own record of what it declared: each row of Installed on Settings opens the statelens.json it was installed from, as uploaded, before the bin that removes it.

The default is App_Data/extensions under the content root. Point Extensions:Directory somewhere else to give an instance its own uploads — a scratch directory for a second instance, or a mounted volume so a container keeps what was uploaded across restarts:

Extensions__Directory=/var/lib/statelens/extensions dotnet StateLens.dll

Three behaviours worth knowing:

• Each archive has a directory of its own. Its assemblies are stored there by file name alone: an entry at bin/Release/Contoso.dll and one at ../../Contoso.dll both land on lib/pack.zip/Contoso.dll, which is what stops a crafted archive writing outside the store. Two archives may each carry a Contoso.dll, and each keeps its own. Uploading an archive again under the same name replaces its directory, files and all.
• Removing an archive removes its directory and nothing else. What another archive brought is untouched, an assembly both carry included.
• Each archive loads into a context of its own. One AssemblyLoadContext per archive, so two archives can carry two versions of one library without either seeing the other's, and a reload lets go of what the previous one loaded. Within an archive the rule is one line long: anything the tool carries is the tool's; the archive supplies the rest. The tool's copy of Memoria, of the serializers and of the framework is what its pages work with, so an archive's copy of any of those is not loaded, whatever its version. The service's sheet on Settings, under Types, lists the files this happened to, and names the two versions when they differ — the line to read when a page misbehaves over a domain built against a newer library than the tool carries.

Every installed archive's directory is loaded at start-up, so what was uploaded survives a restart as long as the directory does. A directory under lib/ that no archive in zips/ accounts for is not loaded.

What to put in a zip

Three things: a manifest, the assemblies holding your domain types, and any dependency of theirs that the tool does not already carry — a validation library, say. The loader resolves those from the archive's own directory; one archive never sees another's.

Managed assemblies only. A native library — a Windows .dll with no managed code in it, or the .so and .dylib a NuGet package ships under runtimes/ — is not loaded: the service's sheet lists such a file as "native libraries, which the tool does not load", and a fold that turns out to need one says so on its page. A domain zip should not need one: the database drivers are the tool's own, and events and aggregates are plain classes. A file that is neither a managed assembly nor a native library is a problem, and is listed as one on the Types tab and on the service's sheet, with the reason.

The manifest is required. A file called statelens.json at the root of the archive — not in a folder — declaring the services the zip brings. A zip without one is refused, and so is one whose manifest breaks a rule below; the Settings page says which.

A service says three things: which assemblies hold its types, what its domain is — which of those types are the streams, the events, the aggregates and the projections, how each is named in the store, how a model folds and how a payload is read back — and where its store is: the table or SELECT that holds the events, and the ones that hold the snapshots, if it keeps any. The tool has no notion of any framework beyond these words. A store Memoria wrote, a store Marten wrote and a store your own code wrote are declared the same way, and read by the same code; the only thing built in is the dynamic consistency boundary, which is Memoria's own and which a service opts into with "dcb": true.

This is a service over a store Memoria wrote, declared in full — also kept as a ready-to-edit file under docs/manifests, beside one for a store Eventuous wrote:

{
  "services": [
    {
      "name": "orders",
      "description": "Orders placed in the shop, one stream a customer.",
      "assemblies": ["Contoso.Orders.Domain.dll", "Contoso.Orders.Contracts.dll"],
      "connectionString": "Orders",
      "roles": { "read": ["orders-team"] },
      "dcb": true,
      "domain": {
        "streams":     [ { "implements": "Memoria.EventSourcing.Domain.IStreamId" } ],
        "events":      [ { "implements": "Memoria.EventSourcing.Domain.IEvent" } ],
        "aggregates":  [ { "implements": "Memoria.EventSourcing.Domain.IAggregateRoot",
                           "id": { "genericInterface": "Memoria.EventSourcing.Domain.IAggregateId`1" } } ],
        "projections": [ { "implements": "Memoria.EventSourcing.Domain.IProjection",
                           "id": { "genericInterface": "Memoria.EventSourcing.Domain.IProjectionId`1" } } ],
        "naming": {
          "events":      { "attribute": "Memoria.EventSourcing.Domain.EventType", "property": "Name", "version": "Version" },
          "aggregates":  { "attribute": "Memoria.EventSourcing.Domain.AggregateType", "property": "Name", "version": "Version" },
          "projections": { "attribute": "Memoria.EventSourcing.Domain.ProjectionType", "property": "Name", "version": "Version" }
        },
        "fold":       { "methods": ["Apply"], "version": "Version", "applies": "EventTypeFilter" },
        "serializer": { "kind": "newtonsoft" },
        "ids":        { "property": "Id", "claim": "EventPropertyFilter", "keySeparator": ":" }
      },
      "store": {
        "events": { "table": "DomainEvents", "streamId": "StreamId", "sequence": "Sequence", "type": "EventType",
                    "payload": "Data", "written": "CreatedDate", "writtenBy": "CreatedBy", "id": "Id" },
        "snapshots": [
          { "table": "DomainAggregates", "streamId": "StreamId", "id": "Id", "type": "AggregateType", "version": "Version",
            "appliedUpTo": "LatestEventSequence", "payload": "Data", "created": "CreatedDate", "createdBy": "CreatedBy",
            "updated": "UpdatedDate", "updatedBy": "UpdatedBy" },
          { "table": "DomainProjections", "kind": "projection", "streamId": "StreamId", "id": "Id", "type": "ProjectionType",
            "version": "Version", "appliedUpTo": "LatestEventSequence", "payload": "Data", "created": "CreatedDate",
            "createdBy": "CreatedBy", "updated": "UpdatedDate", "updatedBy": "UpdatedBy" }
        ]
      }
    }
  ]
}

And this is one over a store nothing of Memoria's ever touched — plain records, a table with its own column names, and a read model kept behind a SELECT:

{
  "services": [
    {
      "name": "Things",
      "assemblies": ["Contoso.Things.dll"],
      "connectionString": "Things",
      "domain": {
        "streams":     [ { "type": "Contoso.Things.ThingId" } ],
        "events":      [ { "namespace": "Contoso.Things.Events" } ],
        "aggregates":  [ { "type": "Contoso.Things.Thing", "id": "Contoso.Things.ThingId" } ],
        "projections": [ { "type": "Contoso.Things.ThingSummary" } ],
        "naming":      { "events": "snake-case", "aggregates": "lower-case", "projections": "snake-case" },
        "labels":      { "projections": "view" },
        "fold":        { "methods": ["Apply"], "version": "Version" },
        "hidden":      ["Id", "Version"]
      },
      "store": {
        "events": { "table": "evts", "streamId": "stream", "sequence": "seq", "type": "kind", "payload": "body", "written": "at", "writtenBy": "who", "metadata": "meta" },
        "snapshots": [
          { "table": "snaps", "typeName": "thing", "streamId": "stream", "id": "id", "version": "ver", "appliedUpTo": "upto", "payload": "body", "created": "made", "updated": "changed" },
          { "sql": "SELECT stream, id, kind, ver, body, at FROM views WHERE kind <> 'draft'", "kind": "projection",
            "streamId": "stream", "id": "id", "type": "kind", "version": "ver", "payload": "body", "updated": "at" }
        ]
      }
    }
  ]
}

Keys the manifest carries that the tool does not read are ignored, so a later version may add to the shape without an older tool refusing what it wrote.

The domain block

Each of streams, events, aggregates and projections is a list of selectors, and a type plays that part when any selector in the list picks it. A selector says one thing:

An aggregate's or projection's selector may add an id, saying which types address that model — what the streamed pages call its identifiers, and what the events tab narrows a shared stream by: a type by full name; { "genericInterface": "…IAggregateId1" }, for a type closing that interface over the model; or { "suffix": "Id" }, for the type named after the model with that suffix. A model with no id` is addressed by the store's own id column alone.

naming says how each part's stored name is made, so a row's type column meets the type that opens it. One rule per part, or one rule for all three:

fold says how a model is rebuilt from its events for the compare tab: methods are the names of the methods that apply an event, matched by the event's type (an overload per event, one generic method, or one taking object that applies whatever comes); version is the property that counts the events applied, incremented by the tool once per event a method accepted (a method returning false declines one); applies is the property listing the event types the model folds, when the model keeps one, and narrows the read to them. A model that is a value — a record whose apply returns the next state rather than changing this one — is folded by taking what each call returns. A model is built from its parameterless constructor, or from one whose parameters all have defaults. A model whose apply methods take the event wrapped — Marten's Apply(IEvent<OrderPlaced>) — folds once fold.wrapper names the wrapper: its type is the open generic type's full name (JasperFx.Events.Event, with or without the `1), constructed around each event from a constructor taking the event or, failing one, put on the property event names; facts lists the wrapper's properties to fill from the stored row, each from sequence, written, streamId or id — Marten's Version from the sequence, its Timestamp from when the row was written. Methods taking the bare event on the same model are called as before. serializer is system-text-json — the default, case-insensitive — or newtonsoft, with caseInsensitive and enumsAsStrings beside it. ids says, of an identifier type, which property holds the value the store writes, which claim property lists what it narrows a shared stream by, and the keySeparator the store joins something after — the type's version, say — so the id can be read back off the key. hidden lists property names the state tab leaves out.

labels says what the domain calls its models, when it does not call them aggregates and projections: the bar, the section pages and the tiles under that service take the word, and its sheet says it back. One entry per kind, aggregates or projections, each the singular in lower case — "projections": "view" shows Views, View Types, "3 views" — with the plural made by adding an s; a plural not made that way is given as both, { "one": "summary", "many": "summaries" }. The addresses do not change: the pages stay under streamed/projections whatever they are called. A store's snapshots keep their kind of aggregate or projection, which says which list the rows join; the label is the list's name.

The store block

events is one source; snapshots is a list of them, or absent for a store that keeps none. A source is a table or a sql — a SELECT the tool reads through as a subquery, so a view, a join or a WHERE that hides archived rows all fit — and the names of the columns the tool needs from it:

A store in Cosmos DB

A store in Cosmos DB is the same block with "kind": "cosmos", the database the containers are in, and a container in place of each table. A column is a property of the document — written plain, streamId, or as a path, meta.version — or anything else, taken to be an expression in Cosmos DB's own SQL over c, the document: c.data.version, StringToNumber(c.v), c._ts. A where is such an expression too. There is no sql: Cosmos reads no SELECT as a subquery.

"store": {
  "kind": "cosmos",
  "database": "shop",
  "events": { "container": "events", "streamId": "stream", "sequence": "seq", "type": "kind", "payload": "body", "written": "at", "id": "id" },
  "snapshots": [
    { "container": "snapshots", "typeName": "Order", "streamId": "stream", "id": "id", "version": "version", "payload": "state", "updated": "updatedAt" }
  ]
}

A payload may be a JSON string or a nested document; either is read as text. A date may be text in any form the framework parses, or the seconds since the epoch that Cosmos itself stamps a document's _ts with. The lists order by the one date alone, since ordering by several properties needs a composite index the container may not have; several snapshot sources of one kind are read whole and paged by the tool, since Cosmos has no UNION; and the streams are grouped on the server and paged by the tool. Against the emulator — an endpoint on the same machine, or one over plain HTTP — the tool goes through the gateway and takes the emulator's certificate. dcb is refused over a cosmos store, as over a kurrentdb one. Memoria's own Cosmos store is one such layout: docs/manifests/cosmos-memoria.json declares it.

A store in KurrentDB

A store with no tables — KurrentDB, formerly EventStoreDB — is declared with "kind": "kurrentdb" in the same block, and read over a connection string in the client's own URL form, esdb://host:2113?tls=false or kurrentdb://…, which the tool recognises by its scheme. Its events source names the streams that are the service's, by prefix; its snapshot sources name families of streams whose last event is a model's current snapshot:

"store": {
  "kind": "kurrentdb",
  "events": { "streams": ["order-", "return-"] },
  "snapshots": [
    { "streams": "snapshot-order-", "typeName": "Order", "streamId": "order-{id}",
      "version": "data.version", "appliedUpTo": "metadata.appliedUpTo", "payload": "data" }
  ]
}

A path starts from data or metadata, whole or followed by property names matched without regard to case, or is one of the event's own facts: eventType, eventNumber, created. Events are read as the store keeps them: the event's number in its stream is its sequence, its type string its type, its data and metadata as text, and its key stream:number.

Two things worth knowing about how a stream store is read. The store answers two questions cheaply — one stream in order, and every event in log order narrowed by stream prefix — and nothing else, so a page over one stream reads that stream and every other page reads the log through the store's prefix filter, narrowing by type, text and pattern as the events go past; a count is such a read that keeps nothing, kept between pages by the caching window. And the dynamic consistency boundary is Memoria's own tables, so dcb is refused over a kurrentdb store.

Nothing here changes the database. The tool reads what it is shown, in the engine's own dialect, and never creates, alters or writes a row.

The service's sheet on the Settings page says all of this back in words — where the events and the snapshots are read from, which types play which part, how they are named, folded and read back — so a manifest that was read other than as meant is caught there rather than on a data page.

An archive already in the directory without a manifest — from before one was required — stays listed, marked No manifest, registers nothing, and its row says why. Add a manifest to the zip and upload it again. One whose manifest breaks a rule is marked Manifest refused instead, and its row still opens the manifest, since what it says is how the rule was broken.

Never include a Memoria* assembly. Uploaded types must bind to the ones the process already loaded, or nothing they declare satisfies IEvent or IAggregateRoot. An assembly compiled against a different Memoria version loads and then fails to yield types at all; the Settings page reports it:

Contoso.Domain.dll: Could not load file or assembly 'Memoria, Version=…'

Rebuild against the version the tool was built from and upload again.

Branding

An Administrator can put their own name and logo in the header in place of Memoria's, on the Branding tab of the Settings page. Both are kept in files, not in any store — the tool reads over the stores it is pointed at and owns none of them:

<Branding:Directory>/
  branding.json   your own name, which name and which logo are drawn, and a version each save moves on
  logo.png        or logo.jpg, or logo.webp — only while an uploaded logo is drawn

The default is App_Data/branding under the content root. They are read once at start-up and held in memory, so drawing the header never reaches the disk; a save replaces what is held at once.

• The logo is a PNG, JPEG or WebP of 512 KB or less, told apart by its first bytes rather than its name. SVG is refused: one can carry script, and the logo is served from the tool's own origin.
• The name and the logo are each Memoria's, your own, or none. Either can stand alone, and with neither the header draws no brand at all and starts with its links. Choosing Memoria's mark or none for the logo deletes an uploaded one; a file chosen is taken as your own logo whichever option is ticked. Your own name is kept while another is chosen, so it is there to choose again.
• The name is at most 60 characters.
• The logo is served at /branding/logo to anyone, signed in or not, because the signed-out page draws the header too — beside the name, which it shows already. The header asks for it by the version, so a browser caches it for good and still fetches a new one after the next save.
• A file that cannot be read is drawn as Memoria. A branding.json broken by hand does not stop the tool starting; the next save writes over it.
• Restore StateLens's own on the tab puts the default name and mark back.
• The About page is always headed Memoria, with Memoria's mark: it describes the tool, whatever this deployment is called.

Like uploads, the copy held in memory is the process's own: a second instance over the same directory sees a save when it is next restarted.

Caching

The tool counts in two places, and both count what one page's own filter reaches: a Data page's total, over the rows it lists, and the total over the events tab of a detail page. Nothing else scans a store to be drawn. Home, a service's own page, the model overviews, the section pages, the Types pages and the Streams page are all tiles and lists over what the uploaded assemblies declare: they lead to a section rather than reporting on it, so none of them asks a store anything and none of them can be held up by one.

The aggregates and projections Data tables mark each row whose stored snapshot is behind its history with a clock, the rule the detail page's Info tab warns by: more events of the types the model applies than the version it was folded to, in the stream the identifier claims, or in the boundary a DCB model was keyed by. A row the rule cannot be applied to — a stream shared by several models whose identifier cannot be recovered, a boundary that cannot be read back — is not marked, rather than marked wrongly. The table is drawn first and the marks follow; each is its own read of one stream or boundary, a few at a time.

What is read is kept for one while, set on the Caching tab of the Settings page:

• Figures kept for, in seconds: a data page's total and a detail page's events tab total — how many rows the filter reaches — whether a row is behind its history, and when the newest was written where the store has to search for it: nothing orders a relational streamed log by date alone. From 0, which reads them on every visit, to 3600, an hour. It is 30 until it is changed. Where the store finds the newest at once — the DCB log, ordered by its key — it is asked on every visit and never kept: it is the figure that shows a service is alive.

A service's sheet under Services on the Settings page is the one page that asks a store how it is: whether it answers, how quickly, and when its last event was written. A service over both models is both logs together, and its last event is the newer of the two. The sheet is drawn before the store is asked; the two rows follow once it has answered, and a store is given 5 seconds before they say it could not be read. A store that is not configured says so instead, and is not asked.

Stores:WarmAtStartUp set to false leaves out the reads the tool makes of each store as it starts. They are there so that nobody's first data page pays for Entity Framework Core building its model and compiling the page's queries; without them, whoever opens the first page that reads a store waits for that once.

How long that is, is Stores:Patience, in whole seconds — a deployment setting rather than one of these, because it is a fact about how far away the store is and not a preference. Five seconds is a number for a store on the same machine. One reached over a network is routinely slower than that and is not broken for being so: a deployment whose sheets keep saying a healthy store could not be read is a deployment that should raise this. It stays the default because raising it costs every deployment and not only the slow ones — a read still running is a scope, a context and a pooled connection still held, and a store that has stopped answering holds one per read for as long as this allows.

A store that runs out of it says it did not answer in that many seconds, whatever its driver made of being abandoned mid-read. PostgreSQL in particular does not report the abandoning as a cancellation: Npgsql tears the connection down, the socket read fails on its own account, and EF Core wraps that as a failure likely to be transient. Reading that sentence off the sheet would send whoever is looking after a fault the store does not have.

The settings are kept in a file, not in any store, for the reason the branding is:

<Settings:Directory>/
  caching.json   how long figures are kept, in seconds

The default is App_Data/settings under the content root. It is read once at start-up and held in memory; a save is felt from the next visit. A file that cannot be read is taken as the defaults, and the next save writes over it. Every figure is forgotten when an upload or a removal changes the services, since it may then be of another store. Like the branding, the copy held in memory is the process's own.

Logging and hosting

Standard ASP.NET Core settings apply. The defaults in appsettings.json are:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "AllowedHosts": "*"
}

Two loggers of the tool's own are worth raising or quieting by name: StateLens.Settings (uploads, removals, refreshes, branding and the caching settings) and StateLens.Streamed / StateLens.Dcb (snapshot refreshes).

What each write logs

Every write an operator can make is logged under an event of its own, with a fixed id and name, so it can be found by the name rather than by its wording:

Each line names the operator who asked, as the name the provider showed and the subject it keys them by, and says what it was about. Every write is to the tool's own files: it writes to no store.

The operator is three columns: Operator is the two together as the wording says them, and OperatorName and OperatorSubject are each apart, so everything one subject did can be asked for without matching text, and is still found after a rename. Running open, Operator says nobody (running open) and the other two are empty.

Application Insights

Set APPLICATIONINSIGHTS_CONNECTION_STRING — the setting App Service sets when Application Insights is connected to it, so a deployment there has it already — and every line above is exported through OpenTelemetry to that resource, along with the request it was written in. In the portal, each is a row in the traces table: the wording in message, and the named values — FileName, Model, Instance, Operator, OperatorName, OperatorSubject, Error — with the event's EventId and EventName in customDimensions, so a query filters on the name rather than the wording:

traces
| where customDimensions.EventName in ("SnapshotRefreshed", "ExtensionInstalled", "ExtensionRemoved", "ExtensionsReread")
| project timestamp,
    event    = tostring(customDimensions.EventName),
    subject  = tostring(customDimensions.OperatorSubject),
    operator = tostring(customDimensions.OperatorName),
    model    = tostring(customDimensions.Model),
    instance = tostring(customDimensions.Instance),
    file     = tostring(customDimensions.FileName)
| order by timestamp desc

Everything one person did is | where subject == "3f1c…", whatever the provider showed as their name at the time.

The requests themselves, in the requests table, carry the operator too: the subject as user_AuthenticatedId, which the portal's own views filter and chart by, and OperatorName and OperatorSubject in customDimensions under the same names as the write lines. So the request a write was made in, and every page the same person opened, answer to the same clause. A request made running open carries none of the three.

Left unset, nothing is exported and the start-up log says so. The Logging levels above apply to what is exported as much as to the console, so a logger quieted there is quiet in the portal too.

Addresses come from ASPNETCORE_URLS; the image listens on 8080 — see Deployment.

Related

• StateLens — what each page shows
• StateLens: deployment
• Try it with sample data