🚧 This documentation is for the developer preview of m-ld.
The Javascript engine can be used in a modern browser or a server engine like Node.js.
The Javascript clone engine conforms to the m-ld specification. Its support for transaction pattern complexity is detailed below. Its concurrency model is based on immutable states.
npm install @m-ld/m-ld
To see some executable code, have a look at the Node.js starter project.
m-ld uses levelup to interface with a LevelDB-compatible storage backend.
A m-ld clone uses a 'remotes' object to communicate with other clones.
MqttRemotes.AblyRemotes.🚧 If your architecture includes some other publish/subscribe service like AMQP, or you would like to use a fully peer-to-peer protocol, please contact us to discuss your use-case.
The clone function initialises the m-ld engine with a leveldb back-end and the clone configuration.
import MemDown from 'memdown';
import { clone, uuid } from '@m-ld/m-ld';
import { MqttRemotes, MeldMqttConfig } from '@m-ld/m-ld/dist/mqtt';
const config: MeldMqttConfig = {
'@id': uuid(),
'@domain': 'test.example.org',
genesis: true,
mqtt: { hostname: 'mqtt.example.org' }
};
const meld = await clone(new MemDown, MqttRemotes, config);The clone function returns control as soon as it is safe to start making data
transactions against the domain. If this clone has has been re-started from
persisted state, it may still be receiving updates from the domain. This can
cause a UI to start showing these updates. If instead, you want to wait until
the clone has the most recent data, you can add:
await meld.status.becomes({ online: true, outdated: false });MQTT is a machine-to-machine (M2M)/"Internet of Things" connectivity protocol. It is convenient to use it for local development or if the deployment environment has an MQTT broker available. See below for specific broker requirements.
The MqttRemotes class and its companion configuration class MeldMqttConfig
can be imported or required from '@m-ld/m-ld/dist/mqtt'.
The configuration interface adds an mqtt key to the base
MeldConfig. The content of this key is a client
options object for MQTT.js. It must
not include the will and clientId options, as these are set internally. It
must include a hostname or a host and port.
MqttRemotes requires broker support for:
A good choice for local development is Aedes.
Ably provides infrastructure and APIs to power realtime experiences at scale. It is a managed service, and includes pay-as-you-go developer pricing. It is also convenient to use for global deployments without the need to self-manage a broker.
The AblyRemotes class and its companion configuration class MeldAblyConfig
can be imported or required from '@m-ld/m-ld/dist/ably'.
The configuration interface adds an ably key to the base
MeldConfig. The content of this key is an Ably
client options
object. It
must not include the echoMessages and clientId options, as these are set
internally.
If using token
authentication,
ensure that the clientId the token is generated for corresponds to the @id
given in the MeldConfig.
A m-ld transaction is a json-rql pattern, which represents a data read or a data write. See the m-ld specification for a walk-through of the syntax.
Supported pattern types for this engine are (follow the links for examples):
🚧 If you have a requirement for an unsupported pattern, please contact us to discuss your use-case. You can browse the full json-rql syntax at json-rql.org.
A m-ld clone contains realtime domain data in principle. This means that any clone operation may be occurring concurrently with operations on other clones, and these operations combine to realise the final convergent state of every clone.
The Javascript clone engine API supports bounded procedures on immutable state, for situations where a query or other data operation may want to operate on a state that is unaffected by concurrent operations. In general this means that in order to guarantee data consistency, it is not necessary for the app to use the clone's local clock ticks (which nevertheless appear in places in the API for consistency with other engines).
An immutable state can be obtained using the read and write methods. The state is passed to a procedure which operates on it. In both cases, the state remains immutable until the procedure's returned Promise resolves or rejects.
In the case of write, the state can be transitioned to a new state from within
the procedure using its own write method,
which returns a new immutable state.
In the case of read, changes to the state following the procedure can be
listened to using the second parameter, a handler for new states. As well as
each update in turn, this handler also receives an immutable state following the
given update.
await clone.read(async (state: MeldReadState) => {
// State in a procedure is locked until sync complete or returned promise resolves
let currentData = await state.read(someQuery);
populateTheUi(currentData);
}, async (update: MeldUpdate, state: MeldReadState) => {
// The handler happens for every update after the proc
// State in a handler is locked until sync complete or returned promise resolves
updateTheUi(update); // See §Handling Updates, below
});ui.on('action', async () => {
clone.write(async (state: MeldState) => {
let currentData = await state.read(something);
let externalStuff = await doExternals(currentData);
let todo = decideWhatToDo(externalStuff);
// Writing to the current state creates a new live state
state = await state.write(todo);
await checkStuff(state);
});
});ui.on('show', async () => {
clone.read((state: MeldReadState) => {
let currentData = await state.read(something);
showTheNewUi(currentData);
});
});Clone updates obtained from a read handler specify the exact Subject property values that have been deleted or inserted during the update. Utility methods are provided to help update app views of data based on updates notified via the read method:
Used to express an ordered or unordered container of data.
A JSON-LD context for some JSON content such as a Subject. m-ld does not require the use of a context, as plain JSON data will be stored in the context of the domain. However in advanced usage, such as for integration with existing systems, it may be useful to provide other context for shared data.
An JSON-LD expanded term definition, as part of a domain Context.
Convenience return type for reading data from a clone. Use as a
Promise
with .then or await to obtain the results as an array of
Subjects. Use as an
Observable with .subscribe
(or other RxJS methods) to be notified of individual Subjects as they arrive.
Note that all reads operate on a data snapshot, so results will not be affected by concurrent writes, even outside the scope of a read procedure.
A reference to a Subject. Used to disambiguate an IRI from a plain string. Unless a custom Context is used for the clone, all references will use this format.
Captures m-ld data semantics
as applied to an app-specific subject type T. Applies the following changes
to T:
@id property, as the subject identitysingle-valued constraint, which is applied at
runtime)undefinedSince any property can have zero to many values, it may be convenient to combine use of this type with the array utility when processing updates.
Note that a Resource always contains concrete data, unlike Subject, which can include variables and filters as required for its role in query specification.
the app-specific subject type of interest
Result declaration of a Select query.
Use of '*' specifies that all variables in the query should be returned.
A function type specifying a 'procedure' during which a clone state is available as immutable. Strictly, the immutable state is guaranteed to remain 'live' until the procedure's return Promise resolves or rejects.
can be MeldReadState (default) or MeldState. If the latter, the state can be transitioned to another immutable state using MeldState.write.
The allowable types for a Subject property value, named awkwardly to avoid
overloading Object. Represents the "object" of a property, in the sense of
the object of discourse.
A m-ld update notification, indexed by Subject.
A function type specifying a 'procedure' during which a clone state is available as immutable following an update. Strictly, the immutable state is guaranteed to remain 'live' until the procedure's return Promise resolves or rejects.
A query variable, prefixed with "?", used as a placeholder for some value in a query, for example:
{
"@select": "?name",
"@where": { "employeeNo": 7, "name": "?name" }
}A query pattern that writes data to the domain. A write can be:
Note that this type does not fully capture the details above. Use isWrite to inspect a candidate pattern.
A utility to generate a variable with a unique Id. Convenient to use when generating query patterns in code.
Utility to normalise a property value according to m-ld
data semantics, from a missing
value (null or undefined), a single value, or an array of values, to an
array of values (empty for missing values). This can simplify processing of
property values in common cases.
the value to normalise to an array
Indexes a m-ld update notification by Subject.
By default, updates are presented with arrays of inserted and deleted subjects:
{
"@delete": [{ "@id": "foo", "severity": 3 }],
"@insert": [
{ "@id": "foo", "severity": 5 },
{ "@id": "bar", "severity": 1 }
]
}In many cases it is preferable to apply inserted and deleted properties to app data views on a subject-by-subject basis. This method transforms the above into:
{
"foo": {
"@delete": { "@id": "foo", "severity": 3 },
"@insert": { "@id": "foo", "severity": 5 }
},
"bar": {
"@delete": {},
"@insert": { "@id": "bar", "severity": 1 }
}
}a m-ld update notification obtained via the follow method
Create or initialise a local clone, depending on whether the given LevelDB database already exists. This function returns as soon as it is safe to begin transactions against the clone; this may be before the clone has received all updates from the domain. You can wait until the clone is up-to-date using the MeldClone.status property.
an instance of a leveldb backend
a driver for connecting to remote m-ld clones on the domain.
This can be a configured object (e.g. new MqttRemotes(config)) or just the
class (MqttRemotes).
the clone configuration
constraints in addition to those in the configuration. 🚧 Experimental: use with caution.
Includes the given value in the Subject property, respecting m-ld data semantics by expanding the property to an array, if necessary.
the subject to add the value to.
the property that relates the value to the subject.
the value to add.
Determines whether the given property object from a well-formed Subject is a
graph edge; i.e. not a @context or the Subject @id.
the Subject property in question
the object (value) of the property
Determines if the given pattern will read data from the domain.
Determines if the given pattern can probably be interpreted as a logical write of data to the domain.
This function is not exhaustive, and a pattern identified as a write can
still turn out to be illogical, for example if it contains an @insert with
embedded variables and no @where clause to bind them.
Returns true if the logical write is a trivial no-op, such as {},
{ "@insert": {} } or { "@graph": [] }.
Utility to generate a short Id according to the given spec.
If a number, a random Id will be generated with the given length. If a string, a stable obfuscated Id will be generated for the string with a fast hash.
a string identifier that is safe to use as an HTML (& XML) element Id
Applies a subject update to the given subject, expressed as a Resource. This method will correctly apply the deleted and inserted properties from the update, accounting for m-ld data semantics.
the app-specific subject type of interest
the resource to apply the update to
the update, obtained from an asSubjectUpdates transformation
Utility to generate a unique UUID for use in a MeldConfig
Generated using TypeDoc. Delivered by Vercel. @m-ld/m-ld - v0.5.0 Source code licensed MIT. Privacy policy
Configuration of the clone data constraint. The supported constraints are:
single-valued: the given property should have only one value. The property can be given in unexpanded form, as it appears in JSON subjects when using the API, or as its full IRI reference.