Plaid Manual

In Plaid a project is a collection of documents which all have the same data model. Plaid’s data model is configurable on a per-project basis, allowing you to have as many or as few annotation types as you would like. For example, perhaps in one project you might only annotate a single part-of-speech tag per word, while in another you might want a POS tag, a lemma, and a gloss for each word. These projects would have different configurations in order to appropriately accommodate the annotation needs of each.

The means by which this configuration is performed is the layer. (If you have used ELAN before, note that this is similar to ELAN’s tiers.) Loosely speaking, a layer corresponds to a single "type" of annotation: for instance, in the example above where we want to collect a POS tag, lemma, and gloss for each word, we would have three span layers associated with the project. We will discuss layers more shortly.

Plaid offers a built-in user system which allows individuals to log in with a password. By default, each project is private, and one of three permissions levels is needed in order to interact with one:

Additionally, a global admin role exists, which allows the user to see and edit all data in the Plaid instance.

Plaid’s functionality is exposed for development as a REST API. This allows you to use Plaid from any programming language with an HTTP client library.

Additionally, we provide official clients for two programming languages, Python and JavaScript. These clients provide an API for interacting with Plaid that is idiomatic for each programming language and frees you from concern about low-level details of HTTP requests. Here is an example of how to use the Python client:

The same, in JavaScript:

Plaid uses the database XTDB to store data. An unusual feature of this database is that it is immutable. This means that whenever a value inside the database changes, the old value is not lost: rather, the old state of the database is still accessible for reading, even after a new version has been created. This allows Plaid to provide time travel, allowing you to access all historical states of your documents and projects.

It is simple to make use of this in the client: simply include an ISO-8601 time as the final argument of any .get method. (In Python, you may also use the keyword argument as_of.) Consider a simple example where we are modifying the name of a document:

Additionally, an audit log provides an account of who performed every write to the database:

Plaid offers a simple system for real-time communication on a per-project basis. This is intended to support two purposes:

This functionality is exposed in two simple functions in the client. The send_message/sendMessage function allows a client to broadcast a message to all clients in the project:

Note that the second positional argument, the body, can be any JSON value.

On the other end, a client may listen like so. Note that there are two arguments for the message. event_type is "message" for data sent via send_message/sendMessage by another client, and "audit-log" for audit log notifications. Consider an example of listener setup:

After the send_message invocation we just saw, this on_event function would produce the following output:

And suppose that another client executed the following code:

The listener’s code above would print this:

Suppose we’re working on a project where all we are doing is POS-tagging. The configuration of the project’s layers (in a simplified JSON representation) would look something like this:

This layer structure prescribes the structure of individual documents. Consider a document where we have POS tagged the sentence "Fido barks":

Notice the following:

We will continue discussing this example in more detail below.

A project is the root of a layer configuration and has a name.

A project has many documents, and each has a name and a unique ID:

For each text layer, each document may have at most one text, which consists of a single string. This string holds all the text which is to be analyzed in dependent layers. A text object looks something like this:

For each token layer, each document may have many tokens, which are defined as substrings of a text:

Note the following:

Tokens are intended to serve ast he basic units for further linguistic analysis using spans and relations.

For each span layer, each document may have many spans, which are groupings of one or more tokens which have a single value:

There are no restrictions on spans, other than that they must hold at least one token, and that they all must belong to the span layer’s parent token layer.

For each relation layer, each document may have many relations, which are directed edges between two spans with a label. Both spans must belong to the relation layer’s parent span layer. For example, if we wanted to extend the example above with a syntactic dependency relation between "Fido" and "barks" expressing that "Fido" is the subject, we could have a relation like this:

The four basic layer types (text, token, span, and relation) are all project-specific and cannot be used in more than one project. The fifth layer type, the vocab layer, can be used across projects. As its name suggests, this layer is intended for recording occurrences of lexical entries.

The vocab layer itself has a name:

The vocab layer has vocab items, which represent lexical entries, each with a canonical form:

Finally, vocab links are used to indicate occurrences of lexical entries. Recall the tokens from before:

We can create links between them and the above vocab items with vocab links like so:

Notice that multiple tokens may be specified, allowing for multi-word and non-contiguous lexical items.

It is often desirable to enrich an entity with additional information—​for instance, you might want to record some information about the annotator’s confidence in whether a certain span value is correct. Additionally, you might want to do the same with a layer in order to e.g. specify what values are acceptable for spans in a given layer. To accommodate this, Plaid allows arbitrary data to be stored in the config attribute for layer types (project, text layer, token layer, span layer, relation layer, vocab layer) and in the metadata attribute for data types (document, text, token, span, relation, vocab item, vocab link).

You might use a config to store legal tags for a given layer:

As for metadata, one use could be to store information about whether a human or an AI system produced the annotation, and in the latter case, store additional information about the system’s predictions:

In the previous section, we noted the constraints which Plaid enforces on each data type. Plaid guarantees that the database will never violate these, no matter what, by ensuring that invalid entities are never created, and often by deleting structures which are indirectly rendered invalid by another change. Consider these examples:

These invariants have been incorporated into Plaid because of their broad desirability in linguistic annotation. However, some invariants will vary by annotation framework. For example, it is quite common to want a span layer’s spans to be in one-to-one correspondence with tokens in the parent token layer. This is not directly enforced by Plaid, but Plaid provides you with three mechanisms which allow you to enforce your own data integrity constraints.

Multiple users may edit the same document simultaneously, and in some cases, undesirable conflicts may occur as users fail to take into account each other’s work. Suppose, for example, that one user is editing a sentence’s lemmas, and the other is editing a sentence’s POS tags. If the lemma editor doesn’t know that a certain POS tag has changed, they might make the wrong decision about which lemma to assign. Plaid clients' optional strict mode causes edits to fail when someone other than the current user has made an edit. Consider this exact scenario in code:

When client 1 executes the second lemma span’s value, unless they happened to have loaded the document anew after client 2’s change, they will not be aware of the new POS tag for the second word.

Strict mode causes requests to fail when someone other than the user in strict mode has edited a document since strict mode began. If client 1 had initiated strict mode at the beginning, then the second request would have failed:

This failure gives client 1 the opportunity to reload the document only when it is necessary, allowing them to reconsider the current state of the document before making changes.

Sometimes a more heavyweight solution is needed. A lock gives a user exclusive permission to write to a document, preventing all other users from changing its contents. Locks have a 60 second expiration timer by default, and they may be released early or renewed by either explicit renewal or any write to the locked document. Consider:

Locks are useful for situations where a concurrent edit by another user could yield an invalid state with respect to data integrity constraints beyond what is enforced in Plaid’s core. They should be used only where necessary in order to minimize the risk of invariant violations stemming from concurrent modifications.

Finally, you may also submit multiple requests in batches. Batches are atomic meaning that we guarantee that either they will all succeed or all fail. This is a very useful guarantee whenever you have sophisticated data integrity requirements that must be orchestrated using more than one request.

Here is an example of how to submit a batch using the JavaScript client:

Or in Python:

Owing to some implementation details, no other writes may be executed while a batch is being processed, so only use them where necessary.

For all constraints which you might attempt to enforce using the three mechanisms described above, there is, of course, nothing stopping a malicious user from circumventing them and submitting changes which invalidate your formalism-specific data integrity constraints. For example, if you have a document where you want all spans to be in one-to-one correspondence with tokens, a malicious user could simply circumvent your UI code entirely and craft a malicious request to e.g. create multiple spans associated with a single token. Since there is no server-side validation for this constraint, the server will happily execute it and advance into a database state that does not violate any core data integrity constraints but does violate your formalism-specific constraints.

This is a fundamental limitation of Plaid which was deliberately adopted in order to support frontend-only development. You therefore must only grant write privileges only to users who you trust not to circumvent client-side validation guardrails. Fortunately, we think this is not an onerous imposition in most real-world circumstances.