Plaid UD Guide

From the Projects screen you can sort and search the list, and create a new project with New UD project. A new project is ready for Universal Dependencies annotation right away. Click it to open its documents.

If you already have annotated data, you can import it instead of typing it in.

Open the project’s Import and export tab and drop .conllu files on it, one or many, or a ZIP of them. Each # newdoc boundary inside a file becomes its own document. The same tab exports the whole project as a ZIP of .conllu files.

The DEPS column is read into the enhanced dependency graph (see Enhanced Dependencies). An enhanced dependency whose head is an empty node is dropped along with the node. A word whose only enhanced heads were empty nodes keeps the relation its tree gives it, which for a gapped clause is orphan. A word whose DEPS is empty, in a sentence where other words have one, has no head in the enhanced graph: its tree relation is one the graph leaves out. In a sentence with no DEPS at all, the file simply says nothing about the enhanced graph and every word follows its tree.

Paste or type your text into the editor, one sentence per line. Use Save to save edits to the raw text without re-tokenizing. Each change is saved where you typed it.

When someone else saved the text after you started editing it, your save puts your changes onto their text. When you both changed the same passage, the save is refused and the editor says Changed elsewhere in the same passage. Discard changes then shows the saved text, and you make the edit again. The save is also refused when the text has passages that read the same and your change could be in either of them.

Tokens already made follow an edit of the saved text. A change inside a token, or touching it with no space between, changes that token, and it keeps its words and annotations. Typing a space inside a token splits it there: the token and its annotations stay on the part holding most of its letters, and the rest is untokenized text. Spaces a token already had do not split it. On a project whose tokens are shared with another app, the split holds for that app too. Deleting the space between two tokens keeps them two tokens. Merge them to make one. A token goes, along with its annotations, only when all its text is deleted. Text typed right before a sentence’s first character joins that sentence, unless it holds a line break, and text typed right after a sentence’s last character joins that one.

There are two ways to turn saved text into tokens, and both are runs (see Services):

Each button opens a dialog naming the method it will use and the options that method offers. A document that already has tokens cannot be tokenized again. Tokenize says to clear the tokens first, and Clear tokens beside it does that, after asking. Clearing takes the annotations on those tokens with them, and cannot be undone.

After tokenizing, the text is shown with each token as a clickable chip:

Hover over a token to open its panel, where you can:

You can also select a span of text directly to create a new token from it.

Editing a token’s words with the same number of words keeps their annotations and relations, and changes only the forms. Changing the number of words replaces them, and asks first when annotations would go. Deleting a token asks first when anything on its words would go, including the split of a multi-word token or a word’s changed form.

Each word has cells for:

Click a cell to edit it. The UPOS, XPOS and Features cells, and the dependency-relation labels in the tree, offer autocomplete from your project’s controlled vocabularies. By default those lists are suggestions and a value off the list is accepted, but a maintainer can close any one of them so only listed values are (see Suggestions or rules).

Move around the grid with Tab, Enter, Escape and the arrow keys. Arrow Up out of the top row jumps into the dependency tree. The full list is in Keyboard Shortcuts.

Below the grid, each sentence’s dependency tree is drawn as arcs between words.

Every sentence has an enhanced dependency graph beside its tree. The enhanced graph starts out equal to the tree, and you record only where it differs. A project that never does so exports a DEPS column that restates the tree.

A change to the tree carries over to the enhanced graph by itself, because only the differences are stored. Splitting a sentence removes the enhanced relations that would cross the new boundary, as it does the tree’s.

On export, DEPS lists every relation of the enhanced graph for each word.

A project made before enhanced dependencies existed gains them the first time one of its maintainers opens a document or runs an import. Until then its editor has no enhanced gestures, and an import into it drops DEPS with a notice. Empty nodes are not supported, so an elided predicate cannot be added, and gapping stays as the tree’s orphan relation.

A treebank records more than annotations. CoNLL-U keeps those notes as comment lines, and Plaid UD edits them in two places.

About a document, on its Details tab: where the text came from, its genre, its licence, whatever the project decides to keep.

About a sentence, under the sentence in the Annotate view, behind Edit metadata in the row of actions below the grid. Every sentence carries sent_id, CoNLL-U’s own identifier for it, whether the project declares anything else or not. A free translation is conventionally text_en, naming the language it is in.

Which fields a project offers is a maintainer’s choice, under Settings → UD settings. A field name cannot contain a dot, and a sentence field cannot be called sent_id or text, which the exporter writes itself.

Fields save as you leave them, and clearing one removes the note rather than storing a blank. A value already stored under a name the project no longer offers is still shown, marked, so it can be read and cleared instead of quietly exporting forever.

Everything here goes out with the document: # source = …​ above a document, # sent_id and # text_en above a sentence.

A document’s Details tab also holds Text direction, which says which way its tokens are laid out.

It is set to Automatic, which reads the document’s own text. A treebank in Arabic, Hebrew, Thaana, N’Ko or any other right-to-left script lays its tokens out right to left in the annotation grid, with the row labels on the right and the dependency arcs over the words where they now are. Set it by hand for a document the text cannot speak for, such as a right-to-left language written here in a Latin transliteration.

Direction affects the way tokens are laid out, not the values in them. Every field decides for itself, so a UPOS tag or an English lemma reads left to right in a column standing under an Arabic word. Arrow keys follow the words: in a right-to-left sentence the next token is the one to the left, so it is the left arrow that moves forward through the sentence.

In a Lemma, XPOS or Features cell, Alt+Down lists the values this project has already used for a word like this one, with how many times each.

The lemma cell asks about the form, meaning what the project has called this string before. The other two ask about the lemma, because once that is settled the tag and the features usually follow, and the same string can be two lemmas.

Pick one with the arrows and Enter, or click it. Escape goes back to the cell. A word with nothing to compare against simply shows nothing.

This is consistency help for hand annotation. A connected parser writes a first draft of most of this already.

Tokenizing and annotating are one job done in two places, so each hands over to the other at the sentence you are looking at.

From Annotate: Edit text under a sentence, or Alt+click any word in it, opens the Text Editor scrolled to that sentence.

From the Text Editor: Alt+click any token opens Annotate at that token’s sentence. A plain click still toggles the sentence boundary.

Either way the sentence you arrived at is briefly outlined, so you can see where you landed.

Annotations nobody has checked yet are marked, so you can see at a glance what is left to review. There are two marks, and they say where the annotation came from:

Anything a trusted member has entered or confirmed is shown plainly, in its own colour. Hovering a cell or a relation names its state and its producer. Any edit you make to a marked cell or relation confirms it and clears the mark. If your own work is reviewed, the mark stays on what you enter until a maintainer or another trusted member confirms it, and accepting a prediction records it as your contribution rather than confirming it.

Typing a value again is an edit, even where the value does not change. In a Features cell that means writing the same Key=Value pair, which confirms the feature rather than adding a second one. Passing through a cell with Tab or an arrow key, and leaving one with Escape, confirm nothing.

If a prediction is already correct, you don’t have to retype it:

Gestures and marks, above the first sentence, opens a legend of these marks and every gesture in the editor.

A cell is saved when you leave it, and a relation when you draw or change it. There is no save button. A save that gets no answer, because the connection is down or the server does not reply, is sent again until the server answers, and the edits made after it wait their turn. Meanwhile the page reads Offline, retrying, or Can’t reach the server, retrying while the browser is online, and closing the tab asks first. A save that reached the server before its answer was lost is not saved twice.

Two people can have one document open, and neither page shows the other’s edits as they are made. An edit is refused when someone else changed the same kind of annotation in the document since the page loaded (a lemma, a head, the words), even on another word. A change to another kind of annotation, on other words, does not refuse it.

When someone else changed the same cell first, the cell shows their value, with yours under it: Yours: X · Enter to keep yours. A message names the change, such as b changed this to NOUN. Enter in the cell saves yours, Escape or typing another value drops it, and leaving the cell saves nothing. The same happens when the word was split or joined by someone else before your value was saved. When the word is gone by the time your value is refused, a message says Not saved: and the value. A value refused because of a change elsewhere in the document stays in its cell, and leaving the cell again sends it. The message reads Changed elsewhere. Your value is in its cell, not saved. Any other edit that is refused says Changed elsewhere. Now showing the latest version. Redo your edit., and the document shows the other person’s change.

A repair is made when a document that needs one is opened. When someone saves while it is being made, it starts again by itself. If it fails again, or the server does not answer, a message asks you to reload the page. A document with no sentences to show until then says The document could not be repaired. Reload the page to try again.

When a reply rests on a particular sentence, that sentence is shown, and it can be drawn three ways. Tree gives the dependency arcs over the words, with the relation on each arc and an arrowhead on the word it points at, as the UD documentation draws them. Grid gives only the columns the point is about. Table gives every column the sentence fills.

The assistant picks whichever fits what it is saying, and you can switch. A sentence nobody has parsed is not offered a tree. The heading of each example opens that sentence in the annotation editor.

The Assistant button in the header opens the same assistant in a panel beside whatever you are looking at. It is there wherever there is a project for it to be about, and on the general screens, where it asks which project to use. The screens for making a project and for importing one have neither, so it is not offered there. A handle on the right edge of the window, halfway down, opens it too: it widens under the pointer to show the assistant’s mark. The panel stays where it is as you move around, so a question can be asked without leaving what you are reading and an answer can be read while you get on with something else. The page narrows to make room for it. The panel can be dragged wider or narrower and hidden again, and its width and whether it is open are remembered. It starts closed until you first open it. A narrow window has no room for it, and neither the button, the handle nor the panel is offered there.

One conversation runs per project. The panel picks up the last one you had in the project you are in and keeps it as you move from document to document. Each question records where it was asked from, so a question about "this sentence" still means the right document when the conversation is read again later.

The panel’s history lists the conversations you have had in the project, with All projects to widen the list past it. Opening one from there keeps you on the screen you are on. ⤢, beside it, moves the open conversation to the Assistant tab and closes the panel.

A question asked on a document is about that document: name no document and that is the one it reads, though it can still look at the rest of the project when the question calls for it, to compare or to count. Ask, beside Edit text under a sentence, puts that sentence into the question, and you can take it out again before sending.

Typing @ in the message box names something without leaving the message: the sentences of the document you have open, and the documents of the project. Sentences are matched on what they say, so @ followed by a word finds the sentence that contains it. Arrow keys move through the list, Enter takes the one that is highlighted, and Escape closes the list and leaves what you typed. An example that names a sentence of the open document scrolls the annotation editor to it rather than opening a new tab.

On a screen with no project of its own, the panel keeps the conversation it has. Before you have opened a project at all it asks which one to use, because the assistant reads one project at a time.

A plan approved in the panel re-reads the document, so the annotation shows the result without losing your place.

The assistant is a service that your server’s operator runs and picks the model for (plaid-ud-agent, installed from a checkout of the source tree). When none is connected there is nothing to open, so the tab, the button, the handle and Ask are not shown. Conversations already saved are unaffected and open from their own links.

Each project has a Search screen that finds structures across all its documents using Grew-style patterns. For example, this finds verbs with a nominal subject:

Type a pattern and run it with the Search button or Ctrl/Cmd+Enter. Results are grouped by document. Click a match to jump straight to that sentence in the Annotate view. A built-in syntax reference with more examples is available on the search screen.

Every change is recorded. History at the end of a document’s tab row, on any tab, opens its history, where you can select an earlier point to view the document exactly as it was then. Each entry is one action as you performed it (for example "Update upos", "Create relation", or a parser run), even when it was made up of several underlying changes. A parser run is listed under the service’s name and names the person who ran it: Stanza UD parse (en), requested by Ana. Selecting an entry shows the document as it was right after that action. Entries made of several changes have an arrow you can expand to see, and select, each individual change. Historical views are read-only, on every tab that can show them: Annotate, Export and Details. The Text Editor and Comments are not shown at a past state. Return to current goes back to the live document.

An entry that is one of these kinds is marked with it:

The Activity tab marks its entries the same way. An assistant plan that stopped partway is listed as Assistant, partly applied: followed by the changes it wrote.

Projects are shared by adding members with a role:

Add members from the project’s Access settings by searching for a name or email address. When you only have Reader access, the editor shows everything but its controls are inert.

Each member also has a Review work checkbox, independent of their role. Tick it and everything that person enters is marked as needing review until someone else confirms it (see Machine vs. Human Annotations). The mark is shared with every Plaid app on the project.

A project’s maintainers can tailor the annotation scheme from the project’s Settings tab:

A document has a Comments tab: every thread on it, the document’s own first, then one per sentence.

Comments are for talking about the work, not annotating it. They are not part of the data: they never change a document, they do not appear in an export, and they are not in its history.

Where they can be attached is deliberately limited to a sentence and the document. There are no per-word threads, and a note about one word belongs in the sentence’s thread where the context is.

From the Annotate view, each sentence carries a small badge under its grid: the number of comments on it, or an invitation to write the first. It opens the thread in place, so a remark can be made without leaving the text.

A thread is collapsed to its latest comment, and opens on click. Bodies are Markdown. You may edit and delete your own, and a maintainer may delete anyone’s. Readers can read comments but not write them.

A comment outlives what it is about. If the sentence it hangs on is merged away or re-segmented, the comment stays, marked Outdated, under the caption it was posted with. Those are shown separately from the current ones.

While a thread is open, comments from other people appear as they are written.

Maintainers get a Validation tab: every value the project has stored that its own vocabularies do not list, with how many times it occurs.

It exists because an open list takes any value, and a closed one still keeps values from an import, a copy or a restore, and a parser’s predictions nobody has accepted (see Suggestions or rules). A closed feature inventory holds only where a person types. This is where you see what got in.

Each value opens to the sentences it occurs in, and each of those is a link straight to that sentence in the editor. Features open the same way, on the whole Key=Value pair.

Where the list is simply incomplete rather than the values wrong, Add N values puts them on it. That is offered, never done for you: a project may well want the list as it is and the values changed instead.

Two things it deliberately does not report: a relation subtype whose base is listed, since nsubj:pass is already legal wherever nsubj is, and a feature value under a key listed with no values, since such a key accepts any value.

A project’s maintainers get an Activity tab: who has been working on it, and on what.

It shows a tally per person over a window (7, 30 or 90 days, or all time) with the changes they made, the documents they touched, and when they were last seen, alongside the members who have not appeared in that window at all. Below it is the project’s own history, newest first, with each entry linking to the document it changed.

The server administers itself through one area, and it lives in Plaid IGT: the release bundles both apps together, so a second one here would be a second answer to the same questions. Administrators reach it from Admin in the header.

It covers accounts (creating people, deactivating them, issuing password-reset links), invitations, every project on the server and who is on each, the services connected, and server-wide activity. A project set up for Universal Dependencies is labelled UD in its project list, and its name opens it back here.

Plaid UD can call external services for certain tasks. A service is a small program, often a Python script wrapping a model, that connects to your project when it starts and advertises which task it handles. Once connected, it appears wherever its task is offered, and a maintainer can make it the default for that task under Settings → Services. Some tasks ship with a built-in method that needs no service at all.

Both of them work the same way. A button names the run, and it opens a dialog with the Method to use, that method’s options, and the button that starts the run. A run keeps going if you close the dialog, and the button that opened it shows how long it has been going and how far it has got.

A run that writes to the document takes it read-only until it finishes, and the document says so at the top, with the clock still running: a run ends by re-reading the document, which would discard anything typed underneath it. Only one run goes at a time.

A run outlives the page that started it. Close the tab or reload it and the service carries on. Reopen the document and the run is picked up where it got to, editing still paused, and its result lands as usual. A run that has been finished and collected, or that is more than fifteen minutes past, is gone, and the document simply opens with whatever it wrote. Stop, on the banner or in the dialog, ends a run. It takes effect at the next point the work reports progress, and a run part-way through writing always finishes writing, so a stop never leaves half a document. Whatever it wrote stays.

If the page loses contact with a service, because the connection dropped or the service went quiet for a long time, it says so rather than reporting the run as failed. The service is still working. Reload the document to pick the run back up. When the server does not answer one of a run’s writes, the run says that the change may or may not have been saved.

There are two integration points, both in the Text Editor (see Entering and Tokenizing Text), and Parse is also in the Annotate toolbar:

Every annotation and relation a service produces is stamped unverified, in the same violet state as any other machine value (see Machine vs. Human Annotations), so you stay in control of what gets confirmed. Where a service draws the sentence and token boundaries, those are not marked. History names the run that drew them. A service reports what it could not do, and a run that changed nothing says so rather than reporting success.

Writing a service of your own is not hard. The Services chapter of the Plaid manual explains how.

From the project’s Access settings, and from your user profile, you can create named API tokens for scripts and services. A token carries your permissions and does not expire, and actions taken with one are attributed to it in the history.

For writing your own scripts against a project, see the Python client and JavaScript client references.