Annotation Queues help you organise and streamline the human evaluation process. You can easily set-up queues, add data for review and provide an accessible and efficient ui for domain experts to score, tag and add to datasets crucial insights.
You can access your queues on the Annotation Queues
page, and create one by clicking the +
button in the top right corner of the table.
Annotation Queues Page
This will open a modal form, that you can fill with a name to identify the annotation queue and a description to convey the queue meaning.
Once you have created a queue, you can start adding data to it. You can add runs/generations at once from the logs page.
Add Multiple Items to the Annotation Queue
You can also add a single item to the queue from a thread/run/generation detail page.
Add a Single Item to the Annotation Queue
On the Annotation Queues
page, a reviewer can open a specific queue and start reviewing added items.
Annotation Queue Item Page
A reviewer can:
Finally, once the review is done, they can mark the item as Reviewed.
Admins can monitor the progress of the queues on the Annotation Queues
page by clicking on the Admin View
button.
Annotation Queue Items List Page
In this view, the Admin can switch between the Pending
and Reviewed
tabs to see how many items are left to review and how many have been reviewed.
Annotation Queues help you organise and streamline the human evaluation process. You can easily set-up queues, add data for review and provide an accessible and efficient ui for domain experts to score, tag and add to datasets crucial insights.
You can access your queues on the Annotation Queues
page, and create one by clicking the +
button in the top right corner of the table.
Annotation Queues Page
This will open a modal form, that you can fill with a name to identify the annotation queue and a description to convey the queue meaning.
Once you have created a queue, you can start adding data to it. You can add runs/generations at once from the logs page.
Add Multiple Items to the Annotation Queue
You can also add a single item to the queue from a thread/run/generation detail page.
Add a Single Item to the Annotation Queue
On the Annotation Queues
page, a reviewer can open a specific queue and start reviewing added items.
Annotation Queue Item Page
A reviewer can:
Finally, once the review is done, they can mark the item as Reviewed.
Admins can monitor the progress of the queues on the Annotation Queues
page by clicking on the Admin View
button.
Annotation Queue Items List Page
In this view, the Admin can switch between the Pending
and Reviewed
tabs to see how many items are left to review and how many have been reviewed.