Data Annotation for Machine Learning Tutorial: Definition, Tools, Datasets

Data Annotation for Machine Learning Tutorial: Definition, Tools, Datasets
Data annotation

Data annotation for machine learning often requires collaboration between humans and computers. Specifically, humans annotate the data by adding metadata about what each item represents or how to use it.

Computers learn from these human-created labels to recognize similar patterns in new datasets. In this tutorial, you'll learn what image and video annotation is, how it works, and how you can use annotation tools to label your data.

What is data annotation?

The data annotation process is an essential part of machine learning. It's how you transform raw data into meaningful information. The process involves labeling or tagging datasets with descriptors. Then, machines can categorize them for automatic speech recognition or semantic search.

Image annotation is helpful in many industries, like healthcare and research. For example, healthcare can annotate medical images to identify organs in chest X-rays or CT scans to indicate abnormal ones.

Video annotation projects are similar but slightly more complicated. You must identify and mark objects within each video frame. In research, it can label parts of images to train machine learning algorithms.

How to label data for machine learning: step-by-step

First, you need to label data using annotation tools. These are tools that allow users to tag images manually. They often include tools for marking up maps, photos, and videos. For example, you might use an annotation tool to mark a map with labels for streets, buildings, and other features.

Data annotation can be done manually or with the help of an AI data annotation platform. It will help improve accuracy and efficiency. Sometimes, you might use manual and automatic methods to create dataset labels.

Step 1: gather your data

To get started with annotation tool, you will need the following:

  • A dataset containing images or videos that you want to annotate
  • A dataset containing annotation data (e.g., labels like "street," "car," etc.)
  • An account on Keylabs (for example) where you can upload your data

Then, you can use the annotation datasets (data annotation for machine learning algorithms) to train models for image recognition or object detection.

Data annotation tool step-by-step guide

Step 2: segment objects

Segmentation is a way to separate different elements in an image into multiple segments (or image objects). These segments have unique attributes. You can use segmentation for many other purposes, like separating the background from the foreground.

Select the annotation tool that best suits your project:

  • Bounding box and rotated bounding box – for object tracking and counting
  • Cuboid image annotation– specify the position and size of the cuboids in your vision and resize them manually
  • Polygon – Draw polygons around objects or parts of an object. Label things like buildings or other objects with sides
  • Semantic segmentation – This tool allows you to apply multiple labels using semantic descriptions like "person" or "building" instead of labeling each pixel with a bounding box or polygon. You can use them with other types of annotations
Semantic segmentation
Semantic segmentation
  • Skeletal – Use skeletal annotation in medical imaging or to point out specific human body areas
  • Key points – Mark critical points on an image or video, such as certain parts that need attention
  • Lane – Mark rows and columns with the lane tool. For example, you can use this tool to label rows of plants in a field
  • Instance segmentation – Use color or texture information to isolate objects so you can edit them without affecting other parts of the image
  • Bitmask – This tool allows you to create a bitmap mask over your image or video by choosing which pixels to show
  • Custom – If none of the tools above are ideal, you can create custom boundaries with this tool

These tools allow you to create labels on an image or video. You can use data annotation for machine learning.

Step 3: create classification labels

In this step, you define the categories for your dataset and then use them for image labeling within your dataset. For example, suppose you're creating environmental monitoring project. Categories might be "seasonal," "urban," "forest," etc. You could also label files "pond" or "bridge."

If you want to tag similar images together as one big batch manually, then select "Identify Similar Images" from the dropdown list of available labels (e.g., "Product"). For example, you identify similar product photos as one big batch instead of doing it individually by clicking on each photo. Then click "Apply."


Step 4: leverage shape interpolation

Define a start and end point. Keylabs supports shape interpolation and object tracking across multiple frames. In addition, it can use object localization to create a smooth path between them. You can also use built-in object tracker to track objects across various boundaries and generate paths.


As you've seen, data annotation for machine learning is a crucial part of a project’s success. You must know how to label your data, which can be time-consuming. Professional data annotators can help you keep your machine learning project on track.