AI prompts and models
Start with Try AI-assisted annotation for a worked example. This reference covers the other prompt choices in Labelme 7.6.
Download and select a model
In a model picker, choose Download a model… or Manage models… to open the Models dialog. Download the model, wait for Downloaded, then select it in the picker.
Models run locally. After downloading a model, you can run it without an internet connection. Choose the output shape in AI-Assisted Annotation: Polygon, Mask, Rectangle, Oriented Rectangle, or Circle.
Adjust polygon detail
With Polygon output selected, use the sliders button beside AI-Assisted Annotation, or Settings > AI assist > Polygon detail. The range is 0 to 100. Lower values produce smoother outlines with fewer vertices and may remove small regions. Higher values retain more boundary detail.
Set the value before creating a new AI polygon. It does not simplify shapes you have already accepted. The AI lesson uses 20 so the points are easier to edit; check the outline against your image before keeping it.
Points and boxes
- AI-Points: click inside the object to include it; Shift+click to exclude an unwanted region. Press Enter/Space or Ctrl+click to accept the result.
- AI-Box: click one corner and then the opposite corner of a box around the object.
EfficientSam, Sam, and Sam2 support point prompts and box prompts. Their size variants trade model size and speed for accuracy; the choice depends on your images and machine. Sam3 supports box prompts, not point prompts, so choose AI-Box for Sam3.
Inspect the proposed boundary before accepting it, then edit any errors. Results can include more than one shape.
Text prompts
The AI Text-to-Annotation field is enabled in Polygon, Rectangle, AI-Points, and AI-Box modes. Choose a compatible model and output before running the prompt.
- YOLO-World (fast) detects boxes. Choose Rectangle, Circle, or Oriented Rectangle output. Polygon and Mask output need segmentation and will report "Mask Output Unavailable".
- SAM3 (smart) also produces masks, so it can create Polygon and Mask output.
For YOLO-World, separate labels with commas, such as dog,car. Spaces are retained in the labels, so do not put a space after the comma unless you want one in the label. Use one description at a time with SAM3.
In Polygon or Rectangle mode, that drawing mode determines the text output shape. In AI-Points or AI-Box mode, the output-shape picker determines it. Review, edit, or undo the results before using them as training data.