Reading time: 10 minutes.

Analysing images using AI

AI is transforming accessibility by providing accurate and detailed image descriptions. It is giving people with a sight impairment the opportunity to independently explore visual content without the need to seek clarification from others.

The SuperNova "Ask AI" feature enhances this experience by offering a simple, quick, and accessible way to ask questions about images you may encounter whilst using your computer.

AI usage policy

You can query images using Ask AI provided you do not exceed your limit of one hundred questions per month. This fair use limit is reset at the start of each month.

Tip: The number of questions remaining appears at the top of the "Ask AI" HTML window. You can navigate to this information using the Dolphin Cursor.

Dolphin is constantly monitoring AI benchmark results along with user feedback to ensure users get the best experience using AI. At time of writing, Ask AI is using OpenAI for image analysis.

Watch - Describing images with Ask AI

How to ask questions about images

Ask questions about an image file using the SuperNova Control Panel

1. Open the "Media" menu (ALT + M).
2. Open the "Ask AI" submenu.
3. Select "Image file". The "Open" dialog box appears enabling you to browse and select your image file. Once selected, the "Ask AI" window opens showing the image.
4. Type your question into the edit box and press ENTER. Within a few seconds an answer will be provided.
5. Continue to ask questions as required.
6. When done, select the "Close" button or press ALT + F4 to close the window.

Ask questions about an image copied to the clipboard using the SuperNova Control Panel

1. Open the "Media" menu (ALT + M).
2. Open the "Ask AI" submenu.
3. Select "Image from clipboard". The "Ask AI" window opens showing the image. If this option is disabled, then there is no image currently available on the clipboard.
4. Type your question into the edit box and press ENTER. Within a few seconds an answer will be provided.
5. Continue to ask questions as required.
6. When done, select the "Close" button or press ALT + F4 to close the window.

Ask questions about an image using Layered hotkey mode

You can use Layered hotkey mode to ask questions about images. To do this, press CAPS LOCK + SPACEBAR, Q, and then one of the following:

• F to describe image file.
• C to describe image from clipboard.
• W to describe image of whole screen.
• A to describe image of application window.
• R to describe image of Doc Reader page.

In addition, you can press CAPS LOCK + SPACEBAR, Q, Q to open the SuperNova Help window. Here, you can learn about SuperNova features and settings by asking AI to provide answers to questions.

Refining your questions

In the context of AI, questions are referred to as prompts. The way you phrase your prompts has a significant impact on the answer you receive and the amount of additional questions you may need to ask.

The most important thing to remember when writing a prompt is to be clear and focused. Your aim should be about guiding the AI to give you the most useful, accurate, and accessible responses.

When crafting a prompt, consider:

• Context - Be Specific About the Image. Your prompt should include where the image has come from, for example, a webpage, photo, screenshot, part of a user interface, etc.
• Purpose – be clear on what you want to know. Your prompt should state what you are seeking, for example, "Describe the objects", "Identify the text", “Is there an animal?”, etc.
• Be specific – state the information you want to focus on. Your prompt should ask about elements like colours, shapes, facial expressions, layout, etc.
• Actionable phrases - use Clear, Direct terms such as "Describe…", "Identify…", "List…”, etc. and avoid vague terms like "Tell me about…”.

The answers you receive may lead to further questions. You should ensure you continue to be clear in what you are seeking in your follow-up questions.

Here are some example prompts:

• Describe the image.
• Identify the text in the image.
• List the elements and their position in the application window.
• State the type of chart used to represent the data.

And some example follow-up prompts:

• Describe the position of the person in the image and what they are wearing.
• State whether the background is indoors or outdoors.
• Describe the shape and colour of the third item in the toolbar and what the graphic usually represents in user interfaces.
• Report the best and worst selling product during the month of May.

As your experience using prompts grows, and you increasingly become clear on the responses you require, you should continue to craft your prompts. This will reduce the number of questions you may have to ask and thus help you be even more efficient with your fair use allowance.

For example, consider:

• Describe the number and position of the people in the image, what they are wearing, and whether the background is an indoor setting.
• Summarise the data presented in the chart. This must include the chart type, the months represented in the chart, and state the best selling product for each month.
• Describe in detail the neckless in the image. If the image includes a pendant describe the shape of the pendant and state whether the pendant has a gemstone. If a gemstone is present, include its colour.

Remember, if the AI gives too much or too little detail, adjust your prompt accordingly. For example, if the response is too vague, add more context. If the response is too verbose, ask for a summary. If the response is too technical, ask for simpler language.