Overview

Welcome to lesson 17 of 20 in the Trainer’s Course. In this lesson you will review the ways you can examine images using AI technology within SuperNova and Dolphin ScreenReader.

Lesson objectives

After studying this lesson, you will be able to:

• Accurately state the licence requirement to use the AI image description feature and its fair use policy.
• Correctly name the six sources from which you can retrieve and query images and the steps you follow to begin the process.
• Correctly describe how questions may be phrased for best and most efficient responses.

This lesson should take no more than fifteen minutes to complete.

This lesson applies to:

• SuperNova Magnifier.
• SuperNova Magnifier & Speech.
• SuperNova Magnifier & Screen Reader.
• Dolphin ScreenReader.

Analysing images using AI technology

AI is transforming accessibility by providing accurate and detailed image descriptions. It is giving people with a sight impairment the opportunity to independently explore visual content without the need to seek clarification from others.

The SuperNova and Dolphin ScreenReader "Ask AI" feature enhances this experience by providing a simple, quick, and accessible way to ask questions about images you encounter whilst using a computer.

Fair-use policy

You can query images you encounter on your computer using the SuperNova and Dolphin ScreenReader Ask AI feature. This feature is available provided you do not exceed your limit of one hundred questions per month. This fair use limit is reset at the start of each month.

Dolphin is constantly monitoring AI benchmark results along with user feedback to ensure users get the best experience using AI. At time of writing, Ask AI is using OpenAI for image analysis. This may change in the future.

Tip: Organisations restricting the use of AI technology can disable its use in SuperNova through the Enterprise Configuration Utility. See How to disable AI features in SuperNova for instructions.

How to ask questions about images

Ask about an image file using the SuperNova and Dolphin ScreenReader Control Panel

1. Open the "Media" menu (ALT + M).
2. Open the "Ask AI" submenu.
3. Select "Image file". The "Open" dialog box appears enabling you to browse and select your image file. Once selected, the "Ask AI" window opens showing the image.
4. Type your question into the edit box and press ENTER. Within a few seconds an answer will be provided.
5. Continue to ask questions as required.
6. When done, select the "Close" button or press ALT + F4 to close the window.

Ask about an image from a scanner using the SuperNova and Dolphin ScreenReader Control Panel

1. Open the "Media" menu (ALT + M).
2. Open the "Ask AI" submenu.
3. Select "Image from scanner". The scanner will initialise and scan the current page. Once scanning is complete, the "Ask AI" window opens showing the image. 
4. Type your question into the edit box and press ENTER. Within a few seconds an answer will be provided.
5. Continue to ask questions as required.
6. When done, select the "Close" button or press ALT + F4 to close the window.

Note that this feature is not available in SuperNova Magnifier.

Ask about an image copied to the clipboard using the SuperNova and Dolphin ScreenReader Control Panel

1. Open the "Media" menu (ALT + M).
2. Open the "Ask AI" submenu.
3. Select "Image from clipboard". The "Ask AI" window opens showing the image. If this option is disabled, then there is no image currently available on the clipboard.
4. Type your question into the edit box and press ENTER. Within a few seconds an answer will be provided.
5. Continue to ask questions as required.
6. When done, select the "Close" button or press ALT + F4 to close the window.

Ask about an image using Layered hotkey mode

You can use Layered hotkey mode to ask questions about images. To do this, press CAPS LOCK + SPACEBAR, Q, and then one of the following:

• F to describe image file.
• S to describe image from scanner.
• C to describe image from clipboard.
• W to describe image of whole screen.
• A to describe image of application window.
• D to describe image at Dolphin Cursor.
• R to describe image of Doc Reader page.

In addition, you can press CAPS LOCK + SPACEBAR, Q, Q to open the SuperNova and Dolphin ScreenReader Help window. Here, you can learn about SuperNova and Dolphin ScreenReader features and settings by asking AI to provide answers to your questions.

How to ask questions about images

In the context of AI, questions are referred to as prompts. The way you phrase your prompts has a significant impact on the answer you receive and the amount of additional questions you may need to ask.

The most important thing to remember when writing a prompt is to be clear and focused. Your aim should be about guiding the AI to give you the most useful, accurate, and accessible response.

When crafting a prompt, consider:

• Context - Be Specific About the Image. Your prompt should include where the image has come from, for example, a webpage, photo, screenshot, part of a user interface, etc.
• Purpose – be clear on what you want to know. Your prompt should state what you are seeking, for example, "Describe the objects", "Identify the text", “Is there an animal?”, etc.
• Be specific – state the information you want to focus on. Your prompt should ask about elements like colours, shapes, facial expressions, layout, etc.
• Actionable phrases - use Clear, Direct terms such as "Describe…", "Identify…", "List…”, etc. and avoid vague terms like "Tell me about…”.

The answers you receive may lead to further questions. You should ensure you continue to be clear in your follow-up questions.

Here are some example prompts:

• Describe the image.
• Identify the text in the image.
• List the elements in the app window.
• State the type of chart used to represent the data.

And some example follow-up prompts:

• Describe the position of the person in the image and what they are wearing.
• State whether the background is indoors or outdoors.
• Describe the shape and colour of the third item in the toolbar and what the graphic usually represents in a user interface.
• Report the best and worst selling product during the month of May.

As your experience using prompts grows, and you increasingly become clear on the responses you require, you should continue to craft your prompts. This will reduce the number of questions you may have to ask, making you even more efficient with your fair use allowance.

For example, consider:

• Describe the number and position of the people in the image, what they are wearing, and whether the background is an indoor setting.
• Summarise the data presented in the chart. This must include the chart type, the months represented in the chart, and state the best selling product for each month.
• Describe in detail the neckless in the image. If the image includes a pendant describe the shape of the pendant and state whether the pendant has a gemstone. If a gemstone is present, include its colour.

Remember, if AI gives too much or too little detail, adjust your prompt accordingly. For example, if the response is too vague, add more context. If the response is too verbose, ask for a summary. If the response is too technical, ask for simpler language.

Exercises

Please complete Exercise 16 in the SuperNova for Trainers Workbook.

If you have not downloaded this supplementary document, please click here to download the SuperNova for Trainers Workbook. 

Summary

In this lesson you have discovered how AI technology can help you learn more about the images you encounter on your computer. With SuperNova and Dolphin ScreenReader, you can use AI technology to ask questions about photos, graphs, pictures, and user interfaces without the need to seek assistance.

You also learnt about the different ways you can access "Ask AI", Dolphin’s fair-use policy, and what to consider when crafting your questions.

Finally, it is important to remember that AI can make mistakes. AI mistakes, called hallucinations, can be caused by a number of factors including insufficient training data, incorrect assumptions made by the model, or biases in the data used to train the model.

Quiz

Below are three sample quiz questions based on this lesson. Please spend no more than five minutes considering the answers to these questions.

1. The AI image description feature is available in all editions of SuperNova plus Dolphin ScreenReader. Is this statement true?
2. You can use natural language to ask questions about an image when using AI technology. Is this statement true?
3. Asking vague and general questions about an image guarantees accurate responses from AI. Is this statement true?

End of lesson

Thank you for completing this lesson. Please select 'Next Lesson' below to continue to the next part of this course.

To revisit a previous lesson, please scroll to the top of this page and choose the lesson from the Course Outline menu.

But before you move on, here are the answers to the quiz questions:

1. True.
2. True.
3. False.