Apple Intelligence now lets blind users ‘ask' what their camera sees

Apple has unveiled a significant expansion of its accessibility tools, adding Apple Intelligence to bring advanced capabilities to VoiceOver, Magnifier, Voice Control, and Accessibility Reader.

The new feature will roll out later this year, aiming to empower users who are blind, have low vision, or live with physical disabilities.

Through the use of Apple Intelligence, the Image Explorer feature of VoiceOver provides comprehensive descriptions of pictures, bills, and other images. The user only needs to click on the Action button to ask questions about what the camera sees and get an immediate response. Similarly, the Magnifier app will be able to answer questions such as “How much is the bill?”

Voice Control will now integrate natural language, enabling users to navigate iPhone and iPad by describing onscreen elements conversationally, such as “tap the purple folder” instead of memorising exact labels.

Moreover, accessibility readers can now manage complex documents such as scientific articles with several columns and tables, providing on-demand summaries and built-in translation while preserving custom formatting.

Apple’s other innovation was the inclusion of subtitle generation technology within iPhones, iPads, Macs, Apple TVs, and Vision Pros. Subtitles can be automatically generated for videos that have no captioning by automatically transcribing the speech heard in the video.

Most notably, Apple Vision Pro will enable users to control compatible power wheelchairs using their eyes. 

Apple’s CEO Tim Cook stated, “Apple’s approach to accessibility is unlike any other. Now with Apple Intelligence, we are bringing powerful new capabilities into our accessibility features while maintaining our foundational commitment to privacy by design.”

For differently abled individuals, Apple is also introducing an adaptive Hikawa Grip and Stand for iPhone, a MagSade accessory built for people with grip, strength, or mobility challenges, which is available worldwide.