(Current status: This work was integrated into Android Studio 1.4.)
The source editor in Android Studio 1.0 and 1.1 does not support screen readers. This is something we're trying to fix, and this document describes the technical issues involved.
The source editor is custom component (EditorComponentImpl extends JComponent), which means it doesn't have any builtin accessibility handling.
To fix this, we need to
- Make its getAccessibleContext() return an accessibility helper
- That AccessibilityContext needs to report its role as AccessibleRole.TEXT, and
- Its getAccessibleText() method should return an AccessibleText method which
- responds to all the methods for querying caret and selection positions, reading back text at given positions, and so on, and
- fires accessible events when the caret, selection or text changes
That's all pretty straightforward. But unfortunately, when I tested this on OSX, it didn't work. The screen reader reads the text at the current caret, but when you move the caret around, or enter text, nothing happens!
It turns out that while Swing is accessible, at least on Mac OSX that's not fully true: The support for text is
hardcoded to JTextComponents. And the IntelliJ source editor is
not a JTextComponent (which is how it's able to do a lot of advanced features like folding text within a single line etc). Take a look at the
source code for CAccessible.java, line 102:
|
|
|
| // currently only supports text components |
| public void addNotificationListeners(Component c) { |
| if (c instanceof JTextComponent) { |
| JTextComponent tc = (JTextComponent) c; |
| ... |
| tc.getDocument().addDocumentListener(listener); |
| tc.addCaretListener(listener); |
| } |
| ... |
| } |
In other words, the accessibility support is not responding to caret and document modification events by listening to properties being fired, as the documentation suggests it should. Instead, it only listens for these events by directly casting to a JTextComponent and directly listening on the document and carets, and doing nothing for non-JTextComponents.
I tried to fix this in two ways.
The first way is to create a "dummy" JTextComponent, which basically wraps the real source editor's mode. The methods to get the caret position and text contents delegate to calls in the IntelliJ source editor, and change events in the source editor are mapped back to Swing JTextComponent events. The second step here is to pretend to the accessibility system that this document component is the real source editor. I tried to do this by inserting the JTextComponent's accessibility handler into the accessibility hierarchy (e.g. the getAccessibleChildrenCount() and getAccessibleChild(int index) methods). However, I could not get this to work. I suspect there's some code in the accessibility framework or screen reader code which checks that the focused component (which has to be the real editor) is really the same as the accessibility component.
The second way, which does work, is to make the source editor component (EditorComponentImpl) actually extend JTextComponent. Note that I'm not talking about actually making the editor really be a JTextComponent. I mean make the class extend JTextComponent, and then lobotomize all the functionality such that we don't end up doing a lot of extra work. We do this by overriding all the applicable methods from JTextComponent such that they become no-ops. With the notable exception of course of the getDocument() and addCaretListeners() methods, where we return objects that can map through to the real source editor. The final challenge is that the JTextComponent constructor unconditionally does some operations that we can't avoid (since they're done in the constructor which we can't replace). This requires some extra care such that the work done is cheap (e.g. it tries to install a UI delegate, so make that a no op etc).
This is not a great solution for obvious reasons:
- To the outside, it now looks like EditorComponentImpl is a JTextComponent. Somebody not familiar with the reasons may try to use it in a real JTextComponent context (e.g. calling many of the lobotomized methods which may lead to problems). This is probably not a huge deal since EditorComponentImpl is a deep implementation class, not a general widget, so it's unlikely somebody outside of the core IntelliJ developer group would use it. (In the current CL I've made all the unsupported methods throw an exception to make it clear that this is not a functional JTextComponent.)
- A bigger problem is that we're now depending on implementation details of JTextComponent. What if in some upcoming JDK version they add additional semantics (e.g. new methods we need to override to ensure that they are no-ops, or worse yet, the constructor calls overridden methods and assumes that they return results that does not happen correctly in our lobotomized context.)
This is proposed in the following CL:
There are a couple of other wrinkles:
Just like CAccessible.java makes JTextComponent assumptions, so does CAccessibleText.java, so we have to make our JTextComponent implement some additional methods used there, e.g. getVisibleRect(), viewToModel(), and even Swing's Document model, getDefaultRootElement() and getElementIndex(). This is shown in the changelist.
Furthermore, also in the JDK OSX integration for accessibility, AccessibleRole.TEXT is hardcoded to be read out as a "text field", but we can create a new AccessibleRole with the exact text "textarea" which will be handled correctly.
With those changes, the source editor properly reads out source text, responds to edits and navigation around in the document.
Remaining Issues
- I've only tested this on OSX for now. Next I need to see what happens on Windows and Linux, and whether additional changes are required. I'm optimistic that things will work better on Windows (since back when Swing was adding accessibility, Windows was the main platform they targeted with the Accessibility Bridge etc), so hopefully simply implementing the AccessibleContext interfaces and firing property events will be adequate. But until we test it, I'm going to assume it's broken.
- Note (October 2015): It turns out the things are not working under Windows + NVDA, because the "textarea" role is non standard, and NVDA does not understand it. By "not working", we mean that the screen reader does not read line text when the caret moves in the editor. although the reader can "read" the text line when moving the mouse. Given than MacOS support is broken in other ways, we will revert to returning a supported role (e.g. "text").
- I've marked some TODOs in the code: I'm not sure whether we need to hold a write lock for the AccessibleText callbacks for updating the text. I'm not sure how to make the screen reader invoke those, to see whether they need to be wrapped in command actions or if these will function the same way a Swing event originating a text change does.
- The text reader interface makes a distinction between a "sentence" and a "line". For now, I've implemented line semantics for both types of request. It might be that in some languages (e.g. text files) we can do better than that; if there's an editor lexer I can use to classify characters by sentence as opposed to line, that would be good. Perhaps for code, such as in Java, we should map the sentence request to a complete statement, rather than the current line? It would be good to get feedback from blind developers to find out what they would prefer here.
- The accessibility framework supports actions. The AccessibleContext can list how many actions they have, and return each one - as well as invoke them. For a simple app, I can see how it would be useful to find out that there are 4 available actions, to listen to them and to invoke them. But for the IntelliJ source editor, with his huge number of available editing actions (hundreds?), this doesn't seem very useful. I'm not sure how we should solve this, or what users expect, but we can address this by making the editor AccessibleContext also implement AccessibleAction and then implementing its three methods to delegate to the real editor actions: getAccessibleActionCount(), getAccessibleActionDescription(index), and doAccessibleAction(int(index).
- Testing. The whole AccessibleText implementation could use some unit tests; this should be added to platform/platform-tests/testSrc/com/intellij/openapi/editor/impl/EditorImplTest.java (I'm having trouble running those tests at the moment; the test case fails to initialize early on in PluginManagerCore.initializePlugins.
