Microsoft UI Automation
Microsoft UI Automation (UIA) is an application programming interface (API) that allows one to access, identify, and manipulate the user interface (UI) elements of another application. UIA is targeted at providing UI accessibility and it is a successor to Microsoft Active Accessibility. It also facilitates GUI test automation, and it is the engine upon which many test automation tools are based. RPA tools also use it to automate applications in business processes. UIA's property providers support both Win32 and .NET programs. The latest specification of UIA is found as part of the Microsoft UI Automation Community  Promise Specification. Microsoft claims that portability to platforms other than Microsoft Windows was one of its design goals. It has since been ported to Mono. History. In 2005, Microsoft released UIA as a successor to MSAA framework. Managed UI Automation API was released as a part of .NET Framework 3.0. The native UI Automation API (provider) is included as part of the Windows Vista and Windows Server 2008 SDK and is also distributed with the .NET Framework. UIA is available out of the box in Windows 7 as a part of Windows Automation API 3.0 and as a separate download  for Windows XP, Windows Vista, and Windows Server 2003 and 2008. Motivation and goals. As a successor to MSAA, UIA aims to address the following goals: Technical overview. At client side, UIA provides a .NET interface in codice_1 assembly and a COM interface implemented directly in codice_2. At server side, codice_2 is injected into all or selected processes on the current desktop to perform data retrieval on behalf of a client. The DLL can also load UIA plugins (called "providers") into its host process to extract data using different techniques. UIA has four main provider and client components, as shown  in the following table. Elements. UIA exposes every piece of the UI to client applications as an Automation Element. Elements are contained in a tree structure, with the desktop as the root element. Automation Element objects expose common properties of the UI elements they represent. One of these properties is the control type, which defines its basic appearance and functionality as a single recognizable entity (e.g., a button or check box). In addition, elements expose control patterns that provide properties specific to their control types. Control patterns also expose methods that enable clients to get further information about the element  and to provide input. Clients can filter the raw view of the tree as a control view or a content view. Applications can also create custom views. Tree. Within the UIA tree there is a root element that represents the current desktop and whose child elements represent application windows. Each of these child elements may contain elements representing pieces of UI such as menus, buttons, toolbars, and list boxes. These elements, in turn, can contain other elements, such as list items. The UIA tree is not a fixed structure and is seldom seen in its totality because it might contain  thousands of elements. Parts of the tree are built as they are needed, and the tree can undergo changes as elements are added, moved, or removed. Control types. UIA control types are well-known identifiers that can be used to indicate what kind of control a particular element represents, such as a combo box or a button. Having a well-known identifier allows assistive technology (AT) devices to more easily determine what types of controls are available in the user interface (UI) and how to interact with the controls. A human-readable representation of the UIA control type information is  available as a codice_4 property, which can be customizable by control or application developers. Control patterns. Control patterns provide a way to categorize and expose a control's functionality independent of the control type or the appearance of the control. UIA uses control patterns to represent common control behaviors. For example, the codice_5 control pattern is used for controls that can be invoked (such as buttons) and the codice_6 control pattern is used for controls that are scrollable viewports (such as list boxes, list views, or combo boxes). Because each control pattern represents a separate functionality, they can be combined  to describe the full set of functionality supported by a particular control. Properties. UIA providers expose properties on UIA elements and the control patterns. These properties enable UIA client applications to discover information about pieces of the user interface (UI), especially controls, including both static and dynamic data. Events. UIA event notification is a key feature for assistive technologies (AT) such as screen readers and screen magnifiers. These UIA clients track events that are raised by UIA providers that occur within the UIA, and use the information to notify end users. Efficiency is improved by allowing provider applications to raise  events selectively, depending on whether any clients are subscribed to those events, or not at all, if no clients are listening for any events. TextPattern. UIA exposes the textual content, including format and style attributes, of text controls in UIA-supported platforms. These controls include, but are not limited to, the Microsoft .NET Framework codice_7 and codice_8 as well as their Win32 equivalents. Exposing the textual content of a control is accomplished through the use of the codice_9 control pattern, which represents the contents of a text container as a text stream. In turn, codice_9 requires the support of the  codice_11 class to expose format and style attributes. codice_11 supports codice_9 by representing a contiguous text span in a text container with the codice_14 and codice_15 endpoints. Multiple or disjoint text spans can be represented by more than one codice_11 objects. codice_11 supports functionality such as clone, selection, comparison, retrieval and traversal. UI Automation for automated testing. UIA can also be useful as a framework for programmatic access in automated testing scenarios. In addition to providing more refined solutions for accessibility, it is also specifically designed to provide robust functionality for automated testing. Programmatic access provides the ability to imitate , through code, any interaction and experience exposed by traditional user interactions. UIA enables programmatic access through five components: Availability. UIA was initially available on Windows Vista and Windows Server 2008, and it was also made available to Windows XP and Windows Server 2003 as part of .NET Framework 3.0. It has been integrated with all subsequent Windows versions, up to and including Windows 7. Besides Windows platforms, the Olive project (which is a set of add-on libraries for the Mono core aiming for the .NET Framework support) includes a subset of WPF (codice_18 and codice_19) and UI Automation. Novell 's Mono Accessibility project is an implementation of the UIA Provider and Client specifications targeted for the Mono framework. Additionally, the project provides a bridge to the Accessibility Toolkit (ATK) for Linux assistive technologies (ATs). Novell is also working on a bridge for UIA-based ATs to interact with applications that implement ATK.