Automate
Make your phone or tablet smarter with automation
Make your phone or tablet smarter with automation
A decision block that interacts with the on-screen user interface.
If the proceed option is set to Immediately, then the action will be performed on the UI elements currently visible on screen, without pausing the fiber. If set to When UI element appeared, then the fiber will pause until the UI element has appeared on screen.
Only use this block as a last resort if there’s no other way of performing the desired automation.
There are two types of interactions:
The NO path is executed if the action is allowed but failed to be performed, and also when the UI element isn’t visible on screen when using proceed Immediately.
Use the App start or App in foreground block to ensure that the correct activity is shown on screen. A delay between sequential interactions may be needed to ensure the UI has been fully loaded and redrawn.
The XML element “tag” name is the UI element view class. Attribute availability depends on type of UI element and Android version, see table below. Use the Inspect layout button to view the XML layout of a window shown on screen.
| Attribute | Value | Description |
|---|---|---|
@android:checkable | boolean | Can display a check mark |
@android:checked | boolean | Checked state |
@android:choiceMode | text | Choice behavior for collection (Android 5+) |
@android:clickable | boolean | Reacts to clicks |
@android:columnCount | number | Column count of collection (Android 5+) |
@android:contentDescription | text | Associated content description |
@android:contextClickable | boolean | Reacts to context clicks (Android 6+) |
@android:editable | boolean | Has an input method (Android 4.3+) |
@android:enabled | boolean | Allows interaction |
@android:error | text | Error text (Android 5+) |
@android:focusable | boolean | Can take focus |
@android:focused | boolean | Currently has focus |
@android:hint | text | Hint text (Android 8+) |
@android:id | text | Identifier name (Android 4.3+) |
@android:inputType | text | Allowed type of text input, separated by '|' (Android 4.4+) |
@android:labelFor | text | Identifier name of element it’s a label for (Android 4.3+) |
@android:labeledBy | text | Identifier name of label element (Android 4.3+) |
@android:layout_column | number | Column index of collection item (Android 5+) |
@android:layout_columnSpan | number | Column span of collection item (Android 5+) |
@android:layout_height | (number) | Height |
@android:layout_row | number | Row index of collection item (Android 5+) |
@android:layout_rowSpan | number | Row span of collection item (Android 5+) |
@android:layout_width | (number) | Width |
@android:layout_x | (number) | Horizontal position on screen |
@android:layout_y | (number) | Vertical position on screen |
@android:longClickable | boolean | Reacts to long clicks |
@android:max | number | Maximum value for range (Android 5+) |
@android:maxLength | number | Maximum length of text input (Android 5+) |
@android:min | number | Minimum value for range (Android 5+) |
@android:password | boolean | Input characters displayed as password dots |
@android:progress | number | Current value of range (Android 5+) |
@android:rowCount | number | Row count of collection (Android 5+) |
@android:scrollable | boolean | Can scroll |
@android:selected | boolean | Currently selected |
If the floating record window doesn’t generate an event, e.g. when clicking a button, it’s likely due to the UI layout XML changing before its XPath expression can be created. A workaround is to instead use the search feature (magnifying glass icon) to pick the UI element shown on screen, then change the action, e.g. to Click.
On Android 11+ the record interactions and inspect layout windows are hidden on system Settings unless the Allow screen overlays on settings option is enable in system Developer settings.
Some targeted actions take additional input arguments:
1 for character (default), 2 word, 4 line, 8 paragraph or 16 page.
argY is a boolean whether to extend selection (Android 4.3+).
"INPUT".0.Note! Interacting with Automate isn’t permitted, for security reasons. The Close system dialogs action do not work on Android 12+. The Quick settings action may not work prior to Android 6. The Show input method picker action requires the “appear on top of other apps or parts of the screen” privilege on Android 9+, may not work on Android 12. The Show soft input action requires the “appear on top of other apps or parts of the screen” privilege.