# Keyboard Manager UI

## Table of Contents:
1. [XAML Implementation](#xaml-implementation)
2. [UI Structure](#ui-structure)
3. [EditKeyboardWindow / EditShortcutsWindow](#editkeyboardwindow--editshortcutswindow)
    1. [OK and Cancel button](#ok-and-cancel-button)
    2. [Delete button](#delete-button)
    3. [Handling common modifiers in EditKeyboardWindow](#handling-common-modifiers-in-editkeyboardwindow)
4. [SingleKeyRemapControl](#singlekeyremapcontrol)
5. [ShortcutControl](#shortcutcontrol)
6. [KeyDropDownControl](#keydropdowncontrol)
    1. [Localized key names](#localized-key-names)
    2. [Single Key ComboBox Selection Handler](#single-key-combobox-selection-handler)
    3. [Shortcut ComboBox Selection Handler](#shortcut-combobox-selection-handler)

## XAML Implementation
The KBM UI was originally implemented as a XAML Island, but in order to easily support Mica, which is [still an issue](https://github.com/microsoft/microsoft-ui-xaml/issues/5319), the [`XamlBridge`](https://github.com/microsoft/PowerToys/blob/b3f27057d43445abc59aa04405f7c24bb895a61c/src/modules/keyboardmanager/KeyboardManagerEditorLibrary/XamlBridge2.cpp) was rewritten to use a [`FrameworkView`](https://github.com/microsoft/PowerToys/blob/b3f27057d43445abc59aa04405f7c24bb895a61c/src/modules/keyboardmanager/KeyboardManagerEditorLibrary/XamlBridge2.cpp#L47C11-L49) object which is how a traditional UWP app behaves:
1. A `CoreWindow` is created by [calling a function inside of `Windows.UI.dll` with an ordinal number of `1500`](https://github.com/microsoft/PowerToys/blob/b3f27057d43445abc59aa04405f7c24bb895a61c/src/modules/keyboardmanager/KeyboardManagerEditorLibrary/XamlBridge2.cpp#L35-L42).
2. A [stubbed implementation of `CoreApplicationView`](https://github.com/microsoft/PowerToys/blob/b3f27057d43445abc59aa04405f7c24bb895a61c/src/modules/keyboardmanager/KeyboardManagerEditorLibrary/XamlBridge2.cpp#L10-L18) was created.
3. Then [both objects are passed on to the `FrameworkView`](https://github.com/microsoft/PowerToys/blob/b3f27057d43445abc59aa04405f7c24bb895a61c/src/modules/keyboardmanager/KeyboardManagerEditorLibrary/XamlBridge2.cpp#L47-L49) to initialize the XAML framework.
4. Lastly, the `CoreWindow` is [attached to the editor window](https://github.com/microsoft/PowerToys/blob/b3f27057d43445abc59aa04405f7c24bb895a61c/src/modules/keyboardmanager/KeyboardManagerEditorLibrary/XamlBridge2.cpp#L54-L61) and its `HWND` is used in-place of `DesktopWindowXamlSource`'s.

Mica is then achieved by calling [`BackdropMaterial::SetApplyToRootOrPageBackground()`](https://github.com/microsoft/PowerToys/blob/b3f27057d43445abc59aa04405f7c24bb895a61c/src/modules/keyboardmanager/KeyboardManagerEditorLibrary/EditKeyboardWindow.cpp#L388-L400) in both of the editor windows, or falls back to the `ApplicationPageBackgroundThemeBrush` background if Mica isn't available.

The UI was also updated to use WinUI 2.8 to match the look and feel of the Fluent design language of Windows 11 and the rest of PowerToys. There has been talk about [migrating the implementation to XAML files instead of code-behind](https://github.com/microsoft/PowerToys/issues/2027) and [utilizing WinUI 3 going forward](https://github.com/microsoft/PowerToys/issues/15870). More about the update can be read in [[Keyboard Manager] Modernize the editor UI - PR#28473](https://github.com/microsoft/PowerToys/pull/28473).

[**Link to the original documentation**](https://github.com/microsoft/PowerToys/blob/b3f27057d43445abc59aa04405f7c24bb895a61c/doc/devdocs/modules/keyboardmanager/keyboardmanagerui.md#c-xaml-islands)

## UI Structure
The KBM UI consists of a [`Grid` with several columns](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L200-L218). Rows are added dynamically when [the add button is pressed](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L305-L309). [A vector of vector of unique pointers to `SingleKeyRemapControl`/`ShortcutControl`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L248-L249) is created so that references to the UI components and their data are not lost until the window is closed. [`SingleKeyRemapControl`](https://github.com/microsoft/PowerToys/blob/main/src/modules/keyboardmanager/ui/SingleKeyRemapControl.cpp) is the UI class for each row of the Remap keys table, and [`ShortcutControl`](https://github.com/microsoft/PowerToys/blob/main/src/modules/keyboardmanager/ui/ShortcutControl.cpp) is the UI class for each row of the Remap shortcuts table. [`KeyDropDownControl`](https://github.com/microsoft/PowerToys/blob/main/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp) is used for handling the ComboBox operations. Each of these two classes [have vectors of unique pointers to the `KeyDropDownControl` objects](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/ShortcutControl.h#L44-L45) so that references to the objects are active until the control is deleted.

When the UI windows are activated the `KeyboardManagerState` object [sets the `UIState` variable](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L251-L252) which is used for distinguishing if the UI is up from the keyboard hook thread. The [states are also updated](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/SingleKeyRemapControl.cpp#L53) on opening and closing the Type window.

Clicking the Type Button [opens a content dialog](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/SingleKeyRemapControl.cpp#L206-L380) which registers key delays using the [`KeyDelay` class](keyboardmanagercommon.md#KeyDelay) for Enter and Esc keys and sets the UI states such that when a key event occurs the TextBlocks on the ContentDialog are updated accordingly. On accepting the dialog, the selected keys are copied into the ComboBoxes from the TextBlocks, and on closing the window, the key delays are unregistered and UI states are reset.

Since ComboBoxes are added dynamically, handlers have been added which [update the accessible names for these controls](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L69-L74), which get executed whenever a drop down is added or removed.

When the `EditKeyboardWindow`/`EditShortcutsWindow` is created, [we iterate through the remappings](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L254-L262) stored in `KeyboardManagerState` and add rows to the UI Grid. For both the windows we have `static` buffers [`singleKeyRemapBuffer`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/SingleKeyRemapControl.h#L39-L40) and [`shortcutRemapBuffer`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/ShortcutControl.h#L42-L43) which store the corresponding key/shortcuts as per the selections in the UI if they are valid with no warnings.

## EditKeyboardWindow / EditShortcutsWindow

### OK and Cancel button
[On pressing the OK button](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditKeyboardWindow.cpp#L66-L89) in `EditKeyboardWindow`, first the [`CheckIfRemappingsAreValid` method](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L10-L44) is executed which performs basic validity checks on the current remappings in the remap buffer (`static SingleKeyRemapControl::singleKeyRemapBuffer`), such as if there are no NULL columns and none of the source keys are repeated. All other validity checks are assumed to happen while the user adds the remapping. If this is found to be invalid a ContentDialog is displayed which shows that some remappings are invalid and if the user proceeds only the valid ones will be applied. If it is valid [`GetOrphanedKeys`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L46-L75) is executed which checks if any keys are orphaned (i.e. the key has been remapped and no other key has been remapped to it, so there is no way to send that key code), and a dialog is shown for notifying the user with a list of orphaned keys. After this the settings are [applied by adding it to the `KeyboardManagerState.singleKeyReMap` member](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L102-L164) and they are saved to the JSON file. `EditShortcutsWindow` differs slightly from this, as there is no orphaned keys check, and [on pressing OK](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/EditShortcutsWindow.cpp#L32-L47) both the global and app-specific shortcuts are validated and [updated](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L166-L223).

The code used for updating the remapping tables in `KeyboardManagerState` can be found in [KeyboardManagerState.cpp lines 104-183](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.cpp#L104-L183). For shortcut remaps, the `sortedKeys` vectors are updated and re-sorted whenever an element is added to them (like [this code in `KeyboardManagerState::AddOSLevelShortcut`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/common/KeyboardManagerState.cpp#L135-L136)).

On pressing OK (after confirmation dialogs) or Cancel, the window is closed and UI states are reset.

### Delete button
Since there is no single method to delete the elements in a row for a Grid, [the logic](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/SingleKeyRemapControl.cpp#L143-L188) we use involves decrementing the rowIndex for all the UI controls that appearing after the row to be deleted, and removing each of the items of the row from the Grid, followed by deleting that row definition. We also update the accessible names for all the rows since the indexing has changed. After this the corresponding row in the remap buffer is also deleted, and `SingleKeyRemapControl`/`ShortcutControl` objects are deleted from the vector.

### Handling common modifiers in EditKeyboardWindow
In the SingleKeyRemap table for a remapping of the form Ctrl->X, where Ctrl is the common version and not L/R, we can't store it directly as Ctrl->X because when the hook receives the key event it only gets LCtrl or RCtrl specifically and not `VK_CONTROL`. To simplify the backend code, when [single key remappings are applied](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L116-L143), any remapping of the form Ctrl->X is split into Ctrl(L)->X and Ctrl(R)->X (i.e. both L and R versions are remapped to the same target), and when remappings are loaded in EditKeyboardWindow, we [pre-process the remap table](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/LoadingAndSavingRemappingHelper.cpp#L77-L100) such that if the L and R versions of a modifier are remapped to the same key/shortcut we combine them. This also results in the behavior where a user adds LCtrl->X and RCtrl->X and after closing and re-opening KBM UI it appears combined as Ctrl->X.

## SingleKeyRemapControl
The left drop down column uses a single ComboBox and the Type button is linked to `createDetectKeyWindow`, whereas the right column is linked to `createDetectShortcutWindow` as the column can accept a key or a shortcut (required to support key to key and key to shortcut). The `KeyDropDownControl` for the left column in the window uses a smaller key list, without `None`.

## ShortcutControl
Both the columns in are linked to `createDetectShortcutWindow`, however the drop down selection handlers differ in their logic as the left column only allows shortcuts, and the drop downs do not contain the `Disable` key, whereas the right column allows you to select both shortcuts and keys (to support shortcut to shortcut and shortcut to key), and it allows selection of `Disable`.

For the app-specific shortcut target app text-box, we had to validate that the shortcut row is still valid when the target app is changed (for example, <kbd>Ctrl+A</kbd> is remapped for Chrome, and another remapping for <kbd>Ctrl+A</kbd> was remapped to Edge, but the target was changed to Chrome.). For this we didn't use the TextChanged handler as every time a letter is typed it would get executed. Instead we used the [`LostFocus` handler](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/ShortcutControl.cpp#L108-L167) which gets executed whenever you focus into the box (by clicking or tabbing) and then tabbing or clicking out. In the method we perform the same shortcut buffer validation used for selections in the drop down menus and update the buffers accordingly.

## KeyDropDownControl
Each ComboBox has a linked flyout, which is used to show warnings to the users whenever a user selects an invalid key from the drop down. When the warning is displayed the ComboBox is also reset to -1, i.e. no selection.

For selection handlers on the ComboBoxes we couldn't use just the SelectionChanged handler directly as it gets executed even on searching for elements in the drop down. Instead we used DropDownClosed (when a user opens the drop down and searches and selects something) and SelectionChanged when the drop down is not open (for setting selections programmatically or selection made by searching with tab focus on the drop down without opening it). This was required because if we execute the selection handlers while users are searching, it could cause false positive flyout warnings if the search causes an invalid value to be selected, and flyouts cause the drop down to close leading to bad UI experience.

### Localized key names
For getting localized key names and symbols for each virtual key code, whenever the key lists are accessed, i.e. [whenever the drop down is opened](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L56-L60) or when `GetKeyName` is called in the Type window, the current `KeyboardLayout` is retrieved to ensure that the displayed key names are always updated. Since the `WM_INPUTLANGCHANGED` event was having some issues with XAML islands we weren't able to use this to update the keyboard layout. In addition to this we do not refresh the UI, so the key lists get updated only on opening/interacting with them.

### Single Key ComboBox Selection Handler
On making a selection in the drop down, [the selection handler](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L91-L130) validates the input with the buffer from the other column and other rows. Error messages are shown using flyouts if the selection is not considered valid and the drop down and buffer for that entry are reset to empty selection. The errors that can occur on the single key ComboBox are:
- Remap to same key (A->A)
- Same key previously remapped (A->B and A->C)
- Conflicting modifier previously remapped (Ctrl->A and Ctrl(left)->B, since Ctrl also includes Ctrl(left))
If the selection is found to be valid, the `singleKeyRemapBuffer` is updated accordingly.
For handling `Shortcut` and key in the remap buffer for the right column, we use `std::variant`, which allows us to store either of the two types and check which one of them is present in the buffer by using the `index` method.
[`ValidateAndUpdateKeyBufferElement`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/BufferValidationHelpers.cpp#L8-L66) does not reference any UI components and instead takes all the relevant data as arguments. This method [has tests](https://github.com/microsoft/PowerToys/blob/main/src/modules/keyboardmanager/test/BufferValidationTests.cpp) which covers all the cases that could arise from making selections on the UI.

### Shortcut ComboBox Selection Handler
On making a selection in the drop down, [the selection handler](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L215-L295) validates the input with the buffer from the other column and other rows. Error messages are shown using flyouts if the selection is not considered valid and the drop down and buffer for that entry are reset to empty selection.
This differs from the Single Key ComboBox handler in the sense that after validating the current selection we may perform [a set of actions](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L160-L204) such as:
- Adding a drop down (if a modifier is selected)
- Removing a drop down (if None is selected)
- Clearing terminal empty drop downs (if an action key is selected in a non-last drop down and the remaining ones are empty)

After performing the corresponding action, if any, we check if the drop down resulted in an error, in which case we do [a second level of validation](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L224-L229) on all the drop downs in that list of drop downs. This is done because there can be cases where an error in one drop down results us setting it to empty, and the remaining selection is also invalid. For example, you have Ctrl+A -> Ctrl+Shift+A, and you change Shift to Ctrl. This would show a warning for having two Ctrls in the shortcut being invalid, after which setting that to empty would result in Ctrl+A->Ctrl+Empty+A, which is a case of remapping a shortcut to itself.

Once this second level of validation is done, we proceed with [updating the buffer](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L231-L251). Depending on the number of drop downs with valid values, this could be either a key or a shortcut (for the right columns). We also [set the buffer value for the target app](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L252-L267) while doing this.

Unlike the Single Key handler, there is a different set of errors that can occur here which are related to making a selection that is considered as a valid shortcut. The `isHybridControl` argument is used to distinguish between the differing behaviors for the two types of columns (shortcut only or shortcut/key column). The errors that can occur for this handler are:
- Shortcut must start with modifier (selecting A on the first drop for the left column is invalid)
- Shortcut can't have a repeated modifier (Ctrl+Ctrl(left)+A is not a shortcut)
- Shortcut can only have up to 2 modifiers (Ctrl+Shift+Alt is not supported as we have enforced a 3 key constraint (**not a backend limitation, there is [an issue](https://github.com/microsoft/PowerToys/issues/3936) requesting to remove this**))
- Shortcut must contain an action key (Ctrl+A and change A to None, only for left column)
- Shortcut must have at least two keys (Ctrl+A and change Ctrl to None, only for left column)
- Disable can't be a modifier or action key (Ctrl+Disable is invalid)
- Shortcut can't have more than one action key (Ctrl+Shift+A, change Shift to B)
- Remap to same shortcut(Ctrl+A->Ctrl+A)
- Same shortcut previously remapped for same target app (Ctrl+A->B and Ctrl+A->C)
- Conflicting shortcut previously remapped for same target app (Ctrl+A->B and Ctrl(left)+A->C, since Ctrl also includes Ctrl(left))
- Illegal shortcut remaps like Win+L or Ctrl+Alt+Del (since these cannot be remapped using LL hooks)

[`ValidateShortcutBufferElement`](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/BufferValidationHelpers.cpp#L68-L304) does not reference any UI components and instead takes all the relevant data as arguments. This method [has tests](https://github.com/microsoft/PowerToys/blob/main/src/modules/keyboardmanager/test/BufferValidationTests.cpp) which covers all the cases that could arise from making selections on the UI.

**Note:** After updating the buffer we have [code to handle a special case](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L269-L279), which was required to prevent scenarios where a drop down can get deleted but the corresponding `KeyDropDownControl` object isn't deleted. The code checks if the drop down is still linked to the parent and accordingly deletes the `KeyDropDownControl` object from the vector.

**IgnoreKeyToShortcutWarning special case:**
[An additional](https://github.com/microsoft/PowerToys/blob/b80578b1b9a4b24c9945bddac33c771204280107/src/modules/keyboardmanager/ui/KeyDropDownControl.cpp#L177-L181) check was added to ignore the Map to Same key error when an existing remapping is loaded.
This was because a remapping like Ctrl->Ctrl+A has an intermediate step of Ctrl->Ctrl, which could lead to an error of invalid input, even though Ctrl+A is valid.
The only way to actually add this is from the Type button or by adding them in a different order (like typing Shift+A and then changing Shift to Ctrl).
Since the intermediate check could fail, this was causing the app to crash since the Xaml Island wouldn't be completely loaded at that point and the Flyout can't be displayed.
[Issue #6695](https://github.com/microsoft/PowerToys/issues/6695) is the linked issue which describes the repro scenario.
