Mixed Reality Toolkit
Mixed Reality Toolkit (MRTK) is an open-source software development kit (SDK) developed by Microsoft in 2016 for the development of mixed reality (MR) and augmented reality (AR) software applications.[1][2] It consists of a collection of components and features designed to enhance the mixed reality user and developer experiences.[3] The toolkit was originally developed by Microsoft for the release of the HoloLens 1 augmented reality headset, although it supports a variety of platforms.[2]
 | |
| Other names | HoloToolkit (HTK) |
|---|---|
| Developer(s) | Microsoft |
| Initial release | 2017 |
| Stable release | v2.8.3
/ November 7, 2022 |
| Repository | github |
| Written in | C# |
| License | MIT License |
| Website | aka |
| As of | March 2023 |
History
Initial release
Known as the "HoloToolkit" (HTK) prior to its initial public release, the software development kit was created to aid in the development of the new Microsoft HoloLens augmented reality device. Mixed Reality Toolkit was released to the public as an open-source SDK in December 2017, one year after the release of Microsoft's HoloLens. The software has had multiple successful generations, including MRTK, MRTK2, and the soon-to-be-released MRTK3.
Layoffs
On January 18, 2023, Microsoft announced that it would be laying off approximately 10,000 employees.[4] Days later, these layoffs were found to include the entire team behind the development of MRTK, along with significant portions of the HoloLens hardware team.[5] Currently, the future of MRTK is uncertain, although it is expected that the extended reality (XR) community will continue to maintain the software due to it being released as open-source under the MIT License.
Supported platforms
Mixed Reality Toolkit is considered to be a platform-agnostic tool as it features a wide variety of supported platforms.[1] This high level of compatibility allows developers to quickly and easily build mixed reality applications tailored to the specific platform they are working on.[3] The following is a list of supported platforms:
| Supported Platform | Minimum Version | Devices |
|---|---|---|
| OpenXR | Unity 2020.3.8+ | HoloLens 2, Windows Mixed Reality headsets, Meta Quest, SteamVR devices |
| Windows XR Plugin | N/A | HoloLens, HoloLens 2, Windows Mixed Reality headsets |
| Oculus XR Plugin | Unity 2019.3+ | Meta Quest, Meta Quest 2 |
| ARCore XR Plugin | N/A | Android devices (via AR Foundation) |
| ARKit XR Plugin | N/A | iOS devices (via AR Foundation) |
| OpenVR | N/A | Windows Mixed Reality headsets, HTC Vive, Oculus Rift |
| Ultraleap Hand Tracking | N/A | Ultraleap Leap Motion controller |
| Unreal Engine | N/A | HoloLens2, Windows mixed Reality, SteamVR |
Components and features
Mixed Reality Toolkit is equipped with a range of components and features that allow developers to create immersive mixed reality applications. These provide developers with a broad range of tools and technologies, including low-level APIs and high-level development frameworks.[3] These components and features are designed to be used in combination with additional software libraries to create engaging and interactive mixed reality experiences.
| Name | Description |
|---|---|
| Input System | Provides developers with the ability to read and handle device-level input events from a wide range of sources |
| Hand Tracking (HoloLens 2) | Enables users to control their HoloLens 2 device with their hands |
| Eye Tracking (HoloLens 2) | Interprets user eye movements to provide accurate interactions |
| Profiles | Configurable profiles used to set up and manage the toolkit in each scene |
| Hand Tracking (Ultraleap) | Enables users to control their device by interfacing with the Leap Motion Data Provider |
| UI Controls | Pre-made UI components used to quickly create user interfaces |
| Solvers | Components to solve complex problems in mixed reality, such as object placement and 3D manipulation |
| Multi-Scene Manager | System for managing multiple scenes in mixed reality, including scene loading, unloading, and transition effects |
| Spatial Awareness | Detects physical environment elements to enhance user experience and interactions |
| Diagnostic Tool | Interface which provides critical runtime verification data to simplify system debugging and analysis |
| MRTK Standard Shader | Shader system designed to implement high quality, yet performant, visuals on mixed reality devices |
| Speech & Dictation | Utilizes speech recognition and natural language processing to interpret user keywords and commands |
| Boundary System | Defines the boundaries of a user's area for increased safety and awareness |
| In-Editor Simulation | Interface designed to test a range of MRTK interactions within the editor without creating a new build |
| Experimental Features | Features currently in development or unfinished, which may not have been thoroughly tested |
MRTK contains UX building block components, which include buttons, object manipulators, system keyboards, object collections, and more.[3] These tools are provided to developers with the intention of simplifying the development process and encouraging innovation in mixed reality.
References
- What is the Mixed Reality Toolkit, Microsoft, 2023-03-12, retrieved 2023-03-12
- Team, XR Today (2022-06-30). "What is MRTK-Unity? - Mixed Reality ToolKit". XR Today. Retrieved 2023-03-12.
- polar-kev. "MRTK2-Unity Developer Documentation - MRTK 2". learn.microsoft.com. Retrieved 2023-03-12.
- "Subject: Focusing on our short- and long-term opportunity". The Official Microsoft Blog. 2023-01-18. Retrieved 2023-03-12.
- Viswanathan, Pradeep. "Microsoft has laid off entire Mixed Reality Toolkit (MRTK) development team | BigTechWire". Retrieved 2023-03-12.
