Once viewed as purely futuristic, immersive technologies like virtual reality (VR), augmented reality (AR) and mixed reality (MR) are transforming manufacturing workflows as business leaders recognize the value of visualizing complex data dimensionally and supporting more information-rich experiences.

When talking about these three technologies, it is important to understand the differences between them. Virtual reality is a fully immersive technology in which the user only sees the digital content displayed by the device. With augmented reality, the physical and virtual worlds are combined by overlaying digital information into the user’s environment, usually through a 2D display. Mixed reality blends the digital and physical worlds and allows the digital content to interact with the real-world in 3D.

All of these technology innovations provide benefits such as increased productivity and reduced risk of error; however, they have mostly been used in the manufacturing world for visualization and communication scenarios. But that is changing.

 Paradigm Shift

While the use of AR, VR and MR for communication and visualization is still important, the way manufacturers are taking advantage of these solutions has started to shift. Now more than ever, manufacturers are using AR and MR technology in production environments  and bringing it to frontline workers to be used for more sophisticated and demanding tasks, such as quality control, production control and actual manufacturing in assembly lines.

This is especially true when it comes to mixed reality, where improved hardware and software capabilities enable hands free interaction and compliance with safety requirements. The main improvements on the hardware side compared to previous-generation devices include a wider field of view, better performance and better calibrated optics for improved alignment accuracy--a critical aspect when it comes to production. In addition, a specialized, production-targeted mixed reality device can address safety requirements like hardhat integration and bone-conduction headsets that keep the user aware of the surroundings. 

Significant improvements on the software side include two-hands interaction as a natural, more efficient interface, eye tracking, and advanced features such as sequencing for improved context-based interaction.  

The market has also matured to the point where technology is being more widely adopted and is expanding to the production floor. Until now, most digital transformation has consisted of changes in the office. Engineers have embraced 3D modeling solutions and mixed reality for the design and evaluation of projects and products in the office, but workers in the manufacturing facility were still using paper drawings or interacting with models or 2D representations on a computer screen for production. This painful gap motivated companies to look for solutions to improve efficiency by extending the use of mixed reality on the production floor.

New Use CasesMixed reality can directly translate the time invested in the 3D digital content during the design stage to the production side to improve efficiency. Manufacturers can justify their investment by using the digital data—such as 3D models—not just for the design phase, but also for the manufacturing phase.

The combination of market maturity and technology innovation has enabled a new set of use cases for the manufacturing industry. including:

Employee Training: Rather than reading a set of drawings or flipping through a manual for passive learning, workers can immerse themselves in the training material while physically interacting with the assets needed for the job. This is especially useful with spatially complex products were traditional training methods take time due to the high cognitive load.   

Quality Assurance: Blending digital information into real world views enables on-site teams to validate the quality of their work, identify discrepancies in real-time, and benefit from a short communication loop with the office team. Mixed reality can help embed the quality assurance in the production process by enabling real-time visual analysis of the assembled product.Visual Sequencing: Visual sequencing provides step-by-step instructions of an assembly process delivered to the line worker during assembly. For complex assemblies, 3D views of each step with accompanying instructions help even experienced workers to avoid potential errors. As an added benefit, the wearable devices allow them to keep their hands free while working. A concrete example for such a scenario is an assembly of rebar cages in a prefab factory. The ability to filter the displayed information based on production sequence helps both novice and experienced team members avoid costly errors.

Production Control: Prefabricated construction elements are becoming more complex. Mixed reality provides manufacturers with the ability to monitor the production process in the context of the physical assembly and determine whether production is ahead, according, or behind schedule.

Look for integration of mixed reality with advanced technologies such as artificial intelligence, computer vision and machine learning to further increase its value.

Aviad Almagor is senior director of mixed reality and BCI at Trimble.