Work and immersive technologies

ByMarcus Carter, Ben Egliston and Kate Euphemia Clark
An image of the factory floor. There are a line of workers in orange vests in a production line. The perspective makes it look like the factory goes on forever

Both augmented reality and virtual reality have increasingly pushed to become key players in the workplace. AR has had a much longer history within the workplace, particularly as an assistive technology in manufacturing industries. There is a body of research in the field of Human Computer Interaction (HCI) that presents experimental cases – with Funk et al. (2017) noting that few have made the leap from lab to industry (although citing some exceptions, such as the in-situ projection-based Light Guide System from OPS Solutions). As Funk et al. (2017) point out, AR interfaces have been used in some capacity since the early 1990s, specifically in aerospace engineering (see Caudell et al. 1992 cited in Funk et al., 2017). A range of research to date has addressed the potential benefits of AR in manual assembly work – underlining its benefits in reducing error and cognitive load (Tang et al., 2003), in both abled and disabled workers, (see Funk et al., 2015), providing task-relevant information (Henderson and Feiner, 2009). Little work has been done on the long-term evaluation of in-situ projection with the exception of Funk et al.’s (2017) study, which provides an account of in-situ AR interfaces in manual assembly workplaces, finding that such interfaces hindered the assembly speed of expert workers (e.g. increasing cognitive load), yet enhanced the efficiency of untrained workers.

Beyond experimental applications emerging from HCI research, technology companies have variously come to develop AR applications for the workplace. A range of AR hardware (e.g. Google Glass Enterprise Edition, Vuzix M400) and software (e.g. UpSkill) have emerged for use within workplace settings. As the Google Glass Enterprise edition website notes, the device is used within contexts of manufacturing (in the emerging ‘smart factory’) but also in logistics, and healthcare. We see this through examples like Boeing and DHL’s respective uses of Google Glass. In one testimonial for the Google Glass, by American aerospace company Boeing, AR is framed as appending the limited capacities of the human worker (specifically, those installing electrical wiring on aircraft) – offering “real-time, hands-free, interactive 3D wiring diagrams – right before their eyes” (Boeing, 2018, n.p.).

Yet, despite big tech companies staking out a space in industrial settings, there is no existing literature (including non-academic writing) that critically addresses the implications of AR in the workplace, nor providing an account of the scale at which this technology is being used. As such, in order to think speculatively about AR’s implications for work, we adopt a more permissive definition of AR here – such as that used by Wolf et al. 2018 – to include wearable sensor technologies (such as FitBits and Apple Watches) rather than squarely focusing on head-mounted AR. We believe that this wider, existing literature shows the clear potentials for AR’s impact on the workplace. Many of these current issues are pertinent in a consideration of AR’s future in the workplace.

Wearable devices that track user activity are a key part of contemporary work. As Moore (2018) suggests, this includes wearables as efficiency-tracking mechanisms for manual work in warehouses, or for corporate ‘wellness’ programs in white collar or office contexts. Such technologies were particularly widely problematised following news of Amazon’s use of tracking wearables for employees in warehouses, specifically, a wristband “which gives it the ability to track and record employees’ hands in real time” (Salame, 2018, n.p.). In recent years, Amazon has been criticised for the use of monitoring their employees schedules to such an extent that many employees did not have time to use the bathroom, and had to urinate in bottles (Paul, 2021). In April of 2019, Amazon introduced a system that would monitor employees ‘time off task’ (Williams, 2021), and Wholefoods (owned by Amazon), tracked data about what stores were ‘at risk’ of unionisation (Williams, 2021). Employees face an unprecedented level of surveillance and control that negatively impact workplace conditions.

caucasian people, worker, asian-1822531.jpg
Worker in a factory

A range of work has developed critical responses to the movement toward tracking and quantification in the workplace. From the perspective of sociology and critical theory, Till (2019) suggests that the deployment of smart tech in corporate workplaces – and the veneer of corporate ‘wellness’ that they carry with them – works as a mechanism for control. Drawing from French philosopher Bernard Stiegler, Till argues that we might think of corporate tracking as a mechanism for shaping people’s psychic and libidinal energy (accruing benefits to companies) rather than for stated material health outcomes. Critics elsewhere have compared quantified work to Taylorism (Morozov, 2013; Salame, 2018), that is, an approach to applying experimental measures to enhance managerial control of worker efficiency – or an ‘updated’ form of Taylorism (as Moore, 2018 has it). On the effects of such regimes of quantification and tracking in the workplace, Moore writes “we can speak of declining welfare for workers and the associated regime of total mobilization and surveillance as they corrode workers’ health and safety, and create anxiety, burnout and overwork” (2018, p.55). Moreover, she notes “Capital is tempted to invest in new technology not because it may improve the public good, regardless of the rhetoric of wellness that informs current wellbeing initiatives for workers, but rather because it can increase its profit ratios” (2018, p.56). Offering a different take, O’Neill (2016) suggests that quantified work resembles not Taylorism, but rather a distinct style of worker management referred to as the ‘European Science of Work’. The distinction, O’Neill argues is that the wearables and sensors mobilised in quantified workplaces do not simply attempt to intensify the body as to be more productive (as per Taylorism), but rather to manipulate the production process as to “better accommodate biological processes of fatigue and regeneration” (2016, p. 610), that is, better harnessing productivity through attention to the labouring body’s rhythms.

“[w]earables and sensors mobilised in quantified workplaces do not simply attempt to intensify the body as to be more productive (as per Taylorism), but rather to manipulate the production process.”

At Meta’s 2022 conference, Meta Connect, Mark Zuckerberg laid out Meta’s intention to bring virtual reality to our workplaces (Meta, 2022). Zuckerberg suggested that virtual workspaces have a unique benefit over platforms such as Zoom, because people do not have to be physically present in the office space, but co-workers can still get all the subtle communication cues that come from body language. Meta also announced a partnership with Microsoft to bring VR capability to Microsoft Teams (Microsoft, 2022). The critical reception to this follows many of the concerns of the use of AR technology in the workplace, including issues around oppressive surveillance of employees behaviour, an increased capacity to ‘program out’ behaviour that managers deem unproductive, and increased marginalisation of disabled and other minority workers (Egliston, Clark, and Heemsbergen, 2022). 

Despite the lack of research on head mounted AR hardware and software marketed for industrial use, there is clear potential for many of the same issues described in existing research on the quantification and rationalisation of work through wearable sensor technology to be played out through the introduction of AR into workplace settings. An overemphasis in the workplace on productivity has concerning implications for the future of VR and AR technologies. 

Amazon, surveillance and control: The future of work?

In recent years, the material conditions of warehouse workers servicing Amazon’s e-commerce arm have been subject to increasingly intense scrutiny. Workers in Amazon’s so-called ‘fulfilment centres’ – the sites for the storage and delivery of products to customers – have widely reported conditions of surveillance and strict control. In an Australian context, Burin reports that warehouse workers are expected to work at ‘Amazon pace’ – “somewhere between walking and jogging” (Burin, 2019, n.p.). She notes that workers – many of whom are precarious, casual workers, hired by Amazon through labour hire agencies – are at risk of losing further employment should they not meet this standard. Intensifying this are reports of Amazon’s use of surveillance technologies (such as wearable wristbands, see Salame, 2018) which tracks various forms of bodily activity – data which can be operationalised to say whether or not employees are being efficient enough. 

Workers in a factory wearing VR headsets

MR technologies, largely HMDs and smart glasses, are beginning to enter workplaces – particularly within the setting of manual labour. In addition to catering to a commercial market, MR manufacturers like Microsoft, Google, Vuzix, and Kopin have ‘enterprise’ versions of their technologies aimed largely at the manufacturing and logistics industries. While these are often framed by industry boosters as a net positive – reducing human error, increasing efficiency and so forth, it is not difficult to see them harnessed (much like Amazon’s wearables) to discipline and dominate, to maintain productivity through intimate surveillance of the worker’s body.

HMDs are very new to training and survelling workers. These tasks have a long history from which they are drawing, but they also have some new concerns that should be addressed in future critical research. For example, while an employee is being trained in a VR headset device, vast amounts of data about their body and eye movements are collected (Big Think, 2022), from which decisions about their performance is made. However, these things are all culturally dependent, and may lead to an automation of discrimination in hiring. 

References

Big Think. (2022). VR training is the future. Here’s why and how companies are using it. Retrieved from https://bigthink.com/plus/vr-training/.

Boeing. (2018). Boeing tests augmented reality in the factory. Retrieved from https://www.boeing.com/features/2018/01/augmented-reality-01-18.page

Burin, M. (2019). ‘They resent the fact that I’m not a robot’. Retrieved from https://www.abc.net.au/news/2019-02-27/amazon-australia-warehouse-working-conditions/10807308?nw=0

Egliston, B, Clark, KE, and Heemsbergen, L. (2022). Is the metaverse really the future of work? Retrieved from https://cavrn.org/is-the-metaverse-really-the-future-of-work/.

Funk, M., Bachler, A., Bachler, L., Kosch, T., Heidenreich, T. & Schmidt, A. (2017). Working with Augmented Reality? A long-term analysis of in-situ instructions at the assembly workplace. In PETRA ’17: Proceedings of the 10th International Conference on Pervasive Technologies Related to Assistive Environments (pp. 222–229). New York, NY: ACM.

Funk, M., Mayer, S. & Schmidt, A. (2015). Using in-situ projection to support cognitively impaired workers at the workplace. In ASSETS ’15: Proceedings of the 17th international ACM SIGACCESS Conference on Computers and Accessibility (pp. 185–192). New York, NY: ACM.

Henderson, S. & Feiner, S. (2009). Evaluating the benefits of augmented reality for task localization in maintenance of an armored personnel carrier turret. In ISMAR’09 (pp. 135–144). New York, NY: IEEE.

Meta (2022).  Meta Connect Keynote 2022. Retrieved from https://www.youtube.com/watch?v=hvfV-iGwYX8.

Microsoft. (2022). Microsoft and Meta partner to deliver immersive experiences for the future of work and play. Retrieved from  https://blogs.microsoft.com/blog/2022/10/11/microsoft-and-meta-partner-to-deliver-immersive-experiences-for-the-future-of-work-and-play/. 

Moore, P. (2018). Tracking affective labour for agility in the quantified workplace. Body and Society, 24(3), 39–67.

Morozov, E. (2013). To Save Everything, Click Here: The Folly of Technological Solutionism. New York, NY: Public Affairs.

O’Neill, C. (2016). Taylorism, the European Science of Work, and the Quantified Self at Work. Science, technology and human values, 42(4), 600–621.

Paul, K. (2021). Leaked memo shows that Amazon knows delivery drivers resort to urinating in bottles. Retrieved from https://www.theguardian.com/technology/2021/mar/25/amazon-delivery-workers-bathrooms-memo.

Salame, R. (2018). The New Taylorism. Retrieved from https://www.jacobinmag.com/2018/02/amazon-wristband-surveillance-scientific-management

Tang, A., Owen, C., BIocca, F. & Mou, W. (2003). Comparative effectiveness of augmented reality in object assembly. In Proceedings of CHI’03 (pp. 73–80). New York, NY: ACM.

Till, C. (2019). Creating ‘automatic subjects’: Corporate wellness and self-tracking. Health, 23(4), 418–435.

Williams, A. (2021). 5 ways Amazon monitors its employees, from AI cameras to hiring a spy agency. Retrieved from https://www.businessinsider.com/how-amazon-monitors-employees-ai-cameras-union-surveillance-spy-agency-2021-4?op=1

Wolf, K., Marky, K. & Funk, M. (2018) We should start thinking about Privacy Implications of Sonic Input in Everyday Augmented Reality! In Mensch und Computer 2018 – Workshopband. Gesellschaft für Informatik e.V., Bonn, 353–359. DOI: http://dx.doi.org/10.18420/muc2018-ws07-0466

Recommended citation

Carter, M, Egliston, B, Clark, K.E. (November, 2022) Work and immersive technologies. Critical Augmented and Virtual Reality Researchers Network (CAVRN). https://cavrn.org/work-and-immersive-technologies/

An earlier version of this literature review was published for the Socio-Tech Futures Lab in 2020. This version has been updated to include more recent literature.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.