Real. But difficult to perceive. Part 02.
Extended Reality interfaces and speculations on a screenless world.
I have an odd relationship with extended reality technologies. I don’t use them. At least not yet. My reluctance to strap a computer to my face, as VR and AR headsets like the Apple Vision Pro require, has not been overcome by the applications currently on offer. During the pandemic, I spent time gathering over zoom with other women also interested in Extended Reality (XR) tech, and at one point completely embarrassed myself by admitting to the assembled VR game developers and AR app aficionados, that I don’t own a headset.
“What — at all?” one esteemed leader of the field enquired?
This essay was written for the xtended substack — it’s free for everyone to read. Subscribe here to read, join the conversation, and support!
“No…” I was quickly backtracking here, cursing myself for ever having spoken “I have access to one of course but I’m not currently using it.” I wasn’t lying. I did have access to one. Technically. I had played Beat Sabre at my neighbor’s apartment and the university that my husband was teaching at had multiple Quests, which I could have borrowed if I so desired and spent hours within, voyaging into VR realms. I did not so desire. I don’t like VR headsets. I like the sun and going outside. I don’t want to wear an AR hololens because I think it makes my look like a doofus and AR phone applications drain my battery and rapidly lose their novelty.
So what on earth was I doing gathered with all these XR experts on zoom?
Well, I write about mixed reality. Like, a lot. I had recently (in conjunction with collaborators) won a prize for a piece titled X-reality: A manifesto for mixed reality futures in which I described three scenarios from a speculative near future where XR experiences are so seamless and integrated with daily life that completely novel use-cases for the technology have emerged.
[from X-Reality: A manifesto for mixed reality future] I organised a work dinner last night. Getting ready, I chose a color palette for my skin texture taken from the rainforest my organization is currently campaigning to protect. I’m personally trying to raise awareness about ecological biodiversity, so I used as an augmentation around my eyes this pattern from a lichen that is native to the region. Someone asked me about it on the underground — and I got a bit of public awareness raising done, talking about our campaign.
Another lady heard us chatting and actually joined the conversation — pretty incredible for London and actually sort of hilarious because (as she explained to me) she uses a filter in public spaces that color codes people’s overlays based upon their likely covid-transmission risk. I was deemed low-risk apparently — so I already looked green to her.
When it was time for dinner, I found myself sitting opposite our boss. Like most people, she carries around with her a projected environment; yesterday it was a field of wildflowers from an at-risk ecosystem our organisation is working with. During dinner, the flowers mingled with a projected wall mural that our CFO was sitting in front of, and some kelp from the underwater environment that our boss’s little daughter had around herself.
When our boss gave a short presentation about the mission of our organization. The lighting in our views all adapted to show her spotlit as if on a stage. I didn’t notice that as she talked, everyone else was gradually adding those same kinds of wildflowers to their own environments to indicate that they were listening and agreeing.
One-on-one, I actually pride myself on my social skills; I’m good at fluttering my skin matte between different colors to show empathy or enthusiasm — and I’m good at names. I almost never turn on name-tagging because I think people can tell (it’s the quick glance above your head) and hence appreciate it if you remember them. But there’s a time and a place for social support services, and last night I had dialled in a coach who helps me navigate social anxiety. I shared my view with her and, watching remotely, she gave real-time advice about conversation starters, or how to use environmental cues. I only caught on to what everyone else was doing with the flowers becasue she saw and prompted me.
I never again joined that zoom meeting. I felt too awkward. My interest in and engagement with XR lies much more comfortably in the realm of science and speculative fiction. I need to skip a few steps in interface development, to a time when we have XR lenses or some other solution that allows the seamless integration of digital and physical realities. In this future, the entire physical world becomes a potential computing interface. While some visions of XR futures present proliferating screenage and a horrifying assortment of pop-up windows obscuring our fields of view, my speculations of XR futures are tactile, connecting us more to physical realities, not less.
This essay was written for the xtended substack — it’s free for everyone to read. Subscribe here to read, join the conversation, and support!
We live in an age of remote sensing and IOT data collection. There are realities all around us that are undeniably real — like the amount of energy being used by a specific appliance. I know this is real because every month I really hold my breath in anticipatory horror at the gigantic energy bill I am about to receive due to how much real energy my real appliances are really using.
But this is something I can’t easily see.
My favorite of the three XR speculations I wrote was about the transformation of domestic items into storytelling devices about the relationship that a person has with resources, economies, and ecological futures.
There are multiple ways that domestic-scale XR can make visible realities that are otherwise difficult to perceive. Environmental data (like how much of a coastal property will be above water in 15 years’ time) could be mapped directly onto the property’s walls — not hidden away in an assessment report.
Users could track energy usage and emissions with patterns, labels, and symbols on their devices, appliances, walls, and doors; or even as an environmental overlay — filling the house (say) with a pink cloud representing the collective carbon emissions of the day’s activites. Maybe that would be counter-productive; too visually compelling, a potentially peverse incentive to emit just a little bit more to see the carbon cloud gather and swirl.
Designing XR overlays is a design industry that I’m relatively confident will soon emerge. For a screenless world to really work and to be a place that makes digital and other invisible realities more visible, accessible, and chanegable; the language of XR needs to be as intuitive and communicative as, I guess, the best product labels. I honestly never thought I would write a sentence that communicated some level of excitement about product labels, but that’s the best analogy I can think of for communication innovations that are today quotidien but were at one time revolutionary.
Real. And possible to be perceived. That is my vision for an extended reality future. The pre-requisite for this might ironically be a device interface that is much less perceivable, and until that time I suspect I will keep writing and musing about XR rather than joining the ranks of VR experts, but when my fridge can easily tell me just how much it’s costing every time I stand there absent-mindingly waiting for more food to magically appear within….I will be the first to overlay that application.
Thanks for reading! Explore the xtended substack here — we write speculative fictions and essays about extended reality tech.
