Multisensory Storytelling Debriefs: Worldbuilding

WORLDBUILDING

The first conversations were convened under the heading of “worldbuilding” with an invitation to think about the bottlenecks encountered in taking projects from ideation to development. The conversations took place on the 23rd and 24th of March 2026 and featured 10 artists across 2 sessions.

The rationale behind convening a debrief under the thematic heading of “Worldbuilding” was a chance to begin by talking about beginnings. In my conceptualisation of bottlenecks, the beginning of a process is where a number of bottlenecks might occur (which software to use? which recording hardware to adopt? which XR format to design for?) and therefore where a range of project-critical decisions might cluster.

Whilst this suspicion was borne out to a degree, the conversations quickly revealed the limits of my framing for both worldbuilding and bottlenecks.

For example, was taking a project into development a matter of expanding or contracting the scope of the piece and what was possible within the project? Likewise, were bottlenecks exclusively, or even primarily, technical? After all, the biggest influence over whether a technology or pipeline can deliver is not the individual artist, or the technology in isolation, but the collaborative dynamic between them.

Worldbuilding: I asked the participants in these projects how they took their projects from the ideational phase and into tech development and how did the available technologies inform that process.

When learning to operate within new technologies, the learning curves were occasionally steeper than expected:

I’m a visual artist, and my background is in photography, and I’m really familiar with photographic technology and software, and I really thought that I wouldn’t particularly struggle with the VR. I figured I’d get this VR lens, I’d put it on my existing camera, I’d be using very similar software, and it would all be fine, and it wasn’t at all. It was really difficult, it was very different. And I was able to, kind of, muddle through, but it was really intensive, every step of that, getting to the next step and to the next point and solving that problem.

As a result, for some artists this process was a matter of re-scaling and re-scoping in response to the technical constraints and freedoms uncovered through worldbuilding:

But I knew I had ideas of what I wanted to do, and then it turned out, as I went through the process, that my ideas were a lot more expensive than I’d realised, and so I had to scale down massively.

So we had to figure out, “Well, which of these 5,000 different directions do we go in?” And the first session we had, which was online, was very much that: a lot of very explosive or iterative developmental discussion. And it was, at least from my point of view, it was very advantageous.

The proof of concept is actually just to connect, and figure out, what are the processes, what do we need to do to make this actually work? Because what I envisaged was something much more, you know, huge, and 3D, and colourful, but the actual R&D will be very small… I’ve had to learn how can I scale down, so what I’m learning, what I’m developing, is still a part of the larger idea I have?

In some processes of worldbuilding, however, this opportunity to either scale up or scale down the nature of the work was inhibited by the interaction with the technology itself.

On marker-less motion capture:

When I was looking at it, when I was writing the bid, it looked like you needed iPhones, which can be quite cheap, like, there were specific models, and you’re like, “Okay, brilliant, this’ll be great.” The moment it started, all of a sudden, it was like, “No, no, you need GoPros,” which are so much more expensive. So we needed six GoPros.

On combining real-time captioning, BSL interpretation and AI translation tools in a mixed reality context: 

And I thought all I was trying to do- the tech existed, I was trying to bring it together. And I was very naïve. So I made many, many mistakes during my R&D, all genuinely generative now, because I know exactly what I couldn’t make, and the kinds of people I don’t want to work with.

A common experience, when engaging with a new technology was encountering unexpected demands on time and resources, either through steeper than anticipated learning curves, additional expenditures, additional labour or just the time required to make the output from a capture session useable within the specific context of a project.

There’s a lot of overpromise, there’s a lot of hidden costs.

But why?

The commercial context for the technologies in question was often a sticking point.

On spatial audio set-ups:

it’s one of the immense struggles I feel in spatial audio… the leading voice for the spatial audio field, is a commercial brand. So at the beginning, I felt really against spatial audio, because it’s like, who is selling you spatial audio? They just want to sell you speakers.

Offers of help from commercial partners come with risks:

They’d be interested to offer us one of their suits on loan for us to experiment with. But they wanted to know the timeline and they wanted to know if they could use some of the work in their marketing…. So there’s a definite, I think maybe a slight mistrust between myself as a creative and the commercialisation of something or the instrumentalisation of artists in that process.

However, it was underlined throughout that collaboration was not always a given.

Several artists struggled to build relationships with technical partners. For some, emails went unanswered and the process of getting a project into development devolved into the process of re-pitching it again and again to potential collaborators.

The networks available to the creatives produced some really positive interactions but weren’t always well timed. It was widely acknowledged that Immersive Arts was just finding its feet when these projects were green-lit, but that in any event the scheduling of the Big Thing early in the development cycle may have been highly beneficial.

And then at the Big Thing, which was brilliant, we had a meet-the-technologist afternoon, which I went to, which was really good, but by this time, I’d kind of done the bulk of my project… And I didn’t want to wait until then, to meet people who may or may not be able to help me with my project, that seemed like cutting it a bit fine.

Finding the right collaborator emerged as key to the process of translating a project out of its ideational phase and into development.

Whilst some participants came to this conclusion through negative experiences, there was ample evidence that effective and timely collaboration was able to transform potentially inhibiting technical challenges into developmental progress.

it was really, really wonderful to be able to work with these people. And it sounds cliché, but it changes absolutely everything…. And then when you come up to those bottlenecks and whatnot or decide how you’re going to move forward, it’s like, “Ah, okay. Well, we’ve created this kind of modus operandi or working method, we’re all so comfortable with one another that it’s just open, so open.” And that was really the springboard to make things move forward, in that respect.

In fact, on the topic of relationship and collaborators, this is where some of the chief bottlenecks for development were found.

Key issues involved how to best manage collaborative relationships: when money was involved, when access to facilities was involved, where the lines were blurred between who was the creative and who was the producer, where technical roles needed to be clarified across the team.

The challenge of building productive collaborations across the creative industries is greater for artists with marginalized identities. Talking about structural barriers based on prejudice encountered not only during the initial phase of development but before, one artists said: 

I know that a lot of artists from my [Black/femme/trans] background have learnt how in the application [to] overemphasise how much support, how many people of expertise are involved in the project, just so they can’t say, “Are you sure you can do it?” It’s about understanding the terrain. It’s like, what equipment do I need to navigate this particular structural issue which exists in the society we live in? 

In the context of cutting-edge production technologies – the access to facilities and the skillsets needed to operate within them – one of the main bottlenecks is not technical but structural: structural inequities determine who has access to expensive and complex production environments and who has is given the freedom to fail.

The pressure to pre-empt the question “are you sure you can do it?” is indicative of the degree to which prejudices inflect the distribution of resources at all stages and for whom the bottlenecks present in the early stages of development are even tighter. 

Many of these insights will be picked up in the following discussion on Accessibility. But, what emerged from this conversation about the beginning of the development process is that it is never a matter of a truly blank slate, or fresh start.

How an artist is situated, what networks and resources they have access to before undertaking the worldbuilding process will directly impact how they navigate bottlenecks at this crucial stage.

This might be a question of understanding the mundanity of productivity tools – the software before the software – that shape a project before it enters technical development. Or it might be a question of the relationships that are in place across the bidding team and their extended networks, whether or not new working relationships need to incorporated into the process.

In any case, the structural conditions that precede and determine the technological ones will be impactful and need to be recognised. This discussion will be expanded in a following report.

“we didn’t know what we didn’t know,”

In a creative landscape where technologies overpromise, and a funding landscape where artists overemphasise those aspects of a project amenable to funding, being given the “freedom to fail” in an artistic and R&D context comes with a range of unwritten conditions, only some of which are to do with the technologies.

A phrase that occurred across the conversation was “we didn’t know what we didn’t know,” applied to tools, technologies, workflows and relationships.

The range of responses to – pivoting, reframing, rescaling – and the variety of experiences – positive, negative, generative, frustrating – arising from this working through of unknown unknowns prompts two final questions. They are pertinent to the experiences shared in these conversations but might be useful discussion starters for artists across the programme as a whole.

  1. What preparatory work can be done to mitigate the impact of unexpected technical constraints?
  2. Conversely, what can be done to ensure that encountering the unknown in development is fruitful for the overall project?