When designing multi-device web artwork, interactive artwork, and system artwork, we tend to depart from the moving images - videos - as this is the most widely accessible reference point in media art/aesthetics. This often forces MDWA to contaminate linear and/or progressive storytelling.
HOWEVER, interactive artwork, multi-device web artwork, and system artwork, does not have to necessarily have linear structure - where user lands on and when interacting, the state/stages moves progressively… This is actually awkward somehow (to force user to be in a certain state), and structurally limits the degree of freedom in user interaction - thus leading to very childish not really provocative user interaction and limited user’s effect to the system dynamics (Take babel room vr artwork for example…) Mostly in this kind of linear storytelling, interaction is mostly used to progress user from one state to the next, and that’s their only objective… User cannot be provocative and actively participate within the system to change and alter the system.
The ideal interaction: Is not user playing as a puppet in a pre-defined system and linear storyboard scenario, but more, user affecting the system as simultaneously the system affects the user. It’s inter-action, not a one-way action. Ideal two-way user interaction: as user interacts the system, the user changes and alternates the core crucial system dynamics (both in short-term or long-term) rather than just changing mere visual effects and/or progressing towards pre-defined states. And thus the user changes the system, and this newly changed system iteratively changes the user. This creates a non-linear narrative and itself the system.
Easier said than done… Biggest caveat of these systems is that 1) It might overcomplicate user and confuse user - user might not interact with anything before understanding the complexity (or confronting with semantic interactions), 2) System design, 3) System unexpectability (If there is too much uncertainty within the system - both engineering-wise and conceptual-wise it might face trouble)
Therefore crucial to understand what I will here define ‘state-based approach’ to create system art, multi-device web artwork-based interactive systematic approach. Here I write about linear storytelling, progressive storytelling, and state-based approach.
Linear Storytelling: Linear states, no crucial difference with traditional moving images. Interaction is merely used for progressing to the next linear step and/or minor visual changes. Progressive Storytelling: Bit of non-linearity but yet still progressive/pre-defined storytelling - There is a set of non-linear parallel states which can be adjusted through interaction, but the overall structure of the artwork itself is still defined on linear characteristic, and user’s interaction is not powerful enough to change the overall system/state-based dynamics. State-Based Approach: Different state and inter-state relationships define the interface that user confronts/interacts with. Each state is non-linearly combined, sometimes even generated. User takes a power to not only navigate through different states, but redefine inter-state dynamics and even create new states.
GPT’s refined definition:
Below is a concise set of definitions for these three storytelling/interaction modes, derived directly from the essay: Linear Storytelling: A single, pre-defined sequence of states or scenes, much like traditional moving images. Interaction only advances the user from one fixed step to the next (e.g., clicking “next” in a slideshow). The user’s role is passive or minor, with no real power to alter the overall flow—just to trigger the next segment. Progressive Storytelling: A mostly pre-defined structure with branching or parallel states, allowing some non-linear navigation. The user can choose which state to visit (e.g., selecting from parallel paths), but the overall shape of the narrative remains controlled by the designer. The user’s role is a navigator, not a co-creator: interactions reveal or select from existing states rather than transform them. State-Based Approach: A dynamic network of states, where inter-state relationships (and even new states) can emerge or be redefined through interaction. The user wields genuine power to modify system dynamics, not just move between pre-set scenes. This yields non-linear, two-way interaction: the user changes the system, and the system changes the user in return—potentially altering the entire state-space over time.
HAHA
Linear storytelling sounds very clear. (Babel Room) Progressive Storytelling: Good example is SoTA. SoTA - three big states (Idle, Frontend, Backend), defined by user interaction (no user, user active, user inactive) & sub-states in Frontend (118 different NN architectures) - and user scrolls through to define one of these 118 states. Non-linear, circulatory, double-layered state systems. YET, user chooses/navigates within these pre-defined states, acting as a mere navigator rather than an active state-changer / state-dynamics changer. User interaction does define the state, and user interaction is non-linear somehow, but it does not change the state or inter-state dynamics, nor its non-linearity is complex enough (it can be also interpreted as linear somehow: Idle - Frontend - Backend - Idle).
To construct good state-based approach, graph-based system analysis might be needed. Also should be lean approach of heading from progressive storytelling to state-based approach gradually.
HOLA
Linear/Progressive Narrative: Clear Start/Ending, mostly accommodate 1 audience State-based approach: No clear start/ending, blurred start/ending, can accommodate N multiple audiences.
What about the audience ‘controlling’ the system?: Can be both progressive storytelling and state-based approach. But the state-based approach offers audience a change to be the PART of the system.
No media art is a perfectly state-based approach as of now: Rafael Lozano Hemmer has a bit of hint but it is too simple to be a state-based approach: It is more of mere use of data and projecting it simply (Cause-and-effect). AI Art? Don’t even talk about it - it is very narrow and shallow.
Platformatic approach? Sort of might help. Should define the role on what data plays.
From my knowledge, it is almost impossible to reach the pure state-based approach as of now. But it is an attractive goal that a truly two-way interactive system art should head.
Text written by Jeanyoon Choi
Ⓒ Jeanyoon Choi, 2025