đź‘ľ Metaverse Research

Communication in the metaverse

Introduction

Defining the metaverse

Nobody’s nailed down an exact definition of the metaverse—likely because it doesn’t exist. But we can rule out what the metaverse we want to design isn’t.

Games like Fortnite and Roblox do have their own internal metaverses, but on their own, they don’t qualify as the metaverse. Instead, Fortnite and Roblox are more like walled-garden universes where developers create their own worlds within, say, the Fortnite universe, but users have no means of interfacing with other universes. So it’s critical that the metaverse enables communication between universes.

We can also say that a universe in the metaverse isn’t constrained by its neighboring universes. This means that if the universes of Facebook and Microsoft interface, they don’t need to be on the same rendering engine or have the same UI, art style, etc. What we want to identify is the glue that can hold unique universes together to compose the metaverse, should such a bonding agent even exist.

Furthermore, the metaverse doesn’t need to be accessed through head-mounted display (HMD) VR. Prof. Mike Zyda has noted that it’s unlikely someone wants to spend their day with a screen strapped to their face. This isn’t to say that one can’t explore the metaverse with HMD VR, but just that the Ready Player One approach isn’t the only way.

Instead of these approaches, we want to design the metaverse such that it enables users to carry their information between universes, while also enabling each universe to be distinct and developed independently of other universes. Additionally, it should be accessible through multiple digital mediums. How the described metaverse might be designed is what will be explored.

Existence of the metaverse

The next pressing question beyond the definition of the metaverse is whether the metaverse already exists. There are a number of articles on the internet contesting that it does and that the metaverse is a smoke-and-mirrors term for the internet. With our flexible definition of the metaverse, that’s both true and not true: true in the sense that if we replace universe with domain and metaverse with web browser, we mostly have the internet, but not true when considering the desire to interface between universes.

Today, communication between web pages of different domains is largely prohibited without explicit approval. This is a matter of security, as each domain wants to be protected from cross-site scripting. Take cookies for example. These chunks of information that live in a client’s browser certainly look like a means of transporting user information between “universes,” but they’re actually constrained to the domain that issued them. This means that not only can cookies not be viewed by another domain, but another domain can’t issue cookies for any domain but itself. The garden walls are tall here!

But this digresses from the original point that while the internet does provide a means of traveling between worlds, it lacks an extensible means for those worlds to share data with each other safely and succinctly. Because with safe data sharing, a user can go from one universe to the other, and the destination universe can easily access the user’s information. This hits on the specific problem this research tackles: communication.

Communication

Communication between environments is the problem that’s really captured my interest. How can two different domains communicate user data in an extensible, consistent way? I like to consider the following test for such a system. Suppose a user brings a hot dog from their previous universe to a Halo universe. What’s a hot dog in Halo? Is it food? Is it a weapon? Is it a wearable? A sound means of communication between universes should be able to resolve this question efficiently.

Standardization

One approach to communication is standardization. Rather than allowing each universe to interpret user data differently, they should all agree on a standard interpretation and data format. With this, a hot dog in Halo means the same thing it did in the last universe and will mean the same thing in the next. Similarly, its art style will persist across universes.

This solves the issue of each universe having to interpret all other universes’ data, but it introduces another issue: locking people into a standard limits expressibility. If one universe wanted to add new user data, they’d be at the mercy of the standard accepting it. Otherwise, other universes wouldn’t know how to interpret the new data.

Epic Games is trending towards creating a standardized metaverse through control of the space. But this brings with it the cons of a monolithic architecture: a loss of diverse expression. Epic Games’s approach is a little like the Henry Ford quote: “Any customer can have a car painted any color that he wants, so long as it is black.” It’s not difficult to picture Epic Games saying, “Any developer can build a universe any way they want, as long as it’s with Unreal Engine.”

But building a Bell Systems of the metaverse is likely not the solution. It’s akin to General Magic partnering with AT&T to build a closed internet for the Magic Link. A more accessible metaverse infrastructure can supercede a monolithic metaverse in the public, just as the open internet did the Magic Link’s closed internet.

Translation

Languages are a parallel place where we can see this communication issue arise. Each language offers a different form of expression that promotes a different mode of thought. For example, Russian distinguishes between light blue and dark blue with different words, and that translates to a better ability in the speaker to differentiate between light and dark blue colors. A standard language would limit that expression and experience.

Yet in a world of myriad languages, we’re still able to communicate—for the most part—and maintain our unique language expressions. This is thanks to translation.

Consequently, I propose translation as a solution to communication in the metaverse. And instead of requiring each universe to translate the data of other universes, translation services could serve as intermediaries between universes. Thus, when a universe requests user data from other universes, the user data passes through a translation service which resolves the meaning of a hot dog in Halo.

Naturally, this discussion intersects with the topics like semantic interopability, data integration, ontology alignment, and the semantic web of Web3. However, these topics are much too deep to dive into here, but it’s worth noting that resarch in those fields will be foundational to this translation.

Extended web browser

To further extend the accessbility of the metaverse, we can look to a web browser. The primary property of the browser we want to leverage is encapsulation.

Encapsulation

There’s a common paradigm of encapsulation in computer science, one that a web browser implements, which is applicable to the metaverse. A web browser partitions domains, just as an operating system partitions users, and a hypervisor partitions operating systems.

Web browser
Very high-level web browser structure

Within our definition of the metaverse, universes must be partitioned, so they can be developed independently, but also traversable, so user data can be transferred seamlessly. Translation can address the latter, but to address the former, we need something of a metaverse browser.

Metaverse browser

What a metaverse browser can offer for a developer of a universe is ease of integration. If translation services are part of the metaverse infrastructure, we don’t want every universe to have to write the same boilerplate communication with the translation service. What would be much nicer is to abstract that into a common message, like getItem, that a universe could send to request a piece of user data. The metaverse browser would be a perfect place to handle such a message.

Upon sending a getItem message, the metaverse browser can call the translation service, which will get the requested item, translate it, and send the result back to the metaverse browser, which forwards it to the callee universe. Such an infrastructure is pictured in the following diagram.

Proposed metaverse infrastructure

Implementation

The aforementioned infrastructure is demoed in this research’s GitHub repository. In this implementation, we have two universes, universe A and universe B. User John leaves universe A for universe B. When John enters universe B, his avatar is loaded from universe A through the translation service and an extended version of the open-source web browser Min. In universe A, the avatar was represented in PNG, but the translation service translates it to JPEG for universe B.

This is a simple example of a metaverse, but it’s not difficult to see how this foundation can be extended. Through getItem, we could also fetch a friend’s list, a user’s attire, their stats, and so on. And with a more powerful translation service, we can translate between not just data formats but potentially art styles as well.

While implementing a fully-functioning metaverse is out of scope for this research, my hope is that this architecture can serve as a platform for future additions and modifications to enable fully traversable and easily programmable universes.

Next steps

Data storage

As with most of my experience so far researching the metaverse, answering one question seems to spawn many more: if we know how to share data, where should it go? It can’t go on the client side, because a user should have the same experience independent of the machine they’re on. So it has to be stored somewhere that universes can easily retrieve it. But is it the metaverse browser’s responsibility to store that data? Or is it the responsibility of the universe? I don’t have the solution to this, but the decentralized graph data protocol gun may be part of the answer.

Advanced translation service

Without an enormous deal of manual intervention, scaling translation beyond similar data formats is an open problem, predicated on semantic interopability; there is still a need for an extensible means to translate user data from one universe to another, past what this research demos. A viable solution should be able to translate user data from a universe like Minecraft to another like Mario without a human explicity specifying how each piece of user data is translated.

At present, semantic interapobility has outstanding issues resolving inconsistency and ambiguity. But in the future, with an advanced-enough AI, semantic interoperability may become more realistic, and with it, advanced translation services.

Credits and inspiration

A huge thanks to Prof. Mike Zyda for his wealth of experience and knowledge regarding the metaverse. From his hand in metaverse’s beginnings as distributed virtual environments to today, Prof. Zyda has been an invaluable resource in this research.

Below is a list of writings and open-source software that made this possible: