04/03/25

Ask the Developer Vol. 17: GameChat — Part 2

This article has been translated from the original Japanese content.
This interview was conducted before the system was released.

In this 17th volume of Ask the Developer, an interview series in which developers convey in their own words Nintendo's thoughts about creating products and the unusual details they hone in on, we're talking to the developers behind GameChat, a feature of Nintendo Switch™ 2.

Check out the rest of the interview

Part 2: Festival vibes

You said that you valued ease of use. I feel like that's easier said than done. Could you tell us about the trial and error you went through to achieve this?

Tokunaga: The more we focus on making it easy to use, the more things we as developers need to consider. For example, not only did we want a built-in microphone, but we also wanted it to be used without any setup. However, implementing a built-in microphone is actually quite difficult. If we don't make any adjustments, the microphone would naturally pick up sounds from the game, such as music. But if those aren't removed, then the most important sounds, the players' voices, might be drowned out. Especially in handheld mode, controller sounds, such as buttons being pressed and rumble, would also get picked up, so those need to be removed too.

On the other hand, in TV mode, when players are far away from the console, their voices need to be picked up without them having to shout. This means achieving two conflicting concepts: removing the loud sounds coming from the TV, while picking up the faint voices of players sitting far away from the console.

Ono: We kept asking you the impossible, Tokunaga-san. Like removing as much noise as possible that isn't someone's voice, but leaving in the sound of people clapping.

Tokunaga: We installed a high-performance audio chip dedicated to voice processing and fine-tuned it relentlessly until it only picked the sounds we wanted it to. (Laughs)

Ono: You have my eternal gratitude. (Laughs) One time I was asked, “Is it just one person's voice we need to pick up, or is it everyone if it’s being used by a family? It'd make our lives easier if you could narrow it down to one or the other.” To which I answered, “It’s not an either-or question. It’s both.” (Laughs) After all, we wanted to take various factors into account, like how far the couch is from the TV and where players are sitting on the couch. Tokunaga-san once told me to please stop being so vague, but if you're going to say “no setup required,” then it needs to be able to handle various situations.

What about the speakers? Did they also require some trial and error?

Ono: To hit our goal of “no setup required,” we worked hard to balance the speaker's volume. Since the game and voice chat sounds play together out of the same speaker, we worked with the sound team to determine which one should stand out more. Even without manual setup, voice chat stands out more when someone's talking, and when no one's speaking, game sound will be more prominent. There's a natural transition between the two.

Tokunaga: The microphone automatically detects which sounds are essential and adjusts the audio balance of voices so they're comfortable to hear. So, for example, if you switch from TV mode, which picks up voices at a distance, to handheld mode, which picks up voices nearby, the voices won't suddenly get louder for the person on the other side of the voice chat, and you can continue using the feature smoothly.

Ono: We regard the ability to seamlessly transition between TV mode and handheld mode while using voice chat as one of the unique features of Nintendo Switch 2. If you’re using voice chat in TV mode and a family member wants to use the TV, you can just remove the console from the dock and continue to voice chat as-is in handheld mode.

Tokunaga: You can take it anywhere in handheld mode, so you can now, for example, voice chat and play games while lying on your bed.

I see. So, when you want to chat, it'll make your voice come through clearly, and when no one's talking, you can concentrate on your game. By the way, the camera used for video chat isn't built into the system, right?

Tamura: We created the Nintendo Switch 2 camera as an accessory, which is sold separately. With TV mode in mind, we deliberately avoided having a built-in camera. I thought that in TV mode, the positioning of the console would differ for each person. So, it'd be better to create something that could be used immediately without any setup, just by connecting it whenever and wherever it's needed, so that people who wanted one could get one.

You can also use other compatible USB-C cameras, right? What's the difference between the Nintendo Switch 2 camera and other cameras?

Tamura: One of the strengths of the Nintendo Switch 2 camera is that it uses a very wide-angle lens to capture the atmosphere in a player's living room, and it’s preset to provide the optimal viewing angle. The camera is designed to capture players whether they're near or far from it, alone or in a group, or even if they're playing a game controlled with body movements. But what we value the most is how clearly their facial expressions are shown. We thought that in order to convey a game's atmosphere, we needed to make players' facial expressions clearly visible to others. So, we added a feature to automatically recognize people’s faces and show them in the highest resolution possible.

Ono: For many USB cameras (11), if the person is far away from the lens, they’ll appear small and become blurry when zoomed in. But the Nintendo Switch 2 camera has a software feature that automatically processes the image to minimize blurring as much as possible when zoomed in and display facial expressions clearly.

(11) Some devices may not be compatible. Compatibility can be checked on the Nintendo Switch 2 system.

So players' faces are captured clearly without them having to worry about how far they are away from the camera.

Tamura: Yes, that’s correct. In terms of clearly capturing facial expressions, we were also conscious of the lighting in the room where Switch 2 would be played. In the evening especially, I think some people play games with the lights dimmed to help them relax. Even in those kinds of situations, the brightness is automatically adjusted so that facial expressions can be seen clearly.

It sounds like there are benefits to choosing the Nintendo Switch 2 camera if it allows you to enjoy GameChat in the most optimized way without any setup.

Ono: Another request I made regarding the camera was to hide the background and display just the players. I thought there might be some people who would be hesitant to use the camera if their room was visible. So, the default setting hides the background. Now, if you combine this with sharing your game screen...

Wow, it looks like you're livestreaming.

Ono: Precisely. In reality, it's just a video chat with friends, but the way it’s presented makes it look like YouTubers streaming their gameplay. I reckon it would be fun for friends to play together as if they're livestreamers.

Tokunaga: You may have seen PC and smartphone cameras that hide backgrounds. But for game systems, the background needs to be hidden while the system is also using processing power to run the game, which is a real challenge. However, I think that whether the background is visible or not also affects safety and security, so we kept on fine-tuning it to use the least amount of processing power to show only the players. Finally, we achieved something we could be confident in.

Ono: The background is properly hidden even with multiple people, not just when you're playing alone. I don’t think I’ve seen that very often.

Ah, that's interesting. Network technology is also an important part of communicating players' voices and expressions, right? How are voices and video streamed to other players?

Tokunaga: Voices and videos are streamed to other players via servers. To help ensure voice chat works smoothly all over the world, we’ve set up servers not only in Japan, but in various other countries and regions.

To make it simple and comfortable to use, you must have had to fine-tune it in various ways.

Tokunaga: Since Switch 2 is a gaming system, we needed to dedicate a significant percentage of its resources to playing games, while using what's left to run GameChat. On top of that, it needs to run stably to avoid impacting the game negatively. Lots of processes are happening at once—removing noise from the audio, hiding the video background, sending and receiving data over the network, compressing and decompressing data... It's packed with various technologies from various teams, so it was challenging to integrate everything, organize it, and make it all work. Because of this, we often described the development as having “festival vibes.” (Laughs)

Festival vibes?

Tokunaga: Yes. Whenever someone came up against a challenge during development, various people would gather around and say, “Are you having trouble? Let me help,” then we’d all work together to solve it. As mentioned earlier, GameChat is a combination of various technologies. And since it has to run alongside a game, if too much processing resource is spent on any part of GameChat, it could affect the game's performance. Therefore, we had to fine-tune all the technologies involved. That necessitated the involvement of people from various teams, but everyone was so reliable and said, “If it’s for GameChat, we'll make it work!” (Laughs)

Everyone likes solving problems with technology, and they have fun doing it. Those festival vibes lasted more than two years. And with more than 100 people involved, what a festival it was. (Laughs)

More than 100 developers! Yep, that's a festival alright.

Ono: During development, we held daily meetings with everyone using GameChat to test, verify, and improve it. We fine-tuned every last detail while soaking in those festival vibes.