Using Ornaments for Debug Displays Such as WebSocket Status
When playing with a local WebSocket server, I want to check each time whether it is currently connected and what kind of data is being passed. At first I output that to Xcode logs, but I started wanting something more graphical, and it became tedious to look at the Mac screen every time.
This entry is for day 23 of the visionOS Advent Calendar 2024. The content is about using Ornaments for debug displays such as WebSocket status, and it also acts as a sequel to the following entry.
So this proceeds on the assumption that the WebSocket server and the connection environment from Vision Pro are already ready. For the details of the prerequisite parts, please see the presentation materials and article. Briefly, they are as follows.
- Vapor, a Swift web application framework, can handle WebSocket, so I used it to build a WebSocket server.
- From the visionOS side, I connected to the WebSocket using daltoniam/Starscream.
This is exactly the stage where I am about to connect Vision Pro to Vapor over WebSocket in a local environment and try various things.
When handling WebSocket in a web browser, you can monitor the connection with Chrome DevTools and similar tools, but that is not possible with Vision Pro. The quickest approach may be to use Mac Virtual Display on Vision Pro and check while displaying Xcode logs.
At first, I was actually doing that, but developing with Mac Virtual Display becomes fairly tiring if you do it for a long time. So the main theme of this entry is whether there is an easier way to view that kind of connection information for debugging.
How Can Development Information Be Displayed When Building Apps for Vision Pro?
The first thing that comes to mind is displaying it in a separate window. Apps for visionOS can have multiple windows, so one use would be to display development information in that separate window.
In visionOS 2 and later, you can specify a window's position using the defaultWindowPlacement instance method. So one option is to create a dedicated connection information window using this approach.
Another option, if you do not mind affecting the app layout itself, is to place a debug information area inside the window as a panel.
I wanted a slightly easier way to create a small information area, which led me to try using Ornaments.
Ornaments are like small toolbars attached around a window. They are displayed slightly in front of the window and subtly protrude from it. However, their display position can be controlled, so it is also possible to show them outside the window.
The official documentation describes them as floating on a plane parallel to the related window, slightly in front of it. When the related window moves, the ornament moves while maintaining its positional relationship to the window. Controls and information in an ornament are not affected even if the window content scrolls.
To put it a little imprecisely, Ornaments feel like small windows attached to a Window. Ornaments can contain various elements, and you can use things like VStack and HStack.
This time, I used Ornaments to create a small display that simply shows whether WebSocket is connected or disconnected.
When connected:
When disconnected:
It looks like this. It can be made relatively easily. The area labeled "Display Log" is intended for eventually displaying JSON and similar content.
The code is below.
import RealityKit
import RealityKitContent
import SwiftUI
struct ContentView: View {
@ObservedObject var client = WebSocketController()
init() {
client = WebSocketController()
}
var body: some View {
VStack {
if client.isConnected {
Text("Connected.")
.padding(.top, 40)
.font(.largeTitle)
List {
ForEach(client.messages, id: \.self) { m in
Text(m)
}
}
}
}
.onAppear {
client.connect()
}
.onDisappear {
client.disconnect()
}
.ornament(
attachmentAnchor: .scene(.top),
contentAlignment: .bottom
) {
VStack {
if client.isConnected {
Label {
Text("Connected.")
.multilineTextAlignment(.center)
.font(.title)
} icon: {
Image(systemName: "bolt.horizontal.circle.fill")
.foregroundColor(.green)
.font(.title)
}
} else {
Label {
Text("Disconnected.")
.multilineTextAlignment(.center)
.font(.title)
} icon: {
Image(systemName: "bolt.horizontal.circle")
.foregroundColor(.red)
.font(.title)
}
}
Text("Display Log")
}
.padding()
.glassBackgroundEffect()
}
}
}
#Preview(windowStyle: .automatic) {
ContentView()
}The .ornament(attachmentAnchor: .scene(.top), contentAlignment: .bottom) { ... } part is where Ornaments are specified.
attachmentAnchor lets you specify the placement position.
The static method scene(_:) is available.
static func scene(_ anchor: UnitPoint) -> OrnamentAttachmentAnchorAnchors are provided so that you can control the position to some degree without specifying numeric values, so you can choose the position you like.
The following documentation explains anchor types in detail.
By the way, if you want to hide it, you can prevent it from displaying by specifying .hidden for visibility.
.ornament(
visibility: .hidden,
attachmentAnchor: .scene(.top),
contentAlignment: .bottom
) {
...
}Is It Okay to Use It This Way?
I used it because it was convenient, but is this kind of usage actually okay? Let us look at the Ornaments documentation. There is also Japanese documentation, so I will refer to that.
The documentation says that ornaments can be placed on any edge of a window and can include UI components such as buttons, segmented controls, and other views. The system uses ornaments to create and manage components such as toolbars, tab bars, and video playback controls. Developers can use ornaments to create custom components.
It also recommends considering ornaments when presenting frequently used controls or information in a consistent location that does not interfere with the window. Because ornaments are always near the related window, users do not lose sight of them.
Connection information can be classified as frequently used information, so this kind of usage seems acceptable. The fact that it is attached to the window and hard to lose sight of is also good. If it were a separate window, it could become hidden when it overlaps behind something else.