Immerse The Bay 2024 Project Ideas / プロジェクトのアイデア

# Project Ideas : Immerse The Bay 2024

Thing out loud about project ideas for the event this week: https://immersethebay.stanfordxr.org/

## About Me

I work with startups in the US on mobile and embedded systems oriented projects, with a focus on IoT and Medical devices.

In Japan, I am looking to build products that intersect AR/AI/IoT

- https://www.linkedin.com/in/dkords/
- https://razortooth.biz/
- https://iotonesite.ai


## Ideas

Here is a short list without filter.  I have hardware for the various project ideas.

### Q-task AR - a task sprint overlay (rissoto timer style) / focus mode

Concept is to build a task timer to allow one to go into focus mode for a fixed amount of time, then switch to the next task, until all tasks are burned down.  The unique aspect of this is to have it be "offline first", and just be a passthrough AR experience with some overlay for notifications and indicator of what to focus on.

Concept notes: https://1drv.ms/i/s!AuIL2jCvMCqRjKlHM9szpJfSZfeBZQ?e=1zLcBo

### Book2.0 - a rethinking of books in XR

The concept is to go deep and explore alternative immersive experiences with books.  Ideas:

- every book has its own server
- every page of a book has an AR marker
- immersive sound
- built in readers
- sky book liner "annotation"

Requires:
- XR skills
- design thinking mindset
- ability to ingest text content
- audio sound design

### ConkyXR - a Conky experience available in XR

See: https://github.com/brndnmtthws/conky for the inspirations and background.  Design approaches:
- use a raspi in the background
- serve the visuals in three.js or other, in a mix reality experience
- target client is XR compatible

Requires:
- designer or artist
- UI widget designer
- network protocols
- linux

### PANKitXR - a personal area network for XR applications

The concept here is to take the great learning from ecosystems like Meshtastic, and build portable mesh networks that can work with any AR/XR capable device, allowing people to build:

- P2P communications servers, think an office network
- P2P gaming, ad-hoc
- P2P emergency services networks

Instead of using expensive ($40+) meshtastic devices, use low cost XiaoESP32C6 or M5NanoC6 devices (both around $7). These give us two tech stacks.  We get Zigbee, and we get Wifi6. 

What we can do with that: 
- Zigbee gives us a mesh network design that is flexible, and secure enought, with support for a theoretical 254 nodes.
- the WIFI6 gives us an ultra fast standard protocol for communication

Design would involve:
- PanKit device: the MCU ESP32C6 will be designated as the PanRouter or PanController, and will serve a Web Server over Wifi, and and act as a router or controller for the Zigbee net
- each host device (PC or HMD or Phone) pairs up with the WIFI AP served on the PanKit device
- the host device subscribes to updates using some standard notification protocol (either web based, UDP, or other)
- host displays activity on the PANKit network in a HUD

Application ideas:
- a simple Team server (send pictures, messages, location sharing)
- a simple game server
- an AR art experience server, where every PANKit node can serve its data via its own web server, and also interact over IoT protocols with other zigbee devices (great for lighting)

Limitations
- high latency possible, so not great for a FPS
- range of Zigbee varies, the more nodes involved, the stronger the network becomes though.  Somewhere between 0-100m is possible
- These devices don't have BLE

Challenges
- Zigbee protocol is fairly new on the ESP32 ecosystem, and a bit difficult
- The alternative that can be used is Matter on top of Thread.  This is even newer and perhaps the most difficult to start up.
- There may not be enough PSRAM to store the server and a web server and AP code.


Requires:
- network protocols
- some basic XR skills based on web stack
- willingness to ideate


### OpenWearXR

Wearable devices configurable via XR UI.  Requires:
- ble knowledge
- ability to prototype simple but elegant UX in XR
- arduino skills
- hardware

I would like to be able to have pop up interfaces to configure all of my wearables without going into a phone app for each device.  

Concept is being explored in open source, a write up here: https://github.com/IoTone/OpenWR/issues/7

## My Approach

- Design thinking / ideation
- small starwman prototypes
- cross functional team

## Skills Available

I can code in:

- swift
- flutter
- c/c++
- python
- node.js

Generally speaking, I can do the low level MCU coding, IoT protocol work, and mobile apps. I have done a limited amount of work on the AR side so am looking for help in this realm.

## Skills Needed

See the projects list!

DM me on Linkedin, Discord, or email.

## What's Next

I am looking for collaborators on AR/XR experiences to build in Japan.  I am also looking to build products around Open Hardware and OSS with a focus on Japan as a market.  Please follow up with me if interested.