A tour of web capabilities
Tags: beginners
Published: 2024-04-03 21:34:11

This post compiles notes from Max Firtman's [course](https://frontendmasters.com/courses/device-web-apis/) in Frontendmasters. 

- What is a capability? The ability of a web browser to perform an action using typically a built-in API. Compatibility can be an issue, of course. So some of the capabilities may not be available in some browsers. 
- Maturity of the capabilities: green (mature), light-green (may not be available in every browse. Normally chromium-based browsers vs Firefox/Safari), yellow (not mature and only available in some browsers), red (you can't use them and may be added in the long term).
- Resources to check the capabilities: MDN, caniuse.com, web.dev,  webkit.org/blog (specifically for Safari), [chromestatus.com](URL), web.dev/baseline (multivendor, define a list of stable capabilities, stable version for the web, one list per year starting in 2022).
- Core capabilities NOT covered in this course: `fetch`, `web workers` (for threading), `WebAssembly`, `WebSocket`, `WebRTC`, `WebPerformance APIs`, `Network information` (see course [here](https://frontendmasters.com/courses/web-app-performance/)), Device memory, WebOTP (one-time password), Web Crypto (cryptography), storage (another [course](https://frontendmasters.com/courses/web-storage-apis/)), Web Components, CSS, 2D Canvas, WebGL, Pointer Lock (for gaming), screen capture, PWA ([course](https://frontendmasters.com/courses/pwas/)), page visibility, background sync, background fetch, web push, and notifications, media session (all these in the [course](https://frontendmasters.com/courses/background-javascript/)), Web Authentication, passkeys, credential management, (all these in the [course](https://frontendmasters.com/courses/web-auth-apis/)).
- Permissions: some are harmless, others have a privacy risk or have a cost. If there is a cost or risk some browsers may limit it: user engagement requirements or permission dialog to the user. Most will require HTTPS. Some capabilities will need the user interaction to be enabled (aka you can't trigger it on page load). Permissions are granted on an origin base (domain). If the user denies permission, the API won't be able to ask again, manual re-enablement is required. Permission may have no time limit or it may be limited by time or session or usage. They are enabled for the main navigation (the HTML document that was loaded) - it won't be available for iframes. If you want to turn a capability off: permission policy spec, it's an HTTP header (`Permissions-Policy`), for an iframe it's an HTML attribute.

### Capabilities

### Permissions API

####1. Permissions API.

MDN Docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Permissions_API)

### Sensors, Geolocation, and Input Devices

- Sensors on mobile devices: accelerometer (3 axis - x, z, y), gyroscope (turning the phone - alpha, beta, gamma), magnetometer (compass), proximity (to the user), light sensor.

####2. Two ways to consume them on the web: old APIs (global DOM APIs, that is how you need to do it in iPhones and iPads. It only supports magnetometer, gyroscope, and accelerometer. Done with event listeners. i.e. `devicemotion` and `deviceorientation` (MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Window/devicemotion_event)) and [here](https://developer.mozilla.org/en-US/docs/Web/API/Window/deviceorientation_event). They need user permission), Sensor API (not yet in Safari nor Firefox, see [MDN](https://developer.mozilla.org/en-US/docs/Web/API/Sensor_APIs#browser_compatibility)).

####3. Geolocation API (one of the first capabilities on the web). 

Google, Apple, etc. know your location because of WiFi - your devices know which WiFis are around and these companies can locate you because they have a database of SSIDs and their location. The API will give you the location, it's provider-agnostic (wifi, GPS, etc.). Works only in the foreground - won't work in the background/workers. Since the API is old, it's callback-based (no promises). MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Geolocation_API). 

####4. Screen orientation API. 

Green availability: if the device is in `portrait` or `landscape` mode. Other stuff is not yet widely available: `lock` the screen.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w3ftvjvtjgf3atzushdd.png)

####5. Touch events API. 

They work with touch screens: `touchstart`, `touchend`, `touchcancel`, etc. Apple proprietary. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Touch_events#browser_compatibility). That is why the following API was created:

####6. Pointer events API. 

MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Pointer_events). Based on mouse events, and multi-pointer (they work with mouse, trackpad, touch pen, stylus, etc). YOu receive one event per touch interaction (ie. 3 fingers you get 3 events vs 1 event in touch event API above). 

####7. VirtualKeyboard API. 

MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/VirtualKeyboard_API). This API lets us show/hide the virtual keyboard and know how much space it is taking in the screen.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5pg3k8eyqxjj9vclg7de.png)

####8. Gamepad API. 

MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Gamepad_API/Using_the_Gamepad_API). Greenlight API. High-level API (I don't need to know the specifics of the gamepad, and it's the minimum that all gamepads have i.e. if a gamepad has a speaker that is not common, so it is not supported via the API) and low latency. You create a requestAnimationFrame loop (60 fps) and you check the status of each button with a `boolean`. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/88pbjw43zd8p7wi32i6d.png)

####9. Web HID API (HID = Human Interface Device). 

MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/WebHID_API). Limited availability (light-green). 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9v8mggtda0i3ebwa0foe.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/smnymp0z4sed7l0a1lvj.png)

### Speech, voice, and camera

####10. Web Speech API.

- Speech recognition API

MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecognition).
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yhvtmvm8m1cdxfqrugya.png)

- Speech synthesis API.

We can make the web app speak.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wb0mklanfwhqh7zztjwl.png)

####11. Shape detection API. Between yellow and red.

- Barcode detector API. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/BarcodeDetector). Experimental, it is only available for Chrome on Android as of this writing. Chromestatus [here]((https://developer.mozilla.org/en-US/docs/Web/API/BarcodeDetector))

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bcdopt9tuq6fel75nx94.png)

- Face detector API. Experimental, it is only available for Chrome on Android as of this writing.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3mi61qofuqabswi7v0l2.png)

- Text detector API (OCR). Experimental, it is only available for Chrome on Android as of this writing.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r8ol0js44c9svrozrwdn.png)

####12. Media devices. It lets you open the camera and microphone, and get the stream.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7ld61mvqjeax7pp6w43d.png)

####13. Advanced control camera (aka Camera PTZ - Pan, Tilt, Zoom). Let's you manipulate the camera.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/06kdxh89hz257lhc4mrx.png)

####13. Augmented reality (AR, XR). 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qv29x9c7tqvydpgdzlmc.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vwbz2ue6pv51wcymut27.png)

####14. Screen Wake Lock API. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Screen_Wake_Lock_API). Light green (not supported on Safari on iOS).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/id17kapeev482xzzzg44.png)

### External hardware and Devices

####15. Web Bluetooth API. Limited availability. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Web_Bluetooth_API). FF and Safari don't implement it and looks like they will never do it, because of reasons. So only on Chromium-based browsers for now. Only with BLE devices (Bluetooth Low Energy).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0me2cw8npesjz41328t5.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xgjs3jo7en116z038u2e.png)

####16. Web Audio API. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API). Green. Low-level API that lets you generate dynamic audio, 3D audio, and ultrasound communication with devices. Sonic socket [library](https://smus.com/ultrasonic-networking/) for ultrasound communication between devices.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xt2anbwgjf59waa6hz95.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e307g51qg54x7owgr0bu.png)

####17. Web Midi API. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Web_MIDI_API). Limited availability (Chromium-based browsers only). It's similar but much older than HID.

![Image description]
(https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h8oslv78eq96d1tbfhwx.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vqpqegajg64i8g5ia13y.png)

####18. Web serial API. Very low level API. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Web_Serial_API). Experimental.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/em7t5zonzuq5f40p5jxk.png)

####19. Web USB API. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/WebUSB_API). Experimental. Low-level API for device vendors. You probably won't be using this API.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8k9exobuoxu17jm25m1l.png)

####19. Vibration API. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Vibration_API). Targeting phones only.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/okze2h5my5apokrxa1rz.png)

####20. Battery status. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Battery_Status_API#browser_compatibility). Doesn't work in many browsers.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/boqt4jtu336ook6z1abq.png)

####21. Idle detection. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Idle_Detection_API). Limited availability.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fygk8dm18vl2p6na1nig.png)

####22. Web NFC. MDN docs [here](https://developer.mozilla.org/en-US/docs/Web/API/Web_NFC_API). Limited availability.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t2r84ya3g7n498mrg1yn.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4cggc2sor5ycxdqmo5sn.png)