anubhavmalhotra/r2d2-tv: Native house display screen, non-public distant browser, and push-to-talk voice assistant for webOS TVs · GitHub

Demo: https://anubhavmalhotra.com/vibing/r2d2-webos/r2d2-demo.mp4

Read the story behind the venture: R2D2 for webOS.

An area house display screen, non-public distant browser, and push-to-talk voice assistant for webOS TVs. A Linux compute host runs Chromium and the voice fashions; the TV supplies the show, purposes, and distant management.

• The keyfilter patch, Home/idle routing, and native voice seize change or hook vendor companies. A mistake can disable the distant, Home, or voice till the TV is restored.
• Firmware SHA-256 pins, code offsets, image names, and patch anchors goal one reference construct. Re-derive them on your personal firmware; by no means pressure a mismatched pin.
• The software program is offered as is, with out guarantee; see LICENSE.md.
• This is an unbiased venture. It isn’t affiliated with, endorsed by, or sponsored by LG Electronics, Lucasfilm Ltd., The Walt Disney Company, or any streaming service. Product and character names are emblems of their respective homeowners.

Reference deployment on one TV/firmware: Home, the browser, and distant voice instructions work finish to finish there. Other TV, firmware, distant, and host combos are unvalidated. This repository supplies supply and offline checks, not a common root, firmware, or host installer.

• Compute host (spark/ in paths): an aarch64 Linux machine. It renders a sandboxed Chromium session in a personal show and audio session and streams it to the TV over the LAN. It additionally runs the native speech recognition and intent fashions and a closed command registry.
• TV (television/, voice/television/): the Home utility and the browser viewer, which sends distant enter again to the host. A root seize proprietor takes the distant’s voice key and sends bounded audio to the host over mutual TLS.
• Content safety (safety/): pinned filter guidelines loaded into every disposable browser profile.

Model inference stays on the host. Requested web sites, searches, streaming apps, and elective climate nonetheless contact their suppliers; see SECURITY.md.

• Build machine: Python 3.10+ and Node.js 20+.
• Compute host: aarch64 Linux with a non-root service account (UID 1000); the Chromium snap with its sandbox enabled; Xvfb for a personal X show; PipeWire, WirePlumber, and the GStreamer plugins for the WebRTC path; FFmpeg with {hardware} HEVC encoding for the MSE and AirPlay paths; a CUDA-capable GPU and Python 3.12 for the pinned native fashions (about 10 GB of weights).
• TV: a webOS TV you might have system (root) entry to and a certified solution to set up packages on. Native seize additionally wants a toolchain and headers matching the precise firmware construct.
• Network: TV and host on the identical LAN. Measure throughput and jitter; a nominal hyperlink fee doesn’t set up playback high quality.

cp config/instance.json config.native.json
# Edit the ignored file on your personal units.
python3 instruments/construct.py --config config.native.json --output dist/native --include-tests

The output incorporates an unsigned Home package deal, rendered host companies, TV service sources, and a file manifest. Building doesn’t set up software program or contact a tool. The output listing have to be new.

To allow the browser shortcut, create a personal functionality key and move it to the construct:

umask 077
python3 -c 'import secrets and techniques; print(secrets and techniques.token_hex(32), finish="")' > browser.native.key
python3 instruments/construct.py --config config.native.json --browser-key-file browser.native.key --output dist/local-browser --include-tests

Brand logos usually are not shipped. Home reveals textual content tiles until you move --home-logos-file, a personal JSON object mapping app IDs to information: URI logos that you’ve the precise to make use of. Keep that file and any construct made with it non-public.

Keep the rendered output non-public: it incorporates deployment addresses and should include the browser functionality.

python3 -B -m unittest uncover -s checks
node checks/test_protection_proof.js
node checks/test_home_core.js
python3 instruments/privacy_check.py

UI checks use Playwright’s Chromium:

npm ci --ignore-scripts
npx --no-install playwright set up chromium
npm run take a look at:house
npm run take a look at:browser

The full offline checks are listed in the CI workflow. Offline checks don’t set up bodily video cadence, microphone high quality, or compatibility with one other firmware construct.

Stage an entire rendered spark/ tree as a versioned listing owned by the service account, then run its preflight earlier than enabling the session supervisor:

python3 -B "$RENDERED_SPARK/runtime/trial.py" --run --preflight-only

RENDERED_SPARK factors to the staged listing. Preflight checks the sandbox, show/audio, content material safety, DNS, and cleanup. It doesn’t set up packages or confirm TV playback. There is not any basic host installer; dependency set up and repair supervision are host administration.

The runtime retains its functionality key and elective voice configuration below $HOME/.native/share/baaz-browser/. baaz-browser is a legacy runtime listing title stored for compatibility. Keep these recordsdata mode 0600 in non-public directories. Resident voice additionally wants pinned mannequin snapshots, TLS credentials, and domestically paired TV management; see the voice backend.

Install the Home package deal by your webOS package deal workflow and take a look at it as an atypical utility first. Changing Home and idle routes is a separate step: the offline Home planner produces candidates and rollback payloads out of your firmware’s captured state.

Voice seize follows the local-capture activation contract. The public builder doesn’t acquire system entry, patch system recordsdata, or activate companies. Keep unique recordsdata and a restoration path outdoors any course of {that a} UI restart terminates. Then examine distant controls, present apps, browser playback, consecutive voice presses, standby/resume, and energy biking.

Keep system configuration, keys, firmware photos, and runtime information outdoors model management. Security boundaries and reporting are in SECURITY.md.

Project code is MIT-licensed (LICENSE.md). The browser extension, filter tooling and information, and two runtime adapters are GPL-3.0-or-later; THIRD_PARTY_NOTICES.md lists these elements and upstream attributions.