Testing Espframe

Espframe has several types of checks. They are split so day-to-day changes can be tested quickly, while release checks still cover the slower firmware and publishing safeguards.

Recommended Checks

Before opening a pull request, run:

npm run check:pr

This is the normal confidence check for feature branches. It verifies generated files, product metadata, backup compatibility, web UI behavior, firmware helper logic, timezone data, and the documentation build. GitHub runs this automatically on every pull request as the Validate PR Gate check.

Before publishing a firmware release, run:

npm run check:release-ready-with-compile

This runs the full release validation, checks the working tree is clean, and compiles both factory and OTA firmware with ESPHome Docker. If you only need the non-compile release gate while iterating, run npm run check:release.

For the fastest local check while editing metadata, generated assets, or compatibility fixtures, run:

npm run check:fast

This skips browser and documentation build work, so it is useful while iterating.

Test Groups

Fast Project Checks

npm run check:fast

This group validates the standard product-contract schema, then checks generated files, product metadata, backup configuration, compatibility fixtures, source ownership, embedded web size budgets, and generated firmware fields. These checks catch mistakes where the user-visible settings contract, generated web files, firmware fields, or docs metadata drift apart.

Build Budgets

npm run check:budgets

The limits in product/budgets.json protect the embedded web app's raw and compressed sizes. Full firmware builds additionally check the actual ESPHome flash report, static internal-RAM usage, and factory/OTA binary sizes. The ram_static_bytes_warn value reports growth without failing the build; ram_static_bytes_max is the hard project ceiling. Neither value is a runtime heap limit or a factory-firmware configuration setting. The current ceiling is above the previously measured P4 boot-failure region, so it is not evidence that builds up to the ceiling are boot-safe. A hard-ceiling increase must be an intentional contract change backed by device testing, not just a larger number added after a build fails.

Web Type Checks

npm run webserver:typecheck checks the assembled application and its imports, including settings, networking, backup, and rendering code. Setting field types come from the product contract; runtime-only fields are declared separately. The contracts and standalone save module additionally use strict checking. Legacy assembled functions still allow implicit parameter types while they are migrated incrementally to explicit modules. New modules should use strict types. npm run test:web-types verifies that invalid setting names and values fail the application check, and npm run test:web-saves covers overlapping writes, rollback, legacy-write ordering, and live updates during edits.

Web UI Checks

npm run test:web

This group checks the generated web app bundle, compatibility helpers, focused smoke-test command options, and browser smoke coverage. The browser smoke test opens the web UI in Chrome or Chromium and exercises the main setup, settings, firmware update, and backup import flows.

When you are working on one browser flow, run a single smoke scenario while iterating:

npm run test:web-smoke -- --scenario wizard-connection-save

Use npm run test:web-smoke -- --list to see the available scenario names. Run the full npm run test:web-smoke suite before committing web UI behavior changes.

Firmware Logic Checks

npm run test:firmware-logic

The helper tests include the production slideshow model; only platform logging and time are stubbed. npm run test:parsers additionally exercises the production Immich JSON parsers with response fixtures and ArduinoJson 7.4.3. The first run downloads checksum-verified test headers into .esphome/host-tests; subsequent runs use that cache.

This group compiles and runs host-side C++ tests for firmware helper logic, then checks timezone data. It is much faster than a full ESPHome compile and is the right place to cover slideshow decisions, Immich request building, date handling, duration parsing, and other logic that can be tested without a device.

Automation controller checks

npm run test:automation-controller covers ordered worker cancellation, panel rotation mappings and all 4,608 combinations of the slot-fetch gates. It runs as part of npm run check:pr.

For changes to script cancellation or rotation sequencing, also run the pinned ESPHome host integration test:

docker run --rm -v "${PWD}:/config" --entrypoint python \
  ghcr.io/esphome/esphome:2026.8.2 /config/tests/automation_runtime_tests.py

This loads the production rotation script and select callback, and the production recovery worker list. ESPHome runs the real actions, scripts and scheduler; only the display/controller endpoints are replaced. It checks self-stop, cancellation of 22 pending workers, rapid rotation changes, all four rotation options and reentrant developer-mode clamping. It uses no device or Wi-Fi credentials. Physical rendering, touch alignment and live HTTP callbacks still need device testing.

Full Firmware Compile

Pull requests run the normal validation gate automatically. Full ESPHome firmware builds are slower, so they run weekly on the dedicated Espframe CI runner and remain available from the PR Validation workflow's manual run button. Run that workflow against a feature branch when you need firmware files to test on a device before merging.

The manual workflow builds both factory and OTA firmware, enforces the flash, RAM, and binary budgets, and uploads a firmware-test-<device> artifact containing:

• <device>.factory.bin for USB/browser flashing
• <device>.ota.bin for OTA testing
• <device>.version.txt with the branch build version and source commit

To run the same factory compile locally with Docker:

docker run --rm -v "${PWD}:/config" ghcr.io/esphome/esphome:2026.8.2 compile /config/builds/guition-esp32-p4-jc8012p4a1.factory.yaml

Use a full compile before firmware releases, after changing ESPHome YAML, after changing C++ code that is not covered by the host-side helper tests, and whenever you want a branch firmware build to flash to a test display.

When To Add Tests

Add or update tests with the feature change, especially when changing:

• device settings, backup import/export, or compatibility behavior
• Immich request building, photo source filtering, portrait pairing, or slideshow flow
• generated firmware fields, product metadata, or public docs tables
• the web settings UI, startup wizard, firmware update controls, or backup restore flow
• release files, firmware manifests, or changelog behavior

Prefer fixture-based tests for saved settings and compatibility changes. Add host-side C++ helper tests for firmware decisions that do not need hardware. Add browser smoke coverage when the user-facing web flow changes.

Manual Device Checks

Automated checks do not prove everything that happens on the physical display. Do a short device check when changing display behavior, touch behavior, networking setup, image rendering, OTA updates, or anything that depends on ESPHome runtime behavior.

A useful manual pass is:

• run the manual PR Validation workflow for the branch and flash the downloaded firmware artifact to a test display
• confirm WiFi setup and Immich setup still work
• confirm the slideshow starts and advances photos
• check touch wake, sleep, next-photo, previous-image-set, and next-image-set gestures, including the on-screen feedback
• open the device web UI and change a setting
• export a backup, restore it, and confirm important settings survive
• check firmware update status if release/update behavior changed

Record any manual device testing in the pull request so reviewers know what was verified outside automation.

Internal Heap Checks

The memory log records byte-addressable internal free bytes, allocator low-water bytes, the current largest free block, its sampled minimum, and free PSRAM. Reports run every minute and after Immich responses and image decoding. The largest-block minimum is sampled once per second and at these callbacks; it is not a guarantee that shorter fragmentation peaks were captured. Existing Home Assistant memory entities retain their names and sampling intervals.

Compare the same device, HTTPS server, settings, and browser/API connections before and after allocation changes. Exercise JPEG and WebP photos, portrait pairs, sleep/wake, reconnects, and OTA. Keep enough runtime to observe repeated photo changes; different images and lazy decoder allocation affect the results.

Background prefetch pauses below 48 KiB free internal RAM or a 16 KiB largest block, and resumes at 64 KiB free with a 24 KiB largest block. Look for prefetch-paused and prefetch-resumed reports. Verify foreground photo requests still run under pressure and prefetch resumes without changing saved settings. TLS allocations prefer dedicated PSRAM, general allocations above 1 KiB prefer PSRAM, and the 32 KiB internal reserve remains available to internal-only users.

Frame naming

npm run test:identity tests the production name parser and startup/storage adapter with platform stubs, including UTF-8 validation, unchanged first boot, persistence, clearing, corruption, failed writes and stable running names. Browser scenarios prefixed frame-name cover saving, retry, byte limits, named exports and optional restore with the destination MAC suffix.

The startup adapter in components/espframe/frame_identity.cpp is pinned to ESPHome 2026.8.2. Review it when upgrading: it rebinds Application's non-const StringRef members after entity registration and before network component setup. It must not change registered entity names, hashes, or preference keys. Names use an independent namespace in the existing 448 KiB NVS partition; no partition-table change or erasure is performed. This differs from Espcontrol PR #1924's dedicated data-partition storage because Espframe has no equivalent spare data partition.

Before merging naming changes, OTA-update an already paired frame, verify its original identity and saved settings, then save a name, restart, and check DHCP, mDNS, the native API and Home Assistant. Confirm it reconnects to the existing device without duplicate entities or changed entity IDs and retains manually assigned Home Assistant names. Test clearing, power cycling, another OTA update, and restoring a backup onto a second frame. No automated test proves those network and Home Assistant integration behaviors on hardware.

The naming UI and optional backup dialog intentionally increase the raw web app budget to 228,000 bytes, CSS to 22,500 bytes, combined raw to 250,500 bytes and combined gzip to 56,000 bytes. The individual gzip limits are 51,000 bytes for the app and 5,200 bytes for CSS. Firmware flash/RAM limits remain unchanged; full builds still enforce them.

The naming form and reconnect dialog reuse Espcontrol’s UI structure and styles, with an inline Save & Restart action, live hostname preview and reconnect links. Browser coverage also checks that failed saves do not restart and restart failures leave the saved name available for retry.

LVGL allocation diagnostics

Normal firmware redirects capability-neutral aligned allocations to PSRAM only on the ESPHome setup task between priorities 401 and 399, around pinned LVGL's priority 400. This prevents its internal-first draw-buffer attempt from consuming 250 KiB of internal heap. Alignment, buffer size, explicit capabilities, other tasks, and allocations outside this setup window are preserved. Allocation failure is returned to LVGL's existing setup failure handling; there is no internal-RAM fallback. Review the setup ordering and allocator call when upgrading ESPHome 2026.8.2. Do not add unrelated components in this priority window. The allocation policy and probes are enabled only when both LVGL and PSRAM are configured. Helper configurations without PSRAM retain LVGL's existing allocator. With the policy enabled, configurations overriding lvgl.setup_priority to anything other than 400 are rejected during validation to prevent silently bypassing the allocation policy. The parent espframe.setup_priority can still be overridden independently.

Use an opt-in diagnostic build to verify graphics placement. In a local build wrapper, add:

espframe:
  memory_diagnostics: true

For example, create the ignored builds/dev.yaml alongside the release wrappers:

packages:
  base: !include guition-esp32-p4-jc8012p4a1.yaml
espframe:
  memory_diagnostics: true

Use guition-esp32-p4-jc8012p4a1-v2.yaml for the V2 board. Compile with the pinned ESPHome Docker image as above and -s firmware_version v0.0.0. Flash only the artifact for the matching board when hardware testing is authorized. Keep the full boot log; the allocation trace runs only during setup. This flag defaults to false and does not alter saved preferences or normal firmware allocation. Diagnostic logging consumes additional memory, so these builds are measurement artifacts, not release candidates subject to the production static-size budget.

memory.lvgl reports:

• before-lvgl and after-lvgl: internal free bytes, allocator low-water mark, largest internal block, free PSRAM and largest PSRAM block.
• setup-net: signed net usage between those snapshots. Concurrent tasks and instrumentation affect the delta; it is not an exact LVGL allocation total.
• setup-aligned-allocation: address, requested bytes, actual memory region and requested capability bits for each aligned allocation on the setup task in the window around LVGL's setup priority. Failed allocations are also logged.
• draw-buffer: the active buffer obtained through LVGL's public API. Match its address to the allocation trace to determine its requested capabilities.

For ESPHome 2026.8.2 the configured buffer_size: 6% becomes a one-eighth buffer: 256,000 bytes at 1280×800 RGB565. The same-sized allocation following the draw buffer is the expected rotation workspace in this version. The trace deliberately labels it as an allocation rather than accessing ESPHome's private rotation pointer: verify the sequence against the pinned LvglComponent::setup() before attributing other allocations to it, especially after an ESPHome upgrade.

memory.stack reports the calling ESPHome loop task's minimum free stack bytes after LVGL setup and alongside the existing periodic/response/decode memory reports. It is not a report of all networking or driver task stacks. Enabling diagnostics does not change the production allocation policy or buffer size.

Compare the same diagnostic build on each board with landscape/portrait JPEG and WebP photos, HTTPS, browser and Home Assistant connections, reconnects, sleep/wake and OTA. Record buffer placement, minimum internal heap, largest blocks and loop stack headroom. Verify that the draw buffer and rotation allocation report PSRAM, and compare rendering, rotation, touch and OTA before shipping. Lowering the YAML percentage alone does not reduce this buffer in the pinned ESPHome version.

Photo buffer reclamation

npm run test:photo-buffers runs with check:pr. It covers visible and preloaded pair ownership, pending downloads/actions, partial pairs, queued commands and repeated release. All three full-frame slots and previous-photo metadata remain outside the reclamation policy.

After a firmware build, test the production descriptor adapter against its LVGL source (currently 9.5):

cmake -S tests/photo_buffer_lvgl -B .esphome/photo-buffer-lvgl \
  -DLVGL_SOURCE="${PWD}/builds/.esphome/build/immich-frame-10inch/managed_components/lvgl__lvgl"
cmake --build .esphome/photo-buffer-lvgl -j 4
.esphome/photo-buffer-lvgl/photo_buffer_lvgl_tests

This verifies live-source protection, detachment before release and image/header cache invalidation across descriptor reuse. The test uses real LVGL with host pixel allocations; it does not measure ESP32 PSRAM.

For an explicitly requested device test, an ignored local ESPHome config can include tests/photo_buffer_device_probe.h and call esphome::espframe::run_photo_buffer_device_probe() from a lambda after a 45-second boot delay. The optional probe allocates four independent 640×800 RGB565 images and hidden widgets, verifies a pinned pair survives reclamation, and exercises 20 allocation/release cycles. Logs tagged photo-buffer-test report actual PSRAM and largest-block measurements. Remove the probe or restore the prior firmware after testing; never include it in release configuration.

Normal firmware logs photo-buffer only when allocations are reclaimed. Savings are conditional: up to 4,096,000 bytes (3.906 MiB) when both portrait pairs are unused. The fully occupied 9.766 MiB ceiling is unchanged. This is the ownership and reclamation stage; sharing live full-frame/paired representations needs a separate transition and fallback design. Before merge, exercise mixed photos, consecutive pairs, previous during refill, rotation, failed companions and recovery on hardware. Compare largest free PSRAM blocks over a longer mixed-photo run as well as free bytes; repeated large allocations can still fragment memory.