## Summary
* **What is the goal of this PR?**
Add a debugging script to help developers monitor the ESP32 serial port
directly from a PC.
* **What changes are included?**
Added a new script: scripts/debugging_monitor.py
## Additional Context
While working on a new Crosspoint-Reader feature, it quickly became
clear that watching the ESP32 serial output without any visual cues was
inconvenient and easy to mess up.
This script improves the debugging experience by reading data from the
serial port and providing:
1. A timestamp prefix for every log line (instead of milliseconds since
power-up)
2. Color-coded output for different message types
3. A secondary window displaying a live graph of RAM usage, which is
especially useful for tracking the memory impact of new features
<img width="1916" height="1049" alt="Screenshot_20260126_183811"
src="https://github.com/user-attachments/assets/6291887f-ac17-43ac-9e43-f5dec8a7097e"
/>
---
### AI Usage
Did you use AI tools to help write this code? _**< PARTIALLY >**_
I wrote the initial version of the script. Gemini was used to help add
the Matplotlib-based graphing and threading logic.
## Summary
* Truncating chapter titles using utf8 safe functions (Cyrillic titles
were split mid codepoint)
* refactoring of lib/Utf8
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< PARTIALLY >**_
## Summary
* Fixing custom sleep behaviour where the first image in the /sleep
directory is not shown
* image at index 0 is not being rendered when more than 1 image is
stored in /sleep directory, because `APP_STATE.lastSleepImage` is always
0.
## Additional Context
* `APP_STATE.lastSleepImage` is reset to 0 when a epub is open, this
value is only used to compare it to the randomly selected one in
`renderCustomSleepScreen()` that should always be a valid index, since
the list of valid bmp images is colected from scratch. -> no need to
reset it and block image @ index 0 from being rendered
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
Co-authored-by: Oyster <Oyster@home>
## Summary
* This change hides the button hints from overlapping chapter titles
when in landscape CW mode.
Before
After
## Additional Context
* I initially considered implementing an offset fix, but with potential
UI changes on the horizon, hiding the button hints appears to be the
simplest solution for now.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? < Partially >
## Summary
* Clarify strings on Wifi connection error screens
* I have confirmed on device that these are short enough not to overflow
screen margins
## Additional Context
* Several screens give duplicative text (e.g., header "Connection
Failed" with contents text "Connection failed") or slightly confusing
text (header "Forget Network?" with text "Remove saved password?")
---
### AI Usage
Did you use AI tools to help write this code? **No**
## Summary
* refactors Indexing popups into ScreenComponents (they had different
implementations in different files)
* removes Indexing popup for small chapters
* only show Indexing popup (without progress bar) for large chapters
(using same minimum file size condition as for progress bar before)
## Additional Context
* Having to show even single popup message and redraw the screen slows
down the flow significantly
* Testing results:
* Opening large chapter with progress bar - 11 seconds
* Same chapter without progress bar, only single Indexing popup - 5
seconds
---
### AI Usage
Did you use AI tools to help write this code? _**< PARTIALLY>**_
## Summary
* Adds a menu in the Epub reader
* The Chapter selection is moved there to pos 1 (so it can be reached by
double tapping the confirm button)
* A Go Home is there, too
* Most significantly, a function "Delete Book Cache" is added. This
returns to main (to avoid directly rebuilding cached items, eg. if this
is used to debug/develop other areas - and it's also easier ;))
Probably, the Sync function could now be moved from the Chapter
selection to this menu, too.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY**_
## Summary
Extracted some changes from
https://github.com/crosspoint-reader/crosspoint-reader/pull/500 to make
reviewing easier
This PR adds HAL (Hardware Abstraction Layer) for display and GPIO
components, making it easier to write a stub or an emulated
implementation of the hardware.
SD card HAL will be added via another PR, because it's a bit more
tricky.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **NO**
* **What is the goal of this PR?** Implement a metadata viewer for the
Recents screen
* **What changes are included?**
| Recents | Files |
| --- | --- |
| <img alt="image"
src="https://github.com/user-attachments/assets/e0f2d816-ddce-4a2e-bd4a-cd431d0e6532"
/> | <img alt="image"
src="https://github.com/user-attachments/assets/3225cdce-d501-4175-bc92-73cb8bfe7a41"
/> |
For the Files screen, I have not made any changes on purpose. For the
Recents screen, we now display the Book title and author. If it is a
file with no epub metadata like txt or md, we display the file name
without the file extension.
---
Did you use AI tools to help write this code? _**< YES >**_
Although I went trough all the code manually and made changes as well,
please be aware the majority of the code is AI generated.
---------
Co-authored-by: Eliz Kilic <elizk@google.com>
## Summary
* Correctly render italics on image alt placeholders
* Parser incorrectly handled depth of self-closing tags
* Self-closing tags immediately call start and end tag
## Additional Context
* Previously, it would incorrectly make the whole chapter bold/italics,
or not italicised the image alt
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? No
## Summary
* **What is the goal of this PR?**
Quality of Life
* **What changes are included?**
Add compile_commands.json & .cache to .gitignore .
Both are use by clangd that can help IDE support.
Run `pio run --target compiledb` to generate `compile_commands.json`.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
Finally, I have received my device and got to chance to work on OTA.
https://github.com/crosspoint-reader/crosspoint-reader/issues/176
* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
Existing OTA functionality is very buggy, many of times (I would say 8
out of 10) are end up with fail for me. When the time that it works it
is very slow and take ages. For others looks like end up with crash or
different issues.
* **What changes are included?**
To be honest, I'm not familiar with Arduino APIs of OTA process, but
looks like not good as much esp-idf itself. I always found Arduino APIs
very bulky for esp32. Wrappers and wrappers.
## Additional Context
Right now, OTA takes ~ 3min 10sec (of course depends on size of .bin
file). Can be tested with playing version info inside from
`platform.ini` file.
```
[crosspoint]
version = 0.14.0
```
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< NO >**_
## Summary
Optimizes EPUB metadata indexing for large books (2000+ chapters) from
~30 minutes to ~50 seconds by replacing O(n²) algorithms with O(n log n)
hash-indexed lookups.
Fixes#134
## Problem
Three phases had O(n²) complexity due to nested loops:
| Phase | Operation | Before (2768 chapters) |
|-------|-----------|------------------------|
| OPF Pass | For each spine ref, scan all manifest items | ~25 min |
| TOC Pass | For each TOC entry, scan all spine items | ~5 min |
| buildBookBin | For each spine item, scan ZIP central directory | ~8.4
min |
Total: **~30+ minutes** for first-time indexing of large EPUBs.
## Solution
Replace linear scans with sorted hash indexes + binary search:
- **OPF Pass**: Build `{hash(id), len, offset}` index from manifest,
binary search for each spine ref
- **TOC Pass**: Build `{hash(href), len, spineIndex}` index from spine,
binary search for each TOC entry
- **buildBookBin**: New `ZipFile::fillUncompressedSizes()` API - single
ZIP central directory scan with batch hash matching
All indexes use FNV-1a hashing with length as secondary key to minimize
collisions. Indexes are freed immediately after each phase.
## Results
**Shadow Slave EPUB (2768 chapters):**
| Phase | Before | After | Speedup |
|-------|--------|-------|---------|
| OPF pass | ~25 min | 10.8 sec | ~140x |
| TOC pass | ~5 min | 4.7 sec | ~60x |
| buildBookBin | 506 sec | 34.6 sec | ~15x |
| **Total** | **~30+ min** | **~50 sec** | **~36x** |
**Normal EPUB (87 chapters):** 1.7 sec - no regression.
## Memory
Peak temporary memory during indexing:
- OPF index: ~33KB (2770 items × 12 bytes)
- TOC index: ~33KB (2768 items × 12 bytes)
- ZIP batch: ~44KB (targets + sizes arrays)
All indexes cleared immediately after each phase. No OOM risk on
ESP32-C3.
## Note on Threshold
All optimizations are gated by `LARGE_SPINE_THRESHOLD = 400` to preserve
existing behavior for small books. However, the algorithms work
correctly for any book size and are faster even for small books:
| Book Size | Old O(n²) | New O(n log n) | Improvement |
|-----------|-----------|----------------|-------------|
| 10 ch | 100 ops | 50 ops | 2x |
| 100 ch | 10K ops | 800 ops | 12x |
| 400 ch | 160K ops | 4K ops | 40x |
If preferred, the threshold could be removed to use the optimized path
universally.
## Testing
- [x] Shadow Slave (2768 chapters): 50s first-time indexing, loads and
navigates correctly
- [x] Normal book (87 chapters): 1.7s indexing, no regression
- [x] Build passes
- [x] clang-format passes
## Files Changed
- `lib/Epub/Epub/parsers/ContentOpfParser.h/.cpp` - OPF manifest index
- `lib/Epub/Epub/BookMetadataCache.h/.cpp` - TOC index + batch size
lookup
- `lib/ZipFile/ZipFile.h/.cpp` - New `fillUncompressedSizes()` API
- `lib/Epub/Epub.cpp` - Timing logs
<details>
<summary><b>Algorithm Details</b> (click to expand)</summary>
### Phase 1: OPF Pass - Manifest to Spine Lookup
**Problem**: Each `<itemref idref="ch001">` in spine must find matching
`<item id="ch001" href="...">` in manifest.
```
OLD: For each of 2768 spine refs, scan all 2770 manifest items
= 7.6M string comparisons
NEW: While parsing manifest, build index:
{ hash("ch001"), len=5, file_offset=120 }
Sort index, then binary search for each spine ref:
2768 × log₂(2770) ≈ 2768 × 11 = 30K comparisons
```
### Phase 2: TOC Pass - TOC Entry to Spine Index Lookup
**Problem**: Each TOC entry with `href="chapter0001.xhtml"` must find
its spine index.
```
OLD: For each of 2768 TOC entries, scan all 2768 spine entries
= 7.6M string comparisons
NEW: At beginTocPass(), read spine once and build index:
{ hash("OEBPS/chapter0001.xhtml"), len=25, spineIndex=0 }
Sort index, binary search for each TOC entry:
2768 × log₂(2768) ≈ 30K comparisons
Clear index at endTocPass() to free memory.
```
### Phase 3: buildBookBin - ZIP Size Lookup
**Problem**: Need uncompressed file size for each spine item (for
reading progress). Sizes are in ZIP central directory.
```
OLD: For each of 2768 spine items, scan ZIP central directory (2773 entries)
= 7.6M filename reads + string comparisons
Time: 506 seconds
NEW:
Step 1: Build targets from spine
{ hash("OEBPS/chapter0001.xhtml"), len=25, index=0 }
Sort by (hash, len)
Step 2: Single pass through ZIP central directory
For each entry:
- Compute hash ON THE FLY (no string allocation)
- Binary search targets
- If match: sizes[target.index] = uncompressedSize
Step 3: Use sizes array directly (O(1) per spine item)
Total: 2773 entries × log₂(2768) ≈ 33K comparisons
Time: 35 seconds
```
### Why Hash + Length?
Using 64-bit FNV-1a hash + string length as a composite key:
- Collision probability: ~1 in 2⁶⁴ × typical_path_lengths
- No string storage needed in index (just 12-16 bytes per entry)
- Integer comparisons are faster than string comparisons
- Verification on match handles the rare collision case
</details>
---
_AI-assisted development. All changes tested on hardware._
## Summary
* **What is the goal of this PR?** Add Spanish language hyphenation
support to improve text rendering for Spanish books.
* **What changes are included?**
- Added Spanish hyphenation trie (`hyph-es.trie.h`) generated from
Typst's hypher patterns
- Registered `spanishHyphenator` in `LanguageRegistry.cpp` for language
tag `es`
- Added Spanish to the hyphenation evaluation test suite
- Added Spanish test data file with 5000 test cases
## Additional Context
* **Test Results:** Spanish hyphenation achieves 99.02% F1 Score (97.72%
perfect matches out of 5000 test cases)
* **Compatibility:** Works automatically for EPUBs with
`<dc:language>es</dc:language>` (or es-ES, es-MX, etc.)
<img width="115" height="189" alt="imagen"
src="https://github.com/user-attachments/assets/9b92e7fc-b98d-48af-8d53-dfdc2e68abee"
/>
| Metric | Value |
|--------|-------|
| Perfect matches | 97.72% |
| Overall Precision | 99.33% |
| Overall Recall | 99.42% |
| Overall F1 Score | 99.38% |
---
### AI Usage
Did you use AI tools to help write this code? _**PARTIALLY**_
AI assisted with:
- Guiding and compile
- Preparing the PR description
## Summary
* If long-press chapter skip is disabled, turn pages on button pressed,
not released
* Makes page turning snappier
* Refactors MappedInputManager for readability
---
### AI Usage
Did you use AI tools to help write this code? _**< PARTIALLY>**_
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
* **What is the goal of this PR?** This PR introduces a new "Status Bar"
mode that displays a visual progress bar at the bottom of the screen,
providing readers with a graphical indication of their position within
the book.
* **What changes are included?**
* **Settings**: Updated SettingsActivity to expand the "Status Bar"
configuration with a new option: Full w/ Progress Bar.
* **EPUB Reader**: Modified EpubReaderActivity to calculate the global
book progress and render a progress bar at the bottom of the viewable
area when the new setting is active.
* **TXT Reader**: Modified TxtReaderActivity to implement similar
progress bar rendering logic based on the current page and total page
count.
## Additional Context
* The progress bar is rendered with a height of 4 pixels at the very
bottom of the screen (adjusted for margins).
* The feature reuses the existing renderStatusBar logic but
conditionally draws the bar instead of (or in addition to) other
elements depending on the specific implementation details in each
reader.
* Renamed existing 'Full' mode to 'Full w/ Percentage'
* Added new 'Full w/ Progress Bar' option
<img
src="https://github.com/user-attachments/assets/08c0dd49-c64c-4d4d-9fbb-f576c02d05d9"
width="500">
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
Fixes#487
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _** YES **_
Co-authored-by: Dave Allie <dave@daveallie.com>
