# Formula model notice

Printed-formula reader used by Tinystudy, in the browser (transformers.js, image to LaTeX).

## Source

- Model: `breezedeus/pix2text-mfr` on Hugging Face ("Pix2Text-MFR", TrOCR-style DeiT encoder + TrOCR decoder),
  https://huggingface.co/breezedeus/pix2text-mfr
- Commit: `bea257edb2653f2ae413b084f2ac0e8299d08df0`
- Licence: MIT. The model card (README.md front matter, `license: mit`) declares MIT for the weights, and the
  Pix2Text code repository (https://github.com/breezedeus/Pix2Text) is MIT as well. The model repository has no
  separate LICENSE file, so the licence text of the code repository applies and is reproduced here:

```
MIT License

Copyright (c) 2022 BreezeDeus

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```

## Parent models

Pix2Text-MFR is built on two parent models; their notices:

- TrOCR (Microsoft Corporation): MIT licence (the MIT text above applies in the same form).
- DeiT (Meta Platforms, Inc.): Apache License 2.0. The full text is served at `/vendor/LICENSES/transformers.js.txt` (the standard Apache-2.0 text).

Licences of every other third-party part: https://tinystudy.projektai777.eu/legal/licenses

## What was changed

Rebuilt by our build script (reproducible, pinned tool versions listed in the script header).

- `config.json`, `generation_config.json`, `preprocessor_config.json`, `tokenizer.json`, `tokenizer_config.json`,
  `special_tokens_map.json`: copied unchanged from the commit above.
- `onnx/encoder_model_quantized.onnx`: the original fp32 `encoder_model.onnx` with dynamic int8 (uint8 weights)
  quantization of the MatMul operators (onnxruntime 1.20.1). The small patch-embedding Conv stays fp32.
- `onnx/decoder_model_merged_quantized.onnx`: the source repository only has a decoder exported WITHOUT a
  key/value cache, which transformers.js 4.3.0 cannot generate with. The decoder weights were therefore read
  out of the original ONNX decoder graph into a PyTorch TrOCR decoder (161 tensors, logits match the original
  ONNX decoder to 8.5e-4 on random inputs), exported again with optimum 1.23.3 as a decoder with and without
  cache, quantized the same way (dynamic int8, MatMul only) and merged into one file with `use_cache_branch`.
  The weights are the original weights; only precision (int8) and the graph layout (cache inputs) changed.
- No retraining or fine-tuning.

## Files (size, sha256)

| File | Bytes | sha256 |
| --- | --- | --- |
| onnx/encoder_model_quantized.onnx | 23966224 | 4f3818458895c5bf43007fc23619c8d74efcc624862013eebda9818e9d51b89f |
| onnx/decoder_model_merged_quantized.onnx | 9541238 | 3244260dc28cc8fe3ddd3a84dab182e0749717adb86467f5404b1a141ccef3b2 |

Every file is under 24 MiB (Cloudflare static asset limit is 25 MiB). Load with
`pipeline('image-to-text', <model dir>, { dtype: 'q8' })`.

## Known limits

The model reads images of one printed formula (the model card says it may fail on other images). Handwriting
is not covered.