firefly-statement-importer

A small web app that turns Philippine bank and credit card statements into CSV, shows a preview, and imports the confirmed rows into Firefly III through the Firefly III data importer's POST /autoupload endpoint. Parsing is deterministic: it reads the PDF text layer with pdfplumber, with no OCR and no LLM.

Supported inputs

Each parser tags a statement with a source id taken from the document itself.

Input	Source id
BPI credit card statement PDF (BE<yyyymmdd>.pdf)	bpi:card
BDO credit card statement of account PDF	bdo:<product>, the product name printed above the statement date, lowercased with non-alphanumerics as -
UnionBank credit card eStatement PDF	unionbank:card
MariBank statement PDF (MariBank_Statement_*.pdf)	maribank:savings
BDO account CSV export (17 columns, Account number(BBAN),...)	bdo:savings-export
3-column CSV (transaction date,transaction details,amount)	none; the account is chosen on the upload form

Encrypted PDFs take a password on the upload form. The BPI statement date is only printed as an image, so the year comes from the file name.

Accounts

Accounts are not part of the repository. At startup the app reads $ACCOUNTS_DIR/accounts.yaml (default /config/accounts) and refuses to start if it is missing or invalid. It is a list of accounts:

- key: example_card            # lowercase letters, digits, _; part of every reference
  label: Example Bank Card     # shown in the UI
  config: example_card.json    # data importer config, in the same directory
  match: bdo:example-card      # source id that selects this account, or null
- key: example_account
  label: Example Bank Account
  config: example_account.json
  match: null                  # only reachable by choosing it for a 3-column CSV

Each config is a data importer JSON export whose last role is external-id, with duplicate_detection_method: cell, unique_column_index pointing at that last column, unique_column_type: external-id and ignore_duplicate_transactions: false. The output CSV must have exactly as many columns as the config has roles. An amount_negated role labels the amount column Amount (Credit Card), otherwise Amount. A statement whose source id matches no account is rejected with the id in the message, so it can be added as a match. tests/fixtures/accounts is a complete synthetic example.

Duplicate-safe references

Every output row gets a Reference column: the first 16 hex characters of SHA-256 over key|date|amount|n, where key is the account key and n counts rows with the same date and amount earlier in the file. Two genuine identical transactions therefore stay distinct, and re-importing the same statement yields the same references even if a description is parsed slightly differently. The account configs map that column to external-id and use the importer's cell duplicate detection on it.

Running

uv sync
uv run uvicorn firefly_statement_importer.app:app --host 0.0.0.0 --port 8080

The container image runs the same command as uid 1000, needs only a writable /tmp, and expects the accounts directory mounted at /config/accounts.

The pages load nothing from outside the app. Fonts, styles and the small enhancement script are served from /static. The bundled Public Sans font is under the SIL Open Font License; its text is in static/fonts/PublicSans-OFL.txt. Encrypted PDFs and 3-column CSVs work without JavaScript: the server shows the password or account field and asks for the file again.

Variable	Default	Purpose
ACCOUNTS_DIR	/config/accounts	Directory with accounts.yaml and the importer configs
IMPORTER_URL	http://firefly-iii-data-importer.default.svc.cluster.local:8080	Data importer base URL
AUTO_IMPORT_SECRET	unset	Must match the importer's AUTO_IMPORT_SECRET; previews work without it
FIREFLY_III_ACCESS_TOKEN	unset	Personal access token for read-only checks against Firefly; without it the checks are skipped
FIREFLY_III_URL	http://firefly-iii.default.svc.cluster.local:8080	Firefly III base URL for those checks
FIREFLY_III_PUBLIC_URL	unset	Browser-facing Firefly URL; when set, the result page links to the account

With a token, the review page unticks rows whose reference is already on the account in Firefly (matched through the config's default_account). While an import runs, the page polls Firefly to show how many rows have arrived, and the result page lists each row as imported, already existing or missing.

The data importer needs CAN_POST_FILES=true, AUTO_IMPORT_SECRET (at least 16 characters), and a working FIREFLY_III_URL plus FIREFLY_III_ACCESS_TOKEN.

Development

uv run ruff check src/ tests/ scripts/
uv run pyright
uv run pytest tests/ -v
# compares against private reference CSVs, using a private accounts directory
uv run python scripts/golden.py /path/to/statements --accounts /path/to/accounts