XRechnung IMAP Scanner

A Go service that connects to an IMAP server, monitors mailboxes in real-time via IMAP IDLE, detects XRechnung invoices (UBL / UN-CEFACT CII XML and Factur-X hybrid PDFs), and applies a custom IMAP flag to matching messages for easy filtering in mail clients.

User Guide — Thunderbird setup: see and filter tagged invoices

Administration Guide — installation, configuration, deployment

Features

• IMAP IDLE listener: Real-time monitoring of incoming mail; auto-reconnect on connection drop
• Multiple mailboxes: Monitor several mailboxes in parallel, each with its own IMAP connection
• Mailbox heartbeat logs: Every 60 minutes per mailbox, logs a summary window with processed/tagged/errors/reconnects and initial/startup/idle source counters
• XRechnung detection:
  • UBL Invoice (namespace: urn:oasis:names:specification:ubl:schema:xsd:Invoice-2)
  • UN-CEFACT CII (namespace: urn:un:unece:uncefact:data:standard:CrossIndustryInvoice:100)
  • CustomizationID validation against XRechnung profiles
  • Factur-X/ZUGFeRD hybrid PDFs (embedded XML via PDF object model, pdfcpu)
• Persistent state: Tracks UIDVALIDITY + last-scanned UID per mailbox; no re-scan on restart
• Custom IMAP keyword: Configurable keyword name (default: ius-xrechnung)
• Secure deployment: systemd unit with hardening, env var config, graceful shutdown

Quick Start

go build -o xr-invoiced ./cmd/xr-invoiced/
./xr-invoiced --doctor                    # verify IMAP credentials
./xr-invoiced --since "last month"        # preview what would be tagged
./xr-invoiced --apply                     # tag messages for real, enter IDLE

How It Works

1. Startup: Loads config, connects to IMAP (one connection per mailbox)
2. Initial scan: On first run or after UIDVALIDITY change, scans all existing messages; logs are marked scan=initial
3. State/Resume scan: On normal startup, resumes from saved state and scans only messages newer than the last-seen UID; logs are marked scan=startup
4. State persistence: Saves per-mailbox UIDVALIDITY + LastUID to JSON file
5. IDLE loop: Enters IMAP IDLE and listens for new messages; after wake-up, every processed message is logged with scan=idle
6. Message processing: For each new message:
   • Fetches full body + headers
   • Walks MIME attachments
   • Checks each attachment for XRechnung
   • Applies custom flag if match found
7. Reconnection: On IDLE timeout (≈28 min) or connection drop, auto-reconnects and resumes
8. Heartbeat: Every 60 minutes per mailbox, emits heartbeat with counters for the previous 60-minute window:
   • processed, tagged, errors, reconnects
   • initial, startup, idle

Architecture

cmd/xr-invoiced/main.go        Entry point; flag parsing; per-mailbox goroutines; signal handling
internal/config/config.go      Viper-based YAML + env override
internal/imap/client.go        TLS/STARTTLS connection, UID fetch, flag storage
internal/imap/idle.go          IDLE listener with auto-reconnect
internal/scanner/scanner.go    Orchestrator: fetch → walk → detect → flag
internal/scanner/attachment.go MIME part walker
internal/state/state.go        Per-mailbox UIDVALIDITY + LastUID persistence
internal/xrechnung/detector.go XML namespace + CustomizationID validation
internal/xrechnung/pdf.go      Factur-X/ZUGFeRD PDF embedded-file extraction (pdfcpu)

Known Limitations & Future Work

• SASL mechanisms: Uses plain LOGIN; no OAuth2/SASL XOAUTH2 support
• Email notifications: No alerting on scan errors; structured log output only
• Parallel processing: Messages are processed sequentially; parallel fetch/detect is a planned enhancement

License

MIT (or your chosen license)

References

• IMAP RFC 3501 & RFC 2177 (IDLE)
• XRechnung Specification
• Factur-X / ZUGFeRD Standard
• go-imap Documentation

Glossary

Term	Meaning
Flag	Generic IMAP per-message attribute (RFC 3501). Covers system flags and keywords.
System flag	Predefined by the RFC, backslash-prefixed: \Seen, \Answered, \Flagged, \Deleted, \Draft.
Keyword	User-defined flag without backslash, e.g. ius-xrechnung. This is what xr-invoiced sets.
Tag	Thunderbird's UI name for an IMAP keyword. Same data, different vocabulary.