Finlytic Assets Service

Finlytic Assets is a scalable C# microservice designed for asset discovery, metadata ingestion, and quick querying. It interfaces directly with the Trade Republic API via WebSockets and exposes a high-performance RPC interface over MQTT to other services in the Finlytic ecosystem.

Architecture Overview

graph TD
    TR[Trade Republic WebSocket API] <-->|WS Protocol| TRS[TradeRepublicService]
    TRS <-->|Ingest| ADS[AssetsFullScanService]
    ADS <-->|Save / Update| DB[PostgreSQL Database]
    DB -->|Trigger Index Update| AIS[AssetsIndexService]
    AIS -->|Write Cache| Index[assets/index/index.json]

    MS[Other Finlytic Services] <-->|MQTT Request-Reply| MqttClient[AssetsMqttClient]
    MqttClient <-->|Query Cache| DB

Core Features & Workflows

1. Automated WebSocket scraping

Continuous Scan: The background worker (AssetsFullScanService) iterates through all supported asset types:
- Stocks (StockEntity)
- Crypto (CryptoEntity)
- Derivatives (DerivativeEntity)
- Bonds (BondEntity)
- Funds/ETFs (EtfEntity)
Resilient Reconnection: Uses a robust custom socket wrapper (TradeRepublicClient) that closes automatically after 5 minutes of inactivity to mimic human interaction profiles and reconnects dynamically when a request is made.

2. Scanner State Persistence & Crash Recovery

Resilient State: The scan loop continuously persists progress to the database (Settings table), recording CurrentScanningType and CurrentScanningPage.
Exit Recovery: Resumes exactly from the last saved page and asset type upon restart.

3. Stealth Scheduling with Randomized Jitter

Dynamic Delays: Scanner wait times are read dynamically from database configuration (Settings).
Jitter Offset: Randomized offsets between batches and asset types to prevent rate limiting.

4. Active vs. Dead Asset Tracking

Recency Filter: Assets updated within 14 days are active. Inactive assets are flagged.

5. MQTT RPC Interface

Get Asset by ISIN: Subscribes to services/request/assets_Get/#.
Omnibox Search: Subscribes to services/request/assets_Search/#.
JIT Fallback: Performs JIT-lookup directly against the Trade Republic API for unknown ISINs.

6. Local Indexing File (`index.json`)

Automatic Regeneration: Serializes active assets list (ISIN, Name) to shared volume assets/index/index.json.

Feature Status

Implemented Features

Trade Republic WebSocket scanner for Stocks, Crypto, Derivatives, Bonds, ETFs.
State persistence & crash recovery in PostgreSQL.
JIT Trade Republic lookup fallback for new ISINs.
Shared local index file generation (index.json).
Zero-Allocation MQTT RPC handlers.
Pure Worker Service architecture (No Kestrel HTTP server).

Planned Features

Real-time WebSocket price tick streaming over MQTT topic finlytic/assets/ticks/{isin}.
Derivative option-chain Greeks calculator integration.

3.0 KiB Raw Blame History