Finlytic Assets Service

Finlytic Assets is a scalable C# microservice designed for asset discovery, metadata ingestion, and quick querying. It interfaces directly with the Trade Republic API via WebSockets and exposes a high-performance RPC interface over MQTT to other services in the Finlytic ecosystem.

Architecture Overview

graph TD
    TR[Trade Republic WebSocket API] <-->|WS Protocol| TRS[TradeRepublicService]
    TRS <-->|Ingest| ADS[AssetsFullScanService]
    ADS <-->|Save / Update| DB[PostgreSQL Database]
    DB -->|Trigger Index Update| AIS[AssetsIndexService]
    AIS -->|Write Cache| Index[assets/index/index.json]
    
    MS[Other Finlytic Services] <-->|MQTT Request-Reply| MqttClient[AssetsMqttClient]
    MqttClient <-->|Query Cache| DB

Core Features & Workflows

1. Automated WebSocket scraping

Continuous Scan: The background worker (AssetsFullScanService) iterates through all support asset types:
- Stocks (StockEntity)
- Crypto (CryptoEntity)
- Derivatives (DerivativeEntity)
- Bonds (BondEntity)
- Funds/ETFs (EtfEntity)
Resilient Reconnection: Uses a robust custom socket wrapper (TradeRepublicClient) that closes automatically after 5 minutes of inactivity to mimic human interaction profiles and reconnects dynamically when a request is made.

2. Stealth Scheduling with Randomized Jitter

Dynamic Delays: Scanner wait times are read dynamically from database configuration (Settings).
Jitter Offset: High/low jitter values (randomized offsets between batches and asset types) are injected to avoid predictable traffic patterns and prevent Trade Republic rate limiting.

3. Active vs. Dead Asset Tracking

Recency Filter: Assets are continuously updated with their latest metadata. If an asset has not been updated within 14 days, it is considered inactive, de-listed, or "dead".
Implicit Filtration: Queries by other services only return active/valid records unless explicitly requested otherwise.

4. MQTT RPC Interface

Get Asset by ISIN:
- Subscribes to: services/request/assets_Get/#
- Returns a list of all active assets matching the specified ISIN across different instrument types (e.g., matching both a stock and its derivative tracker).
Omnibox Search:
- Subscribes to: services/request/assets_Search/#
- Evaluates search keywords against local records (matching ISIN, Name, or tags like Region, Country, Sector, etc.).
- JIT Fallback: If a search query is formatted as an unknown valid ISIN, it performs a JIT-lookup directly against the Trade Republic API and registers the discovered asset before replying.

5. Local Indexing File (`index.json`)

Automatic Regeneration: On database modifications, AssetsIndexService is triggered to regenerate assets/index/index.json.
Pre-Filtering: This lean index (containing only ISIN and Name properties) is written to a shared volume to allow other services to perform instant pre-filtering without querying the database or sending network requests.

Technical Components

Component	Class / Interface	Responsibility
Ingestion Worker	`AssetsFullScanService`	Runs the main cron loop, scraping and paginating all Trade Republic asset types.
Database Store	IAssetsDbService	Handles CRUD operations, stateful merges, JIT lookups, and active record filtering.
WebSocket Handler	ITradeRepublicService	Wraps the connection and request dispatching for the Trade Republic API.
Local Indexer	IAssetsIndexService	Serializes the active assets list to `index.json` for external service indexing.
Messaging Broker	`AssetsMqttClient`	Listens for incoming MQTT requests, queries the database, and responds to RPC channels.
Config Service	ISettingsDbService	Manages scanner configurations (scan intervals, active pointers, offsets).

4.2 KiB Raw Blame History