68 lines
3.0 KiB
Markdown
68 lines
3.0 KiB
Markdown
# Finlytic Assets Service
|
|
|
|
Finlytic Assets is a scalable C# microservice designed for asset discovery, metadata ingestion, and quick querying. It interfaces directly with the Trade Republic API via WebSockets and exposes a high-performance RPC interface over MQTT to other services in the Finlytic ecosystem.
|
|
|
|
---
|
|
|
|
## Architecture Overview
|
|
|
|
```mermaid
|
|
graph TD
|
|
TR[Trade Republic WebSocket API] <-->|WS Protocol| TRS[TradeRepublicService]
|
|
TRS <-->|Ingest| ADS[AssetsFullScanService]
|
|
ADS <-->|Save / Update| DB[PostgreSQL Database]
|
|
DB -->|Trigger Index Update| AIS[AssetsIndexService]
|
|
AIS -->|Write Cache| Index[assets/index/index.json]
|
|
|
|
MS[Other Finlytic Services] <-->|MQTT Request-Reply| MqttClient[AssetsMqttClient]
|
|
MqttClient <-->|Query Cache| DB
|
|
```
|
|
|
|
---
|
|
|
|
## Core Features & Workflows
|
|
|
|
### 1. Automated WebSocket scraping
|
|
- **Continuous Scan**: The background worker (`AssetsFullScanService`) iterates through all supported asset types:
|
|
- **Stocks** (`StockEntity`)
|
|
- **Crypto** (`CryptoEntity`)
|
|
- **Derivatives** (`DerivativeEntity`)
|
|
- **Bonds** (`BondEntity`)
|
|
- **Funds/ETFs** (`EtfEntity`)
|
|
- **Resilient Reconnection**: Uses a robust custom socket wrapper (`TradeRepublicClient`) that closes automatically after 5 minutes of inactivity to mimic human interaction profiles and reconnects dynamically when a request is made.
|
|
|
|
### 2. Scanner State Persistence & Crash Recovery
|
|
- **Resilient State**: The scan loop continuously persists progress to the database (`Settings` table), recording `CurrentScanningType` and `CurrentScanningPage`.
|
|
- **Exit Recovery**: Resumes exactly from the last saved page and asset type upon restart.
|
|
|
|
### 3. Stealth Scheduling with Randomized Jitter
|
|
- **Dynamic Delays**: Scanner wait times are read dynamically from database configuration (`Settings`).
|
|
- **Jitter Offset**: Randomized offsets between batches and asset types to prevent rate limiting.
|
|
|
|
### 4. Active vs. Dead Asset Tracking
|
|
- **Recency Filter**: Assets updated within 14 days are active. Inactive assets are flagged.
|
|
|
|
### 5. MQTT RPC Interface
|
|
- **Get Asset by ISIN**: Subscribes to `services/request/assets_Get/#`.
|
|
- **Omnibox Search**: Subscribes to `services/request/assets_Search/#`.
|
|
- **JIT Fallback**: Performs JIT-lookup directly against the Trade Republic API for unknown ISINs.
|
|
|
|
### 6. Local Indexing File (`index.json`)
|
|
- **Automatic Regeneration**: Serializes active assets list (`ISIN`, `Name`) to shared volume `assets/index/index.json`.
|
|
|
|
---
|
|
|
|
## Feature Status
|
|
|
|
### Implemented Features
|
|
- [x] Trade Republic WebSocket scanner for Stocks, Crypto, Derivatives, Bonds, ETFs.
|
|
- [x] State persistence & crash recovery in PostgreSQL.
|
|
- [x] JIT Trade Republic lookup fallback for new ISINs.
|
|
- [x] Shared local index file generation (`index.json`).
|
|
- [x] Zero-Allocation MQTT RPC handlers.
|
|
- [x] Pure Worker Service architecture (No Kestrel HTTP server).
|
|
|
|
### Planned Features
|
|
- [ ] Real-time WebSocket price tick streaming over MQTT topic `finlytic/assets/ticks/{isin}`.
|
|
- [ ] Derivative option-chain Greeks calculator integration.
|