# Finlytic Assets Service

Finlytic Assets is a scalable C# microservice designed for asset discovery, metadata ingestion, and quick querying. It interfaces directly with the Trade Republic API via WebSockets and exposes a high-performance RPC interface over MQTT to other services in the Finlytic ecosystem.

---

## Architecture Overview

```mermaid
graph TD
    TR[Trade Republic WebSocket API] <-->|WS Protocol| TRS[TradeRepublicService]
    TRS <-->|Ingest| ADS[AssetsFullScanService]
    ADS <-->|Save / Update| DB[PostgreSQL Database]
    DB -->|Trigger Index Update| AIS[AssetsIndexService]
    AIS -->|Write Cache| Index[assets/index/index.json]
    
    MS[Other Finlytic Services] <-->|MQTT Request-Reply| MqttClient[AssetsMqttClient]
    MqttClient <-->|Query Cache| DB
```

---

## Core Features & Workflows

### 1. Automated WebSocket scraping
- **Continuous Scan**: The background worker (`AssetsFullScanService`) iterates through all support asset types:
  - **Stocks** (`StockEntity`)
  - **Crypto** (`CryptoEntity`)
  - **Derivatives** (`DerivativeEntity`)
  - **Bonds** (`BondEntity`)
  - **Funds/ETFs** (`EtfEntity`)
- **Resilient Reconnection**: Uses a robust custom socket wrapper (`TradeRepublicClient`) that closes automatically after 5 minutes of inactivity to mimic human interaction profiles and reconnects dynamically when a request is made.

### 2. Stealth Scheduling with Randomized Jitter
- **Dynamic Delays**: Scanner wait times are read dynamically from database configuration (`Settings`).
- **Jitter Offset**: High/low jitter values (randomized offsets between batches and asset types) are injected to avoid predictable traffic patterns and prevent Trade Republic rate limiting.

### 3. Active vs. Dead Asset Tracking
- **Recency Filter**: Assets are continuously updated with their latest metadata. If an asset has not been updated within **14 days**, it is considered inactive, de-listed, or "dead".
- **Implicit Filtration**: Queries by other services only return active/valid records unless explicitly requested otherwise.

### 4. MQTT RPC Interface
- **Get Asset by ISIN**:
  - Subscribes to: `services/request/assets_Get/#`
  - Returns a list of all active assets matching the specified ISIN across different instrument types (e.g., matching both a stock and its derivative tracker).
- **Omnibox Search**:
  - Subscribes to: `services/request/assets_Search/#`
  - Evaluates search keywords against local records (matching `ISIN`, `Name`, or tags like `Region`, `Country`, `Sector`, etc.).
  - **JIT Fallback**: If a search query is formatted as an unknown valid ISIN, it performs a JIT-lookup directly against the Trade Republic API and registers the discovered asset before replying.

### 5. Local Indexing File (`index.json`)
- **Automatic Regeneration**: On database modifications, `AssetsIndexService` is triggered to regenerate `assets/index/index.json`.
- **Pre-Filtering**: This lean index (containing only `ISIN` and `Name` properties) is written to a shared volume to allow other services to perform instant pre-filtering without querying the database or sending network requests.

---

## Technical Components

| Component | Class / Interface | Responsibility |
| :--- | :--- | :--- |
| **Ingestion Worker** | `AssetsFullScanService` | Runs the main cron loop, scraping and paginating all Trade Republic asset types. |
| **Database Store** | [IAssetsDbService](file:///e:/Projects/FinlyticAssets/FinlyticAssets/Services/AssetsDbService.cs) | Handles CRUD operations, stateful merges, JIT lookups, and active record filtering. |
| **WebSocket Handler** | [ITradeRepublicService](file:///e:/Projects/FinlyticAssets/FinlyticAssets/Services/TradeRepublicService.cs) | Wraps the connection and request dispatching for the Trade Republic API. |
| **Local Indexer** | [IAssetsIndexService](file:///e:/Projects/FinlyticAssets/FinlyticAssets/Services/AssetsIndexService.cs) | Serializes the active assets list to `index.json` for external service indexing. |
| **Messaging Broker** | `AssetsMqttClient` | Listens for incoming MQTT requests, queries the database, and responds to RPC channels. |
| **Config Service** | [ISettingsDbService](file:///e:/Projects/FinlyticAssets/FinlyticAssets/Services/SettingsDbService.cs) | Manages scanner configurations (scan intervals, active pointers, offsets). |