From 8d6dbcb97062664c13ca2719988cf39399022155 Mon Sep 17 00:00:00 2001 From: Branden J Brown Date: Mon, 31 Mar 2025 09:14:11 -0400 Subject: [PATCH] add architectural info to readme --- README.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/README.md b/README.md index 493967c..409d773 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,34 @@ # 开颜 Kaiyan is an analytics platform for 7TV emotes on Twitch. +It provides various indicators of emote popularity and allows identifying the least used emotes so that streamers can quickly decide which ones to remove when it's time for a cleanup. + +## Architecture + +Kaiyan is built as a collection of small components. + +### Subscription manager + +The subscription manager registers new channels and their 7TV emotes. +It communicates to the indexer to describe emotes which are added or removed in real time. +It also handles channels that revoke their subscription. + +This is the component which handles OAuth2 authentication with Twitch. + +### Message ingest + +Message ingest consumes chat messages, parsing metadata and content. +It emplaces an event for each message onto Kafka for the indexers to process. + +### Indexer + +The indexer records emote usage in ClickHouse, on open-source column store database for online analytical processing. + +### API + +The API computes full metrics on demand from what the indexer has recorded. + +### Load tester + +The load tester is an independent component that pushes a high volume of simulated messages into message ingest. +This produces internal metrics to determine how message ingest, indexers, Kafka, and ClickHouse can scale relative to each other.