zephyr/kaiyan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22 Commits 1 Branch 0 Tags 82 KiB
Go 100%
main
Go to file
Branden J Brown dba10cee1a kaiyan-api: implement api server
2025-04-20 23:16:05 -04:00
cmd kaiyan-api: implement api server 2025-04-20 23:16:05 -04:00
emote emote: add json annotations 2025-04-20 22:22:51 -04:00
ingest ingest: add wire format for incoming messages 2025-04-06 09:20:32 -04:00
queue queue: include send time in messages 2025-04-20 13:55:36 -04:00
twitch twitch: implement webhook challenge handling 2025-04-15 19:21:02 -04:00
go.mod emote/sqlitestore: implement index in terms of sqlite 2025-04-20 10:45:32 -04:00
go.sum emote/sqlitestore: implement index in terms of sqlite 2025-04-20 10:45:32 -04:00
README.md add architectural info to readme 2025-03-31 09:14:11 -04:00

README.md

开颜

Kaiyan is an analytics platform for 7TV emotes on Twitch. It provides various indicators of emote popularity and allows identifying the least used emotes so that streamers can quickly decide which ones to remove when it's time for a cleanup.

Architecture

Kaiyan is built as a collection of small components.

Subscription manager

The subscription manager registers new channels and their 7TV emotes. It communicates to the indexer to describe emotes which are added or removed in real time. It also handles channels that revoke their subscription.

This is the component which handles OAuth2 authentication with Twitch.

Message ingest

Message ingest consumes chat messages, parsing metadata and content. It emplaces an event for each message onto Kafka for the indexers to process.

Indexer

The indexer records emote usage in ClickHouse, on open-source column store database for online analytical processing.

API

The API computes full metrics on demand from what the indexer has recorded.

Load tester

The load tester is an independent component that pushes a high volume of simulated messages into message ingest. This produces internal metrics to determine how message ingest, indexers, Kafka, and ClickHouse can scale relative to each other.