9 Commits

Author	SHA1	Message	Date
Mark Veidemanis	817bfd8835	Time stuff and switch to gensim for tokenisation	2022-10-01 14:46:45 +01:00
Mark Veidemanis	63081f68b7	Use only one Redis key for the queue to make chunk size more precise for thread allocation	2022-09-30 07:22:22 +01:00
Mark Veidemanis	fc7450c33a	Make debug output cleaner	2022-09-22 17:39:29 +01:00
Mark Veidemanis	64ebcedd76	Remove debugging code and fix regex substitution	2022-09-21 12:48:54 +01:00
Mark Veidemanis	cf4aa45663	Normalise fields in processing and remove invalid characters	2022-09-21 10:01:12 +01:00
Mark Veidemanis	6de17063a2	Make CPU threads configurable	2022-09-20 22:29:13 +01:00
Mark Veidemanis	143f2a0bf0	Implement sentiment/NLP annotation and optimise processing	2022-09-16 17:09:49 +01:00
Mark Veidemanis	4ea77ac543	Properly process Redis buffered messages and ingest into Kafka	2022-09-14 18:32:32 +01:00
Mark Veidemanis	fec0d379a6	Ingest into Kafka and queue messages better	2022-09-13 22:17:46 +01:00