 Mark Veidemanis
|
ebaf8c765d
|
Allow disabling modules in environment variables
|
2022-12-22 07:20:26 +00:00 |
|
Mark Veidemanis
|
ce2d7684bc
|
Run ingest task first
|
2022-12-22 10:11:48 +00:00 |
|
Mark Veidemanis
|
02ff44a6f5
|
Use only one Redis key for the queue to make chunk size more precise for thread allocation
|
2022-09-30 07:22:22 +01:00 |
|
Mark Veidemanis
|
6b549dee6a
|
Reformat
|
2022-09-30 15:23:00 +01:00 |
|
Mark Veidemanis
|
a2f88e29e6
|
Implement uvloop
|
2022-09-23 07:20:30 +01:00 |
|
Mark Veidemanis
|
5ebae02bf2
|
Remove commented code for debugging
|
2022-09-21 10:02:05 +01:00 |
|
Mark Veidemanis
|
ced3a251b2
|
Normalise fields in processing and remove invalid characters
|
2022-09-21 10:01:12 +01:00 |
|
Mark Veidemanis
|
a89b5a8b6f
|
Implement sentiment/NLP annotation and optimise processing
|
2022-09-16 17:09:49 +01:00 |
|
Mark Veidemanis
|
f432e9b29e
|
Properly process Redis buffered messages and ingest into Kafka
|
2022-09-14 18:32:32 +01:00 |
|
Mark Veidemanis
|
c5f01c3084
|
Ingest into Kafka and queue messages better
|
2022-09-13 22:17:46 +01:00 |
|
Mark Veidemanis
|
5c3b338017
|
Implement threshold writing to Redis and manticore ingesting from Redis
|
2022-09-07 07:20:30 +01:00 |
|
Mark Veidemanis
|
aaf2d58d86
|
Remove development dotenv loading
|
2022-09-05 07:20:30 +01:00 |
|
Mark Veidemanis
|
d7e49777ef
|
Add debug statement
|
2022-09-05 07:20:30 +01:00 |
|
Mark Veidemanis
|
6e00f70184
|
Reformat code
|
2022-09-04 21:40:04 +01:00 |
|
Mark Veidemanis
|
db23b31f30
|
Implement aiohttp
|
2022-09-04 19:44:25 +01:00 |
|
Mark Veidemanis
|
f7860bf08b
|
Begin implementing aiohttp
|
2022-09-04 13:47:32 +01:00 |
|
Mark Veidemanis
|
734a2b7879
|
Implement running Discord and 4chan gathering simultaneously
|
2022-09-02 22:30:45 +01:00 |