 Mark Veidemanis
|
ebaf8c765d
|
Allow disabling modules in environment variables
|
1 year ago |
|
Mark Veidemanis
|
ce2d7684bc
|
Run ingest task first
|
1 year ago |
|
Mark Veidemanis
|
02ff44a6f5
|
Use only one Redis key for the queue to make chunk size more precise for thread allocation
|
2 years ago |
|
Mark Veidemanis
|
6b549dee6a
|
Reformat
|
2 years ago |
|
Mark Veidemanis
|
a2f88e29e6
|
Implement uvloop
|
2 years ago |
|
Mark Veidemanis
|
5ebae02bf2
|
Remove commented code for debugging
|
2 years ago |
|
Mark Veidemanis
|
ced3a251b2
|
Normalise fields in processing and remove invalid characters
|
2 years ago |
|
Mark Veidemanis
|
a89b5a8b6f
|
Implement sentiment/NLP annotation and optimise processing
|
2 years ago |
|
Mark Veidemanis
|
f432e9b29e
|
Properly process Redis buffered messages and ingest into Kafka
|
2 years ago |
|
Mark Veidemanis
|
c5f01c3084
|
Ingest into Kafka and queue messages better
|
2 years ago |
|
Mark Veidemanis
|
5c3b338017
|
Implement threshold writing to Redis and manticore ingesting from Redis
|
2 years ago |
|
Mark Veidemanis
|
aaf2d58d86
|
Remove development dotenv loading
|
2 years ago |
|
Mark Veidemanis
|
d7e49777ef
|
Add debug statement
|
2 years ago |
|
Mark Veidemanis
|
6e00f70184
|
Reformat code
|
2 years ago |
|
Mark Veidemanis
|
db23b31f30
|
Implement aiohttp
|
2 years ago |
|
Mark Veidemanis
|
f7860bf08b
|
Begin implementing aiohttp
|
2 years ago |
|
Mark Veidemanis
|
734a2b7879
|
Implement running Discord and 4chan gathering simultaneously
|
2 years ago |